autocomplete-model/README.md

# Local Autocomplete

Fast, private, inline code completion for **TypeScript, Vue and Node.js**, from a small model that runs on your own machine.

- **Local:** a 0.5B model (fine-tuned Qwen2.5-Coder) runs through llama.cpp on your CPU or Apple GPU. No code leaves your machine.
- **Fast:** ~90–100 ms per keystroke on a laptop CPU. Prompts are laid out suffix-first so the KV cache survives typing (~2 tokens re-processed per keystroke).
- **Context-aware:** an optional `symbols` mode feeds the model signatures from the TypeScript language service (and, for newer models, nearby TypeScript errors); a missing `this.` is repaired automatically, and half-typed words are completed properly (token healing).
- **Next-edit suggestions** (with models trained for it): edits beyond the cursor, like removing a leftover `)` or updating the uses of a renamed variable. **Tab** applies, **Esc** dismisses.
- **Zero setup:** the VS Code extension downloads the model and llama.cpp on first use and manages the background daemon itself.

| Model (held-out private code, never trained on) | exact match | edit similarity |
|---|---:|---:|
| Qwen2.5-Coder-0.5B base | 0.270 | 0.506 |
| Qwen2.5-Coder-1.5B base | 0.366 | 0.623 |
| **`stage-c` (this repo)** | **0.397** | **0.681** |

See [models/](models/) for details.

## Quick start

1. Download the latest `local-autocomplete-<version>.vsix` from **[Releases](https://git.harvmaster.com/Harvmaster/autocomplete-model/releases)**.
2. Install it: `code --install-extension local-autocomplete-<version>.vsix`, or in VS Code: Extensions → `⋯` → *Install from VSIX…*
3. Open a TypeScript or Vue file. On first use the extension downloads the model (~530 MB) and llama.cpp (~15 MB), then starts. A progress notification shows this.
4. Start typing: ghost text appears. **Tab** accepts, **Ctrl/Cmd+→** accepts word by word.

Turn off other inline-completion providers (Copilot, …) so they don't compete.

**Remote-SSH:** install the extension in the remote window. Everything (model, daemon, symbol lookups) then runs on the remote machine, next to your code.

**Cursor:** the VSIX installs in Cursor too.

### The status bar

`AC: simple · 42 ms`. Click it for the menu:

- **Context mode:**
  - `symbols`: available symbols from the TypeScript language service (best for TS/Vue projects)
  - `simple`: repo + file path only
  - `raw`: no header, for older models

  The menu only offers modes the active model was trained for.
- **Model:** switch between models in the manifest (downloaded on first use).
- **On/off**, **Restart daemon**, **Open daemon log**, **Show last prompt** (exactly what the model saw, for debugging a suggestion), **Check for updates**.

### Settings

| setting | default | |
|---|---|---|
| `localAutocomplete.modelManifestUrl` | this repo's `models/manifest.json` | where models are downloaded from |
| `localAutocomplete.model` | manifest default | which model to use |
| `localAutocomplete.modelPath` | | use a local `.gguf` instead (set `modelProfile` to match its prompt layout) |
| `localAutocomplete.llamaServerPath` | | use your own `llama-server` (e.g. Homebrew's) instead of the downloaded build |
| `localAutocomplete.excludeGlobs` | `.env*`, keys, `secrets/`, … | never sent to the model |
| `localAutocomplete.nextEdit.enabled` | `true` | next-edit suggestions (only with models trained for it) |
| `localAutocomplete.triggerOnCursorMove` | `false` | also suggest when you move the cursor and pause |
| `localAutocomplete.prewarmOnFocus` | `false` | fill the model cache when you switch files (faster first keystroke) |
| `localAutocomplete.recordEdits` | `false` | record your edits (and diagnostics) locally, as future training data |
| `localAutocomplete.daemon.mode` | `managed` | `external` = use a daemon you run yourself (`serverUrl`) |

## How it works

```
VS Code extension ──HTTP──▶ acd daemon ──▶ llama-server (llama.cpp, CPU / Metal)
 ghost text, toggle,         builds the exact prompt layout the model was trained on,
 edit recorder               symbol context (TypeScript language service, worker thread),
                             KV-cache-friendly truncation, cancellation, this.-repair
```

- In **managed mode** (the default) the extension bundles the daemon and runs it on VS Code's own Node runtime. One daemon is shared by all windows, and it exits when idle.
- The daemon can also run standalone ([daemon/](daemon/)); that's how other editors (Zed is planned) and the `acctl` CLI use it.
- Design details: [docs/architecture.md](docs/architecture.md).

## Repository layout

| path | |
|---|---|
| [`models/`](models/) | GGUF models (Git LFS), `manifest.json`, model card, license |
| [`extensions/vscode/`](extensions/vscode/) | the VS Code extension |
| [`extensions/zed/`](extensions/zed/) | Zed setup (settings for the daemon's OpenAI-compatible endpoint) |
| [`daemon/`](daemon/) | `acd`, the completion daemon (bundled into the extension; also runs standalone) |
| [`shared/`](shared/) | prompt layout + symbol extraction, shared with the training pipeline so prompts match training exactly |
| [`scripts/`](scripts/) | publish models and extension releases |
| [`docs/`](docs/) | architecture |

## Building from source

Requires **Node ≥ 22.18** (the daemon and shared code run as TypeScript directly) and **Git LFS** for the models.

```sh
git clone https://git.harvmaster.com/Harvmaster/autocomplete-model.git local-autocomplete
# code only, without the ~1 GB of models:  GIT_LFS_SKIP_SMUDGE=1 git clone …
cd local-autocomplete
npm run setup          # installs shared/, daemon/ and extensions/vscode/ dependencies
npm run build          # → extensions/vscode/local-autocomplete-<version>.vsix
npm test
```

Running the daemon standalone (external mode) is covered in [daemon/README.md](daemon/README.md).

## Releasing

**Extension:**
1. Bump `version` in `extensions/vscode/package.json` and add a `## vscode <version>` section to [CHANGELOG.md](CHANGELOG.md).
2. Run `npm run build`, then commit and push.
3. Run `npm run release:extension`. This creates the `vscode-v<version>` release with the VSIX attached, using a Gitea token (see the script header).

**Models:**
1. Run `node scripts/make-model-release.mjs --from <dir> stage-d stage-c`. It updates `models/` (hard links, manifest, model card).
2. Commit and push. Installed extensions pick up the new manifest on their next start.

## Privacy

- **Nothing leaves your machine.** Completions run locally, and the only network access is the one-time model and llama.cpp download.
- **Excluded files are never read:** anything matching `excludeGlobs` (`.env*`, keys, …) is never sent to the model or recorded.
- **Telemetry and edit recording are off by default.** If you enable them, they're written to local files only.

## Roadmap

- Better multi-line completions: continuing onto the next line, closing blocks, whole methods.
- **Next-edit prediction:** e.g. removing a leftover `)` after unwrapping a call, or updating call sites after adding a parameter.
- **TypeScript errors as context:** the model sees nearby compiler errors and is trained on error → fix.
- **Suggestions on cursor move:** predict what you came to a location to do, not just what you're typing.
- Token healing for completions that start mid-identifier.
- Zed: the daemon's OpenAI-compatible endpoint is in place ([extensions/zed](extensions/zed/)); needs testing against real Zed traffic.

## License

Code: [MIT](LICENSE). Models: Apache-2.0 ([models/LICENSE](models/LICENSE), [models/NOTICE](models/NOTICE)), as derivatives of Qwen2.5-Coder.