autocomplete-model/README.md

Local Autocomplete

Fast, private, inline code completion for TypeScript, Vue and Node.js, from a small model that runs on your own machine.

• Local: a 0.5B model (fine-tuned Qwen2.5-Coder) runs through llama.cpp on your CPU or Apple GPU. No code leaves your machine.
• Fast: ~90–100 ms per keystroke on a laptop CPU. Prompts are laid out suffix-first so the KV cache survives typing (~2 tokens re-processed per keystroke).
• Context-aware: an optional symbols mode feeds the model signatures from the TypeScript language service (and, for newer models, nearby TypeScript errors); a missing this. is repaired automatically, and half-typed words are completed properly (token healing).
• Next-edit suggestions (with models trained for it): edits beyond the cursor, like removing a leftover ) or updating the uses of a renamed variable. Tab applies, Esc dismisses.
• Zero setup: the VS Code extension downloads the model and llama.cpp on first use and manages the background daemon itself.

Model (held-out private code, never trained on)	exact match	edit similarity
Qwen2.5-Coder-0.5B base	0.270	0.506
Qwen2.5-Coder-1.5B base	0.366	0.623
stage-c (this repo)	0.397	0.681

See models/ for details.

Quick start

1. Download the latest local-autocomplete-<version>.vsix from Releases.
2. Install it: code --install-extension local-autocomplete-<version>.vsix, or in VS Code: Extensions → ⋯ → Install from VSIX…
3. Open a TypeScript or Vue file. On first use the extension downloads the model (~530 MB) and llama.cpp (~15 MB), then starts. A progress notification shows this.
4. Start typing: ghost text appears. Tab accepts, Ctrl/Cmd+→ accepts word by word.

Turn off other inline-completion providers (Copilot, …) so they don't compete.

Remote-SSH: install the extension in the remote window. Everything (model, daemon, symbol lookups) then runs on the remote machine, next to your code.

Cursor: the VSIX installs in Cursor too.

The status bar

AC: simple · 42 ms. Click it for the menu:

• Context mode:

  • symbols: available symbols from the TypeScript language service (best for TS/Vue projects)
  • simple: repo + file path only
  • raw: no header, for older models

  The menu only offers modes the active model was trained for.

• Model: switch between models in the manifest (downloaded on first use).

• On/off, Restart daemon, Open daemon log, Show last prompt (exactly what the model saw, for debugging a suggestion), Check for updates.

Settings

setting	default
localAutocomplete.modelManifestUrl	this repo's models/manifest.json	where models are downloaded from
localAutocomplete.model	manifest default	which model to use
localAutocomplete.modelPath		use a local .gguf instead (set modelProfile to match its prompt layout)
localAutocomplete.llamaServerPath		use your own llama-server (e.g. Homebrew's) instead of the downloaded build
localAutocomplete.excludeGlobs	.env*, keys, secrets/, …	never sent to the model
localAutocomplete.nextEdit.enabled	true	next-edit suggestions (only with models trained for it)
localAutocomplete.triggerOnCursorMove	false	also suggest when you move the cursor and pause
localAutocomplete.prewarmOnFocus	false	fill the model cache when you switch files (faster first keystroke)
localAutocomplete.recordEdits	false	record your edits (and diagnostics) locally, as future training data
localAutocomplete.daemon.mode	managed	external = use a daemon you run yourself (serverUrl)

How it works

VS Code extension ──HTTP──▶ acd daemon ──▶ llama-server (llama.cpp, CPU / Metal)
 ghost text, toggle,         builds the exact prompt layout the model was trained on,
 edit recorder               symbol context (TypeScript language service, worker thread),
                             KV-cache-friendly truncation, cancellation, this.-repair

• In managed mode (the default) the extension bundles the daemon and runs it on VS Code's own Node runtime. One daemon is shared by all windows, and it exits when idle.
• The daemon can also run standalone (daemon/); that's how other editors (Zed is planned) and the acctl CLI use it.
• Design details: docs/architecture.md.

Repository layout

path
models/	GGUF models (Git LFS), manifest.json, model card, license
extensions/vscode/	the VS Code extension
daemon/	acd, the completion daemon (bundled into the extension; also runs standalone)
shared/	prompt layout + symbol extraction, shared with the training pipeline so prompts match training exactly
scripts/	publish models and extension releases
docs/	architecture

Building from source

Requires Node ≥ 22.18 (the daemon and shared code run as TypeScript directly) and Git LFS for the models.

git clone https://git.harvmaster.com/Harvmaster/autocomplete-model.git local-autocomplete
# code only, without the ~1 GB of models:  GIT_LFS_SKIP_SMUDGE=1 git clone …
cd local-autocomplete
npm run setup          # installs shared/, daemon/ and extensions/vscode/ dependencies
npm run build          # → extensions/vscode/local-autocomplete-<version>.vsix
npm test

Running the daemon standalone (external mode) is covered in daemon/README.md.

Releasing

Extension:

1. Bump version in extensions/vscode/package.json and add a ## vscode <version> section to CHANGELOG.md.
2. Run npm run build, then commit and push.
3. Run npm run release:extension. This creates the vscode-v<version> release with the VSIX attached, using a Gitea token (see the script header).

Models:

1. Run node scripts/make-model-release.mjs --from <dir> stage-d stage-c. It updates models/ (hard links, manifest, model card).
2. Commit and push. Installed extensions pick up the new manifest on their next start.

Privacy

• Nothing leaves your machine. Completions run locally, and the only network access is the one-time model and llama.cpp download.
• Excluded files are never read: anything matching excludeGlobs (.env*, keys, …) is never sent to the model or recorded.
• Telemetry and edit recording are off by default. If you enable them, they're written to local files only.

Roadmap

• Better multi-line completions: continuing onto the next line, closing blocks, whole methods.
• Next-edit prediction: e.g. removing a leftover ) after unwrapping a call, or updating call sites after adding a parameter.
• TypeScript errors as context: the model sees nearby compiler errors and is trained on error → fix.
• Suggestions on cursor move: predict what you came to a location to do, not just what you're typing.
• Token healing for completions that start mid-identifier.
• Zed support via the daemon's OpenAI-compatible endpoint.

License

Code: MIT. Models: Apache-2.0 (models/LICENSE, models/NOTICE), as derivatives of Qwen2.5-Coder.