Harvmaster/autocomplete-model
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity
Files
d246fb9349a6874e40f61006e816c5f19a06d84e
autocomplete-model/models/README.md
T
HarvmasterandClaude Opus 5.5 d246fb9349
ci / build (push) Canceled after 0s
Details
Models stage-d (default) + stage-e (next-edit companion); extension 0.6.1 
- stage-d: next-line / block-close / multi-line fixes, symbols format 2, TS errors in the header.
  Held-out: 0.426 exact / 0.700 edit sim (stage-c 0.405 / 0.689); block close 0.27 -> 0.79.
- stage-e: next-edit (80% exact region rewrites, abstains on 100% of no-edit samples).
- Manifest: nextEditModel (companion) and nextEditFormat (older extensions ignore it, so they
  never use next-edit with the empty-output deletion bug).
- Daemon: next-edit served by the companion model in a second llama-server, started after the
  main model loads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
2026-10-04 00:23:15 +00:00

87 lines
5.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Models
Q8_0 GGUF fill-in-the-middle models for the [Local Autocomplete](../extensions/vscode) extension, fine-tuned from
[Qwen2.5-Coder-0.5B](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B) for **TypeScript, Vue and Node.js**.
| model | description | prompt order | context modes | size |
|---|---|---|---|---|
| `stage-d` | Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions | spm | symbols, simple | 531 MB |
| `stage-e` | Next-edit prediction (rewrites a region around the cursor or answers "no edit"); also does FIM, slightly below stage-d | spm | symbols, simple | 531 MB |
| `stage-c` | Suffix-first FIM with repo/symbol-context headers (default) | spm | symbols, simple | 531 MB |
| `stage-b` | Suffix-first FIM without context headers (previous stage) | spm | raw | 531 MB |
The extension reads [`manifest.json`](manifest.json), downloads the selected model once, verifies its SHA-256 and
runs it locally with llama.cpp. The files are stored with Git LFS; download links use Gitea's `/media/` route:
https://git.harvmaster.com/Harvmaster/autocomplete-model/media/branch/main/models/<file>
## Evaluation (612 held-out samples from private repos the model never saw, suffix-first prompts)
| | exact match | edit similarity | still parses |
|---|---:|---:|---:|
| Qwen2.5-Coder-0.5B (base) | 0.270 | 0.506 | 0.783 |
| Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 |
| `stage-b` | 0.391 | 0.689 | 0.952 |
| `stage-c` (with symbol context) | 0.397 | 0.681 | 0.954 |
| `stage-c` + token healing (daemon default) | 0.405 | 0.689 | |
| `stage-d` (symbols + TS errors, healed) | **0.426** | **0.700** | 0.954 |
| `stage-e` (symbols + TS errors, healed) | 0.412 | 0.691 | 0.962 |
New behaviours (886 further held-out samples), exact match, stage-c → stage-d: closing a block 0.27 → **0.79**,
next line 0.01 → **0.18**, rest of block 0.08 → 0.11, member access 0.63 → 0.66.
**Next-edit** (`stage-e`, 445 held-out samples): 80% exact region rewrites (edit similarity 0.99) where an edit
was due, and "no edit" on 100% of samples where none was. `stage-d` uses `stage-e` as its next-edit companion
(`nextEditModel` in the manifest): the extension downloads both and the daemon runs it in a second llama-server.
The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts
let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a
laptop CPU.
## Using the files directly
Any llama.cpp-based runtime works, but the prompt must match the training layout. The
[daemon](../daemon) and [`shared/prompt.ts`](../shared/prompt.ts) are the reference.
Suffix-first (`spm`) order:
<|fim_prefix|><|fim_suffix|>{suffix}<|fim_middle|>{prefix} → model generates the middle, then <|endoftext|>
`stage-c` additionally expects a repo-level header: `<|repo_name|>{repo}\n<|file_sep|>{path}\n`, optionally with a
`__symbols__.d.ts` pseudo-file of available symbols before it.
## Training data
Fine-tuned on permissively licensed open-source repositories (below, pinned commits) and on the publisher's own
code (not published). Secrets were scrubbed from all training data; held-out evaluation repos were never trained on.
| repository | commit | license |
|---|---|---|
| [github.com/WiseLibs/better-sqlite3](https://github.com/WiseLibs/better-sqlite3) | `f8e2d54120` | MIT |
| [github.com/CashScript/cashscript](https://github.com/CashScript/cashscript) | `71a243b656` | MIT |
| [gitlab.com/electrum-cash/network](https://gitlab.com/electrum-cash/network) | `c7511725a9` | MIT |
| [github.com/element-plus/element-plus](https://github.com/element-plus/element-plus) | `ad514f79ba` | MIT |
| [github.com/fastify/fastify](https://github.com/fastify/fastify) | `19d5be0daf` | MIT |
| [github.com/h3js/h3](https://github.com/h3js/h3) | `3afe46b170` | MIT |
| [github.com/kysely-org/kysely](https://github.com/kysely-org/kysely) | `4b7287e71b` | MIT |
| [github.com/bitauth/libauth](https://github.com/bitauth/libauth) | `60aec239cc` | MIT |
| [github.com/nuxt/ui](https://github.com/nuxt/ui) | `e8dd339dfd` | MIT |
| [github.com/unjs/ofetch](https://github.com/unjs/ofetch) | `1dbc37fd1c` | MIT |
| [github.com/vuejs/pinia](https://github.com/vuejs/pinia) | `98587ca465` | MIT |
| [github.com/quasarframework/quasar](https://github.com/quasarframework/quasar) | `0fd7ce30e7` | MIT |
| [github.com/unovue/reka-ui](https://github.com/unovue/reka-ui) | `e9230140ed` | MIT |
| [github.com/unovue/shadcn-vue](https://github.com/unovue/shadcn-vue) | `f12a6a098d` | MIT |
| [github.com/vitejs/vite](https://github.com/vitejs/vite) | `10033218d2` | MIT |
| [github.com/vuejs/vitepress](https://github.com/vuejs/vitepress) | `56fa6c9df1` | MIT |
| [github.com/vitest-dev/vitest](https://github.com/vitest-dev/vitest) | `964e1a4404` | MIT |
| [github.com/vuejs/core](https://github.com/vuejs/core) | `4ab865a848` | MIT |
| [github.com/intlify/vue-i18n](https://github.com/intlify/vue-i18n) | `6ef45366e1` | MIT |
| [github.com/vuejs/router](https://github.com/vuejs/router) | `07663eb6be` | MIT |
| [github.com/vbenjs/vue-vben-admin](https://github.com/vbenjs/vue-vben-admin) | `50f4ede309` | MIT |
| [github.com/vueuse/vueuse](https://github.com/vueuse/vueuse) | `efdd69a148` | MIT |
| [github.com/websockets/ws](https://github.com/websockets/ws) | `297202cdae` | MIT |
| [github.com/colinhacks/zod](https://github.com/colinhacks/zod) | `0b216ef674` | MIT |
## License
Apache-2.0 ([LICENSE](LICENSE), [NOTICE](NOTICE)), as a derivative of Qwen2.5-Coder.
Reference in New Issue View Git Blame Copy Permalink