Next-edit: n-gram speculative decoding on the companion server (0.6.2)
ci / build (push) Canceled after 0s
ci / build (push) Canceled after 0s
The rewritten region is mostly a copy of the current one, so prompt-lookup drafts are accepted in long runs. Greedy decoding keeps the output identical. Laptop CPU, 160-token region rewrite: generation 1.76 s -> 0.37 s. Timings now include errorsMs/promptMs/genMs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
This commit is contained in:
@@ -2,6 +2,13 @@
|
||||
|
||||
Extension versions are released as `vscode-v<version>` on the [Releases](https://git.harvmaster.com/Harvmaster/autocomplete-model/releases) page. Models are versioned by their name in [`models/manifest.json`](models/manifest.json).
|
||||
|
||||
## vscode 0.6.2
|
||||
|
||||
- **Next-edit is ~3–5× faster:** the next-edit server uses n-gram speculative decoding (the rewritten region is mostly
|
||||
a copy of the current one, so drafted tokens are verified in batches). Same output, since decoding is greedy.
|
||||
Rewriting a 13-line region on a laptop CPU: ~2.1 s → ~0.7 s (~0.4 s when the prompt is cached).
|
||||
- Next-edit timings in "Show Last Prompt" now split TS errors / prompt / generation time.
|
||||
|
||||
## vscode 0.6.1
|
||||
|
||||
- **New default model `stage-d`**, with **`stage-e` for next-edit**. Both download automatically (~530 MB each); the
|
||||
|
||||
Reference in New Issue
Block a user