Next-edit: n-gram speculative decoding on the companion server (0.6.2)
ci / build (push) Canceled after 0s
ci / build (push) Canceled after 0s
The rewritten region is mostly a copy of the current one, so prompt-lookup drafts are accepted in long runs. Greedy decoding keeps the output identical. Laptop CPU, 160-token region rewrite: generation 1.76 s -> 0.37 s. Timings now include errorsMs/promptMs/genMs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
This commit is contained in:
@@ -32,6 +32,9 @@ export interface DaemonConfig {
|
||||
ctx: number
|
||||
/** Extra flags, e.g. ["-ngl", "99"] on Apple Silicon, ["-t", "16"] on the laptop CPU. */
|
||||
args: string[]
|
||||
/** Extra flags for the next-edit companion server only. Default: n-gram speculative decoding (the output is
|
||||
* mostly a copy of the region, so drafts from the prompt are accepted; exact with greedy sampling). */
|
||||
nextEditArgs?: string[]
|
||||
}
|
||||
models: Record<string, ModelProfile>
|
||||
defaultModel: string
|
||||
|
||||
Reference in New Issue
Block a user