Models stage-d (default) + stage-e (next-edit companion); extension 0.6.1
ci / build (push) Canceled after 0s

- stage-d: next-line / block-close / multi-line fixes, symbols format 2, TS errors in the header.
  Held-out: 0.426 exact / 0.700 edit sim (stage-c 0.405 / 0.689); block close 0.27 -> 0.79.
- stage-e: next-edit (80% exact region rewrites, abstains on 100% of no-edit samples).
- Manifest: nextEditModel (companion) and nextEditFormat (older extensions ignore it, so they
  never use next-edit with the empty-output deletion bug).
- Daemon: next-edit served by the companion model in a second llama-server, started after the
  main model loads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
This commit is contained in:
2026-10-04 00:23:15 +00:00
co-authored by Claude Opus 5.5
parent 5c99f011e8
commit d246fb9349
12 changed files with 144 additions and 28 deletions
+11
View File
@@ -2,6 +2,17 @@
Extension versions are released as `vscode-v<version>` on the [Releases](https://git.harvmaster.com/Harvmaster/autocomplete-model/releases) page. Models are versioned by their name in [`models/manifest.json`](models/manifest.json).
## vscode 0.6.1
- **New default model `stage-d`**, with **`stage-e` for next-edit**. Both download automatically (~530 MB each); the
daemon runs `stage-e` in a second llama-server, only for next-edit. `stage-d` closes blocks (`}`) and predicts the
next line far better (held-out exact match: block close 0.27 → 0.79, next line 0.01 → 0.18), and sees TypeScript
errors near the cursor. Results in [models/README.md](models/README.md).
- **Fix:** a "no edit" answer from a next-edit model could be applied as deleting the suggested region. llama-server
drops the model's no-edit token, so it arrived as empty text. Next-edit models are only enabled from this version on.
- Manifest: `nextEditModel` (companion model) and `nextEditFormat` (replaces `nextEdit`, which older versions ignore
from now on).
## vscode 0.6.0
- **Next-edit suggestions** (with models trained for it, e.g. the upcoming `stage-e`): after you pause typing, the model may suggest an edit around the cursor, such as removing a leftover `)` or updating a renamed variable's uses. Single-line changes show deleted text struck through and inserted text as ghost text; multi-line changes are highlighted with the diff on hover. **Tab** applies, **Esc** dismisses. Setting: `nextEdit.enabled`.
+3
View File
@@ -18,6 +18,8 @@ export interface ModelProfile {
members?: boolean
/** Trained for next-edit prediction (shared/nextedit.ts), enables /v1/next-edit. */
nextEdit?: boolean
/** Another profile (trained with nextEdit) that serves /v1/next-edit for this model, in a second llama-server. */
nextEditModel?: string
}
export interface DaemonConfig {
@@ -86,6 +88,7 @@ export function loadConfig(configPath: string): DaemonConfig {
if (!raw.models[raw.defaultModel]) throw new Error(`defaultModel ${raw.defaultModel} is not in models`)
for (const [name, m] of Object.entries(raw.models)) {
if (!m.modes.includes(m.defaultMode)) throw new Error(`model ${name}: defaultMode must be one of its modes`)
if (m.nextEditModel && !raw.models[m.nextEditModel]?.nextEdit) throw new Error(`model ${name}: nextEditModel ${m.nextEditModel} must be a profile with nextEdit`)
}
return raw
}
+31 -11
View File
@@ -47,17 +47,34 @@ function loadModel(name: string) {
loadError = null
loading = llama.start(config.models[name].path)
.catch(e => { loadError = (e as Error).message; console.error(loadError) })
.finally(() => { loading = null })
.finally(() => { loading = null; if (config.models[name].nextEditModel) void nextEditServer() })
return loading
}
// Next-edit companion model (profile.nextEditModel): a second llama-server, started on first use.
const neLlama = new LlamaServer({ ...config.llamaServer, port: config.llamaServer.port + 1 })
let neLoading: Promise<void> | null = null
/** The server that answers next-edit for the active model, or an error message. */
async function nextEditServer(): Promise<LlamaServer | string> {
const profile = config.models[state.model]
if (profile.nextEdit) return loading || !llama.running ? (loadError ?? 'model loading') : llama
const companion = profile.nextEditModel
if (!companion) return `model ${state.model} is not trained for next-edit`
const wanted = config.models[companion].path
if (neLlama.model !== wanted && !neLoading) {
neLoading = neLlama.start(wanted).catch(e => console.error(`next-edit model ${companion}: ${(e as Error).message}`)).finally(() => { neLoading = null })
}
if (neLoading) await neLoading
return neLlama.running && neLlama.model === wanted ? neLlama : `next-edit model ${companion} failed to start`
}
let inflight: AbortController | null = null
function describeConfig() {
return {
...state,
effectiveMode: state.mode,
models: Object.fromEntries(Object.entries(config.models).map(([n, m]) => [n, { order: m.order, modes: m.modes, defaultMode: m.defaultMode, nextEdit: m.nextEdit ?? false, errors: m.errors ?? false, contextFormat: m.contextFormat ?? 1 }])),
models: Object.fromEntries(Object.entries(config.models).map(([n, m]) => [n, { order: m.order, modes: m.modes, defaultMode: m.defaultMode, nextEdit: !!(m.nextEdit || m.nextEditModel), nextEditModel: m.nextEditModel ?? null, errors: m.errors ?? false, contextFormat: m.contextFormat ?? 1 }])),
llama: { running: llama.running, loading: loading !== null, error: loadError },
}
}
@@ -245,8 +262,8 @@ async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, c
const t0 = performance.now()
const profile = config.models[state.model]
if (!state.enabled) return json(res, 200, { noEdit: true, disabled: true })
if (!profile.nextEdit) return json(res, 400, { error: `model ${state.model} is not trained for next-edit` })
if (loading || !llama.running) return json(res, 503, { error: loadError ?? 'model loading' })
const server = await nextEditServer()
if (typeof server === 'string') return json(res, server.includes('not trained') ? 400 : 503, { error: server })
if (typeof body.text !== 'string' || typeof body.cursor !== 'number') return json(res, 400, { error: 'text and cursor are required' })
const o = config.nextEditOptions ?? { regionBefore: 4, regionAfter: 8, maxTokens: 400, beforeTokens: 1024, afterTokens: 384 }
@@ -266,7 +283,7 @@ async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, c
const controller = new AbortController()
inflight = controller
clientGone.addEventListener('abort', () => controller.abort())
const result = await llama.complete(ids, { nPredict: o.maxTokens, stop: [NEXT_EDIT_TOKENS.outputEnd, ...STOP_STRINGS] }, controller.signal)
const result = await server.complete(ids, { nPredict: o.maxTokens, stop: [NEXT_EDIT_TOKENS.outputEnd, ...STOP_STRINGS] }, controller.signal)
if (inflight === controller) inflight = null
if (result.stoppedBy === 'aborted') return json(res, 200, { noEdit: true, cancelled: true })
@@ -276,7 +293,7 @@ async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, c
const noEdit = newText === null || result.stoppedBy === 'limit'
const response = {
requestId: body.requestId, noEdit, startLine, endLine, oldText, newText: noEdit ? null : newText,
recentEdits, errors: errors ?? null, model: state.model,
recentEdits, errors: errors ?? null, model: profile.nextEditModel ?? state.model,
timings: { promptTokens: ids.length, processedTokens: result.timings?.prompt_n ?? null, genTokens: result.timings?.predicted_n ?? null, totalMs: round(performance.now() - t0) },
}
remember({ kind: 'next-edit', requestId: body.requestId, path: body.path, model: state.model, prompt: builder.decode(ids), raw, noEdit, timings: response.timings })
@@ -303,7 +320,7 @@ if (config.idleShutdownMinutes) {
setInterval(() => {
if (Date.now() - lastActivity > config.idleShutdownMinutes! * 60_000) {
console.log(`idle for ${config.idleShutdownMinutes} min, shutting down`)
void llama.stop().then(() => process.exit(0))
void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0))
}
}, 30_000).unref()
}
@@ -318,7 +335,7 @@ const server = http.createServer(async (req, res) => {
if (route === 'GET /health') return json(res, 200, { ok: llama.running, loading: loading !== null, model: state.model, error: loadError, instanceTag: config.instanceTag ?? null, pid: process.pid })
if (route === 'POST /v1/shutdown') {
json(res, 200, { ok: true })
void llama.stop().then(() => process.exit(0))
void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0))
return
}
if (route === 'GET /v1/config') return json(res, 200, describeConfig())
@@ -331,7 +348,10 @@ const server = http.createServer(async (req, res) => {
const modelChanged = model !== state.model
Object.assign(state, { model, mode, enabled: body.enabled ?? state.enabled })
saveState(config, state)
if (modelChanged) void loadModel(model)
if (modelChanged) {
void loadModel(model)
if (neLlama.model && neLlama.model !== config.models[config.models[model].nextEditModel ?? '']?.path) void neLlama.stop()
}
return json(res, 200, describeConfig())
}
if (route === 'POST /v1/fim') return await handleFim(await readBody(req) as FimRequest, res, gone.signal)
@@ -357,8 +377,8 @@ const server = http.createServer(async (req, res) => {
}
})
process.on('SIGINT', () => void llama.stop().then(() => process.exit(0)))
process.on('SIGTERM', () => void llama.stop().then(() => process.exit(0)))
process.on('SIGINT', () => void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0)))
process.on('SIGTERM', () => void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0)))
server.listen(config.port, config.host, () => {
console.log(`acd listening on http://${config.host}:${config.port} (model ${state.model}, mode ${state.mode})`)
+1 -1
View File
@@ -3,7 +3,7 @@
"displayName": "Local Autocomplete",
"description": "Inline code completion from a local fine-tuned model via the acd daemon",
"publisher": "harvmaster",
"version": "0.6.0",
"version": "0.6.1",
"license": "MIT",
"engines": {
"vscode": "^1.90.0"
+14 -8
View File
@@ -30,13 +30,19 @@ export interface ModelEntry {
contextFormat?: 1 | 2
errors?: boolean
members?: boolean
/** Old (≤ 0.6.0) flag: ignored, because those daemons applied an empty "no edit" output as a deletion. */
nextEdit?: boolean
/** Trained for next-edit (shared/nextedit.ts format version); read since 0.6.1. */
nextEditFormat?: 1
/** Companion model (another manifest entry) that serves next-edit for this one; downloaded alongside it. */
nextEditModel?: string
}
const profileOf = (m: ModelEntry) => ({
const profileOf = (m: ModelEntry, available: Set<string>) => ({
order: m.order, modes: m.modes, defaultMode: m.defaultMode,
...(m.contextFormat ? { contextFormat: m.contextFormat } : {}), ...(m.errors ? { errors: true } : {}),
...(m.members !== undefined ? { members: m.members } : {}), ...(m.nextEdit ? { nextEdit: true } : {}),
...(m.members !== undefined ? { members: m.members } : {}), ...(m.nextEditFormat ? { nextEdit: true } : {}),
...(m.nextEditModel && available.has(m.nextEditModel) ? { nextEditModel: m.nextEditModel } : {}),
})
export interface Manifest {
@@ -229,12 +235,12 @@ export async function prepare(storage: string, daemonDir: string, extensionVersi
if (!entry) throw new Error(`model "${s.model || manifest.default}" is not in the manifest (available: ${manifest.models.map(m => m.name).join(', ')})`)
const file = await ensureFile(entry.url, path.join(storage, 'models', entry.file), entry.sha256, `model ${entry.name}`, progress, signal)
defaultModel = entry.name
models[entry.name] = { path: file, ...profileOf(entry) }
// Other models already on disk stay switchable from the menu.
for (const m of manifest.models) {
const f = path.join(storage, 'models', m.file)
if (m.name !== entry.name && fs.existsSync(`${f}.verified`)) models[m.name] = { path: f, ...profileOf(m) }
}
const companion = entry.nextEditModel ? manifest.models.find(m => m.name === entry.nextEditModel && m.nextEditFormat) : undefined
if (companion) await ensureFile(companion.url, path.join(storage, 'models', companion.file), companion.sha256, `next-edit model ${companion.name}`, progress, signal)
// The selected model, its companion and other models already on disk (switchable from the menu).
const onDisk = manifest.models.filter(m => m.name === entry.name || fs.existsSync(`${path.join(storage, 'models', m.file)}.verified`))
const available = new Set(onDisk.filter(m => m.nextEditFormat).map(m => m.name))
for (const m of onDisk) models[m.name] = { path: path.join(storage, 'models', m.file), ...profileOf(m, available) }
}
const binary = s.llamaServerPath ? s.llamaServerPath.replace(/^~(?=$|\/)/, os.homedir()) : await ensureLlamaServer(storage, llamaCpp, progress, signal)
+12 -3
View File
@@ -36,20 +36,29 @@ const step = (name, fn) => fn().then(r => { console.log(`PASS ${name}`); return
// 1. Simulate an interrupted model download: first 100 MB already on disk.
fs.mkdirSync(path.join(storage, 'models'), { recursive: true })
const modelSrc = path.join(releaseDir, 'stage-c-q8_0.gguf')
const modelSrc = path.join(releaseDir, 'stage-d-q8_0.gguf')
const fd = fs.openSync(modelSrc, 'r'); const buf = Buffer.alloc(100e6); fs.readSync(fd, buf, 0, buf.length, 0); fs.closeSync(fd)
fs.writeFileSync(path.join(storage, 'models', 'stage-c-q8_0.gguf.part'), buf)
fs.writeFileSync(path.join(storage, 'models', 'stage-d-q8_0.gguf.part'), buf)
let t0 = Date.now()
const prepared = await step('first run: manifest, resumed + verified model download, llama.cpp download + unpack', () => core.prepare(storage, daemonDir, '0.4.0-test', settings, progress))
console.log(` ${((Date.now() - t0) / 1000).toFixed(1)} s; range requests: ${rangeRequests}; model verified: ${fs.existsSync(path.join(storage, 'models', 'stage-c-q8_0.gguf.verified'))}`)
console.log(` ${((Date.now() - t0) / 1000).toFixed(1)} s; range requests: ${rangeRequests}; model verified: ${fs.existsSync(path.join(storage, 'models', 'stage-d-q8_0.gguf.verified'))}`)
assert.ok(rangeRequests >= 1, 'download should have resumed with a Range request')
const cfg = JSON.parse(fs.readFileSync(prepared.configPath, 'utf8'))
assert.ok(fs.existsSync(cfg.llamaServer.binary), 'llama-server unpacked')
await step('next-edit companion downloaded and wired (stage-d → stage-e)', async () => {
assert.equal(cfg.defaultModel, 'stage-d')
assert.equal(cfg.models['stage-d'].nextEditModel, 'stage-e')
assert.equal(cfg.models['stage-e'].nextEdit, true)
assert.ok(fs.existsSync(path.join(storage, 'models', 'stage-e-q8_0.gguf.verified')))
})
await step('daemon starts on the editor runtime', async () => assert.equal(await core.ensureRunning(storage, daemonDir, prepared, settings.port, progress), 'started'))
const fim = await (await fetch('http://127.0.0.1:7111/v1/fim', { method: 'POST', body: JSON.stringify({ repo: 'e2e', path: 'src/a.ts', prefix: "import { z } from 'zod'\nexport const s = z.object({ a: z.string() })\nexport type S = ", suffix: '\n' }) })).json()
await step(`completion through the managed daemon (${JSON.stringify(fim.text)})`, async () => assert.match(fim.text, /z\.infer<typeof s>/))
const neText = "export function f(s: string) {\n const t = s.trim()\n return parse(t))\n}\n"
const ne = await (await fetch('http://127.0.0.1:7111/v1/next-edit', { method: 'POST', body: JSON.stringify({ repo: 'e2e', path: 'src/f.ts', text: neText, cursor: neText.indexOf('(t)') + 1 }) })).json()
await step(`next-edit through the companion model (${JSON.stringify(ne.newText ?? ne.error ?? ne)})`, async () => assert.ok(ne.newText?.includes('return parse(t)\n')))
t0 = Date.now()
const again = await step('second window: no re-download, daemon reused', async () => {
+2 -1
View File
@@ -2,6 +2,7 @@ These model files are modified versions of Qwen2.5-Coder-0.5B (https://huggingfa
Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE).
Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion
(and, for stage-c, a repository/symbol-context header), followed by Q8_0 GGUF conversion.
(and, from stage-c on, a repository/symbol-context header; stage-d adds TypeScript errors; stage-e adds
next-edit prediction), followed by Q8_0 GGUF conversion.
Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and
the publisher's own code. Secrets were scrubbed from the training data.
+12
View File
@@ -5,6 +5,8 @@ Q8_0 GGUF fill-in-the-middle models for the [Local Autocomplete](../extensions/v
| model | description | prompt order | context modes | size |
|---|---|---|---|---|
| `stage-d` | Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions | spm | symbols, simple | 531 MB |
| `stage-e` | Next-edit prediction (rewrites a region around the cursor or answers "no edit"); also does FIM, slightly below stage-d | spm | symbols, simple | 531 MB |
| `stage-c` | Suffix-first FIM with repo/symbol-context headers (default) | spm | symbols, simple | 531 MB |
| `stage-b` | Suffix-first FIM without context headers (previous stage) | spm | raw | 531 MB |
@@ -21,6 +23,16 @@ runs it locally with llama.cpp. The files are stored with Git LFS; download link
| Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 |
| `stage-b` | 0.391 | 0.689 | 0.952 |
| `stage-c` (with symbol context) | 0.397 | 0.681 | 0.954 |
| `stage-c` + token healing (daemon default) | 0.405 | 0.689 | |
| `stage-d` (symbols + TS errors, healed) | **0.426** | **0.700** | 0.954 |
| `stage-e` (symbols + TS errors, healed) | 0.412 | 0.691 | 0.962 |
New behaviours (886 further held-out samples), exact match, stage-c → stage-d: closing a block 0.27 → **0.79**,
next line 0.01 → **0.18**, rest of block 0.08 → 0.11, member access 0.63 → 0.66.
**Next-edit** (`stage-e`, 445 held-out samples): 80% exact region rewrites (edit similarity 0.99) where an edit
was due, and "no edit" on 100% of samples where none was. `stage-d` uses `stage-e` as its next-edit companion
(`nextEditModel` in the manifest): the extension downloads both and the daemon runs it in a second llama-server.
The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts
let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a
+37 -1
View File
@@ -1,7 +1,43 @@
{
"schema": 1,
"default": "stage-c",
"default": "stage-d",
"models": [
{
"name": "stage-d",
"file": "stage-d-q8_0.gguf",
"url": "stage-d-q8_0.gguf",
"sha256": "8edaef02ae9e794929af0267f0f47f600824454b08df7d386e386eec1123c839",
"size": 531067552,
"order": "spm",
"modes": [
"symbols",
"simple"
],
"defaultMode": "symbols",
"contextFormat": 2,
"errors": true,
"members": true,
"nextEditModel": "stage-e",
"description": "Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions"
},
{
"name": "stage-e",
"file": "stage-e-q8_0.gguf",
"url": "stage-e-q8_0.gguf",
"sha256": "44d5fe87043f2fd3f8fba39c07a653ec7bdc228f83a2da4a5638be3ab77a962f",
"size": 531067552,
"order": "spm",
"modes": [
"symbols",
"simple"
],
"defaultMode": "symbols",
"contextFormat": 2,
"errors": true,
"members": true,
"nextEditFormat": 1,
"description": "Next-edit prediction (rewrites a region around the cursor or answers \"no edit\"); also does FIM, slightly below stage-d"
},
{
"name": "stage-c",
"file": "stage-c-q8_0.gguf",
Binary file not shown.
Binary file not shown.
+15 -3
View File
@@ -23,9 +23,10 @@ const out = path.join(repo, 'models')
// Prompt layout each model was trained with (the extension only offers matching context modes).
const profiles = {
// Stage E: Stage D + next-edit prediction (shared/nextedit.ts)
'stage-e': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEdit: true, description: 'Suffix-first FIM with symbols + TS errors, plus next-edit prediction (rewrites a region around the cursor)' },
// nextEditFormat (not the old nextEdit flag): extensions ≤ 0.6.0 must not use it (they applied "no edit" as a deletion).
'stage-e': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEditFormat: 1, description: 'Next-edit prediction (rewrites a region around the cursor or answers "no edit"); also does FIM, slightly below stage-d' },
// Stage D: next-line / block-close / multi-line fixes, namespace-import symbols (format 2), TS errors in the header
'stage-d': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, description: 'Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions' },
'stage-d': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEditModel: 'stage-e', description: 'Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions' },
'stage-c': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'simple', description: 'Suffix-first FIM with repo/symbol-context headers (default)' },
'stage-b': { order: 'spm', modes: ['raw'], defaultMode: 'raw', description: 'Suffix-first FIM without context headers (previous stage)' },
}
@@ -71,7 +72,8 @@ fs.writeFileSync(path.join(out, 'NOTICE'), `These model files are modified versi
Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE).
Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion
(and, for stage-c, a repository/symbol-context header), followed by Q8_0 GGUF conversion.
(and, from stage-c on, a repository/symbol-context header; stage-d adds TypeScript errors; stage-e adds
next-edit prediction), followed by Q8_0 GGUF conversion.
Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and
the publisher's own code. Secrets were scrubbed from the training data.
`)
@@ -103,6 +105,16 @@ runs it locally with llama.cpp. The files are stored with Git LFS; download link
| Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 |
| \`stage-b\` | 0.391 | 0.689 | 0.952 |
| \`stage-c\` (with symbol context) | 0.397 | 0.681 | 0.954 |
| \`stage-c\` + token healing (daemon default) | 0.405 | 0.689 | |
| \`stage-d\` (symbols + TS errors, healed) | **0.426** | **0.700** | 0.954 |
| \`stage-e\` (symbols + TS errors, healed) | 0.412 | 0.691 | 0.962 |
New behaviours (886 further held-out samples), exact match, stage-c → stage-d: closing a block 0.27 → **0.79**,
next line 0.01 → **0.18**, rest of block 0.08 → 0.11, member access 0.63 → 0.66.
**Next-edit** (\`stage-e\`, 445 held-out samples): 80% exact region rewrites (edit similarity 0.99) where an edit
was due, and "no edit" on 100% of samples where none was. \`stage-d\` uses \`stage-e\` as its next-edit companion
(\`nextEditModel\` in the manifest): the extension downloads both and the daemon runs it in a second llama-server.
The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts
let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a