Models stage-d (default) + stage-e (next-edit companion); extension 0.6.1
ci / build (push) Canceled after 0s

- stage-d: next-line / block-close / multi-line fixes, symbols format 2, TS errors in the header.
  Held-out: 0.426 exact / 0.700 edit sim (stage-c 0.405 / 0.689); block close 0.27 -> 0.79.
- stage-e: next-edit (80% exact region rewrites, abstains on 100% of no-edit samples).
- Manifest: nextEditModel (companion) and nextEditFormat (older extensions ignore it, so they
  never use next-edit with the empty-output deletion bug).
- Daemon: next-edit served by the companion model in a second llama-server, started after the
  main model loads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
This commit is contained in:
2026-10-04 00:23:15 +00:00
co-authored by Claude Opus 5.5
parent 5c99f011e8
commit d246fb9349
12 changed files with 144 additions and 28 deletions
+11
View File
@@ -2,6 +2,17 @@
Extension versions are released as `vscode-v<version>` on the [Releases](https://git.harvmaster.com/Harvmaster/autocomplete-model/releases) page. Models are versioned by their name in [`models/manifest.json`](models/manifest.json). Extension versions are released as `vscode-v<version>` on the [Releases](https://git.harvmaster.com/Harvmaster/autocomplete-model/releases) page. Models are versioned by their name in [`models/manifest.json`](models/manifest.json).
## vscode 0.6.1
- **New default model `stage-d`**, with **`stage-e` for next-edit**. Both download automatically (~530 MB each); the
daemon runs `stage-e` in a second llama-server, only for next-edit. `stage-d` closes blocks (`}`) and predicts the
next line far better (held-out exact match: block close 0.27 → 0.79, next line 0.01 → 0.18), and sees TypeScript
errors near the cursor. Results in [models/README.md](models/README.md).
- **Fix:** a "no edit" answer from a next-edit model could be applied as deleting the suggested region. llama-server
drops the model's no-edit token, so it arrived as empty text. Next-edit models are only enabled from this version on.
- Manifest: `nextEditModel` (companion model) and `nextEditFormat` (replaces `nextEdit`, which older versions ignore
from now on).
## vscode 0.6.0 ## vscode 0.6.0
- **Next-edit suggestions** (with models trained for it, e.g. the upcoming `stage-e`): after you pause typing, the model may suggest an edit around the cursor, such as removing a leftover `)` or updating a renamed variable's uses. Single-line changes show deleted text struck through and inserted text as ghost text; multi-line changes are highlighted with the diff on hover. **Tab** applies, **Esc** dismisses. Setting: `nextEdit.enabled`. - **Next-edit suggestions** (with models trained for it, e.g. the upcoming `stage-e`): after you pause typing, the model may suggest an edit around the cursor, such as removing a leftover `)` or updating a renamed variable's uses. Single-line changes show deleted text struck through and inserted text as ghost text; multi-line changes are highlighted with the diff on hover. **Tab** applies, **Esc** dismisses. Setting: `nextEdit.enabled`.
+3
View File
@@ -18,6 +18,8 @@ export interface ModelProfile {
members?: boolean members?: boolean
/** Trained for next-edit prediction (shared/nextedit.ts), enables /v1/next-edit. */ /** Trained for next-edit prediction (shared/nextedit.ts), enables /v1/next-edit. */
nextEdit?: boolean nextEdit?: boolean
/** Another profile (trained with nextEdit) that serves /v1/next-edit for this model, in a second llama-server. */
nextEditModel?: string
} }
export interface DaemonConfig { export interface DaemonConfig {
@@ -86,6 +88,7 @@ export function loadConfig(configPath: string): DaemonConfig {
if (!raw.models[raw.defaultModel]) throw new Error(`defaultModel ${raw.defaultModel} is not in models`) if (!raw.models[raw.defaultModel]) throw new Error(`defaultModel ${raw.defaultModel} is not in models`)
for (const [name, m] of Object.entries(raw.models)) { for (const [name, m] of Object.entries(raw.models)) {
if (!m.modes.includes(m.defaultMode)) throw new Error(`model ${name}: defaultMode must be one of its modes`) if (!m.modes.includes(m.defaultMode)) throw new Error(`model ${name}: defaultMode must be one of its modes`)
if (m.nextEditModel && !raw.models[m.nextEditModel]?.nextEdit) throw new Error(`model ${name}: nextEditModel ${m.nextEditModel} must be a profile with nextEdit`)
} }
return raw return raw
} }
+31 -11
View File
@@ -47,17 +47,34 @@ function loadModel(name: string) {
loadError = null loadError = null
loading = llama.start(config.models[name].path) loading = llama.start(config.models[name].path)
.catch(e => { loadError = (e as Error).message; console.error(loadError) }) .catch(e => { loadError = (e as Error).message; console.error(loadError) })
.finally(() => { loading = null }) .finally(() => { loading = null; if (config.models[name].nextEditModel) void nextEditServer() })
return loading return loading
} }
// Next-edit companion model (profile.nextEditModel): a second llama-server, started on first use.
const neLlama = new LlamaServer({ ...config.llamaServer, port: config.llamaServer.port + 1 })
let neLoading: Promise<void> | null = null
/** The server that answers next-edit for the active model, or an error message. */
async function nextEditServer(): Promise<LlamaServer | string> {
const profile = config.models[state.model]
if (profile.nextEdit) return loading || !llama.running ? (loadError ?? 'model loading') : llama
const companion = profile.nextEditModel
if (!companion) return `model ${state.model} is not trained for next-edit`
const wanted = config.models[companion].path
if (neLlama.model !== wanted && !neLoading) {
neLoading = neLlama.start(wanted).catch(e => console.error(`next-edit model ${companion}: ${(e as Error).message}`)).finally(() => { neLoading = null })
}
if (neLoading) await neLoading
return neLlama.running && neLlama.model === wanted ? neLlama : `next-edit model ${companion} failed to start`
}
let inflight: AbortController | null = null let inflight: AbortController | null = null
function describeConfig() { function describeConfig() {
return { return {
...state, ...state,
effectiveMode: state.mode, effectiveMode: state.mode,
models: Object.fromEntries(Object.entries(config.models).map(([n, m]) => [n, { order: m.order, modes: m.modes, defaultMode: m.defaultMode, nextEdit: m.nextEdit ?? false, errors: m.errors ?? false, contextFormat: m.contextFormat ?? 1 }])), models: Object.fromEntries(Object.entries(config.models).map(([n, m]) => [n, { order: m.order, modes: m.modes, defaultMode: m.defaultMode, nextEdit: !!(m.nextEdit || m.nextEditModel), nextEditModel: m.nextEditModel ?? null, errors: m.errors ?? false, contextFormat: m.contextFormat ?? 1 }])),
llama: { running: llama.running, loading: loading !== null, error: loadError }, llama: { running: llama.running, loading: loading !== null, error: loadError },
} }
} }
@@ -245,8 +262,8 @@ async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, c
const t0 = performance.now() const t0 = performance.now()
const profile = config.models[state.model] const profile = config.models[state.model]
if (!state.enabled) return json(res, 200, { noEdit: true, disabled: true }) if (!state.enabled) return json(res, 200, { noEdit: true, disabled: true })
if (!profile.nextEdit) return json(res, 400, { error: `model ${state.model} is not trained for next-edit` }) const server = await nextEditServer()
if (loading || !llama.running) return json(res, 503, { error: loadError ?? 'model loading' }) if (typeof server === 'string') return json(res, server.includes('not trained') ? 400 : 503, { error: server })
if (typeof body.text !== 'string' || typeof body.cursor !== 'number') return json(res, 400, { error: 'text and cursor are required' }) if (typeof body.text !== 'string' || typeof body.cursor !== 'number') return json(res, 400, { error: 'text and cursor are required' })
const o = config.nextEditOptions ?? { regionBefore: 4, regionAfter: 8, maxTokens: 400, beforeTokens: 1024, afterTokens: 384 } const o = config.nextEditOptions ?? { regionBefore: 4, regionAfter: 8, maxTokens: 400, beforeTokens: 1024, afterTokens: 384 }
@@ -266,7 +283,7 @@ async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, c
const controller = new AbortController() const controller = new AbortController()
inflight = controller inflight = controller
clientGone.addEventListener('abort', () => controller.abort()) clientGone.addEventListener('abort', () => controller.abort())
const result = await llama.complete(ids, { nPredict: o.maxTokens, stop: [NEXT_EDIT_TOKENS.outputEnd, ...STOP_STRINGS] }, controller.signal) const result = await server.complete(ids, { nPredict: o.maxTokens, stop: [NEXT_EDIT_TOKENS.outputEnd, ...STOP_STRINGS] }, controller.signal)
if (inflight === controller) inflight = null if (inflight === controller) inflight = null
if (result.stoppedBy === 'aborted') return json(res, 200, { noEdit: true, cancelled: true }) if (result.stoppedBy === 'aborted') return json(res, 200, { noEdit: true, cancelled: true })
@@ -276,7 +293,7 @@ async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, c
const noEdit = newText === null || result.stoppedBy === 'limit' const noEdit = newText === null || result.stoppedBy === 'limit'
const response = { const response = {
requestId: body.requestId, noEdit, startLine, endLine, oldText, newText: noEdit ? null : newText, requestId: body.requestId, noEdit, startLine, endLine, oldText, newText: noEdit ? null : newText,
recentEdits, errors: errors ?? null, model: state.model, recentEdits, errors: errors ?? null, model: profile.nextEditModel ?? state.model,
timings: { promptTokens: ids.length, processedTokens: result.timings?.prompt_n ?? null, genTokens: result.timings?.predicted_n ?? null, totalMs: round(performance.now() - t0) }, timings: { promptTokens: ids.length, processedTokens: result.timings?.prompt_n ?? null, genTokens: result.timings?.predicted_n ?? null, totalMs: round(performance.now() - t0) },
} }
remember({ kind: 'next-edit', requestId: body.requestId, path: body.path, model: state.model, prompt: builder.decode(ids), raw, noEdit, timings: response.timings }) remember({ kind: 'next-edit', requestId: body.requestId, path: body.path, model: state.model, prompt: builder.decode(ids), raw, noEdit, timings: response.timings })
@@ -303,7 +320,7 @@ if (config.idleShutdownMinutes) {
setInterval(() => { setInterval(() => {
if (Date.now() - lastActivity > config.idleShutdownMinutes! * 60_000) { if (Date.now() - lastActivity > config.idleShutdownMinutes! * 60_000) {
console.log(`idle for ${config.idleShutdownMinutes} min, shutting down`) console.log(`idle for ${config.idleShutdownMinutes} min, shutting down`)
void llama.stop().then(() => process.exit(0)) void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0))
} }
}, 30_000).unref() }, 30_000).unref()
} }
@@ -318,7 +335,7 @@ const server = http.createServer(async (req, res) => {
if (route === 'GET /health') return json(res, 200, { ok: llama.running, loading: loading !== null, model: state.model, error: loadError, instanceTag: config.instanceTag ?? null, pid: process.pid }) if (route === 'GET /health') return json(res, 200, { ok: llama.running, loading: loading !== null, model: state.model, error: loadError, instanceTag: config.instanceTag ?? null, pid: process.pid })
if (route === 'POST /v1/shutdown') { if (route === 'POST /v1/shutdown') {
json(res, 200, { ok: true }) json(res, 200, { ok: true })
void llama.stop().then(() => process.exit(0)) void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0))
return return
} }
if (route === 'GET /v1/config') return json(res, 200, describeConfig()) if (route === 'GET /v1/config') return json(res, 200, describeConfig())
@@ -331,7 +348,10 @@ const server = http.createServer(async (req, res) => {
const modelChanged = model !== state.model const modelChanged = model !== state.model
Object.assign(state, { model, mode, enabled: body.enabled ?? state.enabled }) Object.assign(state, { model, mode, enabled: body.enabled ?? state.enabled })
saveState(config, state) saveState(config, state)
if (modelChanged) void loadModel(model) if (modelChanged) {
void loadModel(model)
if (neLlama.model && neLlama.model !== config.models[config.models[model].nextEditModel ?? '']?.path) void neLlama.stop()
}
return json(res, 200, describeConfig()) return json(res, 200, describeConfig())
} }
if (route === 'POST /v1/fim') return await handleFim(await readBody(req) as FimRequest, res, gone.signal) if (route === 'POST /v1/fim') return await handleFim(await readBody(req) as FimRequest, res, gone.signal)
@@ -357,8 +377,8 @@ const server = http.createServer(async (req, res) => {
} }
}) })
process.on('SIGINT', () => void llama.stop().then(() => process.exit(0))) process.on('SIGINT', () => void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0)))
process.on('SIGTERM', () => void llama.stop().then(() => process.exit(0))) process.on('SIGTERM', () => void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0)))
server.listen(config.port, config.host, () => { server.listen(config.port, config.host, () => {
console.log(`acd listening on http://${config.host}:${config.port} (model ${state.model}, mode ${state.mode})`) console.log(`acd listening on http://${config.host}:${config.port} (model ${state.model}, mode ${state.mode})`)
+1 -1
View File
@@ -3,7 +3,7 @@
"displayName": "Local Autocomplete", "displayName": "Local Autocomplete",
"description": "Inline code completion from a local fine-tuned model via the acd daemon", "description": "Inline code completion from a local fine-tuned model via the acd daemon",
"publisher": "harvmaster", "publisher": "harvmaster",
"version": "0.6.0", "version": "0.6.1",
"license": "MIT", "license": "MIT",
"engines": { "engines": {
"vscode": "^1.90.0" "vscode": "^1.90.0"
+14 -8
View File
@@ -30,13 +30,19 @@ export interface ModelEntry {
contextFormat?: 1 | 2 contextFormat?: 1 | 2
errors?: boolean errors?: boolean
members?: boolean members?: boolean
/** Old (≤ 0.6.0) flag: ignored, because those daemons applied an empty "no edit" output as a deletion. */
nextEdit?: boolean nextEdit?: boolean
/** Trained for next-edit (shared/nextedit.ts format version); read since 0.6.1. */
nextEditFormat?: 1
/** Companion model (another manifest entry) that serves next-edit for this one; downloaded alongside it. */
nextEditModel?: string
} }
const profileOf = (m: ModelEntry) => ({ const profileOf = (m: ModelEntry, available: Set<string>) => ({
order: m.order, modes: m.modes, defaultMode: m.defaultMode, order: m.order, modes: m.modes, defaultMode: m.defaultMode,
...(m.contextFormat ? { contextFormat: m.contextFormat } : {}), ...(m.errors ? { errors: true } : {}), ...(m.contextFormat ? { contextFormat: m.contextFormat } : {}), ...(m.errors ? { errors: true } : {}),
...(m.members !== undefined ? { members: m.members } : {}), ...(m.nextEdit ? { nextEdit: true } : {}), ...(m.members !== undefined ? { members: m.members } : {}), ...(m.nextEditFormat ? { nextEdit: true } : {}),
...(m.nextEditModel && available.has(m.nextEditModel) ? { nextEditModel: m.nextEditModel } : {}),
}) })
export interface Manifest { export interface Manifest {
@@ -229,12 +235,12 @@ export async function prepare(storage: string, daemonDir: string, extensionVersi
if (!entry) throw new Error(`model "${s.model || manifest.default}" is not in the manifest (available: ${manifest.models.map(m => m.name).join(', ')})`) if (!entry) throw new Error(`model "${s.model || manifest.default}" is not in the manifest (available: ${manifest.models.map(m => m.name).join(', ')})`)
const file = await ensureFile(entry.url, path.join(storage, 'models', entry.file), entry.sha256, `model ${entry.name}`, progress, signal) const file = await ensureFile(entry.url, path.join(storage, 'models', entry.file), entry.sha256, `model ${entry.name}`, progress, signal)
defaultModel = entry.name defaultModel = entry.name
models[entry.name] = { path: file, ...profileOf(entry) } const companion = entry.nextEditModel ? manifest.models.find(m => m.name === entry.nextEditModel && m.nextEditFormat) : undefined
// Other models already on disk stay switchable from the menu. if (companion) await ensureFile(companion.url, path.join(storage, 'models', companion.file), companion.sha256, `next-edit model ${companion.name}`, progress, signal)
for (const m of manifest.models) { // The selected model, its companion and other models already on disk (switchable from the menu).
const f = path.join(storage, 'models', m.file) const onDisk = manifest.models.filter(m => m.name === entry.name || fs.existsSync(`${path.join(storage, 'models', m.file)}.verified`))
if (m.name !== entry.name && fs.existsSync(`${f}.verified`)) models[m.name] = { path: f, ...profileOf(m) } const available = new Set(onDisk.filter(m => m.nextEditFormat).map(m => m.name))
} for (const m of onDisk) models[m.name] = { path: path.join(storage, 'models', m.file), ...profileOf(m, available) }
} }
const binary = s.llamaServerPath ? s.llamaServerPath.replace(/^~(?=$|\/)/, os.homedir()) : await ensureLlamaServer(storage, llamaCpp, progress, signal) const binary = s.llamaServerPath ? s.llamaServerPath.replace(/^~(?=$|\/)/, os.homedir()) : await ensureLlamaServer(storage, llamaCpp, progress, signal)
+12 -3
View File
@@ -36,20 +36,29 @@ const step = (name, fn) => fn().then(r => { console.log(`PASS ${name}`); return
// 1. Simulate an interrupted model download: first 100 MB already on disk. // 1. Simulate an interrupted model download: first 100 MB already on disk.
fs.mkdirSync(path.join(storage, 'models'), { recursive: true }) fs.mkdirSync(path.join(storage, 'models'), { recursive: true })
const modelSrc = path.join(releaseDir, 'stage-c-q8_0.gguf') const modelSrc = path.join(releaseDir, 'stage-d-q8_0.gguf')
const fd = fs.openSync(modelSrc, 'r'); const buf = Buffer.alloc(100e6); fs.readSync(fd, buf, 0, buf.length, 0); fs.closeSync(fd) const fd = fs.openSync(modelSrc, 'r'); const buf = Buffer.alloc(100e6); fs.readSync(fd, buf, 0, buf.length, 0); fs.closeSync(fd)
fs.writeFileSync(path.join(storage, 'models', 'stage-c-q8_0.gguf.part'), buf) fs.writeFileSync(path.join(storage, 'models', 'stage-d-q8_0.gguf.part'), buf)
let t0 = Date.now() let t0 = Date.now()
const prepared = await step('first run: manifest, resumed + verified model download, llama.cpp download + unpack', () => core.prepare(storage, daemonDir, '0.4.0-test', settings, progress)) const prepared = await step('first run: manifest, resumed + verified model download, llama.cpp download + unpack', () => core.prepare(storage, daemonDir, '0.4.0-test', settings, progress))
console.log(` ${((Date.now() - t0) / 1000).toFixed(1)} s; range requests: ${rangeRequests}; model verified: ${fs.existsSync(path.join(storage, 'models', 'stage-c-q8_0.gguf.verified'))}`) console.log(` ${((Date.now() - t0) / 1000).toFixed(1)} s; range requests: ${rangeRequests}; model verified: ${fs.existsSync(path.join(storage, 'models', 'stage-d-q8_0.gguf.verified'))}`)
assert.ok(rangeRequests >= 1, 'download should have resumed with a Range request') assert.ok(rangeRequests >= 1, 'download should have resumed with a Range request')
const cfg = JSON.parse(fs.readFileSync(prepared.configPath, 'utf8')) const cfg = JSON.parse(fs.readFileSync(prepared.configPath, 'utf8'))
assert.ok(fs.existsSync(cfg.llamaServer.binary), 'llama-server unpacked') assert.ok(fs.existsSync(cfg.llamaServer.binary), 'llama-server unpacked')
await step('next-edit companion downloaded and wired (stage-d → stage-e)', async () => {
assert.equal(cfg.defaultModel, 'stage-d')
assert.equal(cfg.models['stage-d'].nextEditModel, 'stage-e')
assert.equal(cfg.models['stage-e'].nextEdit, true)
assert.ok(fs.existsSync(path.join(storage, 'models', 'stage-e-q8_0.gguf.verified')))
})
await step('daemon starts on the editor runtime', async () => assert.equal(await core.ensureRunning(storage, daemonDir, prepared, settings.port, progress), 'started')) await step('daemon starts on the editor runtime', async () => assert.equal(await core.ensureRunning(storage, daemonDir, prepared, settings.port, progress), 'started'))
const fim = await (await fetch('http://127.0.0.1:7111/v1/fim', { method: 'POST', body: JSON.stringify({ repo: 'e2e', path: 'src/a.ts', prefix: "import { z } from 'zod'\nexport const s = z.object({ a: z.string() })\nexport type S = ", suffix: '\n' }) })).json() const fim = await (await fetch('http://127.0.0.1:7111/v1/fim', { method: 'POST', body: JSON.stringify({ repo: 'e2e', path: 'src/a.ts', prefix: "import { z } from 'zod'\nexport const s = z.object({ a: z.string() })\nexport type S = ", suffix: '\n' }) })).json()
await step(`completion through the managed daemon (${JSON.stringify(fim.text)})`, async () => assert.match(fim.text, /z\.infer<typeof s>/)) await step(`completion through the managed daemon (${JSON.stringify(fim.text)})`, async () => assert.match(fim.text, /z\.infer<typeof s>/))
const neText = "export function f(s: string) {\n const t = s.trim()\n return parse(t))\n}\n"
const ne = await (await fetch('http://127.0.0.1:7111/v1/next-edit', { method: 'POST', body: JSON.stringify({ repo: 'e2e', path: 'src/f.ts', text: neText, cursor: neText.indexOf('(t)') + 1 }) })).json()
await step(`next-edit through the companion model (${JSON.stringify(ne.newText ?? ne.error ?? ne)})`, async () => assert.ok(ne.newText?.includes('return parse(t)\n')))
t0 = Date.now() t0 = Date.now()
const again = await step('second window: no re-download, daemon reused', async () => { const again = await step('second window: no re-download, daemon reused', async () => {
+2 -1
View File
@@ -2,6 +2,7 @@ These model files are modified versions of Qwen2.5-Coder-0.5B (https://huggingfa
Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE). Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE).
Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion
(and, for stage-c, a repository/symbol-context header), followed by Q8_0 GGUF conversion. (and, from stage-c on, a repository/symbol-context header; stage-d adds TypeScript errors; stage-e adds
next-edit prediction), followed by Q8_0 GGUF conversion.
Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and
the publisher's own code. Secrets were scrubbed from the training data. the publisher's own code. Secrets were scrubbed from the training data.
+12
View File
@@ -5,6 +5,8 @@ Q8_0 GGUF fill-in-the-middle models for the [Local Autocomplete](../extensions/v
| model | description | prompt order | context modes | size | | model | description | prompt order | context modes | size |
|---|---|---|---|---| |---|---|---|---|---|
| `stage-d` | Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions | spm | symbols, simple | 531 MB |
| `stage-e` | Next-edit prediction (rewrites a region around the cursor or answers "no edit"); also does FIM, slightly below stage-d | spm | symbols, simple | 531 MB |
| `stage-c` | Suffix-first FIM with repo/symbol-context headers (default) | spm | symbols, simple | 531 MB | | `stage-c` | Suffix-first FIM with repo/symbol-context headers (default) | spm | symbols, simple | 531 MB |
| `stage-b` | Suffix-first FIM without context headers (previous stage) | spm | raw | 531 MB | | `stage-b` | Suffix-first FIM without context headers (previous stage) | spm | raw | 531 MB |
@@ -21,6 +23,16 @@ runs it locally with llama.cpp. The files are stored with Git LFS; download link
| Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 | | Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 |
| `stage-b` | 0.391 | 0.689 | 0.952 | | `stage-b` | 0.391 | 0.689 | 0.952 |
| `stage-c` (with symbol context) | 0.397 | 0.681 | 0.954 | | `stage-c` (with symbol context) | 0.397 | 0.681 | 0.954 |
| `stage-c` + token healing (daemon default) | 0.405 | 0.689 | |
| `stage-d` (symbols + TS errors, healed) | **0.426** | **0.700** | 0.954 |
| `stage-e` (symbols + TS errors, healed) | 0.412 | 0.691 | 0.962 |
New behaviours (886 further held-out samples), exact match, stage-c → stage-d: closing a block 0.27 → **0.79**,
next line 0.01 → **0.18**, rest of block 0.08 → 0.11, member access 0.63 → 0.66.
**Next-edit** (`stage-e`, 445 held-out samples): 80% exact region rewrites (edit similarity 0.99) where an edit
was due, and "no edit" on 100% of samples where none was. `stage-d` uses `stage-e` as its next-edit companion
(`nextEditModel` in the manifest): the extension downloads both and the daemon runs it in a second llama-server.
The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts
let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a
+37 -1
View File
@@ -1,7 +1,43 @@
{ {
"schema": 1, "schema": 1,
"default": "stage-c", "default": "stage-d",
"models": [ "models": [
{
"name": "stage-d",
"file": "stage-d-q8_0.gguf",
"url": "stage-d-q8_0.gguf",
"sha256": "8edaef02ae9e794929af0267f0f47f600824454b08df7d386e386eec1123c839",
"size": 531067552,
"order": "spm",
"modes": [
"symbols",
"simple"
],
"defaultMode": "symbols",
"contextFormat": 2,
"errors": true,
"members": true,
"nextEditModel": "stage-e",
"description": "Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions"
},
{
"name": "stage-e",
"file": "stage-e-q8_0.gguf",
"url": "stage-e-q8_0.gguf",
"sha256": "44d5fe87043f2fd3f8fba39c07a653ec7bdc228f83a2da4a5638be3ab77a962f",
"size": 531067552,
"order": "spm",
"modes": [
"symbols",
"simple"
],
"defaultMode": "symbols",
"contextFormat": 2,
"errors": true,
"members": true,
"nextEditFormat": 1,
"description": "Next-edit prediction (rewrites a region around the cursor or answers \"no edit\"); also does FIM, slightly below stage-d"
},
{ {
"name": "stage-c", "name": "stage-c",
"file": "stage-c-q8_0.gguf", "file": "stage-c-q8_0.gguf",
Binary file not shown.
Binary file not shown.
+15 -3
View File
@@ -23,9 +23,10 @@ const out = path.join(repo, 'models')
// Prompt layout each model was trained with (the extension only offers matching context modes). // Prompt layout each model was trained with (the extension only offers matching context modes).
const profiles = { const profiles = {
// Stage E: Stage D + next-edit prediction (shared/nextedit.ts) // Stage E: Stage D + next-edit prediction (shared/nextedit.ts)
'stage-e': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEdit: true, description: 'Suffix-first FIM with symbols + TS errors, plus next-edit prediction (rewrites a region around the cursor)' }, // nextEditFormat (not the old nextEdit flag): extensions ≤ 0.6.0 must not use it (they applied "no edit" as a deletion).
'stage-e': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEditFormat: 1, description: 'Next-edit prediction (rewrites a region around the cursor or answers "no edit"); also does FIM, slightly below stage-d' },
// Stage D: next-line / block-close / multi-line fixes, namespace-import symbols (format 2), TS errors in the header // Stage D: next-line / block-close / multi-line fixes, namespace-import symbols (format 2), TS errors in the header
'stage-d': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, description: 'Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions' }, 'stage-d': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEditModel: 'stage-e', description: 'Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions' },
'stage-c': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'simple', description: 'Suffix-first FIM with repo/symbol-context headers (default)' }, 'stage-c': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'simple', description: 'Suffix-first FIM with repo/symbol-context headers (default)' },
'stage-b': { order: 'spm', modes: ['raw'], defaultMode: 'raw', description: 'Suffix-first FIM without context headers (previous stage)' }, 'stage-b': { order: 'spm', modes: ['raw'], defaultMode: 'raw', description: 'Suffix-first FIM without context headers (previous stage)' },
} }
@@ -71,7 +72,8 @@ fs.writeFileSync(path.join(out, 'NOTICE'), `These model files are modified versi
Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE). Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE).
Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion
(and, for stage-c, a repository/symbol-context header), followed by Q8_0 GGUF conversion. (and, from stage-c on, a repository/symbol-context header; stage-d adds TypeScript errors; stage-e adds
next-edit prediction), followed by Q8_0 GGUF conversion.
Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and
the publisher's own code. Secrets were scrubbed from the training data. the publisher's own code. Secrets were scrubbed from the training data.
`) `)
@@ -103,6 +105,16 @@ runs it locally with llama.cpp. The files are stored with Git LFS; download link
| Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 | | Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 |
| \`stage-b\` | 0.391 | 0.689 | 0.952 | | \`stage-b\` | 0.391 | 0.689 | 0.952 |
| \`stage-c\` (with symbol context) | 0.397 | 0.681 | 0.954 | | \`stage-c\` (with symbol context) | 0.397 | 0.681 | 0.954 |
| \`stage-c\` + token healing (daemon default) | 0.405 | 0.689 | |
| \`stage-d\` (symbols + TS errors, healed) | **0.426** | **0.700** | 0.954 |
| \`stage-e\` (symbols + TS errors, healed) | 0.412 | 0.691 | 0.962 |
New behaviours (886 further held-out samples), exact match, stage-c → stage-d: closing a block 0.27 → **0.79**,
next line 0.01 → **0.18**, rest of block 0.08 → 0.11, member access 0.63 → 0.66.
**Next-edit** (\`stage-e\`, 445 held-out samples): 80% exact region rewrites (edit similarity 0.99) where an edit
was due, and "no edit" on 100% of samples where none was. \`stage-d\` uses \`stage-e\` as its next-edit companion
(\`nextEditModel\` in the manifest): the extension downloads both and the daemon runs it in a second llama-server.
The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts
let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a