Models stage-d (default) + stage-e (next-edit companion); extension 0.6.1
ci / build (push) Canceled after 0s
ci / build (push) Canceled after 0s
- stage-d: next-line / block-close / multi-line fixes, symbols format 2, TS errors in the header. Held-out: 0.426 exact / 0.700 edit sim (stage-c 0.405 / 0.689); block close 0.27 -> 0.79. - stage-e: next-edit (80% exact region rewrites, abstains on 100% of no-edit samples). - Manifest: nextEditModel (companion) and nextEditFormat (older extensions ignore it, so they never use next-edit with the empty-output deletion bug). - Daemon: next-edit served by the companion model in a second llama-server, started after the main model loads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
This commit is contained in:
@@ -2,6 +2,17 @@
|
||||
|
||||
Extension versions are released as `vscode-v<version>` on the [Releases](https://git.harvmaster.com/Harvmaster/autocomplete-model/releases) page. Models are versioned by their name in [`models/manifest.json`](models/manifest.json).
|
||||
|
||||
## vscode 0.6.1
|
||||
|
||||
- **New default model `stage-d`**, with **`stage-e` for next-edit**. Both download automatically (~530 MB each); the
|
||||
daemon runs `stage-e` in a second llama-server, only for next-edit. `stage-d` closes blocks (`}`) and predicts the
|
||||
next line far better (held-out exact match: block close 0.27 → 0.79, next line 0.01 → 0.18), and sees TypeScript
|
||||
errors near the cursor. Results in [models/README.md](models/README.md).
|
||||
- **Fix:** a "no edit" answer from a next-edit model could be applied as deleting the suggested region. llama-server
|
||||
drops the model's no-edit token, so it arrived as empty text. Next-edit models are only enabled from this version on.
|
||||
- Manifest: `nextEditModel` (companion model) and `nextEditFormat` (replaces `nextEdit`, which older versions ignore
|
||||
from now on).
|
||||
|
||||
## vscode 0.6.0
|
||||
|
||||
- **Next-edit suggestions** (with models trained for it, e.g. the upcoming `stage-e`): after you pause typing, the model may suggest an edit around the cursor, such as removing a leftover `)` or updating a renamed variable's uses. Single-line changes show deleted text struck through and inserted text as ghost text; multi-line changes are highlighted with the diff on hover. **Tab** applies, **Esc** dismisses. Setting: `nextEdit.enabled`.
|
||||
|
||||
@@ -18,6 +18,8 @@ export interface ModelProfile {
|
||||
members?: boolean
|
||||
/** Trained for next-edit prediction (shared/nextedit.ts), enables /v1/next-edit. */
|
||||
nextEdit?: boolean
|
||||
/** Another profile (trained with nextEdit) that serves /v1/next-edit for this model, in a second llama-server. */
|
||||
nextEditModel?: string
|
||||
}
|
||||
|
||||
export interface DaemonConfig {
|
||||
@@ -86,6 +88,7 @@ export function loadConfig(configPath: string): DaemonConfig {
|
||||
if (!raw.models[raw.defaultModel]) throw new Error(`defaultModel ${raw.defaultModel} is not in models`)
|
||||
for (const [name, m] of Object.entries(raw.models)) {
|
||||
if (!m.modes.includes(m.defaultMode)) throw new Error(`model ${name}: defaultMode must be one of its modes`)
|
||||
if (m.nextEditModel && !raw.models[m.nextEditModel]?.nextEdit) throw new Error(`model ${name}: nextEditModel ${m.nextEditModel} must be a profile with nextEdit`)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
|
||||
+31
-11
@@ -47,17 +47,34 @@ function loadModel(name: string) {
|
||||
loadError = null
|
||||
loading = llama.start(config.models[name].path)
|
||||
.catch(e => { loadError = (e as Error).message; console.error(loadError) })
|
||||
.finally(() => { loading = null })
|
||||
.finally(() => { loading = null; if (config.models[name].nextEditModel) void nextEditServer() })
|
||||
return loading
|
||||
}
|
||||
|
||||
// Next-edit companion model (profile.nextEditModel): a second llama-server, started on first use.
|
||||
const neLlama = new LlamaServer({ ...config.llamaServer, port: config.llamaServer.port + 1 })
|
||||
let neLoading: Promise<void> | null = null
|
||||
/** The server that answers next-edit for the active model, or an error message. */
|
||||
async function nextEditServer(): Promise<LlamaServer | string> {
|
||||
const profile = config.models[state.model]
|
||||
if (profile.nextEdit) return loading || !llama.running ? (loadError ?? 'model loading') : llama
|
||||
const companion = profile.nextEditModel
|
||||
if (!companion) return `model ${state.model} is not trained for next-edit`
|
||||
const wanted = config.models[companion].path
|
||||
if (neLlama.model !== wanted && !neLoading) {
|
||||
neLoading = neLlama.start(wanted).catch(e => console.error(`next-edit model ${companion}: ${(e as Error).message}`)).finally(() => { neLoading = null })
|
||||
}
|
||||
if (neLoading) await neLoading
|
||||
return neLlama.running && neLlama.model === wanted ? neLlama : `next-edit model ${companion} failed to start`
|
||||
}
|
||||
|
||||
let inflight: AbortController | null = null
|
||||
|
||||
function describeConfig() {
|
||||
return {
|
||||
...state,
|
||||
effectiveMode: state.mode,
|
||||
models: Object.fromEntries(Object.entries(config.models).map(([n, m]) => [n, { order: m.order, modes: m.modes, defaultMode: m.defaultMode, nextEdit: m.nextEdit ?? false, errors: m.errors ?? false, contextFormat: m.contextFormat ?? 1 }])),
|
||||
models: Object.fromEntries(Object.entries(config.models).map(([n, m]) => [n, { order: m.order, modes: m.modes, defaultMode: m.defaultMode, nextEdit: !!(m.nextEdit || m.nextEditModel), nextEditModel: m.nextEditModel ?? null, errors: m.errors ?? false, contextFormat: m.contextFormat ?? 1 }])),
|
||||
llama: { running: llama.running, loading: loading !== null, error: loadError },
|
||||
}
|
||||
}
|
||||
@@ -245,8 +262,8 @@ async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, c
|
||||
const t0 = performance.now()
|
||||
const profile = config.models[state.model]
|
||||
if (!state.enabled) return json(res, 200, { noEdit: true, disabled: true })
|
||||
if (!profile.nextEdit) return json(res, 400, { error: `model ${state.model} is not trained for next-edit` })
|
||||
if (loading || !llama.running) return json(res, 503, { error: loadError ?? 'model loading' })
|
||||
const server = await nextEditServer()
|
||||
if (typeof server === 'string') return json(res, server.includes('not trained') ? 400 : 503, { error: server })
|
||||
if (typeof body.text !== 'string' || typeof body.cursor !== 'number') return json(res, 400, { error: 'text and cursor are required' })
|
||||
|
||||
const o = config.nextEditOptions ?? { regionBefore: 4, regionAfter: 8, maxTokens: 400, beforeTokens: 1024, afterTokens: 384 }
|
||||
@@ -266,7 +283,7 @@ async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, c
|
||||
const controller = new AbortController()
|
||||
inflight = controller
|
||||
clientGone.addEventListener('abort', () => controller.abort())
|
||||
const result = await llama.complete(ids, { nPredict: o.maxTokens, stop: [NEXT_EDIT_TOKENS.outputEnd, ...STOP_STRINGS] }, controller.signal)
|
||||
const result = await server.complete(ids, { nPredict: o.maxTokens, stop: [NEXT_EDIT_TOKENS.outputEnd, ...STOP_STRINGS] }, controller.signal)
|
||||
if (inflight === controller) inflight = null
|
||||
if (result.stoppedBy === 'aborted') return json(res, 200, { noEdit: true, cancelled: true })
|
||||
|
||||
@@ -276,7 +293,7 @@ async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, c
|
||||
const noEdit = newText === null || result.stoppedBy === 'limit'
|
||||
const response = {
|
||||
requestId: body.requestId, noEdit, startLine, endLine, oldText, newText: noEdit ? null : newText,
|
||||
recentEdits, errors: errors ?? null, model: state.model,
|
||||
recentEdits, errors: errors ?? null, model: profile.nextEditModel ?? state.model,
|
||||
timings: { promptTokens: ids.length, processedTokens: result.timings?.prompt_n ?? null, genTokens: result.timings?.predicted_n ?? null, totalMs: round(performance.now() - t0) },
|
||||
}
|
||||
remember({ kind: 'next-edit', requestId: body.requestId, path: body.path, model: state.model, prompt: builder.decode(ids), raw, noEdit, timings: response.timings })
|
||||
@@ -303,7 +320,7 @@ if (config.idleShutdownMinutes) {
|
||||
setInterval(() => {
|
||||
if (Date.now() - lastActivity > config.idleShutdownMinutes! * 60_000) {
|
||||
console.log(`idle for ${config.idleShutdownMinutes} min, shutting down`)
|
||||
void llama.stop().then(() => process.exit(0))
|
||||
void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0))
|
||||
}
|
||||
}, 30_000).unref()
|
||||
}
|
||||
@@ -318,7 +335,7 @@ const server = http.createServer(async (req, res) => {
|
||||
if (route === 'GET /health') return json(res, 200, { ok: llama.running, loading: loading !== null, model: state.model, error: loadError, instanceTag: config.instanceTag ?? null, pid: process.pid })
|
||||
if (route === 'POST /v1/shutdown') {
|
||||
json(res, 200, { ok: true })
|
||||
void llama.stop().then(() => process.exit(0))
|
||||
void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0))
|
||||
return
|
||||
}
|
||||
if (route === 'GET /v1/config') return json(res, 200, describeConfig())
|
||||
@@ -331,7 +348,10 @@ const server = http.createServer(async (req, res) => {
|
||||
const modelChanged = model !== state.model
|
||||
Object.assign(state, { model, mode, enabled: body.enabled ?? state.enabled })
|
||||
saveState(config, state)
|
||||
if (modelChanged) void loadModel(model)
|
||||
if (modelChanged) {
|
||||
void loadModel(model)
|
||||
if (neLlama.model && neLlama.model !== config.models[config.models[model].nextEditModel ?? '']?.path) void neLlama.stop()
|
||||
}
|
||||
return json(res, 200, describeConfig())
|
||||
}
|
||||
if (route === 'POST /v1/fim') return await handleFim(await readBody(req) as FimRequest, res, gone.signal)
|
||||
@@ -357,8 +377,8 @@ const server = http.createServer(async (req, res) => {
|
||||
}
|
||||
})
|
||||
|
||||
process.on('SIGINT', () => void llama.stop().then(() => process.exit(0)))
|
||||
process.on('SIGTERM', () => void llama.stop().then(() => process.exit(0)))
|
||||
process.on('SIGINT', () => void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0)))
|
||||
process.on('SIGTERM', () => void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0)))
|
||||
|
||||
server.listen(config.port, config.host, () => {
|
||||
console.log(`acd listening on http://${config.host}:${config.port} (model ${state.model}, mode ${state.mode})`)
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
"displayName": "Local Autocomplete",
|
||||
"description": "Inline code completion from a local fine-tuned model via the acd daemon",
|
||||
"publisher": "harvmaster",
|
||||
"version": "0.6.0",
|
||||
"version": "0.6.1",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"vscode": "^1.90.0"
|
||||
|
||||
@@ -30,13 +30,19 @@ export interface ModelEntry {
|
||||
contextFormat?: 1 | 2
|
||||
errors?: boolean
|
||||
members?: boolean
|
||||
/** Old (≤ 0.6.0) flag: ignored, because those daemons applied an empty "no edit" output as a deletion. */
|
||||
nextEdit?: boolean
|
||||
/** Trained for next-edit (shared/nextedit.ts format version); read since 0.6.1. */
|
||||
nextEditFormat?: 1
|
||||
/** Companion model (another manifest entry) that serves next-edit for this one; downloaded alongside it. */
|
||||
nextEditModel?: string
|
||||
}
|
||||
|
||||
const profileOf = (m: ModelEntry) => ({
|
||||
const profileOf = (m: ModelEntry, available: Set<string>) => ({
|
||||
order: m.order, modes: m.modes, defaultMode: m.defaultMode,
|
||||
...(m.contextFormat ? { contextFormat: m.contextFormat } : {}), ...(m.errors ? { errors: true } : {}),
|
||||
...(m.members !== undefined ? { members: m.members } : {}), ...(m.nextEdit ? { nextEdit: true } : {}),
|
||||
...(m.members !== undefined ? { members: m.members } : {}), ...(m.nextEditFormat ? { nextEdit: true } : {}),
|
||||
...(m.nextEditModel && available.has(m.nextEditModel) ? { nextEditModel: m.nextEditModel } : {}),
|
||||
})
|
||||
|
||||
export interface Manifest {
|
||||
@@ -229,12 +235,12 @@ export async function prepare(storage: string, daemonDir: string, extensionVersi
|
||||
if (!entry) throw new Error(`model "${s.model || manifest.default}" is not in the manifest (available: ${manifest.models.map(m => m.name).join(', ')})`)
|
||||
const file = await ensureFile(entry.url, path.join(storage, 'models', entry.file), entry.sha256, `model ${entry.name}`, progress, signal)
|
||||
defaultModel = entry.name
|
||||
models[entry.name] = { path: file, ...profileOf(entry) }
|
||||
// Other models already on disk stay switchable from the menu.
|
||||
for (const m of manifest.models) {
|
||||
const f = path.join(storage, 'models', m.file)
|
||||
if (m.name !== entry.name && fs.existsSync(`${f}.verified`)) models[m.name] = { path: f, ...profileOf(m) }
|
||||
}
|
||||
const companion = entry.nextEditModel ? manifest.models.find(m => m.name === entry.nextEditModel && m.nextEditFormat) : undefined
|
||||
if (companion) await ensureFile(companion.url, path.join(storage, 'models', companion.file), companion.sha256, `next-edit model ${companion.name}`, progress, signal)
|
||||
// The selected model, its companion and other models already on disk (switchable from the menu).
|
||||
const onDisk = manifest.models.filter(m => m.name === entry.name || fs.existsSync(`${path.join(storage, 'models', m.file)}.verified`))
|
||||
const available = new Set(onDisk.filter(m => m.nextEditFormat).map(m => m.name))
|
||||
for (const m of onDisk) models[m.name] = { path: path.join(storage, 'models', m.file), ...profileOf(m, available) }
|
||||
}
|
||||
|
||||
const binary = s.llamaServerPath ? s.llamaServerPath.replace(/^~(?=$|\/)/, os.homedir()) : await ensureLlamaServer(storage, llamaCpp, progress, signal)
|
||||
|
||||
@@ -36,20 +36,29 @@ const step = (name, fn) => fn().then(r => { console.log(`PASS ${name}`); return
|
||||
|
||||
// 1. Simulate an interrupted model download: first 100 MB already on disk.
|
||||
fs.mkdirSync(path.join(storage, 'models'), { recursive: true })
|
||||
const modelSrc = path.join(releaseDir, 'stage-c-q8_0.gguf')
|
||||
const modelSrc = path.join(releaseDir, 'stage-d-q8_0.gguf')
|
||||
const fd = fs.openSync(modelSrc, 'r'); const buf = Buffer.alloc(100e6); fs.readSync(fd, buf, 0, buf.length, 0); fs.closeSync(fd)
|
||||
fs.writeFileSync(path.join(storage, 'models', 'stage-c-q8_0.gguf.part'), buf)
|
||||
fs.writeFileSync(path.join(storage, 'models', 'stage-d-q8_0.gguf.part'), buf)
|
||||
|
||||
let t0 = Date.now()
|
||||
const prepared = await step('first run: manifest, resumed + verified model download, llama.cpp download + unpack', () => core.prepare(storage, daemonDir, '0.4.0-test', settings, progress))
|
||||
console.log(` ${((Date.now() - t0) / 1000).toFixed(1)} s; range requests: ${rangeRequests}; model verified: ${fs.existsSync(path.join(storage, 'models', 'stage-c-q8_0.gguf.verified'))}`)
|
||||
console.log(` ${((Date.now() - t0) / 1000).toFixed(1)} s; range requests: ${rangeRequests}; model verified: ${fs.existsSync(path.join(storage, 'models', 'stage-d-q8_0.gguf.verified'))}`)
|
||||
assert.ok(rangeRequests >= 1, 'download should have resumed with a Range request')
|
||||
const cfg = JSON.parse(fs.readFileSync(prepared.configPath, 'utf8'))
|
||||
assert.ok(fs.existsSync(cfg.llamaServer.binary), 'llama-server unpacked')
|
||||
await step('next-edit companion downloaded and wired (stage-d → stage-e)', async () => {
|
||||
assert.equal(cfg.defaultModel, 'stage-d')
|
||||
assert.equal(cfg.models['stage-d'].nextEditModel, 'stage-e')
|
||||
assert.equal(cfg.models['stage-e'].nextEdit, true)
|
||||
assert.ok(fs.existsSync(path.join(storage, 'models', 'stage-e-q8_0.gguf.verified')))
|
||||
})
|
||||
|
||||
await step('daemon starts on the editor runtime', async () => assert.equal(await core.ensureRunning(storage, daemonDir, prepared, settings.port, progress), 'started'))
|
||||
const fim = await (await fetch('http://127.0.0.1:7111/v1/fim', { method: 'POST', body: JSON.stringify({ repo: 'e2e', path: 'src/a.ts', prefix: "import { z } from 'zod'\nexport const s = z.object({ a: z.string() })\nexport type S = ", suffix: '\n' }) })).json()
|
||||
await step(`completion through the managed daemon (${JSON.stringify(fim.text)})`, async () => assert.match(fim.text, /z\.infer<typeof s>/))
|
||||
const neText = "export function f(s: string) {\n const t = s.trim()\n return parse(t))\n}\n"
|
||||
const ne = await (await fetch('http://127.0.0.1:7111/v1/next-edit', { method: 'POST', body: JSON.stringify({ repo: 'e2e', path: 'src/f.ts', text: neText, cursor: neText.indexOf('(t)') + 1 }) })).json()
|
||||
await step(`next-edit through the companion model (${JSON.stringify(ne.newText ?? ne.error ?? ne)})`, async () => assert.ok(ne.newText?.includes('return parse(t)\n')))
|
||||
|
||||
t0 = Date.now()
|
||||
const again = await step('second window: no re-download, daemon reused', async () => {
|
||||
|
||||
+2
-1
@@ -2,6 +2,7 @@ These model files are modified versions of Qwen2.5-Coder-0.5B (https://huggingfa
|
||||
Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE).
|
||||
|
||||
Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion
|
||||
(and, for stage-c, a repository/symbol-context header), followed by Q8_0 GGUF conversion.
|
||||
(and, from stage-c on, a repository/symbol-context header; stage-d adds TypeScript errors; stage-e adds
|
||||
next-edit prediction), followed by Q8_0 GGUF conversion.
|
||||
Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and
|
||||
the publisher's own code. Secrets were scrubbed from the training data.
|
||||
|
||||
@@ -5,6 +5,8 @@ Q8_0 GGUF fill-in-the-middle models for the [Local Autocomplete](../extensions/v
|
||||
|
||||
| model | description | prompt order | context modes | size |
|
||||
|---|---|---|---|---|
|
||||
| `stage-d` | Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions | spm | symbols, simple | 531 MB |
|
||||
| `stage-e` | Next-edit prediction (rewrites a region around the cursor or answers "no edit"); also does FIM, slightly below stage-d | spm | symbols, simple | 531 MB |
|
||||
| `stage-c` | Suffix-first FIM with repo/symbol-context headers (default) | spm | symbols, simple | 531 MB |
|
||||
| `stage-b` | Suffix-first FIM without context headers (previous stage) | spm | raw | 531 MB |
|
||||
|
||||
@@ -21,6 +23,16 @@ runs it locally with llama.cpp. The files are stored with Git LFS; download link
|
||||
| Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 |
|
||||
| `stage-b` | 0.391 | 0.689 | 0.952 |
|
||||
| `stage-c` (with symbol context) | 0.397 | 0.681 | 0.954 |
|
||||
| `stage-c` + token healing (daemon default) | 0.405 | 0.689 | |
|
||||
| `stage-d` (symbols + TS errors, healed) | **0.426** | **0.700** | 0.954 |
|
||||
| `stage-e` (symbols + TS errors, healed) | 0.412 | 0.691 | 0.962 |
|
||||
|
||||
New behaviours (886 further held-out samples), exact match, stage-c → stage-d: closing a block 0.27 → **0.79**,
|
||||
next line 0.01 → **0.18**, rest of block 0.08 → 0.11, member access 0.63 → 0.66.
|
||||
|
||||
**Next-edit** (`stage-e`, 445 held-out samples): 80% exact region rewrites (edit similarity 0.99) where an edit
|
||||
was due, and "no edit" on 100% of samples where none was. `stage-d` uses `stage-e` as its next-edit companion
|
||||
(`nextEditModel` in the manifest): the extension downloads both and the daemon runs it in a second llama-server.
|
||||
|
||||
The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts
|
||||
let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a
|
||||
|
||||
+37
-1
@@ -1,7 +1,43 @@
|
||||
{
|
||||
"schema": 1,
|
||||
"default": "stage-c",
|
||||
"default": "stage-d",
|
||||
"models": [
|
||||
{
|
||||
"name": "stage-d",
|
||||
"file": "stage-d-q8_0.gguf",
|
||||
"url": "stage-d-q8_0.gguf",
|
||||
"sha256": "8edaef02ae9e794929af0267f0f47f600824454b08df7d386e386eec1123c839",
|
||||
"size": 531067552,
|
||||
"order": "spm",
|
||||
"modes": [
|
||||
"symbols",
|
||||
"simple"
|
||||
],
|
||||
"defaultMode": "symbols",
|
||||
"contextFormat": 2,
|
||||
"errors": true,
|
||||
"members": true,
|
||||
"nextEditModel": "stage-e",
|
||||
"description": "Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions"
|
||||
},
|
||||
{
|
||||
"name": "stage-e",
|
||||
"file": "stage-e-q8_0.gguf",
|
||||
"url": "stage-e-q8_0.gguf",
|
||||
"sha256": "44d5fe87043f2fd3f8fba39c07a653ec7bdc228f83a2da4a5638be3ab77a962f",
|
||||
"size": 531067552,
|
||||
"order": "spm",
|
||||
"modes": [
|
||||
"symbols",
|
||||
"simple"
|
||||
],
|
||||
"defaultMode": "symbols",
|
||||
"contextFormat": 2,
|
||||
"errors": true,
|
||||
"members": true,
|
||||
"nextEditFormat": 1,
|
||||
"description": "Next-edit prediction (rewrites a region around the cursor or answers \"no edit\"); also does FIM, slightly below stage-d"
|
||||
},
|
||||
{
|
||||
"name": "stage-c",
|
||||
"file": "stage-c-q8_0.gguf",
|
||||
|
||||
BIN
Binary file not shown.
BIN
Binary file not shown.
@@ -23,9 +23,10 @@ const out = path.join(repo, 'models')
|
||||
// Prompt layout each model was trained with (the extension only offers matching context modes).
|
||||
const profiles = {
|
||||
// Stage E: Stage D + next-edit prediction (shared/nextedit.ts)
|
||||
'stage-e': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEdit: true, description: 'Suffix-first FIM with symbols + TS errors, plus next-edit prediction (rewrites a region around the cursor)' },
|
||||
// nextEditFormat (not the old nextEdit flag): extensions ≤ 0.6.0 must not use it (they applied "no edit" as a deletion).
|
||||
'stage-e': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEditFormat: 1, description: 'Next-edit prediction (rewrites a region around the cursor or answers "no edit"); also does FIM, slightly below stage-d' },
|
||||
// Stage D: next-line / block-close / multi-line fixes, namespace-import symbols (format 2), TS errors in the header
|
||||
'stage-d': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, description: 'Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions' },
|
||||
'stage-d': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEditModel: 'stage-e', description: 'Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions' },
|
||||
'stage-c': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'simple', description: 'Suffix-first FIM with repo/symbol-context headers (default)' },
|
||||
'stage-b': { order: 'spm', modes: ['raw'], defaultMode: 'raw', description: 'Suffix-first FIM without context headers (previous stage)' },
|
||||
}
|
||||
@@ -71,7 +72,8 @@ fs.writeFileSync(path.join(out, 'NOTICE'), `These model files are modified versi
|
||||
Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE).
|
||||
|
||||
Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion
|
||||
(and, for stage-c, a repository/symbol-context header), followed by Q8_0 GGUF conversion.
|
||||
(and, from stage-c on, a repository/symbol-context header; stage-d adds TypeScript errors; stage-e adds
|
||||
next-edit prediction), followed by Q8_0 GGUF conversion.
|
||||
Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and
|
||||
the publisher's own code. Secrets were scrubbed from the training data.
|
||||
`)
|
||||
@@ -103,6 +105,16 @@ runs it locally with llama.cpp. The files are stored with Git LFS; download link
|
||||
| Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 |
|
||||
| \`stage-b\` | 0.391 | 0.689 | 0.952 |
|
||||
| \`stage-c\` (with symbol context) | 0.397 | 0.681 | 0.954 |
|
||||
| \`stage-c\` + token healing (daemon default) | 0.405 | 0.689 | |
|
||||
| \`stage-d\` (symbols + TS errors, healed) | **0.426** | **0.700** | 0.954 |
|
||||
| \`stage-e\` (symbols + TS errors, healed) | 0.412 | 0.691 | 0.962 |
|
||||
|
||||
New behaviours (886 further held-out samples), exact match, stage-c → stage-d: closing a block 0.27 → **0.79**,
|
||||
next line 0.01 → **0.18**, rest of block 0.08 → 0.11, member access 0.63 → 0.66.
|
||||
|
||||
**Next-edit** (\`stage-e\`, 445 held-out samples): 80% exact region rewrites (edit similarity 0.99) where an edit
|
||||
was due, and "no edit" on 100% of samples where none was. \`stage-d\` uses \`stage-e\` as its next-edit companion
|
||||
(\`nextEditModel\` in the manifest): the extension downloads both and the daemon runs it in a second llama-server.
|
||||
|
||||
The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts
|
||||
let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a
|
||||
|
||||
Reference in New Issue
Block a user