Author SHA1 Message Date
HarvmasterandClaude Opus 5.5 d246fb9349 Models stage-d (default) + stage-e (next-edit companion); extension 0.6.1
ci / build (push) Canceled after 0s
- stage-d: next-line / block-close / multi-line fixes, symbols format 2, TS errors in the header.
  Held-out: 0.426 exact / 0.700 edit sim (stage-c 0.405 / 0.689); block close 0.27 -> 0.79.
- stage-e: next-edit (80% exact region rewrites, abstains on 100% of no-edit samples).
- Manifest: nextEditModel (companion) and nextEditFormat (older extensions ignore it, so they
  never use next-edit with the empty-output deletion bug).
- Daemon: next-edit served by the companion model in a second llama-server, started after the
  main model loads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
2026-10-04 00:23:15 +00:00
HarvmasterandClaude Opus 5.5 5c99f011e8 Next-edit: treat empty model output as no-edit
ci / build (push) Canceled after 0s
llama-server leaves special tokens out of content, so the NO_EDIT token arrives as an
empty string. The daemon treated that as a rewrite of the region to nothing (Tab would
have deleted the region). shared/nextedit.ts parseNextEditOutput() is now used by the
daemon and the eval.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
2026-10-04 00:15:11 +00:00
HarvmasterandClaude Opus 5.5 f92341d1a2 shared/nextedit: trim before-context in a few encodes instead of one per line
ci / build (push) Canceled after 0s
Same output on 397/398 test prompts, ~6x faster prompt building (daemon and export).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
2026-10-03 18:17:56 +00:00
HarvmasterandClaude Opus 5.5 4c86c2a703 Daemon: OpenAI-compatible /v1/completions for Zed
ci / build (push) Canceled after 0s
Parses Qwen-format FIM prompts (prefix- or suffix-first, optional repo/file headers) back into
prefix/suffix and rebuilds them in the trained layout; mode from the model string (ac:symbols /
ac:simple); files located via workspaceRoots by path or content. extensions/zed/README.md documents
the Zed settings; untested against real Zed traffic yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
2026-10-03 17:25:27 +00:00
HarvmasterandClaude Opus 5.5 87d3c3c3bf Extension 0.6.0: next-edit, token healing, TS errors context, cursor-move, prewarm
ci / build (push) Canceled after 0s
- shared/nextedit.ts: next-edit prompt format (region rewrite or NO_EDIT, Qwen spare special tokens)
- shared/diagnostics.ts: TS errors near the cursor as context; symbols format 2 adds namespace imports
- shared/prompt.ts: token healing (cut the half-typed word, force it via grammar), errors header section
- daemon: model profiles carry contextFormat/errors/members/nextEdit; /v1/next-edit with edit-burst
  tracking; /v1/prewarm; /v1/debug/last; token healing on by default
- extension: next-edit decorations (Tab apply / Esc dismiss), suggest on cursor move, prewarm on focus,
  Show Last Prompt, update check against releases, diagnostics in the edit recorder
- models/README: training-data attribution; release script profiles for stage-d/stage-e
- .gitea/workflows/ci.yml: build + test, publish release on vscode-v* tags (needs a runner + RELEASE_TOKEN)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J39imRDT8L7xYbWzsVWd8A
2026-10-03 17:21:38 +00:00
HarvmasterandClaude Opus 5.5 af302df26a README: roadmap adds TS-error context and cursor-move suggestions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 15:56:53 +00:00
31 changed files with 1389 additions and 71 deletions
+30
View File
@@ -0,0 +1,30 @@
# Runs on any Gitea Actions runner with the ubuntu-latest label (needs Node; no GPU, no models).
name: ci
on:
push:
branches: [main]
tags: ['vscode-v*']
pull_request:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
lfs: false # the models aren't needed to build or test the extension
- uses: actions/setup-node@v4
with:
node-version: 22
- run: npm run setup
- run: npm test
- run: npm run build
- uses: actions/upload-artifact@v3
with:
name: vsix
path: extensions/vscode/*.vsix
# On a vscode-v* tag, publish the release with the VSIX (repo secret RELEASE_TOKEN: write:repository).
- if: startsWith(github.ref, 'refs/tags/vscode-v')
env:
GITEA_TOKEN: ${{ secrets.RELEASE_TOKEN }}
run: node scripts/publish-extension-release.mjs
+22
View File
@@ -2,6 +2,28 @@
Extension versions are released as `vscode-v<version>` on the [Releases](https://git.harvmaster.com/Harvmaster/autocomplete-model/releases) page. Models are versioned by their name in [`models/manifest.json`](models/manifest.json).
## vscode 0.6.1
- **New default model `stage-d`**, with **`stage-e` for next-edit**. Both download automatically (~530 MB each); the
daemon runs `stage-e` in a second llama-server, only for next-edit. `stage-d` closes blocks (`}`) and predicts the
next line far better (held-out exact match: block close 0.27 → 0.79, next line 0.01 → 0.18), and sees TypeScript
errors near the cursor. Results in [models/README.md](models/README.md).
- **Fix:** a "no edit" answer from a next-edit model could be applied as deleting the suggested region. llama-server
drops the model's no-edit token, so it arrived as empty text. Next-edit models are only enabled from this version on.
- Manifest: `nextEditModel` (companion model) and `nextEditFormat` (replaces `nextEdit`, which older versions ignore
from now on).
## vscode 0.6.0
- **Next-edit suggestions** (with models trained for it, e.g. the upcoming `stage-e`): after you pause typing, the model may suggest an edit around the cursor, such as removing a leftover `)` or updating a renamed variable's uses. Single-line changes show deleted text struck through and inserted text as ghost text; multi-line changes are highlighted with the diff on hover. **Tab** applies, **Esc** dismisses. Setting: `nextEdit.enabled`.
- **Token healing:** completions that start mid-word (`this.connections.del▮`) now finish the word instead of skipping it.
- **Suggest on cursor move** (`triggerOnCursorMove`, off by default): with next-edit models it predicts what you came to change; otherwise it only fires on empty lines, line ends and lines with errors.
- **Pre-warm on focus** (`prewarmOnFocus`, off by default): fills the model cache when you switch files, so the first keystroke is fast.
- **Show Last Prompt** command: the exact prompt the model saw, its raw output and timings.
- **Update check** against the Releases page (daily; `checkForUpdates`), with one-click install.
- Edit recorder also records VS Code's diagnostics (TS, Vue, ESLint) for recorded files.
- Daemon: model profiles carry the prompt format they were trained with (`contextFormat`, `errors`, `nextEdit`), so newer models get TS errors and namespace-import symbols in their context.
## vscode 0.5.0
- New repository layout (`models/`, `extensions/vscode/`, `daemon/`, `shared/`). The model manifest moved to `models/manifest.json`, and the default `localAutocomplete.modelManifestUrl` follows it.
+11 -4
View File
@@ -4,7 +4,8 @@ Fast, private, inline code completion for **TypeScript, Vue and Node.js**, from
- **Local:** a 0.5B model (fine-tuned Qwen2.5-Coder) runs through llama.cpp on your CPU or Apple GPU. No code leaves your machine.
- **Fast:** ~90–100 ms per keystroke on a laptop CPU. Prompts are laid out suffix-first so the KV cache survives typing (~2 tokens re-processed per keystroke).
- **Context-aware:** an optional `symbols` mode feeds the model signatures from the TypeScript language service, and a missing `this.` is repaired automatically.
- **Context-aware:** an optional `symbols` mode feeds the model signatures from the TypeScript language service (and, for newer models, nearby TypeScript errors); a missing `this.` is repaired automatically, and half-typed words are completed properly (token healing).
- **Next-edit suggestions** (with models trained for it): edits beyond the cursor, like removing a leftover `)` or updating the uses of a renamed variable. **Tab** applies, **Esc** dismisses.
- **Zero setup:** the VS Code extension downloads the model and llama.cpp on first use and manages the background daemon itself.
| Model (held-out private code, never trained on) | exact match | edit similarity |
@@ -39,7 +40,7 @@ Turn off other inline-completion providers (Copilot, …) so they don't compete.
The menu only offers modes the active model was trained for.
- **Model:** switch between models in the manifest (downloaded on first use).
- **On/off**, **Restart daemon**, **Open daemon log**.
- **On/off**, **Restart daemon**, **Open daemon log**, **Show last prompt** (exactly what the model saw, for debugging a suggestion), **Check for updates**.
### Settings
@@ -50,7 +51,10 @@ Turn off other inline-completion providers (Copilot, …) so they don't compete.
| `localAutocomplete.modelPath` | | use a local `.gguf` instead (set `modelProfile` to match its prompt layout) |
| `localAutocomplete.llamaServerPath` | | use your own `llama-server` (e.g. Homebrew's) instead of the downloaded build |
| `localAutocomplete.excludeGlobs` | `.env*`, keys, `secrets/`, … | never sent to the model |
| `localAutocomplete.recordEdits` | `false` | record your edits locally, as future training data for next-edit prediction |
| `localAutocomplete.nextEdit.enabled` | `true` | next-edit suggestions (only with models trained for it) |
| `localAutocomplete.triggerOnCursorMove` | `false` | also suggest when you move the cursor and pause |
| `localAutocomplete.prewarmOnFocus` | `false` | fill the model cache when you switch files (faster first keystroke) |
| `localAutocomplete.recordEdits` | `false` | record your edits (and diagnostics) locally, as future training data |
| `localAutocomplete.daemon.mode` | `managed` | `external` = use a daemon you run yourself (`serverUrl`) |
## How it works
@@ -72,6 +76,7 @@ VS Code extension ──HTTP──▶ acd daemon ──▶ llama-server (llama.c
|---|---|
| [`models/`](models/) | GGUF models (Git LFS), `manifest.json`, model card, license |
| [`extensions/vscode/`](extensions/vscode/) | the VS Code extension |
| [`extensions/zed/`](extensions/zed/) | Zed setup (settings for the daemon's OpenAI-compatible endpoint) |
| [`daemon/`](daemon/) | `acd`, the completion daemon (bundled into the extension; also runs standalone) |
| [`shared/`](shared/) | prompt layout + symbol extraction, shared with the training pipeline so prompts match training exactly |
| [`scripts/`](scripts/) | publish models and extension releases |
@@ -113,8 +118,10 @@ Running the daemon standalone (external mode) is covered in [daemon/README.md](d
- Better multi-line completions: continuing onto the next line, closing blocks, whole methods.
- **Next-edit prediction:** e.g. removing a leftover `)` after unwrapping a call, or updating call sites after adding a parameter.
- **TypeScript errors as context:** the model sees nearby compiler errors and is trained on error → fix.
- **Suggestions on cursor move:** predict what you came to a location to do, not just what you're typing.
- Token healing for completions that start mid-identifier.
- Zed support via the daemon's OpenAI-compatible endpoint.
- Zed: the daemon's OpenAI-compatible endpoint is in place ([extensions/zed](extensions/zed/)); needs testing against real Zed traffic.
## License
+16
View File
@@ -10,6 +10,16 @@ export interface ModelProfile {
order: FimOrder
modes: ContextMode[]
defaultMode: ContextMode
/** Symbol-context format the model was trained on (shared/symbols.ts): 1 = Stage C, 2 = + namespace imports. */
contextFormat?: 1 | 2
/** Trained with the TS-errors header section (Stage D+). */
errors?: boolean
/** Override symbols.members for this model (member lists after `expr.`). */
members?: boolean
/** Trained for next-edit prediction (shared/nextedit.ts), enables /v1/next-edit. */
nextEdit?: boolean
/** Another profile (trained with nextEdit) that serves /v1/next-edit for this model, in a second llama-server. */
nextEditModel?: string
}
export interface DaemonConfig {
@@ -43,10 +53,15 @@ export interface DaemonConfig {
members: boolean
}
thisRepair: { enabled: boolean; waitMs: number }
/** Cut a half-typed word off the prompt and force the output to start with it (default on). */
tokenHealing?: boolean
nextEditOptions?: { regionBefore: number; regionAfter: number; maxTokens: number; beforeTokens: number; afterTokens: number }
telemetry: { enabled: boolean; dir: string }
/** Where opted-in edit recordings from editors are appended (the opt-in is the editor setting). */
recording: { dir: string }
stateFile: string
/** Workspace roots for clients that can't send one (Zed via /v1/completions): files are looked up under these. */
workspaceRoots?: string[]
/** Exit after this many minutes without any request (0 = never). Editors poll every 15 s while open. */
idleShutdownMinutes?: number
/** Opaque tag reported by /health (the VS Code extension uses it to replace daemons left over from older versions). */
@@ -73,6 +88,7 @@ export function loadConfig(configPath: string): DaemonConfig {
if (!raw.models[raw.defaultModel]) throw new Error(`defaultModel ${raw.defaultModel} is not in models`)
for (const [name, m] of Object.entries(raw.models)) {
if (!m.modes.includes(m.defaultMode)) throw new Error(`model ${name}: defaultMode must be one of its modes`)
if (m.nextEditModel && !raw.models[m.nextEditModel]?.nextEdit) throw new Error(`model ${name}: nextEditModel ${m.nextEditModel} must be a profile with nextEdit`)
}
return raw
}
+36
View File
@@ -0,0 +1,36 @@
import { lineDiff } from '../../shared/nextedit.ts'
/**
* Tracks each file's recent "edit burst" so next-edit prompts can include what the user just changed,
* without editors having to send diffs (works for any client that sends full text, Zed included).
*
* A burst starts when the cursor jumps more than `jumpLines` away or after `idleMs` without requests; its
* baseline is the text at that moment. The recent edit is the line diff from the baseline to now, the same
* compact format as training (shared/nextedit.ts lineDiff).
*/
export class EditTracker {
private files = new Map<string, { baseline: string; last: string; line: number; time: number }>()
private jumpLines: number
private idleMs: number
constructor(opts: { jumpLines?: number; idleMs?: number } = {}) {
this.jumpLines = opts.jumpLines ?? 20
this.idleMs = opts.idleMs ?? 60_000
}
/** Records the file's current text; returns the burst's diff (empty if nothing changed). */
observe(key: string, displayPath: string, text: string, cursorLine: number): string {
const now = Date.now()
const f = this.files.get(key)
if (!f) {
this.files.set(key, { baseline: text, last: text, line: cursorLine, time: now })
if (this.files.size > 50) this.files.delete(this.files.keys().next().value!)
return ''
}
if (Math.abs(cursorLine - f.line) > this.jumpLines || now - f.time > this.idleMs) f.baseline = f.last
f.last = text
f.line = cursorLine
f.time = now
return lineDiff(displayPath, f.baseline, text)
}
}
+2 -2
View File
@@ -70,7 +70,7 @@ export class LlamaServer {
get running() { return this.proc !== null && this.proc.exitCode === null && this.model !== null }
async complete(prompt: number[], opts: { nPredict: number; stop: string[] }, signal: AbortSignal): Promise<CompletionResult> {
async complete(prompt: number[], opts: { nPredict: number; stop: string[]; grammar?: string }, signal: AbortSignal): Promise<CompletionResult> {
let text = ''
let timings: Timings | null = null
let stoppedBy: CompletionResult['stoppedBy'] = 'limit'
@@ -79,7 +79,7 @@ export class LlamaServer {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({
prompt, n_predict: opts.nPredict, stop: opts.stop, stream: true,
prompt, n_predict: opts.nPredict, stop: opts.stop, stream: true, ...(opts.grammar ? { grammar: opts.grammar } : {}),
temperature: 0, top_k: 1, cache_prompt: true, id_slot: 0,
}),
signal,
+180 -22
View File
@@ -9,6 +9,10 @@
* PUT /v1/config { enabled?, model?, mode? } (changing model restarts llama-server)
* POST /v1/feedback { requestId, event: shown|accepted|partial|rejected, acceptedText? }
* POST /v1/edits { events: [...] } opted-in edit recordings from editors -> recording.dir
* POST /v1/completions OpenAI-compatible (Zed): Qwen FIM prompt in, completion text out (prompt rebuilt in our layout)
* POST /v1/prewarm same body as /v1/fim: prefill the prompt in the background (first keystroke warm)
* POST /v1/next-edit { repo, path, workspaceRoot?, languageId?, text, cursor } -> { noEdit, startLine, endLine, newText }
* GET /v1/debug/last?n=1 last requests with decoded prompts and raw output
* POST /v1/shutdown stop llama-server and exit (used by the extension to replace an outdated daemon)
* GET /health
*
@@ -25,6 +29,9 @@ import { Telemetry } from './telemetry.ts'
import { trimSuffixOverlap } from '../../shared/prompt.ts'
import { symbolLang } from '../../shared/symbol-lang.ts'
import { SymbolService, mightNeedThis } from './symbols.ts'
import { EditTracker } from './edits.ts'
import { locateFile, modeFromModel, parseFimPrompt } from './openai-shim.ts'
import { NEXT_EDIT_TOKENS, parseNextEditOutput } from '../../shared/nextedit.ts'
const { values: args } = parseArgs({ options: { config: { type: 'string', default: 'acd.config.json' } } })
const config = loadConfig(args.config!)
@@ -40,31 +47,41 @@ function loadModel(name: string) {
loadError = null
loading = llama.start(config.models[name].path)
.catch(e => { loadError = (e as Error).message; console.error(loadError) })
.finally(() => { loading = null })
.finally(() => { loading = null; if (config.models[name].nextEditModel) void nextEditServer() })
return loading
}
// Next-edit companion model (profile.nextEditModel): a second llama-server, started on first use.
const neLlama = new LlamaServer({ ...config.llamaServer, port: config.llamaServer.port + 1 })
let neLoading: Promise<void> | null = null
/** The server that answers next-edit for the active model, or an error message. */
async function nextEditServer(): Promise<LlamaServer | string> {
const profile = config.models[state.model]
if (profile.nextEdit) return loading || !llama.running ? (loadError ?? 'model loading') : llama
const companion = profile.nextEditModel
if (!companion) return `model ${state.model} is not trained for next-edit`
const wanted = config.models[companion].path
if (neLlama.model !== wanted && !neLoading) {
neLoading = neLlama.start(wanted).catch(e => console.error(`next-edit model ${companion}: ${(e as Error).message}`)).finally(() => { neLoading = null })
}
if (neLoading) await neLoading
return neLlama.running && neLlama.model === wanted ? neLlama : `next-edit model ${companion} failed to start`
}
let inflight: AbortController | null = null
function describeConfig() {
return {
...state,
effectiveMode: state.mode,
models: Object.fromEntries(Object.entries(config.models).map(([n, m]) => [n, { order: m.order, modes: m.modes, defaultMode: m.defaultMode }])),
models: Object.fromEntries(Object.entries(config.models).map(([n, m]) => [n, { order: m.order, modes: m.modes, defaultMode: m.defaultMode, nextEdit: !!(m.nextEdit || m.nextEditModel), nextEditModel: m.nextEditModel ?? null, errors: m.errors ?? false, contextFormat: m.contextFormat ?? 1 }])),
llama: { running: llama.running, loading: loading !== null, error: loadError },
}
}
async function handleFim(body: FimRequest, res: http.ServerResponse, clientGone: AbortSignal) {
const t0 = performance.now()
if (!state.enabled) return json(res, 200, { text: '', disabled: true })
if (loading) return json(res, 503, { error: 'model loading' })
if (!llama.running) return json(res, 503, { error: loadError ?? 'llama-server not running' })
if (typeof body.prefix !== 'string' || typeof body.suffix !== 'string' || !body.path || !body.repo) {
return json(res, 400, { error: 'repo, path, prefix and suffix are required' })
}
/** Everything needed to run a completion: the prompt (symbols, errors, healing, this.-repair applied) and its metadata. */
async function prepareFim(body: FimRequest) {
const profile = config.models[state.model]
const requested = body.mode && profile.modes.includes(body.mode) ? body.mode : state.mode
let mode: ContextMode = requested
@@ -87,18 +104,47 @@ async function handleFim(body: FimRequest, res: http.ServerResponse, clientGone:
}
let symbolContext: string | undefined
let symbolErrors: string | undefined
let symbolInfo: Record<string, unknown> | undefined
if (mode === 'symbols') {
if (root && filePath && lang) {
// Anchor on the original text so a repair doesn't churn the cached context.
const s = await symbols.context({ root, filePath, lang, prefix: body.prefix, suffix: body.suffix, budget: config.symbols.budget, members: config.symbols.members, waitMs: config.symbols.waitMs })
symbolContext = s.result?.context
symbolInfo = { status: s.status, items: s.result?.items ?? 0 }
const s = await symbols.context({
root, filePath, lang, prefix: body.prefix, suffix: body.suffix, budget: config.symbols.budget,
members: profile.members ?? config.symbols.members, waitMs: config.symbols.waitMs, format: profile.contextFormat ?? 1, errors: profile.errors ?? false,
})
symbolContext = s.result?.context || undefined
symbolErrors = s.result?.errors
symbolInfo = { status: s.status, items: s.result?.items ?? 0, errors: symbolErrors ? symbolErrors.split('\n').length : 0 }
} else {
mode = 'simple' // no workspace or not a TS-family file: header without symbols (Stage C saw this via context dropout)
mode = 'simple' // no workspace or not a TS-family file: header without symbols (seen in training via context dropout)
}
}
const prompt = builder.build({ ...body, prefix }, profile, mode, symbolContext)
const prompt = builder.build({ ...body, prefix }, profile, mode, symbolContext, symbolErrors, config.tokenHealing !== false)
return { prompt, mode, requested, repair, symbolInfo, filePath }
}
/** Last requests with their decoded prompts, for "Show last prompt" (GET /v1/debug/last). */
const debugLog: Record<string, unknown>[] = []
function remember(entry: Record<string, unknown>) {
debugLog.push({ t: new Date().toISOString(), ...entry })
if (debugLog.length > 20) debugLog.shift()
}
const edits = new EditTracker()
async function handleFim(body: FimRequest, res: http.ServerResponse, clientGone: AbortSignal) {
const t0 = performance.now()
if (!state.enabled) return json(res, 200, { text: '', disabled: true })
if (loading) return json(res, 503, { error: 'model loading' })
if (!llama.running) return json(res, 503, { error: loadError ?? 'llama-server not running' })
if (typeof body.prefix !== 'string' || typeof body.suffix !== 'string' || !body.path || !body.repo) {
return json(res, 400, { error: 'repo, path, prefix and suffix are required' })
}
// Keep the edit-burst tracker current (only when the client sent the whole file).
if (!body.prefixLineOffset) edits.observe(`${body.workspaceRoot ?? body.repo}/${body.path}`, body.path, body.prefix + body.suffix, body.prefix.split('\n').length - 1)
const { prompt, mode, requested, repair, symbolInfo } = await prepareFim(body)
inflight?.abort()
const controller = new AbortController()
@@ -109,6 +155,7 @@ async function handleFim(body: FimRequest, res: http.ServerResponse, clientGone:
const result = await llama.complete(prompt.ids, {
nPredict: prompt.singleLine ? p.nPredictSingleLine : p.nPredictMultiLine,
stop: prompt.singleLine ? [...STOP_STRINGS, '\n'] : STOP_STRINGS,
grammar: prompt.grammar,
}, controller.signal)
if (inflight === controller) inflight = null
if (result.stoppedBy === 'aborted') {
@@ -116,7 +163,10 @@ async function handleFim(body: FimRequest, res: http.ServerResponse, clientGone:
return
}
let text = trimSuffixOverlap(result.text, body.suffix)
// Token healing: the output starts with the half-typed word that was cut off the prompt; strip it.
let raw = result.text
if (prompt.forced) raw = raw.startsWith(prompt.forced) ? raw.slice(prompt.forced.length) : ''
let text = trimSuffixOverlap(raw, body.suffix)
if (prompt.singleLine) text = text.split('\n')[0]
const maxLines = body.maxLines ?? p.maxLines
const lines = text.split('\n')
@@ -134,6 +184,7 @@ async function handleFim(body: FimRequest, res: http.ServerResponse, clientGone:
mode,
requestedMode: requested,
singleLine: prompt.singleLine,
healed: prompt.forced ?? null,
replaceBefore: repair?.replaceBefore ?? 0,
filterText,
symbols: symbolInfo,
@@ -148,10 +199,107 @@ async function handleFim(body: FimRequest, res: http.ServerResponse, clientGone:
totalMs: round(performance.now() - t0),
},
}
remember({ kind: 'fim', requestId: body.requestId, path: body.path, model: state.model, mode, prompt: builder.decode(prompt.ids), raw: result.text, text, timings: response.timings })
telemetry.request(body, response)
json(res, 200, response)
}
/**
* OpenAI-compatible /v1/completions for editors with built-in providers (Zed: edit_predictions.provider =
* "open_ai_compatible_api", prompt_format "qwen", model "ac:symbols"). The client's prompt is parsed back into
* prefix/suffix and rebuilt in our trained layout; the answer comes back in OpenAI text_completion format.
*/
async function handleOpenAiCompletion(body: Record<string, unknown>, res: http.ServerResponse, clientGone: AbortSignal) {
const prompt = String(body.prompt ?? '')
const parsed = parseFimPrompt(prompt) ?? { prefix: prompt, suffix: '' }
const located = locateFile(config.workspaceRoots ?? [], parsed.path, parsed.prefix)
const fimBody: FimRequest = {
requestId: `oai-${Date.now()}`,
repo: located ? path.basename(located.root) : parsed.repo ?? 'workspace',
path: located?.path ?? parsed.path ?? 'untitled.ts',
workspaceRoot: located?.root,
languageId: /\.vue$/.test(located?.path ?? parsed.path ?? '') ? 'vue' : 'typescript',
prefix: parsed.prefix,
suffix: parsed.suffix,
mode: modeFromModel(body.model),
}
// Reuse the normal pipeline by capturing its JSON response.
let captured: Record<string, unknown> = {}
const fake = { writeHead: () => fake, end: (s: string) => { captured = JSON.parse(s) }, writableEnded: false, destroyed: false } as unknown as http.ServerResponse
await handleFim(fimBody, fake, clientGone)
const text = String(captured.text ?? '')
const out = {
id: `cmpl-${Date.now()}`, object: 'text_completion', created: Math.floor(Date.now() / 1000), model: String(body.model ?? state.model),
choices: [{ text, index: 0, logprobs: null, finish_reason: 'stop' }],
usage: { prompt_tokens: (captured.timings as Record<string, number> | undefined)?.promptTokens ?? 0, completion_tokens: (captured.timings as Record<string, number> | undefined)?.genTokens ?? 0, total_tokens: 0 },
}
if (body.stream) {
res.writeHead(200, { 'content-type': 'text/event-stream', 'cache-control': 'no-cache' })
res.write(`data: ${JSON.stringify(out)}\n\n`)
res.end('data: [DONE]\n\n')
return
}
json(res, 200, out)
}
/** Prefill a file's prompt in the background (e.g. on editor focus) so the first keystroke is warm. */
async function handlePrewarm(body: FimRequest, res: http.ServerResponse) {
if (!state.enabled || loading || !llama.running) return json(res, 200, { warmed: false })
if (inflight) return json(res, 200, { warmed: false, reason: 'busy' }) // never compete with a real completion
const { prompt } = await prepareFim(body)
const controller = new AbortController()
const r = await llama.complete(prompt.ids, { nPredict: 1, stop: STOP_STRINGS }, controller.signal)
json(res, 200, { warmed: true, promptTokens: prompt.ids.length, processedTokens: r.timings?.prompt_n ?? null, promptMs: round(r.timings?.prompt_ms) })
}
interface NextEditRequest { requestId?: string; repo: string; path: string; workspaceRoot?: string; languageId?: string; text: string; cursor: number }
/**
* Next-edit prediction (models with profile.nextEdit): rewrite a small region around the cursor, given the
* current edit burst and nearby TS errors, or answer "no edit".
*/
async function handleNextEdit(body: NextEditRequest, res: http.ServerResponse, clientGone: AbortSignal) {
const t0 = performance.now()
const profile = config.models[state.model]
if (!state.enabled) return json(res, 200, { noEdit: true, disabled: true })
const server = await nextEditServer()
if (typeof server === 'string') return json(res, server.includes('not trained') ? 400 : 503, { error: server })
if (typeof body.text !== 'string' || typeof body.cursor !== 'number') return json(res, 400, { error: 'text and cursor are required' })
const o = config.nextEditOptions ?? { regionBefore: 4, regionAfter: 8, maxTokens: 400, beforeTokens: 1024, afterTokens: 384 }
const lines = body.text.split('\n')
const cursorLine = body.text.slice(0, body.cursor).split('\n').length - 1
const startLine = Math.max(0, cursorLine - o.regionBefore)
const endLine = Math.min(lines.length, cursorLine + o.regionAfter + 1)
const recentEdits = edits.observe(`${body.workspaceRoot ?? body.repo}/${body.path}`, body.path, body.text, cursorLine)
const lang = symbolLang(body.languageId ?? '')
const errors = body.workspaceRoot && lang
? await symbols.errors({ root: body.workspaceRoot, filePath: path.join(body.workspaceRoot, body.path), lang, text: body.text, cursor: body.cursor, waitMs: 300 })
: undefined
const ids = builder.buildNextEdit({ repo: body.repo, path: body.path, file: body.text, regionStartLine: startLine, regionEndLine: endLine, cursor: body.cursor, recentEdits: recentEdits || undefined, errors },
{ beforeTokens: o.beforeTokens, afterTokens: o.afterTokens })
inflight?.abort()
const controller = new AbortController()
inflight = controller
clientGone.addEventListener('abort', () => controller.abort())
const result = await server.complete(ids, { nPredict: o.maxTokens, stop: [NEXT_EDIT_TOKENS.outputEnd, ...STOP_STRINGS] }, controller.signal)
if (inflight === controller) inflight = null
if (result.stoppedBy === 'aborted') return json(res, 200, { noEdit: true, cancelled: true })
const oldText = lines.slice(startLine, endLine).join('\n') + (endLine < lines.length ? '\n' : '')
const raw = result.text
const newText = parseNextEditOutput(raw, oldText)
const noEdit = newText === null || result.stoppedBy === 'limit'
const response = {
requestId: body.requestId, noEdit, startLine, endLine, oldText, newText: noEdit ? null : newText,
recentEdits, errors: errors ?? null, model: profile.nextEditModel ?? state.model,
timings: { promptTokens: ids.length, processedTokens: result.timings?.prompt_n ?? null, genTokens: result.timings?.predicted_n ?? null, totalMs: round(performance.now() - t0) },
}
remember({ kind: 'next-edit', requestId: body.requestId, path: body.path, model: state.model, prompt: builder.decode(ids), raw, noEdit, timings: response.timings })
json(res, 200, response)
}
const round = (x: number | undefined) => (x === undefined ? null : Math.round(x * 10) / 10)
function json(res: http.ServerResponse, status: number, body: unknown) {
@@ -172,7 +320,7 @@ if (config.idleShutdownMinutes) {
setInterval(() => {
if (Date.now() - lastActivity > config.idleShutdownMinutes! * 60_000) {
console.log(`idle for ${config.idleShutdownMinutes} min, shutting down`)
void llama.stop().then(() => process.exit(0))
void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0))
}
}, 30_000).unref()
}
@@ -187,7 +335,7 @@ const server = http.createServer(async (req, res) => {
if (route === 'GET /health') return json(res, 200, { ok: llama.running, loading: loading !== null, model: state.model, error: loadError, instanceTag: config.instanceTag ?? null, pid: process.pid })
if (route === 'POST /v1/shutdown') {
json(res, 200, { ok: true })
void llama.stop().then(() => process.exit(0))
void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0))
return
}
if (route === 'GET /v1/config') return json(res, 200, describeConfig())
@@ -200,10 +348,20 @@ const server = http.createServer(async (req, res) => {
const modelChanged = model !== state.model
Object.assign(state, { model, mode, enabled: body.enabled ?? state.enabled })
saveState(config, state)
if (modelChanged) void loadModel(model)
if (modelChanged) {
void loadModel(model)
if (neLlama.model && neLlama.model !== config.models[config.models[model].nextEditModel ?? '']?.path) void neLlama.stop()
}
return json(res, 200, describeConfig())
}
if (route === 'POST /v1/fim') return await handleFim(await readBody(req) as FimRequest, res, gone.signal)
if (route === 'POST /v1/completions') return await handleOpenAiCompletion(await readBody(req) as Record<string, unknown>, res, gone.signal)
if (route === 'POST /v1/prewarm') return await handlePrewarm(await readBody(req) as FimRequest, res)
if (route === 'POST /v1/next-edit') return await handleNextEdit(await readBody(req) as NextEditRequest, res, gone.signal)
if (route === 'GET /v1/debug/last') {
const n = Math.min(20, Number(new URL(req.url!, 'http://x').searchParams.get('n') ?? 1))
return json(res, 200, debugLog.slice(-n).reverse())
}
if (route === 'POST /v1/edits') {
const body = await readBody(req) as { events?: unknown[] }
return json(res, 200, { recorded: Telemetry.recordEdits(config.recording.dir, body.events ?? []) })
@@ -219,8 +377,8 @@ const server = http.createServer(async (req, res) => {
}
})
process.on('SIGINT', () => void llama.stop().then(() => process.exit(0)))
process.on('SIGTERM', () => void llama.stop().then(() => process.exit(0)))
process.on('SIGINT', () => void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0)))
process.on('SIGTERM', () => void Promise.all([llama.stop(), neLlama.stop()]).then(() => process.exit(0)))
server.listen(config.port, config.host, () => {
console.log(`acd listening on http://${config.host}:${config.port} (model ${state.model}, mode ${state.mode})`)
+69
View File
@@ -0,0 +1,69 @@
import fs from 'node:fs'
import path from 'node:path'
/**
* Parses a FIM prompt built by another client (e.g. Zed's `open_ai_compatible_api` edit-prediction provider with
* prompt_format "qwen") back into prefix / suffix / file path, so the daemon can rebuild the prompt in the layout
* our models were trained on (suffix-first, repo header, symbols, healing).
*
* Accepts, in order of preference:
* [<|repo_name|>repo\n] [<|file_sep|>path\n …] <|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>
* <|fim_prefix|><|fim_suffix|>{suffix}<|fim_middle|>{prefix} (suffix-first)
* Returns null for anything else (e.g. a plain completion prompt): the caller then treats it as prefix-only.
*/
export function parseFimPrompt(prompt: string): { prefix: string; suffix: string; path?: string; repo?: string } | null {
const repo = /<\|repo_name\|>([^\n<]*)\n/.exec(prompt)?.[1]?.trim() || undefined
const seps = [...prompt.matchAll(/<\|file_sep\|>([^\n<]*)\n/g)]
const filePath = seps.length ? seps.at(-1)![1].trim() || undefined : undefined
const psm = /<\|fim_prefix\|>([\s\S]*?)<\|fim_suffix\|>([\s\S]*?)<\|fim_middle\|>\s*$/.exec(prompt)
if (psm && !psm[1].includes('<|fim_suffix|>')) {
if (psm[1] === '') {
// Possibly suffix-first: "<pre><suf>suffix<mid>prefix"
const spm = /<\|fim_prefix\|><\|fim_suffix\|>([\s\S]*?)<\|fim_middle\|>([\s\S]*)$/.exec(prompt)
if (spm) return { prefix: spm[2], suffix: spm[1], path: filePath, repo }
}
return { prefix: psm[1], suffix: psm[2], path: filePath, repo }
}
const spm = /<\|fim_prefix\|><\|fim_suffix\|>([\s\S]*?)<\|fim_middle\|>([\s\S]*)$/.exec(prompt)
if (spm) return { prefix: spm[2], suffix: spm[1], path: filePath, repo }
return null
}
/** "ac:symbols" / "ac:simple" / "ac:raw" select the context mode; anything else keeps the current toggle. */
export function modeFromModel(model: unknown): 'symbols' | 'simple' | 'raw' | undefined {
const m = /^ac:(symbols|simple|raw)$/.exec(String(model ?? ''))
return m ? (m[1] as 'symbols' | 'simple' | 'raw') : undefined
}
/**
* Finds the workspace for a request: the root containing `relPath` (if the client sent a path), else the root
* containing a file whose content starts with the prompt's prefix (Zed may not send a path).
*/
export function locateFile(roots: string[], relPath: string | undefined, prefix: string): { root: string; path: string } | null {
if (relPath) {
for (const root of roots) {
if (path.isAbsolute(relPath) && relPath.startsWith(root + path.sep)) return { root, path: path.relative(root, relPath).split(path.sep).join('/') }
if (fs.existsSync(path.join(root, relPath))) return { root, path: relPath }
}
}
const probe = prefix.slice(0, 400)
if (probe.length < 40) return null
for (const root of roots) {
const stack = [root]
let visited = 0
while (stack.length && visited < 5000) {
const dir = stack.pop()!
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
if (e.name.startsWith('.') || e.name === 'node_modules' || e.name === 'dist') continue
const p = path.join(dir, e.name)
if (e.isDirectory()) stack.push(p)
else if (/\.(ts|tsx|js|jsx|vue|mts|cts)$/.test(e.name) && ++visited) {
try {
if (fs.readFileSync(p, 'utf8').startsWith(probe)) return { root, path: path.relative(root, p).split(path.sep).join('/') }
} catch { /* unreadable */ }
}
}
}
}
return null
}
+19 -4
View File
@@ -1,6 +1,7 @@
import fs from 'node:fs'
import { Tokenizer } from '@huggingface/tokenizers'
import { chunkedPrefixTokens, fimPrompt, headTokens, repoHeader, wantsSingleLine } from '../../shared/prompt.ts'
import { chunkedPrefixTokens, fimPrompt, forcedPrefixGrammar, headTokens, healPrefix, repoHeader, wantsSingleLine } from '../../shared/prompt.ts'
import { buildNextEditPrompt, nextEditTokenIds, type NextEditInput, type NextEditTokenIds } from '../../shared/nextedit.ts'
import type { ContextMode, DaemonConfig, ModelProfile } from './config.ts'
export interface FimRequest {
@@ -28,12 +29,17 @@ export interface BuiltPrompt {
singleLine: boolean
prefixStartLine: number
tokenizeMs: number
/** Token healing: text the output is forced to start with (strip it from the result), and its grammar. */
forced?: string
grammar?: string
}
export const STOP_STRINGS = ['<|endoftext|>', '<|fim_prefix|>', '<|fim_suffix|>', '<|fim_middle|>', '<|fim_pad|>', '<|file_sep|>', '<|repo_name|>', '<|im_start|>', '<|im_end|>']
export class PromptBuilder {
private encodeFn: (s: string) => number[]
private decodeFn: (ids: number[]) => string
readonly nextEditIds: NextEditTokenIds
private fim: { prefix: number; suffix: number; middle: number }
private header: { repoName: number; fileSep: number }
private config: DaemonConfig
@@ -45,6 +51,8 @@ export class PromptBuilder {
JSON.parse(fs.readFileSync(config.tokenizer.config, 'utf8')),
)
this.encodeFn = s => tokenizer.encode(s, { add_special_tokens: false }).ids
this.decodeFn = ids => tokenizer.decode(ids, { skip_special_tokens: false })
this.nextEditIds = nextEditTokenIds(t => tokenizer.token_to_id(t))
const id = (t: string) => {
const v = tokenizer.token_to_id(t)
if (v === undefined) throw new Error(`tokenizer is missing ${t}`)
@@ -55,23 +63,30 @@ export class PromptBuilder {
}
encode(s: string) { return this.encodeFn(s) }
decode(ids: number[]) { return this.decodeFn(ids) }
buildNextEdit(input: NextEditInput, opts: { beforeTokens: number; afterTokens: number }): number[] {
return buildNextEditPrompt(input, this.encodeFn, this.nextEditIds, opts)
}
/**
* Builds the prompt in the model's trained layout. `symbols` context is passed
* in by the caller (empty until the symbol provider exists, which degrades to `simple`).
*/
build(req: FimRequest, profile: ModelProfile, mode: ContextMode, symbols?: string): BuiltPrompt {
build(req: FimRequest, profile: ModelProfile, mode: ContextMode, symbols?: string, errors?: string, heal = false): BuiltPrompt {
const t0 = performance.now()
const p = this.config.prompt
const { tokens: prefix, startLine } = chunkedPrefixTokens(req.prefix, p.prefixTokens, this.encodeFn, p.prefixChunkLines, req.prefixLineOffset ?? 0)
const healed = heal ? healPrefix(req.prefix) : null
const { tokens: prefix, startLine } = chunkedPrefixTokens(healed ? healed.prefix : req.prefix, p.prefixTokens, this.encodeFn, p.prefixChunkLines, req.prefixLineOffset ?? 0)
const suffix = headTokens(req.suffix, p.suffixTokens, this.encodeFn)
const header = mode === 'raw' ? [] : repoHeader(req.repo, req.path, mode === 'symbols' ? symbols : undefined, this.encodeFn, this.header)
const header = mode === 'raw' ? [] : repoHeader(req.repo, req.path, mode === 'symbols' ? symbols : undefined, this.encodeFn, this.header, mode === 'symbols' && profile.errors ? errors : undefined)
return {
ids: [...header, ...fimPrompt(profile.order, prefix, suffix, this.fim)],
mode,
singleLine: req.singleLine ?? wantsSingleLine(req.suffix),
prefixStartLine: startLine,
tokenizeMs: performance.now() - t0,
...(healed ? { forced: healed.forced, grammar: forcedPrefixGrammar(healed.forced) } : {}),
}
}
}
+12 -3
View File
@@ -7,10 +7,12 @@ import fs from 'node:fs'
import { parentPort, workerData } from 'node:worker_threads'
import { Tokenizer } from '@huggingface/tokenizers'
import { WorkspaceSymbols, type SymbolLang } from '../../shared/symbols.ts'
import { formatDiagnostics, relevantDiagnostics } from '../../shared/diagnostics.ts'
export type WorkerRequest =
| { id: number; kind: 'warm'; root: string }
| { id: number; kind: 'context'; root: string; filePath: string; lang: SymbolLang; visible: string; cursor: number; budget: number; members: boolean }
| { id: number; kind: 'context'; root: string; filePath: string; lang: SymbolLang; visible: string; cursor: number; budget: number; members: boolean; format: 1 | 2; errors: boolean }
| { id: number; kind: 'errors'; root: string; filePath: string; lang: SymbolLang; visible: string; cursor: number }
| { id: number; kind: 'thisMember'; root: string; filePath: string; lang: SymbolLang; visible: string; cursor: number }
export interface WorkerResponse { id: number; result?: unknown; error?: string; ms: number }
@@ -31,9 +33,16 @@ parentPort!.on('message', (req: WorkerRequest) => {
try {
let result: unknown = null
if (req.kind === 'warm') workspace(req.root)
if (req.kind === 'context') {
result = workspace(req.root).context({ filePath: req.filePath, lang: req.lang, visible: req.visible, cursor: req.cursor, budget: req.budget, countTokens, members: req.members })
const errorsAt = (w: WorkspaceSymbols, filePath: string, lang: SymbolLang, visible: string, cursor: number) => {
const line = visible.slice(0, cursor).split('\n').length - 1
return formatDiagnostics(relevantDiagnostics(w.diagnostics(filePath, lang, visible), line), visible) || undefined
}
if (req.kind === 'context') {
const w = workspace(req.root)
const symbols = w.context({ filePath: req.filePath, lang: req.lang, visible: req.visible, cursor: req.cursor, budget: req.budget, countTokens, members: req.members, format: req.format })
result = { ...(symbols ?? { context: '', items: 0, hasMembers: false }), errors: req.errors ? errorsAt(w, req.filePath, req.lang, req.visible, req.cursor) : undefined }
}
if (req.kind === 'errors') result = errorsAt(workspace(req.root), req.filePath, req.lang, req.visible, req.cursor)
if (req.kind === 'thisMember') result = workspace(req.root).thisMember(req.filePath, req.lang, req.visible, req.cursor)
parentPort!.postMessage({ id: req.id, result, ms: performance.now() - t0 } satisfies WorkerResponse)
} catch (e) {
+16 -3
View File
@@ -1,6 +1,7 @@
import { Worker } from 'node:worker_threads'
import crypto from 'node:crypto'
import type { SymbolsResult } from '../../shared/symbols.ts'
import type { SymbolsResult as BaseSymbolsResult } from '../../shared/symbols.ts'
export type SymbolsResult = BaseSymbolsResult & { errors?: string }
import type { SymbolLang } from '../../shared/symbol-lang.ts'
import type { WorkerRequest, WorkerResponse } from './symbols-worker.ts'
@@ -49,9 +50,11 @@ export class SymbolService {
void this.call('warm', { root })
}
async context(opts: { root: string; filePath: string; lang: SymbolLang; prefix: string; suffix: string; budget: number; members: boolean; waitMs: number }):
async context(opts: { root: string; filePath: string; lang: SymbolLang; prefix: string; suffix: string; budget: number; members: boolean; waitMs: number; format?: 1 | 2; errors?: boolean }):
Promise<{ result: SymbolsResult | null; status: 'hit' | 'fresh' | 'stale' | 'none' }> {
const key = anchorKey(opts.filePath, opts.prefix, opts.members)
// Errors go stale as you type; refresh them at most every 20 s (a header change costs a cold prefill).
const errorsBucket = opts.errors ? `|e${Math.floor(Date.now() / 20_000)}` : ''
const key = `${anchorKey(opts.filePath, opts.prefix, opts.members)}|f${opts.format ?? 1}${errorsBucket}`
if (this.cache.has(key)) {
const result = this.cache.get(key)!
this.lastForFile.set(opts.filePath, result)
@@ -59,6 +62,7 @@ export class SymbolService {
}
const request = this.inflight.has(key) ? null : this.call('context', {
root: opts.root, filePath: opts.filePath, lang: opts.lang, visible: opts.prefix + opts.suffix, cursor: opts.prefix.length, budget: opts.budget, members: opts.members,
format: opts.format ?? 1, errors: opts.errors ?? false,
})
if (request) {
this.inflight.add(key)
@@ -83,6 +87,15 @@ export class SymbolService {
if (this.cache.size > 500) this.cache.delete(this.cache.keys().next().value!)
}
/** TS errors near the cursor (formatted), for next-edit prompts. */
async errors(opts: { root: string; filePath: string; lang: SymbolLang; text: string; cursor: number; waitMs: number }): Promise<string | undefined> {
const r = await Promise.race([
this.call('errors', { root: opts.root, filePath: opts.filePath, lang: opts.lang, visible: opts.text, cursor: opts.cursor }),
new Promise<null>(r => setTimeout(() => r(null), opts.waitMs)),
])
return (r?.result as string | undefined) ?? undefined
}
async thisMember(opts: { root: string; filePath: string; lang: SymbolLang; prefix: string; suffix: string; waitMs: number }) {
const r = await Promise.race([
this.call('thisMember', { root: opts.root, filePath: opts.filePath, lang: opts.lang, visible: opts.prefix + opts.suffix, cursor: opts.prefix.length }),
+24 -1
View File
@@ -105,6 +105,29 @@ if (envRoot && fs.existsSync(envRoot) && process.env.SMOKE_TS_FILE && process.en
console.log('SKIP symbols / this.-repair checks (set SMOKE_TS_ROOT, SMOKE_TS_FILE, SMOKE_TS_ANCHOR)')
}
// ---------- token healing, debug log, prewarm
const healedRes = (await fim('export class RelayService {\n private readonly connections = new Map<string, WebSocket>()\n\n disconnect(relayUrl: string) {\n const connection = this.connections.get(relayUrl)\n connection?.close()\n this.connections.del', '\n }\n}\n')).body
check('token healing completes a half-typed word', healedRes.healed === 'del' && healedRes.text.startsWith('ete'), JSON.stringify({ healed: healedRes.healed, text: healedRes.text }))
const dbg = (await api('GET', '/v1/debug/last?n=2')).body as unknown as Record<string, string>[]
check('debug log returns decoded prompts', Array.isArray(dbg) && dbg[0]?.prompt?.includes('<|fim_prefix|>'), `${dbg.length} entries`)
const warm = (await api('POST', '/v1/prewarm', { repo: 'smoke', path: 'src/warm.ts', prefix: 'export const answer = ', suffix: '\n' })).body
check('prewarm fills the cache', warm.warmed === true && warm.processedTokens > 0, JSON.stringify(warm))
const ne400 = await api('POST', '/v1/next-edit', { repo: 'smoke', path: 'src/a.ts', text: 'const a = 1\n', cursor: 5 })
check('next-edit refused for models not trained for it', ne400.status === 400)
if (cfg.models?.['nextedit-plumbing']) {
await api('PUT', '/v1/config', { model: 'nextedit-plumbing' })
for (let i = 0; i < 120 && !(await api('GET', '/health')).body.ok; i++) await new Promise(r => setTimeout(r, 500))
const text = 'export function add(a: number, b: number) {\n return a + b\n}\nconst x = add(1)\n'
const ne = (await api('POST', '/v1/next-edit', { repo: 'smoke', path: 'src/add.ts', text, cursor: text.indexOf('add(1)') + 5 })).body
check('next-edit endpoint returns a region decision (plumbing only)', typeof ne.noEdit === 'boolean' && typeof ne.startLine === 'number' && 'newText' in ne, JSON.stringify({ noEdit: ne.noEdit, lines: [ne.startLine, ne.endLine] }))
await api('PUT', '/v1/config', { model: 'stage-c', mode: 'simple' })
for (let i = 0; i < 120 && !(await api('GET', '/health')).body.ok; i++) await new Promise(r => setTimeout(r, 500))
}
// ---------- OpenAI-compatible endpoint (Zed)
const oai = (await api('POST', '/v1/completions', { model: 'ac:simple', prompt: "<|fim_prefix|>import { z } from 'zod'\n\nexport const configSchema = z.object({\n port: z.coerce.number().default(3000),\n})\n\nexport type Config = <|fim_suffix|>\n<|fim_middle|>", max_tokens: 64 })).body
check('OpenAI-compatible /v1/completions (Zed shim)', oai.object === 'text_completion' && String(oai.choices?.[0]?.text).includes('z.infer'), JSON.stringify(oai.choices?.[0]?.text))
// ---------- edit recording endpoint
const recorded = await api('POST', '/v1/edits', { events: [{ t: Date.now(), repo: 'smoke', path: 'src/a.ts', version: 2, changes: [{ offset: 0, length: 0, text: 'x' }], snapshot: 'x' }] })
check('edit recordings are accepted', recorded.body.recorded === 1)
@@ -113,7 +136,7 @@ check('edit recordings are accepted', recorded.body.recorded === 1)
// Optional typing replay over held-out FIM samples from the (private) training workspace.
const dataDir = path.resolve(process.env.SMOKE_DATA_DIR ?? '../../data')
const testFim = path.join(dataDir, 'fim/test-repo.jsonl')
if (fs.existsSync(testFim)) {
if (fs.existsSync(testFim) && Number(args['typing-samples']) > 0) {
const files = new Map(fs.readFileSync(path.join(dataDir, 'corpus/files.jsonl'), 'utf8').trim().split('\n').map(l => JSON.parse(l)).map(r => [r.id, r]))
const rank = (id: string) => crypto.createHash('sha256').update(`smoke:${id}`).digest('hex')
const samples = fs.readFileSync(testFim, 'utf8').trim().split('\n').map(l => JSON.parse(l))
+1 -1
View File
@@ -1,6 +1,6 @@
# Editor integration design (VS Code + Zed)
Status (2026-10-03): the daemon, VS Code extension (incl. managed mode), symbols mode, `this.` repair and the edit recorder are implemented. See the [README](../README.md). Not yet built: token healing, the Zed shim, next-edit prediction.
Status (2026-10-03): the daemon, VS Code extension (incl. managed mode), symbols mode (format 1 and 2), TS errors context, `this.` repair, token healing, next-edit plumbing (`/v1/next-edit`, decorations UI, shared prompt format in `shared/nextedit.ts`), pre-warming, the debug log and the edit recorder are implemented. See the [README](../README.md). Not yet built: the Zed shim, parallel "as if accepted" completions, recent-edits-in-other-files context.
## Goals
+64 -2
View File
@@ -3,7 +3,7 @@
"displayName": "Local Autocomplete",
"description": "Inline code completion from a local fine-tuned model via the acd daemon",
"publisher": "harvmaster",
"version": "0.5.0",
"version": "0.6.1",
"license": "MIT",
"engines": {
"vscode": "^1.90.0"
@@ -37,6 +37,26 @@
{
"command": "localAutocomplete.showLog",
"title": "Local Autocomplete: Show Log"
},
{
"command": "localAutocomplete.applyNextEdit",
"title": "Local Autocomplete: Apply Suggested Edit"
},
{
"command": "localAutocomplete.dismissNextEdit",
"title": "Local Autocomplete: Dismiss Suggested Edit"
},
{
"command": "localAutocomplete.requestNextEdit",
"title": "Local Autocomplete: Suggest Next Edit Here"
},
{
"command": "localAutocomplete.showLastPrompt",
"title": "Local Autocomplete: Show Last Prompt"
},
{
"command": "localAutocomplete.checkForUpdates",
"title": "Local Autocomplete: Check for Updates"
}
],
"configuration": {
@@ -187,9 +207,51 @@
"type": "string",
"default": "",
"description": "If set (e.g. ~/Projects/AutoComplete/daemon/recordings), edit recordings are written here directly and no daemon is needed. Otherwise they are sent to the daemon."
},
"localAutocomplete.nextEdit.enabled": {
"type": "boolean",
"default": true,
"description": "Suggest edits beyond the cursor (e.g. remove a leftover \")\", update call sites) after you pause typing. Only with models trained for next-edit."
},
"localAutocomplete.nextEdit.debounceMs": {
"type": "number",
"default": 700,
"description": "Pause after typing before asking for a next-edit suggestion."
},
"localAutocomplete.triggerOnCursorMove": {
"type": "boolean",
"default": false,
"description": "Also suggest when you move the cursor (arrow keys, mouse) and pause. With next-edit models it predicts what you came to change; otherwise only on empty lines, line ends or lines with errors."
},
"localAutocomplete.cursorMoveDelayMs": {
"type": "number",
"default": 400,
"description": "Pause after moving the cursor before triggerOnCursorMove fires."
},
"localAutocomplete.prewarmOnFocus": {
"type": "boolean",
"default": false,
"description": "Prefill the model cache when you switch to a file, so the first keystroke is fast (uses some CPU/GPU when switching files)."
},
"localAutocomplete.checkForUpdates": {
"type": "boolean",
"default": true,
"description": "Check the Gitea releases page for a newer version once a day."
}
}
}
},
"keybindings": [
{
"command": "localAutocomplete.applyNextEdit",
"key": "tab",
"when": "editorTextFocus && localAutocomplete.nextEditVisible && !inlineSuggestionVisible && !suggestWidgetVisible"
},
{
"command": "localAutocomplete.dismissNextEdit",
"key": "escape",
"when": "editorTextFocus && localAutocomplete.nextEditVisible"
}
]
},
"scripts": {
"compile": "tsc -p . && node build-daemon.mjs",
+187 -9
View File
@@ -4,7 +4,8 @@ import * as fs from 'node:fs'
import * as os from 'node:os'
import * as nodePath from 'node:path'
import { randomUUID } from 'node:crypto'
import { MODE_DESCRIPTIONS, applySelectedSuggestion, repoAndPath, sliceDocument, statusText, type ContextMode } from './logic'
import { MODE_DESCRIPTIONS, applySelectedSuggestion, insertionMakesSense, newerVersion, repoAndPath, sliceDocument, statusText, type ContextMode } from './logic'
import { NextEditController } from './nextedit-ui'
import * as managed from './managed-core'
interface DaemonConfig {
@@ -12,7 +13,7 @@ interface DaemonConfig {
model: string
mode: ContextMode
effectiveMode: ContextMode
models: Record<string, { order: string; modes: ContextMode[]; defaultMode: ContextMode }>
models: Record<string, { order: string; modes: ContextMode[]; defaultMode: ContextMode; nextEdit?: boolean }>
llama: { running: boolean; loading: boolean; error: string | null }
}
@@ -190,7 +191,35 @@ class EditRecorder {
this.timer ??= setTimeout(() => this.flush(), 2000)
}
forget(doc: vscode.TextDocument) { this.counts.delete(doc.uri.toString()); this.lastSelection.delete(doc.uri.toString()) }
/** VS Code's own diagnostics (TS, Vue, ESLint…) for recorded files: real "error appeared → fixed" labels. */
private lastDiagnostics = new Map<string, string>()
onDiagnostics(e: vscode.DiagnosticChangeEvent) {
for (const uri of e.uris) {
const doc = vscode.workspace.textDocuments.find(d => d.uri.toString() === uri.toString())
if (!doc || !this.eligible(doc)) continue
const diagnostics = vscode.languages.getDiagnostics(uri)
.filter(d => d.severity <= vscode.DiagnosticSeverity.Warning)
.map(d => ({
line: d.range.start.line, char: d.range.start.character, endLine: d.range.end.line, endChar: d.range.end.character,
severity: d.severity === vscode.DiagnosticSeverity.Error ? 'error' : 'warning',
code: typeof d.code === 'object' ? d.code.value : d.code, source: d.source, message: d.message,
}))
const key = uri.toString()
const fingerprint = JSON.stringify(diagnostics)
if (this.lastDiagnostics.get(key) === fingerprint) continue
this.lastDiagnostics.set(key, fingerprint)
const folder = vscode.workspace.getWorkspaceFolder(uri)
const { repo, path } = repoAndPath(folder?.name, vscode.workspace.asRelativePath(uri, false), doc.fileName)
this.queue.push({ kind: 'diagnostics', app: this.app, t: Date.now(), repo, path, version: doc.version, diagnostics })
this.timer ??= setTimeout(() => this.flush(), 2000)
}
}
forget(doc: vscode.TextDocument) {
this.lastDiagnostics.delete(doc.uri.toString())
this.counts.delete(doc.uri.toString())
this.lastSelection.delete(doc.uri.toString())
}
flush() {
if (this.timer) { clearTimeout(this.timer); this.timer = null }
@@ -211,6 +240,136 @@ class EditRecorder {
}
const recorder = new EditRecorder()
/** Fields every daemon request carries for a document. */
function documentBase(document: vscode.TextDocument) {
const folder = vscode.workspace.getWorkspaceFolder(document.uri)
const { repo, path } = repoAndPath(folder?.name, vscode.workspace.asRelativePath(document.uri, false), document.fileName)
return { repo, path, workspaceRoot: folder?.uri.scheme === 'file' ? folder.uri.fsPath : undefined, languageId: document.languageId }
}
function documentAllowed(document: vscode.TextDocument) {
const cfg = settings()
return cfg.get<boolean>('enableCompletions', true) && !(daemonConfig && !daemonConfig.enabled)
&& cfg.get<string[]>('languages', []).includes(document.languageId)
&& !cfg.get<string[]>('excludeGlobs', []).some(pattern => vscode.languages.match({ pattern }, document) > 0)
}
const modelSupportsNextEdit = () => !!(daemonConfig && daemonConfig.models[daemonConfig.model]?.nextEdit)
// ---------- next-edit prediction (models trained for it, e.g. Stage E)
const nextEdit = new NextEditController(s => {
api('POST', '/v1/feedback', { event: 'next-edit-accepted', lines: s.newText.split('\n').length }).catch(() => {})
})
let nextEditTimer: NodeJS.Timeout | null = null
let nextEditAbort: AbortController | null = null
async function requestNextEdit(editor: vscode.TextEditor, reason: string) {
const doc = editor.document
if (!modelSupportsNextEdit() || !settings().get<boolean>('nextEdit.enabled', true) || !documentAllowed(doc)) return
nextEditAbort?.abort()
const abort = new AbortController()
nextEditAbort = abort
const version = doc.version
try {
const res = await api<{ noEdit: boolean; startLine: number; endLine: number; oldText: string; newText: string | null }>('POST', '/v1/next-edit', {
...documentBase(doc), text: doc.getText(), cursor: doc.offsetAt(editor.selection.active), reason,
}, abort.signal)
if (abort.signal.aborted || res.noEdit || !res.newText || editor.document.version !== version) return
nextEdit.show(editor, { uri: doc.uri, version, startLine: res.startLine, endLine: res.endLine, oldText: res.oldText, newText: res.newText })
} catch (e) {
if (!abort.signal.aborted) log.appendLine(`next-edit failed: ${(e as Error).message}`)
}
}
function scheduleNextEdit(editor: vscode.TextEditor, reason: string, delay: number) {
if (nextEditTimer) clearTimeout(nextEditTimer)
nextEditTimer = setTimeout(() => void requestNextEdit(editor, reason), delay)
}
// ---------- cursor-move trigger ("why am I here?")
const lastVersions = new Map<string, number>()
let moveTimer: NodeJS.Timeout | null = null
function onCursorMoved(e: vscode.TextEditorSelectionChangeEvent) {
nextEdit.onSelectionChanged(e)
const doc = e.textEditor.document
const key = doc.uri.toString()
const editedNow = lastVersions.get(key) !== doc.version
lastVersions.set(key, doc.version)
// Only plain navigation (keyboard/mouse), not the selection change that comes with typing.
if (editedNow || e.kind === vscode.TextEditorSelectionChangeKind.Command || !settings().get<boolean>('triggerOnCursorMove', false)) return
if (!documentAllowed(doc)) return
if (moveTimer) clearTimeout(moveTimer)
moveTimer = setTimeout(() => {
if (vscode.window.activeTextEditor !== e.textEditor) return
if (modelSupportsNextEdit()) return void requestNextEdit(e.textEditor, 'cursor-move')
// Plain FIM can only insert at the cursor: only where an insertion makes sense.
const pos = e.textEditor.selection.active
const lineText = doc.lineAt(pos.line).text
const lineHasError = vscode.languages.getDiagnostics(doc.uri).some(d => d.severity === vscode.DiagnosticSeverity.Error && d.range.start.line <= pos.line && d.range.end.line >= pos.line)
if (insertionMakesSense(lineText, pos.character, lineHasError)) void vscode.commands.executeCommand('editor.action.inlineSuggest.trigger')
}, settings().get<number>('cursorMoveDelayMs', 400))
}
// ---------- prewarm on focus
function prewarm(editor: vscode.TextEditor | undefined) {
if (!editor || !settings().get<boolean>('prewarmOnFocus', false) || !documentAllowed(editor.document)) return
const doc = editor.document
const cfg = settings()
const slice = sliceDocument(doc.getText(), doc.offsetAt(editor.selection.active), cfg.get<number>('maxPrefixChars', 200000), cfg.get<number>('maxSuffixChars', 20000))
api('POST', '/v1/prewarm', { ...documentBase(doc), ...slice }).catch(() => {})
}
// ---------- show last prompt
async function showLastPrompt() {
const entries = await api<Record<string, unknown>[]>('GET', '/v1/debug/last?n=1').catch(e => { void vscode.window.showErrorMessage(`Local Autocomplete: ${(e as Error).message}`); return null })
if (!entries) return
if (!entries.length) return void vscode.window.showInformationMessage('Local Autocomplete: no requests yet')
const e = entries[0]
const content = [
`# Last ${e.kind} request (${e.t})`, '', `model \`${e.model}\`${e.mode ? `, mode \`${e.mode}\`` : ''}, file \`${e.path}\``, '',
'## Timings', '```json', JSON.stringify(e.timings, null, 2), '```', '',
'## Prompt (decoded, special tokens shown)', '```', String(e.prompt), '```', '',
'## Raw model output', '```', String(e.raw), '```', '',
...(e.text !== undefined ? ['## Shown to you (after post-processing)', '```', String(e.text), '```'] : []),
].join('\n')
const doc = await vscode.workspace.openTextDocument({ language: 'markdown', content })
await vscode.window.showTextDocument(doc, { preview: true })
}
// ---------- update check (Gitea releases)
async function checkForUpdates(manual = false) {
const current = extensionCtx.extension.packageJSON.version as string
const last = extensionCtx.globalState.get<number>('lastUpdateCheck', 0)
if (!manual && (!settings().get<boolean>('checkForUpdates', true) || Date.now() - last < 24 * 3600_000)) return
await extensionCtx.globalState.update('lastUpdateCheck', Date.now())
try {
const res = await fetch('https://git.harvmaster.com/api/v1/repos/Harvmaster/autocomplete-model/releases?limit=20', { signal: AbortSignal.timeout(10000) })
const releases = await res.json() as { tag_name: string; html_url: string; assets: { name: string; browser_download_url: string }[] }[]
const latest = releases
.map(r => ({ r, v: /^vscode-v(\d+\.\d+\.\d+)$/.exec(r.tag_name)?.[1] }))
.filter((x): x is { r: typeof releases[number]; v: string } => !!x.v)
.sort((a, b) => (newerVersion(a.v, b.v) ? -1 : 1))[0]
if (!latest || !newerVersion(latest.v, current)) {
if (manual) void vscode.window.showInformationMessage(`Local Autocomplete ${current} is up to date`)
return
}
const asset = latest.r.assets.find(a => a.name.endsWith('.vsix'))
const pick = await vscode.window.showInformationMessage(`Local Autocomplete ${latest.v} is available (you have ${current}).`, ...(asset ? ['Install'] : []), 'Release notes')
if (pick === 'Release notes') void vscode.env.openExternal(vscode.Uri.parse(latest.r.html_url))
if (pick === 'Install' && asset) {
const file = nodePath.join(extensionCtx.globalStorageUri.fsPath, asset.name)
fs.mkdirSync(nodePath.dirname(file), { recursive: true })
fs.writeFileSync(file, Buffer.from(await (await fetch(asset.browser_download_url)).arrayBuffer()))
await vscode.commands.executeCommand('workbench.extensions.installExtension', vscode.Uri.file(file))
const reload = await vscode.window.showInformationMessage(`Local Autocomplete ${latest.v} installed.`, 'Reload Window')
if (reload) void vscode.commands.executeCommand('workbench.action.reloadWindow')
}
} catch (e) {
log.appendLine(`update check failed: ${(e as Error).message}`)
if (manual) void vscode.window.showErrorMessage(`Local Autocomplete: update check failed (${(e as Error).message})`)
}
}
class Provider implements vscode.InlineCompletionItemProvider {
async provideInlineCompletionItems(document: vscode.TextDocument, position: vscode.Position, context: vscode.InlineCompletionContext, token: vscode.CancellationToken) {
const cfg = settings()
@@ -235,14 +394,12 @@ class Provider implements vscode.InlineCompletionItemProvider {
if (debounce > 0) await new Promise(r => setTimeout(r, debounce))
if (token.isCancellationRequested) return undefined
const folder = vscode.workspace.getWorkspaceFolder(document.uri)
const { repo, path } = repoAndPath(folder?.name, vscode.workspace.asRelativePath(document.uri, false), document.fileName)
const { repo, path, workspaceRoot } = documentBase(document)
const slice = sliceDocument(text, cursor, cfg.get<number>('maxPrefixChars', 200000), cfg.get<number>('maxSuffixChars', 20000))
const requestId = randomUUID()
const abort = new AbortController()
const sub = token.onCancellationRequested(() => abort.abort())
try {
const workspaceRoot = folder?.uri.scheme === 'file' ? folder.uri.fsPath : undefined
const res = await api<{ text: string; cancelled?: boolean; replaceBefore?: number; filterText?: string; timings?: { totalMs: number } }>('POST', '/v1/fim', {
requestId, repo, path, workspaceRoot, languageId: document.languageId, ...slice,
}, abort.signal)
@@ -290,12 +447,16 @@ async function showMenu() {
{ label: '$(file) Open daemon log', action: 'daemonLog' },
] : []),
{ label: '$(output) Show log', action: 'log' },
{ label: '$(debug-console) Show last prompt', action: 'lastPrompt' },
{ label: '$(cloud-download) Check for updates', action: 'updates' },
]
const pick = await vscode.window.showQuickPick(items, { title: 'Local Autocomplete' })
if (pick?.action === 'mode') await selectMode()
if (pick?.action === 'model') await selectModel()
if (pick?.action === 'toggle') await toggleEnabled()
if (pick?.action === 'log') log.show()
if (pick?.action === 'lastPrompt') await showLastPrompt()
if (pick?.action === 'updates') await checkForUpdates(true)
if (pick?.action === 'restart') {
await api('POST', '/v1/shutdown').catch(() => {})
await new Promise(r => setTimeout(r, 1000))
@@ -381,9 +542,24 @@ export async function activate(ctx: vscode.ExtensionContext) {
if (isManaged() && managedKeys.some(k => e.affectsConfiguration(`localAutocomplete.${k}`))) void startManaged()
else void refreshConfig()
}),
vscode.workspace.onDidChangeTextDocument(e => recorder.onChange(e)),
vscode.window.onDidChangeTextEditorSelection(e => recorder.onSelection(e)),
vscode.window.onDidChangeActiveTextEditor(e => recorder.onActiveEditor(e)),
vscode.workspace.onDidChangeTextDocument(e => {
recorder.onChange(e)
nextEdit.onDocumentChanged(e)
// After a pause in typing, ask whether the edit implies a follow-up elsewhere in the region.
const editor = vscode.window.activeTextEditor
if (editor && editor.document === e.document && e.contentChanges.length && modelSupportsNextEdit()) {
scheduleNextEdit(editor, 'edit', settings().get<number>('nextEdit.debounceMs', 700))
}
}),
vscode.window.onDidChangeTextEditorSelection(e => { recorder.onSelection(e); onCursorMoved(e) }),
vscode.window.onDidChangeActiveTextEditor(e => { recorder.onActiveEditor(e); prewarm(e) }),
vscode.languages.onDidChangeDiagnostics(e => recorder.onDiagnostics(e)),
nextEdit,
vscode.commands.registerCommand('localAutocomplete.applyNextEdit', () => nextEdit.apply()),
vscode.commands.registerCommand('localAutocomplete.dismissNextEdit', () => nextEdit.clear()),
vscode.commands.registerCommand('localAutocomplete.requestNextEdit', () => { const ed = vscode.window.activeTextEditor; if (ed) void requestNextEdit(ed, 'manual') }),
vscode.commands.registerCommand('localAutocomplete.showLastPrompt', showLastPrompt),
vscode.commands.registerCommand('localAutocomplete.checkForUpdates', () => checkForUpdates(true)),
vscode.workspace.onDidCloseTextDocument(doc => recorder.forget(doc)),
{ dispose: () => recorder.flush() },
vscode.window.onDidChangeWindowState(s => { if (s.focused) void refreshConfig() }),
@@ -403,8 +579,10 @@ export async function activate(ctx: vscode.ExtensionContext) {
status.text = cfg.get<boolean>('recordEdits', false) ? '$(record) AC: recording only' : '$(circle-slash) AC: completions off'
if (cfg.get<string>('recordingDir', '')) return
}
void checkForUpdates()
await ensureDaemon()
await refreshConfig()
prewarm(vscode.window.activeTextEditor)
}
export function deactivate() {}
+36
View File
@@ -63,3 +63,39 @@ export function statusText(cfg: { enabled: boolean; mode: ContextMode; effective
const mode = cfg.effectiveMode && cfg.effectiveMode !== cfg.mode ? `${cfg.mode}→${cfg.effectiveMode}` : cfg.mode
return `$(sparkle) AC: ${mode}${lastMs !== null ? ` · ${Math.round(lastMs)} ms` : ''}`
}
/** Line-block diff of a region: the first/last changed lines (old and new) after trimming common lines. */
export function lineBlockDiff(oldText: string, newText: string): { oldStart: number; oldEnd: number; newLines: string[]; oldLines: string[] } | null {
const a = oldText.split('\n'), b = newText.split('\n')
let p = 0
while (p < a.length && p < b.length && a[p] === b[p]) p++
if (p === a.length && p === b.length) return null
let s = 0
while (s < a.length - p && s < b.length - p && a[a.length - 1 - s] === b[b.length - 1 - s]) s++
return { oldStart: p, oldEnd: a.length - s, oldLines: a.slice(p, a.length - s), newLines: b.slice(p, b.length - s) }
}
/** Character diff of one line: deleted [start, end) in the old line and the text inserted there. */
export function charDiff(oldLine: string, newLine: string): { start: number; deleteEnd: number; insert: string } {
let p = 0
while (p < oldLine.length && p < newLine.length && oldLine[p] === newLine[p]) p++
let s = 0
while (s < oldLine.length - p && s < newLine.length - p && oldLine[oldLine.length - 1 - s] === newLine[newLine.length - 1 - s]) s++
return { start: p, deleteEnd: oldLine.length - s, insert: newLine.slice(p, newLine.length - s) }
}
/** Where cursor-move triggers make sense for plain FIM: empty line, end of line, or on a line with an error. */
export function insertionMakesSense(lineText: string, column: number, lineHasError: boolean): boolean {
if (!lineText.trim()) return true
if (!lineText.slice(column).trim()) return true
return lineHasError
}
/** Semver-ish comparison of "a.b.c" strings. */
export function newerVersion(candidate: string, current: string): boolean {
const pa = candidate.split('.').map(Number), pb = current.split('.').map(Number)
for (let i = 0; i < Math.max(pa.length, pb.length); i++) {
if ((pa[i] ?? 0) !== (pb[i] ?? 0)) return (pa[i] ?? 0) > (pb[i] ?? 0)
}
return false
}
+24 -7
View File
@@ -26,8 +26,25 @@ export interface ModelEntry {
modes: ('symbols' | 'simple' | 'raw')[]
defaultMode: 'symbols' | 'simple' | 'raw'
description?: string
/** Prompt-format flags the model was trained with (passed through to the daemon's model profile). */
contextFormat?: 1 | 2
errors?: boolean
members?: boolean
/** Old (≤ 0.6.0) flag: ignored, because those daemons applied an empty "no edit" output as a deletion. */
nextEdit?: boolean
/** Trained for next-edit (shared/nextedit.ts format version); read since 0.6.1. */
nextEditFormat?: 1
/** Companion model (another manifest entry) that serves next-edit for this one; downloaded alongside it. */
nextEditModel?: string
}
const profileOf = (m: ModelEntry, available: Set<string>) => ({
order: m.order, modes: m.modes, defaultMode: m.defaultMode,
...(m.contextFormat ? { contextFormat: m.contextFormat } : {}), ...(m.errors ? { errors: true } : {}),
...(m.members !== undefined ? { members: m.members } : {}), ...(m.nextEditFormat ? { nextEdit: true } : {}),
...(m.nextEditModel && available.has(m.nextEditModel) ? { nextEditModel: m.nextEditModel } : {}),
})
export interface Manifest {
schema: 1
default: string
@@ -197,7 +214,7 @@ export interface Prepared { configPath: string; instanceTag: string }
/** Resolves model + llama-server (downloading as needed) and writes the daemon config. */
export async function prepare(storage: string, daemonDir: string, extensionVersion: string, s: ManagedSettings, progress: Progress, signal?: AbortSignal): Promise<Prepared> {
const models: Record<string, { path: string } & Pick<ModelEntry, 'order' | 'modes' | 'defaultMode'>> = {}
const models: Record<string, { path: string } & Partial<ModelEntry>> = {}
let defaultModel: string
let llamaCpp = DEFAULT_LLAMA_CPP
@@ -218,12 +235,12 @@ export async function prepare(storage: string, daemonDir: string, extensionVersi
if (!entry) throw new Error(`model "${s.model || manifest.default}" is not in the manifest (available: ${manifest.models.map(m => m.name).join(', ')})`)
const file = await ensureFile(entry.url, path.join(storage, 'models', entry.file), entry.sha256, `model ${entry.name}`, progress, signal)
defaultModel = entry.name
models[entry.name] = { path: file, order: entry.order, modes: entry.modes, defaultMode: entry.defaultMode }
// Other models already on disk stay switchable from the menu.
for (const m of manifest.models) {
const f = path.join(storage, 'models', m.file)
if (m.name !== entry.name && fs.existsSync(`${f}.verified`)) models[m.name] = { path: f, order: m.order, modes: m.modes, defaultMode: m.defaultMode }
}
const companion = entry.nextEditModel ? manifest.models.find(m => m.name === entry.nextEditModel && m.nextEditFormat) : undefined
if (companion) await ensureFile(companion.url, path.join(storage, 'models', companion.file), companion.sha256, `next-edit model ${companion.name}`, progress, signal)
// The selected model, its companion and other models already on disk (switchable from the menu).
const onDisk = manifest.models.filter(m => m.name === entry.name || fs.existsSync(`${path.join(storage, 'models', m.file)}.verified`))
const available = new Set(onDisk.filter(m => m.nextEditFormat).map(m => m.name))
for (const m of onDisk) models[m.name] = { path: path.join(storage, 'models', m.file), ...profileOf(m, available) }
}
const binary = s.llamaServerPath ? s.llamaServerPath.replace(/^~(?=$|\/)/, os.homedir()) : await ensureLlamaServer(storage, llamaCpp, progress, signal)
+121
View File
@@ -0,0 +1,121 @@
import * as vscode from 'vscode'
import { charDiff, lineBlockDiff } from './logic'
export interface NextEditSuggestion {
uri: vscode.Uri
version: number
startLine: number
endLine: number
oldText: string
newText: string
}
/**
* Shows a predicted region rewrite as decorations (VS Code's stable inline-completion API can't show deletions):
* - one changed line: deleted characters struck through in place, inserted text as ghost text at its position
* - several lines: changed lines highlighted, "Tab to apply" hint, full diff on hover
* Tab applies (only while visible and no other inline suggestion / suggest widget is showing), Escape dismisses.
*/
export class NextEditController implements vscode.Disposable {
private current: NextEditSuggestion | null = null
private readonly deleted = vscode.window.createTextEditorDecorationType({
textDecoration: 'line-through',
backgroundColor: new vscode.ThemeColor('diffEditor.removedTextBackground'),
})
private readonly inserted = vscode.window.createTextEditorDecorationType({})
private readonly block = vscode.window.createTextEditorDecorationType({
isWholeLine: true,
backgroundColor: new vscode.ThemeColor('diffEditor.removedLineBackground'),
})
private readonly onAccepted: (s: NextEditSuggestion) => void
constructor(onAccepted: (s: NextEditSuggestion) => void) {
this.onAccepted = onAccepted
}
get visible() { return this.current !== null }
show(editor: vscode.TextEditor, s: NextEditSuggestion) {
this.clear()
const doc = editor.document
// Only show if the region still holds exactly what the model saw.
const range = new vscode.Range(s.startLine, 0, Math.min(s.endLine, doc.lineCount), 0)
if (doc.version !== s.version || doc.getText(range) !== s.oldText) return
const diff = lineBlockDiff(s.oldText, s.newText)
if (!diff) return
this.current = s
const first = s.startLine + diff.oldStart
if (diff.oldLines.length === 1 && diff.newLines.length === 1) {
const d = charDiff(diff.oldLines[0], diff.newLines[0])
if (d.deleteEnd > d.start) editor.setDecorations(this.deleted, [new vscode.Range(first, d.start, first, d.deleteEnd)])
if (d.insert) {
editor.setDecorations(this.inserted, [{
range: new vscode.Range(first, d.deleteEnd, first, d.deleteEnd),
renderOptions: { before: { contentText: d.insert, color: new vscode.ThemeColor('editorGhostText.foreground'), fontStyle: 'italic', backgroundColor: new vscode.ThemeColor('diffEditor.insertedTextBackground') } },
}])
}
} else {
const hover = new vscode.MarkdownString(['**Suggested edit** (Tab to apply, Esc to dismiss)', '```diff', ...diff.oldLines.map(l => `-${l}`), ...diff.newLines.map(l => `+${l}`), '```'].join('\n'))
const lines = Math.max(1, diff.oldLines.length)
editor.setDecorations(this.block, Array.from({ length: lines }, (_, i) => ({ range: new vscode.Range(first + i, 0, first + i, 0), hoverMessage: hover })))
editor.setDecorations(this.inserted, [{
range: new vscode.Range(first, Number.MAX_SAFE_INTEGER, first, Number.MAX_SAFE_INTEGER),
hoverMessage: hover,
renderOptions: { after: { contentText: ` ⇥ Tab: suggested edit (−${diff.oldLines.length} +${diff.newLines.length} lines)`, color: new vscode.ThemeColor('editorGhostText.foreground'), fontStyle: 'italic' } },
}])
}
void vscode.commands.executeCommand('setContext', 'localAutocomplete.nextEditVisible', true)
}
async apply(): Promise<boolean> {
const s = this.current
const editor = vscode.window.activeTextEditor
this.clear()
if (!s || !editor || editor.document.uri.toString() !== s.uri.toString()) return false
const doc = editor.document
const range = new vscode.Range(s.startLine, 0, Math.min(s.endLine, doc.lineCount), 0)
if (doc.getText(range) !== s.oldText) return false
const ok = await editor.edit(b => b.replace(range, s.newText))
if (ok) {
const diff = lineBlockDiff(s.oldText, s.newText)
if (diff) {
const line = s.startLine + diff.oldStart + Math.max(0, diff.newLines.length - 1)
const pos = new vscode.Position(line, doc.lineAt(Math.min(line, doc.lineCount - 1)).text.length)
editor.selection = new vscode.Selection(pos, pos)
}
this.onAccepted(s)
}
return ok
}
/** Dismiss when the document changes or the cursor leaves the region. */
onSelectionChanged(e: vscode.TextEditorSelectionChangeEvent) {
const s = this.current
if (!s || e.textEditor.document.uri.toString() !== s.uri.toString()) return
const line = e.selections[0].active.line
if (line < s.startLine - 2 || line > s.endLine + 2) this.clear()
}
onDocumentChanged(e: vscode.TextDocumentChangeEvent) {
if (this.current && e.document.uri.toString() === this.current.uri.toString() && e.contentChanges.length) this.clear()
}
clear() {
if (!this.current) return
this.current = null
for (const editor of vscode.window.visibleTextEditors) {
editor.setDecorations(this.deleted, [])
editor.setDecorations(this.inserted, [])
editor.setDecorations(this.block, [])
}
void vscode.commands.executeCommand('setContext', 'localAutocomplete.nextEditVisible', false)
}
dispose() {
this.clear()
this.deleted.dispose()
this.inserted.dispose()
this.block.dispose()
}
}
+25
View File
@@ -36,3 +36,28 @@ test('status text', () => {
assert.match(statusText({ enabled: true, mode: 'symbols', effectiveMode: 'simple', model: 'stage-c' }, 41.6), /symbols→simple · 42 ms/)
assert.match(statusText(null, null), /no daemon/)
})
import { lineBlockDiff, charDiff, insertionMakesSense, newerVersion } from '../dist/logic.js'
test('leftover paren is a one-character deletion', () => {
const d = charDiff('const v = fromString(someString))', 'const v = fromString(someString)')
assert.deepEqual(d, { start: 32, deleteEnd: 33, insert: '' })
})
test('line block diff finds the changed lines', () => {
const d = lineBlockDiff('a\nb\nc\nd\n', 'a\nB\nB2\nd\n')
assert.deepEqual(d, { oldStart: 1, oldEnd: 3, oldLines: ['b', 'c'], newLines: ['B', 'B2'] })
assert.equal(lineBlockDiff('x\n', 'x\n'), null)
})
test('cursor-move gate', () => {
assert.equal(insertionMakesSense(' ', 4, false), true)
assert.equal(insertionMakesSense(' foo()', 7, false), true)
assert.equal(insertionMakesSense(' foo()', 3, false), false)
assert.equal(insertionMakesSense(' foo()', 3, true), true)
})
test('version comparison', () => {
assert.equal(newerVersion('0.10.0', '0.9.3'), true)
assert.equal(newerVersion('0.5.0', '0.5.0'), false)
})
+12 -3
View File
@@ -36,20 +36,29 @@ const step = (name, fn) => fn().then(r => { console.log(`PASS ${name}`); return
// 1. Simulate an interrupted model download: first 100 MB already on disk.
fs.mkdirSync(path.join(storage, 'models'), { recursive: true })
const modelSrc = path.join(releaseDir, 'stage-c-q8_0.gguf')
const modelSrc = path.join(releaseDir, 'stage-d-q8_0.gguf')
const fd = fs.openSync(modelSrc, 'r'); const buf = Buffer.alloc(100e6); fs.readSync(fd, buf, 0, buf.length, 0); fs.closeSync(fd)
fs.writeFileSync(path.join(storage, 'models', 'stage-c-q8_0.gguf.part'), buf)
fs.writeFileSync(path.join(storage, 'models', 'stage-d-q8_0.gguf.part'), buf)
let t0 = Date.now()
const prepared = await step('first run: manifest, resumed + verified model download, llama.cpp download + unpack', () => core.prepare(storage, daemonDir, '0.4.0-test', settings, progress))
console.log(` ${((Date.now() - t0) / 1000).toFixed(1)} s; range requests: ${rangeRequests}; model verified: ${fs.existsSync(path.join(storage, 'models', 'stage-c-q8_0.gguf.verified'))}`)
console.log(` ${((Date.now() - t0) / 1000).toFixed(1)} s; range requests: ${rangeRequests}; model verified: ${fs.existsSync(path.join(storage, 'models', 'stage-d-q8_0.gguf.verified'))}`)
assert.ok(rangeRequests >= 1, 'download should have resumed with a Range request')
const cfg = JSON.parse(fs.readFileSync(prepared.configPath, 'utf8'))
assert.ok(fs.existsSync(cfg.llamaServer.binary), 'llama-server unpacked')
await step('next-edit companion downloaded and wired (stage-d → stage-e)', async () => {
assert.equal(cfg.defaultModel, 'stage-d')
assert.equal(cfg.models['stage-d'].nextEditModel, 'stage-e')
assert.equal(cfg.models['stage-e'].nextEdit, true)
assert.ok(fs.existsSync(path.join(storage, 'models', 'stage-e-q8_0.gguf.verified')))
})
await step('daemon starts on the editor runtime', async () => assert.equal(await core.ensureRunning(storage, daemonDir, prepared, settings.port, progress), 'started'))
const fim = await (await fetch('http://127.0.0.1:7111/v1/fim', { method: 'POST', body: JSON.stringify({ repo: 'e2e', path: 'src/a.ts', prefix: "import { z } from 'zod'\nexport const s = z.object({ a: z.string() })\nexport type S = ", suffix: '\n' }) })).json()
await step(`completion through the managed daemon (${JSON.stringify(fim.text)})`, async () => assert.match(fim.text, /z\.infer<typeof s>/))
const neText = "export function f(s: string) {\n const t = s.trim()\n return parse(t))\n}\n"
const ne = await (await fetch('http://127.0.0.1:7111/v1/next-edit', { method: 'POST', body: JSON.stringify({ repo: 'e2e', path: 'src/f.ts', text: neText, cursor: neText.indexOf('(t)') + 1 }) })).json()
await step(`next-edit through the companion model (${JSON.stringify(ne.newText ?? ne.error ?? ne)})`, async () => assert.ok(ne.newText?.includes('return parse(t)\n')))
t0 = Date.now()
const again = await step('second window: no re-download, daemon reused', async () => {
+32
View File
@@ -0,0 +1,32 @@
# Zed
Zed can't load third-party edit-prediction extensions, but its built-in `open_ai_compatible_api` provider can call
the daemon's OpenAI-compatible endpoint. The daemon parses Zed's Qwen-format FIM prompt back into prefix/suffix and
rebuilds it in the layout our models were trained on (suffix-first, repo header, symbols, token healing).
1. Run the daemon standalone (see [daemon/README.md](../../daemon/README.md)). Add your project folders to
`"workspaceRoots"` in its config, so symbols mode and the `this.` repair can find the files. Zed may not send
the file path; the daemon then locates the file by content.
2. In Zed's `settings.json`:
```jsonc
{
"edit_predictions": {
"provider": "open_ai_compatible_api",
"open_ai_compatible_api": {
"api_url": "http://127.0.0.1:7071/v1/completions",
"model": "ac:symbols", // the context-mode toggle: "ac:symbols" | "ac:simple"
"prompt_format": "qwen"
}
}
}
```
The mode can also be switched with `node daemon/src/acctl.ts mode simple`.
**Status: untested against real Zed traffic.** The parser accepts prefix-first and suffix-first Qwen prompts, with
or without `<|repo_name|>`/`<|file_sep|>` headers, and is covered by the daemon smoke test with a synthetic request.
Still to check with a logging proxy:
- whether Zed sends the file path
- its prefix/suffix truncation
- whether it streams and cancels superseded requests
+2 -1
View File
@@ -2,6 +2,7 @@ These model files are modified versions of Qwen2.5-Coder-0.5B (https://huggingfa
Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE).
Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion
(and, for stage-c, a repository/symbol-context header), followed by Q8_0 GGUF conversion.
(and, from stage-c on, a repository/symbol-context header; stage-d adds TypeScript errors; stage-e adds
next-edit prediction), followed by Q8_0 GGUF conversion.
Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and
the publisher's own code. Secrets were scrubbed from the training data.
+44
View File
@@ -5,6 +5,8 @@ Q8_0 GGUF fill-in-the-middle models for the [Local Autocomplete](../extensions/v
| model | description | prompt order | context modes | size |
|---|---|---|---|---|
| `stage-d` | Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions | spm | symbols, simple | 531 MB |
| `stage-e` | Next-edit prediction (rewrites a region around the cursor or answers "no edit"); also does FIM, slightly below stage-d | spm | symbols, simple | 531 MB |
| `stage-c` | Suffix-first FIM with repo/symbol-context headers (default) | spm | symbols, simple | 531 MB |
| `stage-b` | Suffix-first FIM without context headers (previous stage) | spm | raw | 531 MB |
@@ -21,6 +23,16 @@ runs it locally with llama.cpp. The files are stored with Git LFS; download link
| Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 |
| `stage-b` | 0.391 | 0.689 | 0.952 |
| `stage-c` (with symbol context) | 0.397 | 0.681 | 0.954 |
| `stage-c` + token healing (daemon default) | 0.405 | 0.689 | |
| `stage-d` (symbols + TS errors, healed) | **0.426** | **0.700** | 0.954 |
| `stage-e` (symbols + TS errors, healed) | 0.412 | 0.691 | 0.962 |
New behaviours (886 further held-out samples), exact match, stage-c → stage-d: closing a block 0.27 → **0.79**,
next line 0.01 → **0.18**, rest of block 0.08 → 0.11, member access 0.63 → 0.66.
**Next-edit** (`stage-e`, 445 held-out samples): 80% exact region rewrites (edit similarity 0.99) where an edit
was due, and "no edit" on 100% of samples where none was. `stage-d` uses `stage-e` as its next-edit companion
(`nextEditModel` in the manifest): the extension downloads both and the daemon runs it in a second llama-server.
The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts
let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a
@@ -37,6 +49,38 @@ Suffix-first (`spm`) order:
`stage-c` additionally expects a repo-level header: `<|repo_name|>{repo}\n<|file_sep|>{path}\n`, optionally with a
`__symbols__.d.ts` pseudo-file of available symbols before it.
## Training data
Fine-tuned on permissively licensed open-source repositories (below, pinned commits) and on the publisher's own
code (not published). Secrets were scrubbed from all training data; held-out evaluation repos were never trained on.
| repository | commit | license |
|---|---|---|
| [github.com/WiseLibs/better-sqlite3](https://github.com/WiseLibs/better-sqlite3) | `f8e2d54120` | MIT |
| [github.com/CashScript/cashscript](https://github.com/CashScript/cashscript) | `71a243b656` | MIT |
| [gitlab.com/electrum-cash/network](https://gitlab.com/electrum-cash/network) | `c7511725a9` | MIT |
| [github.com/element-plus/element-plus](https://github.com/element-plus/element-plus) | `ad514f79ba` | MIT |
| [github.com/fastify/fastify](https://github.com/fastify/fastify) | `19d5be0daf` | MIT |
| [github.com/h3js/h3](https://github.com/h3js/h3) | `3afe46b170` | MIT |
| [github.com/kysely-org/kysely](https://github.com/kysely-org/kysely) | `4b7287e71b` | MIT |
| [github.com/bitauth/libauth](https://github.com/bitauth/libauth) | `60aec239cc` | MIT |
| [github.com/nuxt/ui](https://github.com/nuxt/ui) | `e8dd339dfd` | MIT |
| [github.com/unjs/ofetch](https://github.com/unjs/ofetch) | `1dbc37fd1c` | MIT |
| [github.com/vuejs/pinia](https://github.com/vuejs/pinia) | `98587ca465` | MIT |
| [github.com/quasarframework/quasar](https://github.com/quasarframework/quasar) | `0fd7ce30e7` | MIT |
| [github.com/unovue/reka-ui](https://github.com/unovue/reka-ui) | `e9230140ed` | MIT |
| [github.com/unovue/shadcn-vue](https://github.com/unovue/shadcn-vue) | `f12a6a098d` | MIT |
| [github.com/vitejs/vite](https://github.com/vitejs/vite) | `10033218d2` | MIT |
| [github.com/vuejs/vitepress](https://github.com/vuejs/vitepress) | `56fa6c9df1` | MIT |
| [github.com/vitest-dev/vitest](https://github.com/vitest-dev/vitest) | `964e1a4404` | MIT |
| [github.com/vuejs/core](https://github.com/vuejs/core) | `4ab865a848` | MIT |
| [github.com/intlify/vue-i18n](https://github.com/intlify/vue-i18n) | `6ef45366e1` | MIT |
| [github.com/vuejs/router](https://github.com/vuejs/router) | `07663eb6be` | MIT |
| [github.com/vbenjs/vue-vben-admin](https://github.com/vbenjs/vue-vben-admin) | `50f4ede309` | MIT |
| [github.com/vueuse/vueuse](https://github.com/vueuse/vueuse) | `efdd69a148` | MIT |
| [github.com/websockets/ws](https://github.com/websockets/ws) | `297202cdae` | MIT |
| [github.com/colinhacks/zod](https://github.com/colinhacks/zod) | `0b216ef674` | MIT |
## License
Apache-2.0 ([LICENSE](LICENSE), [NOTICE](NOTICE)), as a derivative of Qwen2.5-Coder.
+37 -1
View File
@@ -1,7 +1,43 @@
{
"schema": 1,
"default": "stage-c",
"default": "stage-d",
"models": [
{
"name": "stage-d",
"file": "stage-d-q8_0.gguf",
"url": "stage-d-q8_0.gguf",
"sha256": "8edaef02ae9e794929af0267f0f47f600824454b08df7d386e386eec1123c839",
"size": 531067552,
"order": "spm",
"modes": [
"symbols",
"simple"
],
"defaultMode": "symbols",
"contextFormat": 2,
"errors": true,
"members": true,
"nextEditModel": "stage-e",
"description": "Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions"
},
{
"name": "stage-e",
"file": "stage-e-q8_0.gguf",
"url": "stage-e-q8_0.gguf",
"sha256": "44d5fe87043f2fd3f8fba39c07a653ec7bdc228f83a2da4a5638be3ab77a962f",
"size": 531067552,
"order": "spm",
"modes": [
"symbols",
"simple"
],
"defaultMode": "symbols",
"contextFormat": 2,
"errors": true,
"members": true,
"nextEditFormat": 1,
"description": "Next-edit prediction (rewrites a region around the cursor or answers \"no edit\"); also does FIM, slightly below stage-d"
},
{
"name": "stage-c",
"file": "stage-c-q8_0.gguf",
Binary file not shown.
Binary file not shown.
+33 -2
View File
@@ -13,11 +13,20 @@ import crypto from 'node:crypto'
import { parseArgs } from 'node:util'
const repo = path.resolve(path.dirname(new URL(import.meta.url).pathname), '..')
const { values, positionals } = parseArgs({ options: { from: { type: 'string', default: path.join(repo, '..', 'models') } }, allowPositionals: true })
const { values, positionals } = parseArgs({ options: {
from: { type: 'string', default: path.join(repo, '..', 'models') },
// Lock file of the public fine-tuning corpus (training workspace), for the attribution table.
sources: { type: 'string', default: path.join(repo, '..', 'pipeline', 'config', 'public-sources.lock.json') },
}, allowPositionals: true })
const out = path.join(repo, 'models')
// Prompt layout each model was trained with (the extension only offers matching context modes).
const profiles = {
// Stage E: Stage D + next-edit prediction (shared/nextedit.ts)
// nextEditFormat (not the old nextEdit flag): extensions ≤ 0.6.0 must not use it (they applied "no edit" as a deletion).
'stage-e': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEditFormat: 1, description: 'Next-edit prediction (rewrites a region around the cursor or answers "no edit"); also does FIM, slightly below stage-d' },
// Stage D: next-line / block-close / multi-line fixes, namespace-import symbols (format 2), TS errors in the header
'stage-d': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'symbols', contextFormat: 2, errors: true, members: true, nextEditModel: 'stage-e', description: 'Suffix-first FIM with symbols (incl. namespace imports) + TS errors; better next-line, block closing and multi-line completions' },
'stage-c': { order: 'spm', modes: ['symbols', 'simple'], defaultMode: 'simple', description: 'Suffix-first FIM with repo/symbol-context headers (default)' },
'stage-b': { order: 'spm', modes: ['raw'], defaultMode: 'raw', description: 'Suffix-first FIM without context headers (previous stage)' },
}
@@ -63,11 +72,16 @@ fs.writeFileSync(path.join(out, 'NOTICE'), `These model files are modified versi
Copyright (c) Alibaba Cloud, licensed under the Apache License, Version 2.0 (see LICENSE).
Modifications: full-parameter fine-tuning for suffix-first fill-in-the-middle completion
(and, for stage-c, a repository/symbol-context header), followed by Q8_0 GGUF conversion.
(and, from stage-c on, a repository/symbol-context header; stage-d adds TypeScript errors; stage-e adds
next-edit prediction), followed by Q8_0 GGUF conversion.
Fine-tuning data: permissively licensed (MIT) open-source TypeScript/Vue repositories and
the publisher's own code. Secrets were scrubbed from the training data.
`)
const sources = fs.existsSync(values.sources) ? JSON.parse(fs.readFileSync(values.sources, 'utf8')) : []
const sourcesTable = sources.length
? ['| repository | commit | license |', '|---|---|---|', ...sources.map(src => `| [${src.url.replace(/^https:\/\/|\.git$/g, '')}](${src.url.replace(/\.git$/, '')}) | \`${src.commit.slice(0, 10)}\` | ${src.license} |`)].join('\n')
: '(source list not available when this file was generated)'
const row = m => `| \`${m.name}\` | ${m.description} | ${m.order} | ${m.modes.join(', ')} | ${(m.size / 1e6).toFixed(0)} MB |`
fs.writeFileSync(path.join(out, 'README.md'), `# Models
@@ -91,6 +105,16 @@ runs it locally with llama.cpp. The files are stored with Git LFS; download link
| Qwen2.5-Coder-1.5B (base) | 0.366 | 0.623 | 0.882 |
| \`stage-b\` | 0.391 | 0.689 | 0.952 |
| \`stage-c\` (with symbol context) | 0.397 | 0.681 | 0.954 |
| \`stage-c\` + token healing (daemon default) | 0.405 | 0.689 | |
| \`stage-d\` (symbols + TS errors, healed) | **0.426** | **0.700** | 0.954 |
| \`stage-e\` (symbols + TS errors, healed) | 0.412 | 0.691 | 0.962 |
New behaviours (886 further held-out samples), exact match, stage-c → stage-d: closing a block 0.27 → **0.79**,
next line 0.01 → **0.18**, rest of block 0.08 → 0.11, member access 0.63 → 0.66.
**Next-edit** (\`stage-e\`, 445 held-out samples): 80% exact region rewrites (edit similarity 0.99) where an edit
was due, and "no edit" on 100% of samples where none was. \`stage-d\` uses \`stage-e\` as its next-edit companion
(\`nextEditModel\` in the manifest): the extension downloads both and the daemon runs it in a second llama-server.
The base models were evaluated in their native prefix-first order (they can't do suffix-first). Suffix-first prompts
let the editor reuse the KV cache while typing: about 2 tokens re-processed per keystroke, ~90–100 ms end-to-end on a
@@ -107,6 +131,13 @@ Suffix-first (\`spm\`) order:
\`stage-c\` additionally expects a repo-level header: \`<|repo_name|>{repo}\\n<|file_sep|>{path}\\n\`, optionally with a
\`__symbols__.d.ts\` pseudo-file of available symbols before it.
## Training data
Fine-tuned on permissively licensed open-source repositories (below, pinned commits) and on the publisher's own
code (not published). Secrets were scrubbed from all training data; held-out evaluation repos were never trained on.
${sourcesTable}
## License
Apache-2.0 ([LICENSE](LICENSE), [NOTICE](NOTICE)), as a derivative of Qwen2.5-Coder.
+64
View File
@@ -0,0 +1,64 @@
/**
* TypeScript errors near the cursor, as prompt context ("TS errors as context") and for verifying generated
* training data. Shared by the training pipeline and the daemon so the format is identical in both.
*
* Rendered as a pseudo-file in the repo header:
* <|file_sep|>{dir}/__errors__.txt
* TS2304 Cannot find name 'getU'. | const user = getU
*/
export interface Diagnostic {
/** 0-based line in the visible file. */
line: number
code: number
message: string
category: 'error' | 'warning'
}
/**
* Errors that are noise for completion: environment problems (unresolved modules or types when dependencies
* aren't installed) rather than mistakes in the code being written.
*/
export const IGNORED_CODES = new Set([
2307, // Cannot find module
7016, // Could not find a declaration file for module
2792, // Cannot find module. Did you mean to set moduleResolution…
7006, // Parameter implicitly has an 'any' type
7031, // Binding element implicitly has an 'any' type
2875, // JSX tag requires the module path … to exist
6133, // declared but never read (unused), noise while writing
6196, // declared but never used
])
export function relevantDiagnostics(diags: Diagnostic[], cursorLine: number, opts: { window?: number; max?: number } = {}): Diagnostic[] {
const window = opts.window ?? 20
const max = opts.max ?? 8
return diags
.filter(d => !IGNORED_CODES.has(d.code) && Math.abs(d.line - cursorLine) <= window)
.sort((a, b) => Math.abs(a.line - cursorLine) - Math.abs(b.line - cursorLine) || a.line - b.line)
.slice(0, max)
}
/** One line per diagnostic: code, message (first line), and the trimmed source line it points at. */
export function formatDiagnostics(diags: Diagnostic[], visible: string): string {
const lines = visible.split('\n')
return diags.map(d => {
const message = d.message.split('\n')[0].slice(0, 160)
const src = (lines[d.line] ?? '').trim().slice(0, 100)
return `TS${d.code} ${message} | ${src}`
}).join('\n')
}
/** Diagnostics present in `after` but not in `before`, matched by code + message (positions shift with edits). */
export function newDiagnostics(before: Diagnostic[], after: Diagnostic[]): Diagnostic[] {
const counts = new Map<string, number>()
for (const d of before) counts.set(`${d.code}:${d.message}`, (counts.get(`${d.code}:${d.message}`) ?? 0) + 1)
const out: Diagnostic[] = []
for (const d of after) {
const key = `${d.code}:${d.message}`
const n = counts.get(key) ?? 0
if (n > 0) counts.set(key, n - 1)
else if (!IGNORED_CODES.has(d.code)) out.push(d)
}
return out
}
+165
View File
@@ -0,0 +1,165 @@
/**
* Next-edit prediction prompt format, shared by the training pipeline and the daemon.
*
* <|repo_name|>{repo}\n
* [<|file_sep|>{dir}/__errors__.txt\n{errors}\n] (shared/diagnostics.ts, optional)
* [<|file_sep|>__recent_edits__.diff\n{diff}\n] (most recent edit(s), optional)
* <|file_sep|>{path}\n
* {file before region}<REGION_START>{region, with <CURSOR> at the cursor}<REGION_END>{file after region}<OUTPUT_START>
*
* Target: the rewritten region, then <OUTPUT_END>; or <NO_EDIT><OUTPUT_END> when nothing should change.
* Markers reuse Qwen2.5's vision special tokens, which never occur in code.
*/
import type { Encode } from './prompt.ts'
export const NEXT_EDIT_TOKENS = {
regionStart: '<|quad_start|>',
regionEnd: '<|quad_end|>',
cursor: '<|box_start|>',
outputStart: '<|object_ref_start|>',
outputEnd: '<|object_ref_end|>',
noEdit: '<|vision_pad|>',
} as const
export type NextEditTokenIds = Record<keyof typeof NEXT_EDIT_TOKENS | 'repoName' | 'fileSep', number>
export interface NextEditInput {
repo: string
path: string
/** Current file text (after the user's recent edit). */
file: string
/** Editable region: [startLine, endLine) in `file`, 0-based. */
regionStartLine: number
regionEndLine: number
/** Cursor offset in `file` (must lie inside the region). */
cursor: number
recentEdits?: string
errors?: string
}
/** Character offsets of the start of each line, plus file length. */
export function lineOffsets(text: string): number[] {
const out = [0]
for (let i = text.indexOf('\n'); i !== -1; i = text.indexOf('\n', i + 1)) out.push(i + 1)
return out
}
/** Region text for [startLine, endLine): whole lines, including the final newline when present. */
export function regionText(text: string, startLine: number, endLine: number): string {
const offs = lineOffsets(text)
return text.slice(offs[startLine], endLine < offs.length ? offs[endLine] : text.length)
}
/**
* Compact line diff of one edit for the "recent edits" context:
* @@ path:line
* -old
* +new
*/
export function lineDiff(path: string, before: string, after: string, maxLines = 12): string {
const a = before.split('\n'), b = after.split('\n')
let p = 0
while (p < a.length && p < b.length && a[p] === b[p]) p++
let s = 0
while (s < a.length - p && s < b.length - p && a[a.length - 1 - s] === b[b.length - 1 - s]) s++
const removed = a.slice(p, a.length - s).slice(0, maxLines)
const added = b.slice(p, b.length - s).slice(0, maxLines)
if (!removed.length && !added.length) return ''
return [`@@ ${path}:${p + 1}`, ...removed.map(l => `-${l}`), ...added.map(l => `+${l}`)].join('\n')
}
/** Changed line range of `after` relative to `before`, in `after`'s line numbers: [start, end). */
export function changedLines(before: string, after: string): [number, number] | null {
const a = before.split('\n'), b = after.split('\n')
let p = 0
while (p < a.length && p < b.length && a[p] === b[p]) p++
if (p === a.length && p === b.length) return null
let s = 0
while (s < a.length - p && s < b.length - p && a[a.length - 1 - s] === b[b.length - 1 - s]) s++
return [p, Math.max(p + 1, b.length - s)]
}
/**
* Picks the editable region: covers the cursor line and every line that must change (both in `file`
* coordinates), plus a margin; null if that would exceed maxLines.
*/
export function chooseRegion(file: string, cursorLine: number, mustCover: [number, number][], opts: { margin?: number; maxLines?: number } = {}): [number, number] | null {
const margin = opts.margin ?? 3
const maxLines = opts.maxLines ?? 40
const nLines = file.split('\n').length
let start = cursorLine, end = cursorLine + 1
for (const [a, b] of mustCover) { start = Math.min(start, a); end = Math.max(end, b) }
if (end - start > maxLines) return null
start = Math.max(0, start - margin)
end = Math.min(nLines, end + margin)
return [start, end]
}
export function nextEditTokenIds(tokenToId: (t: string) => number | undefined): NextEditTokenIds {
const id = (t: string) => {
const v = tokenToId(t)
if (v === undefined) throw new Error(`tokenizer is missing ${t}`)
return v
}
return {
regionStart: id(NEXT_EDIT_TOKENS.regionStart), regionEnd: id(NEXT_EDIT_TOKENS.regionEnd), cursor: id(NEXT_EDIT_TOKENS.cursor),
outputStart: id(NEXT_EDIT_TOKENS.outputStart), outputEnd: id(NEXT_EDIT_TOKENS.outputEnd), noEdit: id(NEXT_EDIT_TOKENS.noEdit),
repoName: id('<|repo_name|>'), fileSep: id('<|file_sep|>'),
}
}
/** Prompt tokens up to and including OUTPUT_START. Context before/after the region is truncated at line boundaries. */
export function buildNextEditPrompt(input: NextEditInput, encode: Encode, ids: NextEditTokenIds, opts: { beforeTokens?: number; afterTokens?: number } = {}): number[] {
const beforeTokens = opts.beforeTokens ?? 1024
const afterTokens = opts.afterTokens ?? 384
const offs = lineOffsets(input.file)
const rStart = offs[input.regionStartLine]
const rEnd = input.regionEndLine < offs.length ? offs[input.regionEndLine] : input.file.length
if (input.cursor < rStart || input.cursor > rEnd) throw new Error('cursor outside region')
const out: number[] = [ids.repoName, ...encode(`${input.repo}\n`)]
const dir = input.path.includes('/') ? input.path.slice(0, input.path.lastIndexOf('/')) : '.'
if (input.errors) out.push(ids.fileSep, ...encode(`${dir}/__errors__.txt\n${input.errors}\n`))
if (input.recentEdits) out.push(ids.fileSep, ...encode(`__recent_edits__.diff\n${input.recentEdits}\n`))
out.push(ids.fileSep, ...encode(`${input.path}\n`))
// Before the region: last lines that fit; after: first lines that fit.
// (Window to ~6 chars/token first, then drop whole lines in proportion to the excess: a few encodes, not one per line.)
let before = input.file.slice(0, rStart)
if (before.length > beforeTokens * 6) before = before.slice(before.indexOf('\n', before.length - beforeTokens * 6) + 1)
let beforeIds = encode(before)
while (beforeIds.length > beforeTokens && before.includes('\n')) {
const excess = beforeIds.length - beforeTokens // undershoot (×0.8), then single lines: same cut as line-by-line
const drop = excess > 32 ? Math.floor(0.8 * before.length * excess / beforeIds.length) : 1
const nl = before.indexOf('\n', drop - 1)
before = nl < 0 ? '' : before.slice(nl + 1)
beforeIds = encode(before)
}
let after = input.file.slice(rEnd)
let afterIds = encode(after.slice(0, afterTokens * 8))
if (afterIds.length > afterTokens) {
after = after.slice(0, afterTokens * 8)
while (afterIds.length > afterTokens && after.includes('\n')) {
after = after.slice(0, after.lastIndexOf('\n', after.length - 2) + 1)
afterIds = encode(after)
}
}
out.push(...beforeIds, ids.regionStart,
...encode(input.file.slice(rStart, input.cursor)), ids.cursor, ...encode(input.file.slice(input.cursor, rEnd)),
ids.regionEnd, ...afterIds, ids.outputStart)
return out
}
/** Target tokens: rewritten region (or NO_EDIT), then OUTPUT_END. The caller appends <|endoftext|>. */
export function nextEditTarget(newRegion: string | null, encode: Encode, ids: NextEditTokenIds): number[] {
return newRegion === null ? [ids.noEdit, ids.outputEnd] : [...encode(newRegion), ids.outputEnd]
}
/**
* Model output → rewritten region, or null for "no edit". llama-server leaves special tokens out of `content`, so the
* NO_EDIT token usually arrives as an empty string; never treat empty output as "replace the region with nothing".
*/
export function parseNextEditOutput(raw: string, oldText: string): string | null {
if (raw.startsWith(NEXT_EDIT_TOKENS.noEdit) || raw.trim() === '' || raw === oldText) return null
return raw
}
+25 -6
View File
@@ -29,14 +29,14 @@ export function fimPrompt(order: FimOrder, prefix: number[], suffix: number[], i
/**
* Qwen2.5-Coder repo-level header placed before the FIM prompt:
* <repo_name>repo\n [<file_sep>dir/__symbols__.d.ts\n{context}\n] <file_sep>path\n
* <repo_name>repo\n [<file_sep>dir/__symbols__.d.ts\n{context}\n] [<file_sep>dir/__errors__.txt\n{errors}\n] <file_sep>path\n
*/
export function repoHeader(repo: string, filePath: string, context: string | undefined, encode: Encode, ids: HeaderTokenIds): number[] {
export function repoHeader(repo: string, filePath: string, context: string | undefined, encode: Encode, ids: HeaderTokenIds, errors?: string): number[] {
const out = [ids.repoName, ...encode(`${repo}\n`)]
if (context) {
const dir = filePath.includes('/') ? filePath.slice(0, filePath.lastIndexOf('/')) : '.'
out.push(ids.fileSep, ...encode(`${dir}/__symbols__.d.ts\n${context}\n`))
}
const dir = filePath.includes('/') ? filePath.slice(0, filePath.lastIndexOf('/')) : '.'
if (context) out.push(ids.fileSep, ...encode(`${dir}/__symbols__.d.ts\n${context}\n`))
// TS errors near the cursor (shared/diagnostics.ts), for models trained with them (Stage D+).
if (errors) out.push(ids.fileSep, ...encode(`${dir}/__errors__.txt\n${errors}\n`))
out.push(ids.fileSep, ...encode(`${filePath}\n`))
return out
}
@@ -115,3 +115,22 @@ export function wantsSingleLine(suffix: string): boolean {
const restOfLine = suffix.split('\n')[0]
return /[^\s)\]}'"`;,]/.test(restOfLine)
}
/**
* Token healing for completions that start mid-identifier (`this.connections.del▮`).
*
* Suffix-first prompts end exactly at the cursor, so a half-typed word leaves an unnatural final token and the
* model tends to treat the word as finished (`(relayUrl)` instead of `ete(relayUrl)`). Instead, cut the partial
* word off the prompt and force the output to start with it (llama.cpp grammar); the caller strips it again.
* Bonus: while typing `d`, `de`, `del` the healed prompt is identical, so the KV cache is fully reused.
*/
export function healPrefix(prefix: string): { prefix: string; forced: string } | null {
const m = /(^|[^\w$])([\w$]{1,32})$/.exec(prefix)
if (!m) return null
return { prefix: prefix.slice(0, prefix.length - m[2].length), forced: m[2] }
}
/** GBNF grammar: output must start with `forced`, then anything. */
export function forcedPrefixGrammar(forced: string): string {
return `root ::= "${forced.replace(/\\/g, '\\\\').replace(/"/g, '\\"')}" rest\nrest ::= [^\\x00]*`
}
+74
View File
@@ -16,6 +16,7 @@ import ts from 'typescript'
import { parse as parseSfc } from '@vue/compiler-sfc'
import type { SymbolLang } from './symbol-lang.ts'
import type { Diagnostic } from './diagnostics.ts'
export { symbolLang, type SymbolLang } from './symbol-lang.ts'
export interface SymbolsRequest {
@@ -30,6 +31,11 @@ export interface SymbolsRequest {
countTokens: (s: string) => number
/** Include member completions when the cursor follows `expr.` (default true, as in training). */
members?: boolean
/**
* Context format version. 1 = Stage C's training format (default, byte-identical to before).
* 2 = also namespace imports (`import * as x`): x's members, the ones this file uses first, with signatures.
*/
format?: 1 | 2
}
export interface SymbolsResult { context: string; items: number; hasMembers: boolean }
@@ -181,6 +187,14 @@ export class WorkspaceSymbols {
const names: ts.Identifier[] = []
if (stmt.importClause.name) names.push(stmt.importClause.name)
const bindings = stmt.importClause.namedBindings
if ((req.format ?? 1) >= 2 && bindings && ts.isNamespaceImport(bindings)) {
const ns = this.namespaceMembers(fileName, tsText, bindings.name.text)
if (ns) {
const used = new RegExp(`\\b${bindings.name.text.replace(/\$/g, '\\$')}\\.`).test(near)
items.push({ priority: used ? 1 : 2, text: `// from '${module}' (namespace ${bindings.name.text})\n${ns}` })
}
continue
}
if (bindings && ts.isNamedImports(bindings)) for (const el of bindings.elements) names.push(el.name)
for (const id of names) {
const pos = id.getStart(sf)
@@ -209,6 +223,66 @@ export class WorkspaceSymbols {
return chosen.length ? { context: chosen.join('\n'), items: chosen.length, hasMembers: chosen[0].startsWith('// members of') } : null
}
/**
* Members of a namespace import, as signature lines. Ranked by how often this file already uses each
* member (`ns.member`), then by TypeScript's own completion order; details for the top 12.
*/
private namespaceMembers(fileName: string, tsText: string, ns: string): string | null {
// Probe: ask for completions at `ns.` appended to the file, then restore.
const probe = `${tsText}\n;${ns}.`
this.overrides.set(fileName, { text: probe, version: ++this.version })
try {
const pos = probe.length
const completions = this.service.getCompletionsAtPosition(fileName, pos, {})
if (!completions?.entries.length) return null
const usage = new Map<string, number>()
for (const m of tsText.matchAll(new RegExp(`\\b${ns.replace(/\$/g, '\\$')}\\.([A-Za-z_$][\\w$]*)`, 'g'))) usage.set(m[1], (usage.get(m[1]) ?? 0) + 1)
const entries = [...completions.entries].sort((a, b) => (usage.get(b.name) ?? 0) - (usage.get(a.name) ?? 0) || a.sortText.localeCompare(b.sortText) || a.name.localeCompare(b.name)).slice(0, 40)
const detailed = entries.slice(0, 12).map(e => {
const d = this.service.getCompletionEntryDetails(fileName, pos, e.name, undefined, undefined, undefined, undefined)
return clip(displayText(d?.displayParts).replace(/^\((method|property|function|const|alias|let|var)\) /, ''), 200)
})
const rest = entries.slice(12).map(e => e.name)
return `${detailed.join('\n')}${rest.length ? `\n// also: ${rest.join(', ')}` : ''}`
} finally {
this.overrides.set(fileName, { text: tsText, version: ++this.version })
}
}
/**
* Syntactic + semantic diagnostics of the visible file (Vue: its script blocks), with lines mapped back to the
* visible file. Used for "TS errors as context" and to verify generated next-edit data.
*/
diagnostics(filePath: string, lang: SymbolLang, visible: string): Diagnostic[] {
const loaded = this.load(filePath, lang, visible, 0)
if (!loaded) return []
// Map TS-text offsets back to visible-file lines (Vue script blocks are concatenated with '\n').
const segments: { tsStart: number; visStart: number; length: number }[] = []
if (lang === 'vue') {
let tsPos = 0
for (const r of vueScriptRegions(visible)) {
segments.push({ tsStart: tsPos, visStart: r.start, length: r.end - r.start })
tsPos += r.end - r.start + 1
}
} else {
segments.push({ tsStart: 0, visStart: 0, length: visible.length })
}
const toVisibleLine = (tsOffset: number) => {
const seg = segments.find(g => tsOffset >= g.tsStart && tsOffset <= g.tsStart + g.length) ?? segments[0]
const vis = seg.visStart + (tsOffset - seg.tsStart)
return visible.slice(0, vis).split('\n').length - 1
}
const all = [...this.service.getSyntacticDiagnostics(loaded.fileName), ...this.service.getSemanticDiagnostics(loaded.fileName)]
return all
.filter(d => d.start !== undefined && (d.category === ts.DiagnosticCategory.Error || d.category === ts.DiagnosticCategory.Warning))
.map(d => ({
line: toVisibleLine(d.start!),
code: d.code,
message: ts.flattenDiagnosticMessageText(d.messageText, '\n'),
category: d.category === ts.DiagnosticCategory.Error ? 'error' as const : 'warning' as const,
}))
}
/**
* If the identifier being typed at the cursor only resolves as a member of
* the enclosing class (the user forgot `this.`), returns the member name.