Restructure into a monorepo: models, VS Code extension, daemon, shared
- models/: GGUFs (Git LFS), manifest.json, model card, Apache-2.0 LICENSE + NOTICE - extensions/vscode/: the Local Autocomplete extension (v0.5.0, publisher harvmaster) - daemon/: acd completion daemon (bundled into the extension; also runs standalone) - shared/: prompt layout + symbol extraction shared with the training pipeline - scripts/: make-model-release.mjs, publish-extension-release.mjs (Gitea releases) - README with quick start, settings, architecture, build and release instructions; MIT license for code; CHANGELOG The default model manifest URL moves to models/manifest.json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -0,0 +1,6 @@
|
||||
node_modules/
|
||||
acd.state.json
|
||||
telemetry/
|
||||
recordings/
|
||||
*.local.json
|
||||
test/test*.config.json
|
||||
@@ -0,0 +1,54 @@
|
||||
# acd: local completion daemon
|
||||
|
||||
Builds fill-in-the-middle prompts in exactly the format the models were trained on (it shares
|
||||
[`../shared/`](../shared) with the training pipeline) and serves completions through a llama-server it manages.
|
||||
Editors talk to it over localhost HTTP. Design notes: [../docs/architecture.md](../docs/architecture.md).
|
||||
|
||||
**You usually don't run this yourself:** the VS Code extension bundles it and runs it in *managed mode*. Run it
|
||||
standalone for external mode, for other editors, or when developing.
|
||||
|
||||
## Running standalone
|
||||
|
||||
Requires Node ≥ 22.18 (runs TypeScript directly) and a `llama-server` binary (`brew install llama.cpp` on macOS,
|
||||
or a [llama.cpp release](https://github.com/ggml-org/llama.cpp/releases) on Linux).
|
||||
|
||||
```sh
|
||||
npm install
|
||||
npm start # node src/main.ts --config acd.config.json
|
||||
node src/acctl.ts status # or: mode simple|symbols|raw · model <name> · on · off
|
||||
npm run smoke # end-to-end test (needs the models from ../models, via Git LFS)
|
||||
```
|
||||
|
||||
`acd.config.json` uses `llama-server` from your PATH with `-ngl 99` (Apple GPU), and the models in `../models`.
|
||||
|
||||
- **Linux / CPU:** copy it to `acd.config.local.json` (git-ignored), set `"binary"` to your llama-server and `"args": ["-t", "<physical cores>"]`, then run `node src/main.ts --config acd.config.local.json`.
|
||||
- **Serving another machine** (e.g. an M1 Max serving a laptop): set `"host": "0.0.0.0"` and point the editor's `serverUrl` at it, using `daemon.mode: "external"`. There's no auth, so only do this on a trusted network.
|
||||
- **Start on login:** [`acd.service`](acd.service) is a systemd user unit.
|
||||
|
||||
## Context modes
|
||||
|
||||
| mode | prompt | models |
|
||||
|---|---|---|
|
||||
| `symbols` | repo header + language-service symbols (imports' signatures, repo-local types), from `../shared/symbols.ts`, the same code that built the training data. Requires the editor to send `workspaceRoot`; otherwise the request runs as `simple`. | stage-c |
|
||||
| `simple` | repo name + file path header | stage-c |
|
||||
| `raw` | plain suffix-first FIM | stage-b (spm), base-0.5b (psm) |
|
||||
|
||||
The mode and model are global to the daemon and persisted in `acd.state.json`. VS Code, Zed and `acctl` all see the same toggle.
|
||||
|
||||
**How symbols mode stays fast:**
|
||||
- Context is built in a worker thread (one TypeScript language service per workspace) and cached per *anchor*: the file, the enclosing top-level declaration and the import block. While you type inside one function the header is identical and the KV cache keeps hitting; the smoke test shows the same ~2 re-processed tokens per keystroke as `simple`.
|
||||
- When the anchor changes, the daemon waits up to `symbols.waitMs` for fresh context, else uses the file's previous context (stale-while-revalidate).
|
||||
- Member completions after `.` are off (`symbols.members`): every `.` would change the header and force a cold prefill.
|
||||
|
||||
**`this.` repair** (`thisRepair.enabled`, any mode, needs `workspaceRoot`): when the identifier you're typing only resolves as a member of the enclosing class, the daemon completes as if `this.` were there. It returns `replaceBefore` + `filterText` so the editor replaces `getU` with `this.getUser(id)`.
|
||||
|
||||
**Edit recordings:** `POST /v1/edits` appends opted-in editor edit events to `recording.dir` (default `recordings/`, git-ignored). These are the future training data for next-edit prediction.
|
||||
|
||||
## API
|
||||
|
||||
- `POST /v1/fim`: `{ requestId?, repo, path, languageId?, prefix, suffix, prefixLineOffset?, mode?, singleLine?, maxLines? }` returns `{ text, mode, singleLine, timings, … }`. A newer request cancels the in-flight one; closing the connection cancels too.
|
||||
- `GET|PUT /v1/config`: `{ enabled, model, mode }`. Changing the model restarts llama-server.
|
||||
- `POST /v1/feedback`: editor events, written to the local log only when `telemetry.enabled`.
|
||||
- `POST /v1/edits`: `{ events }` from editors that opted in to edit recording.
|
||||
- The `/v1/fim` request may include `workspaceRoot` (absolute path on the daemon's machine). Responses may include `replaceBefore` / `filterText` (the `this.` repair) and `symbols: { status: hit|fresh|stale|none, items }`.
|
||||
- `GET /health`
|
||||
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"host": "127.0.0.1",
|
||||
"port": 7071,
|
||||
"tokenizer": {
|
||||
"json": "assets/qwen-tokenizer.json",
|
||||
"config": "assets/qwen-tokenizer_config.json"
|
||||
},
|
||||
"llamaServer": {
|
||||
"binary": "llama-server",
|
||||
"port": 7072,
|
||||
"ctx": 8192,
|
||||
"args": [
|
||||
"-ngl",
|
||||
"99"
|
||||
]
|
||||
},
|
||||
"models": {
|
||||
"stage-c": {
|
||||
"path": "../models/stage-c-q8_0.gguf",
|
||||
"order": "spm",
|
||||
"modes": [
|
||||
"symbols",
|
||||
"simple"
|
||||
],
|
||||
"defaultMode": "simple"
|
||||
},
|
||||
"stage-b": {
|
||||
"path": "../models/stage-b-q8_0.gguf",
|
||||
"order": "spm",
|
||||
"modes": [
|
||||
"raw"
|
||||
],
|
||||
"defaultMode": "raw"
|
||||
}
|
||||
},
|
||||
"defaultModel": "stage-c",
|
||||
"prompt": {
|
||||
"prefixTokens": 1536,
|
||||
"suffixTokens": 512,
|
||||
"prefixChunkLines": 32,
|
||||
"maxLines": 8,
|
||||
"nPredictSingleLine": 48,
|
||||
"nPredictMultiLine": 128
|
||||
},
|
||||
"symbols": {
|
||||
"budget": 2048,
|
||||
"waitMs": 150,
|
||||
"members": false
|
||||
},
|
||||
"thisRepair": {
|
||||
"enabled": true,
|
||||
"waitMs": 100
|
||||
},
|
||||
"telemetry": {
|
||||
"enabled": false,
|
||||
"dir": "telemetry"
|
||||
},
|
||||
"recording": {
|
||||
"dir": "recordings"
|
||||
},
|
||||
"stateFile": "acd.state.json"
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
# systemd user service for acd (only needed in external mode; the VS Code extension's
|
||||
# managed mode runs the daemon itself). Edit the two paths below, then:
|
||||
# mkdir -p ~/.config/systemd/user && cp acd.service ~/.config/systemd/user/
|
||||
# systemctl --user daemon-reload && systemctl --user enable --now acd
|
||||
# journalctl --user -u acd -f # logs
|
||||
# To keep it running when you're not logged in: loginctl enable-linger $USER
|
||||
[Unit]
|
||||
Description=acd local completion daemon
|
||||
|
||||
[Service]
|
||||
WorkingDirectory=%h/Projects/AutoComplete/local-autocomplete/daemon
|
||||
# nvm-installed node isn't on PATH for services, so use the absolute path.
|
||||
ExecStart=%h/.nvm/versions/node/v22.22.0/bin/node src/main.ts --config acd.config.json
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,207 @@
|
||||
{
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|endoftext|>",
|
||||
"errors": "replace",
|
||||
"model_max_length": 32768,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null,
|
||||
"add_bos_token": false
|
||||
}
|
||||
Generated
+57
@@ -0,0 +1,57 @@
|
||||
{
|
||||
"name": "acd",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "acd",
|
||||
"dependencies": {
|
||||
"@huggingface/tokenizers": "^0.2.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^26.6.4",
|
||||
"typescript": "5.9"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=22.18"
|
||||
}
|
||||
},
|
||||
"node_modules/@huggingface/tokenizers": {
|
||||
"version": "0.2.0",
|
||||
"resolved": "https://verdaccio.harvmaster.com/@huggingface/tokenizers/-/tokenizers-0.2.0.tgz",
|
||||
"integrity": "sha512-LidMHe1FpcSYH4vcSjXooda34pC0M8m1gmDOv7SQi6Iv+ib1zX7J3sHBCC9/3FPCkn2k1b55FneVqMRxrr99pg==",
|
||||
"license": "Apache-2.0"
|
||||
},
|
||||
"node_modules/@types/node": {
|
||||
"version": "26.6.4",
|
||||
"resolved": "https://verdaccio.harvmaster.com/@types/node/-/node-26.6.4.tgz",
|
||||
"integrity": "sha512-ldVPDCzj7fsaGZrLB0NuHuTvJcsNasysBAqMolr/cgxrLd1xbqxIr3XJiPnHHJUCxj5sNF1vnRj9aWnrVh5Jcg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"undici-types": "~8.9.0"
|
||||
}
|
||||
},
|
||||
"node_modules/typescript": {
|
||||
"version": "5.9.3",
|
||||
"resolved": "https://verdaccio.harvmaster.com/typescript/-/typescript-5.9.3.tgz",
|
||||
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"tsc": "bin/tsc",
|
||||
"tsserver": "bin/tsserver"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=14.17"
|
||||
}
|
||||
},
|
||||
"node_modules/undici-types": {
|
||||
"version": "8.9.0",
|
||||
"resolved": "https://verdaccio.harvmaster.com/undici-types/-/undici-types-8.9.0.tgz",
|
||||
"integrity": "sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"name": "acd",
|
||||
"private": true,
|
||||
"description": "Local completion daemon: builds training-identical FIM prompts and serves them via llama-server",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"start": "node src/main.ts --config acd.config.json",
|
||||
"typecheck": "tsc",
|
||||
"smoke": "node test/smoke.ts"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=22.18"
|
||||
},
|
||||
"dependencies": {
|
||||
"@huggingface/tokenizers": "^0.2.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^26.6.4",
|
||||
"typescript": "5.9"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
/**
|
||||
* acctl: control a running acd from the terminal (the toggle for Zed, scripts, etc.)
|
||||
*
|
||||
* node src/acctl.ts status
|
||||
* node src/acctl.ts mode symbols|simple|raw
|
||||
* node src/acctl.ts model <name>
|
||||
* node src/acctl.ts on|off
|
||||
*
|
||||
* ACD_URL overrides the daemon address (default http://127.0.0.1:7071).
|
||||
*/
|
||||
const url = (process.env.ACD_URL ?? 'http://127.0.0.1:7071').replace(/\/$/, '')
|
||||
const [cmd, arg] = process.argv.slice(2)
|
||||
|
||||
async function call(method: string, route: string, body?: unknown) {
|
||||
const res = await fetch(url + route, { method, headers: { 'content-type': 'application/json' }, body: body ? JSON.stringify(body) : undefined })
|
||||
const json = await res.json() as Record<string, any>
|
||||
if (!res.ok) { console.error(`error: ${json.error ?? res.status}`); process.exit(1) }
|
||||
return json
|
||||
}
|
||||
|
||||
function show(c: Record<string, any>) {
|
||||
const modes = c.models[c.model].modes.join(', ')
|
||||
console.log(`${c.enabled ? 'on' : 'off'} model=${c.model} mode=${c.mode}${c.effectiveMode !== c.mode ? ` (running as ${c.effectiveMode})` : ''} [modes: ${modes}]${c.llama.loading ? ' loading…' : ''}${c.llama.error ? ` error: ${c.llama.error}` : ''}`)
|
||||
}
|
||||
|
||||
try {
|
||||
if (!cmd || cmd === 'status') show(await call('GET', '/v1/config'))
|
||||
else if (cmd === 'mode' && arg) show(await call('PUT', '/v1/config', { mode: arg }))
|
||||
else if (cmd === 'model' && arg) show(await call('PUT', '/v1/config', { model: arg }))
|
||||
else if (cmd === 'on' || cmd === 'off') show(await call('PUT', '/v1/config', { enabled: cmd === 'on' }))
|
||||
else { console.error('usage: acctl status | mode <symbols|simple|raw> | model <name> | on | off'); process.exit(2) }
|
||||
} catch (e) {
|
||||
console.error(`cannot reach acd at ${url}: ${(e as Error).message}`)
|
||||
process.exit(1)
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import type { FimOrder } from '../../shared/prompt.ts'
|
||||
|
||||
export type ContextMode = 'symbols' | 'simple' | 'raw'
|
||||
|
||||
/** What a model was trained on; the toggle only offers these modes. */
|
||||
export interface ModelProfile {
|
||||
path: string
|
||||
order: FimOrder
|
||||
modes: ContextMode[]
|
||||
defaultMode: ContextMode
|
||||
}
|
||||
|
||||
export interface DaemonConfig {
|
||||
host: string
|
||||
port: number
|
||||
tokenizer: { json: string; config: string }
|
||||
llamaServer: {
|
||||
binary: string
|
||||
port: number
|
||||
ctx: number
|
||||
/** Extra flags, e.g. ["-ngl", "99"] on Apple Silicon, ["-t", "16"] on the laptop CPU. */
|
||||
args: string[]
|
||||
}
|
||||
models: Record<string, ModelProfile>
|
||||
defaultModel: string
|
||||
prompt: {
|
||||
prefixTokens: number
|
||||
suffixTokens: number
|
||||
/** Prefix window start moves in multiples of this many lines (KV-cache stability). */
|
||||
prefixChunkLines: number
|
||||
maxLines: number
|
||||
nPredictSingleLine: number
|
||||
nPredictMultiLine: number
|
||||
}
|
||||
symbols: {
|
||||
/** Token budget for the __symbols__.d.ts context (Stage C was trained with up to 2048). */
|
||||
budget: number
|
||||
/** How long a request waits for fresh context when the anchor changes before using stale/none. */
|
||||
waitMs: number
|
||||
/** Member completions after `expr.`: in training, but every `.` changes the header (cold prefill). */
|
||||
members: boolean
|
||||
}
|
||||
thisRepair: { enabled: boolean; waitMs: number }
|
||||
telemetry: { enabled: boolean; dir: string }
|
||||
/** Where opted-in edit recordings from editors are appended (the opt-in is the editor setting). */
|
||||
recording: { dir: string }
|
||||
stateFile: string
|
||||
/** Exit after this many minutes without any request (0 = never). Editors poll every 15 s while open. */
|
||||
idleShutdownMinutes?: number
|
||||
/** Opaque tag reported by /health (the VS Code extension uses it to replace daemons left over from older versions). */
|
||||
instanceTag?: string
|
||||
}
|
||||
|
||||
/** Toggles that survive daemon restarts and are shared by every editor talking to this daemon. */
|
||||
export interface RuntimeState {
|
||||
enabled: boolean
|
||||
model: string
|
||||
mode: ContextMode
|
||||
}
|
||||
|
||||
export function loadConfig(configPath: string): DaemonConfig {
|
||||
const dir = path.dirname(path.resolve(configPath))
|
||||
const raw = JSON.parse(fs.readFileSync(configPath, 'utf8')) as DaemonConfig
|
||||
const abs = (p: string) => (path.isAbsolute(p) || !p.includes('/') ? p : path.resolve(dir, p)) // bare names (e.g. "llama-server") stay on PATH
|
||||
raw.tokenizer = { json: abs(raw.tokenizer.json), config: abs(raw.tokenizer.config) }
|
||||
raw.llamaServer.binary = abs(raw.llamaServer.binary)
|
||||
for (const m of Object.values(raw.models)) m.path = abs(m.path)
|
||||
raw.telemetry.dir = abs(raw.telemetry.dir)
|
||||
raw.recording.dir = abs(raw.recording.dir)
|
||||
raw.stateFile = abs(raw.stateFile)
|
||||
if (!raw.models[raw.defaultModel]) throw new Error(`defaultModel ${raw.defaultModel} is not in models`)
|
||||
for (const [name, m] of Object.entries(raw.models)) {
|
||||
if (!m.modes.includes(m.defaultMode)) throw new Error(`model ${name}: defaultMode must be one of its modes`)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
|
||||
export function loadState(config: DaemonConfig): RuntimeState {
|
||||
const fallback: RuntimeState = { enabled: true, model: config.defaultModel, mode: config.models[config.defaultModel].defaultMode }
|
||||
try {
|
||||
const s = JSON.parse(fs.readFileSync(config.stateFile, 'utf8')) as Partial<RuntimeState>
|
||||
const model = s.model && config.models[s.model] ? s.model : fallback.model
|
||||
const mode = s.mode && config.models[model].modes.includes(s.mode) ? s.mode : config.models[model].defaultMode
|
||||
return { enabled: s.enabled ?? true, model, mode }
|
||||
} catch {
|
||||
return fallback
|
||||
}
|
||||
}
|
||||
|
||||
export function saveState(config: DaemonConfig, state: RuntimeState): void {
|
||||
fs.mkdirSync(path.dirname(config.stateFile), { recursive: true })
|
||||
fs.writeFileSync(config.stateFile, JSON.stringify(state, null, 2) + '\n')
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
import { spawn, type ChildProcess } from 'node:child_process'
|
||||
|
||||
/**
|
||||
* Owns one llama-server process. Completions stream internally so that an
|
||||
* aborted request (the user typed again) closes the connection and llama-server
|
||||
* stops generating immediately.
|
||||
*/
|
||||
|
||||
export interface Timings {
|
||||
prompt_n: number
|
||||
prompt_ms: number
|
||||
predicted_n: number
|
||||
predicted_ms: number
|
||||
cache_n?: number
|
||||
}
|
||||
|
||||
export interface CompletionResult {
|
||||
text: string
|
||||
timings: Timings | null
|
||||
stoppedBy: 'eos' | 'stop' | 'limit' | 'aborted'
|
||||
}
|
||||
|
||||
export class LlamaServer {
|
||||
private proc: ChildProcess | null = null
|
||||
private log = ''
|
||||
model: string | null = null
|
||||
private binary: string
|
||||
private port: number
|
||||
private ctx: number
|
||||
private args: string[]
|
||||
|
||||
constructor(opts: { binary: string; port: number; ctx: number; args: string[] }) {
|
||||
this.binary = opts.binary
|
||||
this.port = opts.port
|
||||
this.ctx = opts.ctx
|
||||
this.args = opts.args
|
||||
}
|
||||
|
||||
get url() { return `http://127.0.0.1:${this.port}` }
|
||||
|
||||
async start(modelPath: string): Promise<void> {
|
||||
await this.stop()
|
||||
this.log = ''
|
||||
const proc = spawn(this.binary, ['-m', modelPath, '--port', String(this.port), '--host', '127.0.0.1', '-c', String(this.ctx), '-np', '1', '--no-webui', ...this.args], {
|
||||
stdio: ['ignore', 'ignore', 'pipe'],
|
||||
})
|
||||
proc.stderr!.on('data', d => { this.log = (this.log + d).slice(-8000) })
|
||||
this.proc = proc
|
||||
for (let i = 0; i < 240; i++) {
|
||||
await new Promise(r => setTimeout(r, 250))
|
||||
if (proc.exitCode !== null) throw new Error(`llama-server exited (code ${proc.exitCode}):\n${this.log}`)
|
||||
try {
|
||||
if ((await fetch(`${this.url}/health`)).ok) { this.model = modelPath; return }
|
||||
} catch { /* not listening yet */ }
|
||||
}
|
||||
await this.stop()
|
||||
throw new Error(`llama-server did not become healthy within 60 s:\n${this.log}`)
|
||||
}
|
||||
|
||||
async stop(): Promise<void> {
|
||||
const proc = this.proc
|
||||
this.proc = null
|
||||
this.model = null
|
||||
if (!proc || proc.exitCode !== null) return
|
||||
const exited = new Promise(r => proc.once('exit', r))
|
||||
proc.kill('SIGTERM')
|
||||
await Promise.race([exited, new Promise(r => setTimeout(r, 3000))])
|
||||
if (proc.exitCode === null) proc.kill('SIGKILL')
|
||||
}
|
||||
|
||||
get running() { return this.proc !== null && this.proc.exitCode === null && this.model !== null }
|
||||
|
||||
async complete(prompt: number[], opts: { nPredict: number; stop: string[] }, signal: AbortSignal): Promise<CompletionResult> {
|
||||
let text = ''
|
||||
let timings: Timings | null = null
|
||||
let stoppedBy: CompletionResult['stoppedBy'] = 'limit'
|
||||
try {
|
||||
const res = await fetch(`${this.url}/completion`, {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
prompt, n_predict: opts.nPredict, stop: opts.stop, stream: true,
|
||||
temperature: 0, top_k: 1, cache_prompt: true, id_slot: 0,
|
||||
}),
|
||||
signal,
|
||||
})
|
||||
if (!res.ok || !res.body) throw new Error(`llama-server ${res.status}: ${await res.text()}`)
|
||||
const decoder = new TextDecoder()
|
||||
let buffer = ''
|
||||
for await (const chunk of res.body) {
|
||||
buffer += decoder.decode(chunk as Uint8Array, { stream: true })
|
||||
let nl: number
|
||||
while ((nl = buffer.indexOf('\n')) !== -1) {
|
||||
const line = buffer.slice(0, nl).trim()
|
||||
buffer = buffer.slice(nl + 1)
|
||||
if (!line.startsWith('data:')) continue
|
||||
const event = JSON.parse(line.slice(5)) as { content?: string; stop?: boolean; timings?: Timings; stop_type?: string }
|
||||
if (event.content) text += event.content
|
||||
if (event.stop) {
|
||||
timings = event.timings ?? null
|
||||
stoppedBy = event.stop_type === 'eos' ? 'eos' : event.stop_type === 'word' ? 'stop' : 'limit'
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
if (signal.aborted) return { text, timings, stoppedBy: 'aborted' }
|
||||
throw e
|
||||
}
|
||||
return { text, timings, stoppedBy }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,229 @@
|
||||
/**
|
||||
* acd: local completion daemon.
|
||||
*
|
||||
* node src/main.ts --config acd.config.json
|
||||
*
|
||||
* Endpoints (JSON over HTTP, localhost by default):
|
||||
* POST /v1/fim { repo, path, prefix, suffix, mode?, singleLine?, ... } -> { text, mode, timings, ... }
|
||||
* GET /v1/config current toggles + the modes each model supports
|
||||
* PUT /v1/config { enabled?, model?, mode? } (changing model restarts llama-server)
|
||||
* POST /v1/feedback { requestId, event: shown|accepted|partial|rejected, acceptedText? }
|
||||
* POST /v1/edits { events: [...] } opted-in edit recordings from editors -> recording.dir
|
||||
* POST /v1/shutdown stop llama-server and exit (used by the extension to replace an outdated daemon)
|
||||
* GET /health
|
||||
*
|
||||
* A new /v1/fim request cancels any in-flight one (there is one llama-server
|
||||
* slot, and the user has moved on anyway); a client disconnect cancels too.
|
||||
*/
|
||||
import http from 'node:http'
|
||||
import path from 'node:path'
|
||||
import { parseArgs } from 'node:util'
|
||||
import { loadConfig, loadState, saveState, type ContextMode, type RuntimeState } from './config.ts'
|
||||
import { LlamaServer } from './llama.ts'
|
||||
import { PromptBuilder, STOP_STRINGS, type FimRequest } from './prompt-builder.ts'
|
||||
import { Telemetry } from './telemetry.ts'
|
||||
import { trimSuffixOverlap } from '../../shared/prompt.ts'
|
||||
import { symbolLang } from '../../shared/symbol-lang.ts'
|
||||
import { SymbolService, mightNeedThis } from './symbols.ts'
|
||||
|
||||
const { values: args } = parseArgs({ options: { config: { type: 'string', default: 'acd.config.json' } } })
|
||||
const config = loadConfig(args.config!)
|
||||
const state: RuntimeState = loadState(config)
|
||||
const builder = new PromptBuilder(config)
|
||||
const llama = new LlamaServer(config.llamaServer)
|
||||
const telemetry = new Telemetry(config.telemetry)
|
||||
const symbols = new SymbolService(config.tokenizer)
|
||||
|
||||
let loading: Promise<void> | null = null
|
||||
let loadError: string | null = null
|
||||
function loadModel(name: string) {
|
||||
loadError = null
|
||||
loading = llama.start(config.models[name].path)
|
||||
.catch(e => { loadError = (e as Error).message; console.error(loadError) })
|
||||
.finally(() => { loading = null })
|
||||
return loading
|
||||
}
|
||||
|
||||
let inflight: AbortController | null = null
|
||||
|
||||
function describeConfig() {
|
||||
return {
|
||||
...state,
|
||||
effectiveMode: state.mode,
|
||||
models: Object.fromEntries(Object.entries(config.models).map(([n, m]) => [n, { order: m.order, modes: m.modes, defaultMode: m.defaultMode }])),
|
||||
llama: { running: llama.running, loading: loading !== null, error: loadError },
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
async function handleFim(body: FimRequest, res: http.ServerResponse, clientGone: AbortSignal) {
|
||||
const t0 = performance.now()
|
||||
if (!state.enabled) return json(res, 200, { text: '', disabled: true })
|
||||
if (loading) return json(res, 503, { error: 'model loading' })
|
||||
if (!llama.running) return json(res, 503, { error: loadError ?? 'llama-server not running' })
|
||||
if (typeof body.prefix !== 'string' || typeof body.suffix !== 'string' || !body.path || !body.repo) {
|
||||
return json(res, 400, { error: 'repo, path, prefix and suffix are required' })
|
||||
}
|
||||
|
||||
const profile = config.models[state.model]
|
||||
const requested = body.mode && profile.modes.includes(body.mode) ? body.mode : state.mode
|
||||
let mode: ContextMode = requested
|
||||
const lang = symbolLang(body.languageId ?? '')
|
||||
const root = body.workspaceRoot
|
||||
const filePath = root ? path.join(root, body.path) : null
|
||||
if (root && lang) symbols.warm(root)
|
||||
|
||||
// Missing `this.`: the identifier being typed only exists as a class member.
|
||||
// Complete as if `this.` were there and tell the editor to replace the typed identifier.
|
||||
let prefix = body.prefix
|
||||
let repair: { replaceBefore: number; typed: string; replacement: string } | null = null
|
||||
if (config.thisRepair.enabled && root && filePath && lang && mightNeedThis(prefix)) {
|
||||
const member = await symbols.thisMember({ root, filePath, lang, prefix, suffix: body.suffix, waitMs: config.thisRepair.waitMs })
|
||||
if (member) {
|
||||
const typed = prefix.slice(-member.identLength)
|
||||
repair = { replaceBefore: member.identLength, typed, replacement: `this.${typed}` }
|
||||
prefix = prefix.slice(0, -member.identLength) + repair.replacement
|
||||
}
|
||||
}
|
||||
|
||||
let symbolContext: string | undefined
|
||||
let symbolInfo: Record<string, unknown> | undefined
|
||||
if (mode === 'symbols') {
|
||||
if (root && filePath && lang) {
|
||||
// Anchor on the original text so a repair doesn't churn the cached context.
|
||||
const s = await symbols.context({ root, filePath, lang, prefix: body.prefix, suffix: body.suffix, budget: config.symbols.budget, members: config.symbols.members, waitMs: config.symbols.waitMs })
|
||||
symbolContext = s.result?.context
|
||||
symbolInfo = { status: s.status, items: s.result?.items ?? 0 }
|
||||
} else {
|
||||
mode = 'simple' // no workspace or not a TS-family file: header without symbols (Stage C saw this via context dropout)
|
||||
}
|
||||
}
|
||||
const prompt = builder.build({ ...body, prefix }, profile, mode, symbolContext)
|
||||
|
||||
inflight?.abort()
|
||||
const controller = new AbortController()
|
||||
inflight = controller
|
||||
clientGone.addEventListener('abort', () => controller.abort())
|
||||
|
||||
const p = config.prompt
|
||||
const result = await llama.complete(prompt.ids, {
|
||||
nPredict: prompt.singleLine ? p.nPredictSingleLine : p.nPredictMultiLine,
|
||||
stop: prompt.singleLine ? [...STOP_STRINGS, '\n'] : STOP_STRINGS,
|
||||
}, controller.signal)
|
||||
if (inflight === controller) inflight = null
|
||||
if (result.stoppedBy === 'aborted') {
|
||||
if (!res.writableEnded && !res.destroyed) json(res, 200, { text: '', cancelled: true })
|
||||
return
|
||||
}
|
||||
|
||||
let text = trimSuffixOverlap(result.text, body.suffix)
|
||||
if (prompt.singleLine) text = text.split('\n')[0]
|
||||
const maxLines = body.maxLines ?? p.maxLines
|
||||
const lines = text.split('\n')
|
||||
if (lines.length > maxLines) text = lines.slice(0, maxLines).join('\n')
|
||||
if (!text.trim()) text = ''
|
||||
// With a repair, the edit replaces the typed identifier: "getU" -> "this.getUser(id)".
|
||||
// filterText starts with what was typed, which is what VS Code requires to show it.
|
||||
const filterText = repair ? repair.typed + text : undefined
|
||||
if (repair) text = repair.replacement + text
|
||||
|
||||
const response = {
|
||||
requestId: body.requestId,
|
||||
text,
|
||||
model: state.model,
|
||||
mode,
|
||||
requestedMode: requested,
|
||||
singleLine: prompt.singleLine,
|
||||
replaceBefore: repair?.replaceBefore ?? 0,
|
||||
filterText,
|
||||
symbols: symbolInfo,
|
||||
stoppedBy: result.stoppedBy,
|
||||
timings: {
|
||||
tokenizeMs: round(prompt.tokenizeMs),
|
||||
promptTokens: prompt.ids.length,
|
||||
processedTokens: result.timings?.prompt_n ?? null,
|
||||
promptMs: round(result.timings?.prompt_ms),
|
||||
genTokens: result.timings?.predicted_n ?? null,
|
||||
genMs: round(result.timings?.predicted_ms),
|
||||
totalMs: round(performance.now() - t0),
|
||||
},
|
||||
}
|
||||
telemetry.request(body, response)
|
||||
json(res, 200, response)
|
||||
}
|
||||
|
||||
const round = (x: number | undefined) => (x === undefined ? null : Math.round(x * 10) / 10)
|
||||
|
||||
function json(res: http.ServerResponse, status: number, body: unknown) {
|
||||
if (res.writableEnded || res.destroyed) return
|
||||
res.writeHead(status, { 'content-type': 'application/json' })
|
||||
res.end(JSON.stringify(body))
|
||||
}
|
||||
|
||||
async function readBody(req: http.IncomingMessage): Promise<unknown> {
|
||||
const chunks: Buffer[] = []
|
||||
for await (const c of req) chunks.push(c as Buffer)
|
||||
const text = Buffer.concat(chunks).toString('utf8')
|
||||
return text ? JSON.parse(text) : {}
|
||||
}
|
||||
|
||||
let lastActivity = Date.now()
|
||||
if (config.idleShutdownMinutes) {
|
||||
setInterval(() => {
|
||||
if (Date.now() - lastActivity > config.idleShutdownMinutes! * 60_000) {
|
||||
console.log(`idle for ${config.idleShutdownMinutes} min, shutting down`)
|
||||
void llama.stop().then(() => process.exit(0))
|
||||
}
|
||||
}, 30_000).unref()
|
||||
}
|
||||
|
||||
const server = http.createServer(async (req, res) => {
|
||||
lastActivity = Date.now()
|
||||
// Cancel the completion if the editor gives up on this request.
|
||||
const gone = new AbortController()
|
||||
res.on('close', () => { if (!res.writableEnded) gone.abort() })
|
||||
try {
|
||||
const route = `${req.method} ${req.url?.split('?')[0]}`
|
||||
if (route === 'GET /health') return json(res, 200, { ok: llama.running, loading: loading !== null, model: state.model, error: loadError, instanceTag: config.instanceTag ?? null, pid: process.pid })
|
||||
if (route === 'POST /v1/shutdown') {
|
||||
json(res, 200, { ok: true })
|
||||
void llama.stop().then(() => process.exit(0))
|
||||
return
|
||||
}
|
||||
if (route === 'GET /v1/config') return json(res, 200, describeConfig())
|
||||
if (route === 'PUT /v1/config') {
|
||||
const body = await readBody(req) as Partial<RuntimeState>
|
||||
if (body.model !== undefined && !config.models[body.model]) return json(res, 400, { error: `unknown model ${body.model}` })
|
||||
const model = body.model ?? state.model
|
||||
const mode = body.mode ?? (body.model ? config.models[model].defaultMode : state.mode)
|
||||
if (!config.models[model].modes.includes(mode)) return json(res, 400, { error: `model ${model} supports modes ${config.models[model].modes.join(', ')}` })
|
||||
const modelChanged = model !== state.model
|
||||
Object.assign(state, { model, mode, enabled: body.enabled ?? state.enabled })
|
||||
saveState(config, state)
|
||||
if (modelChanged) void loadModel(model)
|
||||
return json(res, 200, describeConfig())
|
||||
}
|
||||
if (route === 'POST /v1/fim') return await handleFim(await readBody(req) as FimRequest, res, gone.signal)
|
||||
if (route === 'POST /v1/edits') {
|
||||
const body = await readBody(req) as { events?: unknown[] }
|
||||
return json(res, 200, { recorded: Telemetry.recordEdits(config.recording.dir, body.events ?? []) })
|
||||
}
|
||||
if (route === 'POST /v1/feedback') {
|
||||
telemetry.feedback(await readBody(req))
|
||||
return json(res, 200, { ok: true })
|
||||
}
|
||||
json(res, 404, { error: 'not found' })
|
||||
} catch (e) {
|
||||
console.error(e)
|
||||
json(res, 500, { error: (e as Error).message })
|
||||
}
|
||||
})
|
||||
|
||||
process.on('SIGINT', () => void llama.stop().then(() => process.exit(0)))
|
||||
process.on('SIGTERM', () => void llama.stop().then(() => process.exit(0)))
|
||||
|
||||
server.listen(config.port, config.host, () => {
|
||||
console.log(`acd listening on http://${config.host}:${config.port} (model ${state.model}, mode ${state.mode})`)
|
||||
})
|
||||
await loadModel(state.model)
|
||||
if (llama.running) console.log(`llama-server ready: ${config.models[state.model].path}`)
|
||||
@@ -0,0 +1,77 @@
|
||||
import fs from 'node:fs'
|
||||
import { Tokenizer } from '@huggingface/tokenizers'
|
||||
import { chunkedPrefixTokens, fimPrompt, headTokens, repoHeader, wantsSingleLine } from '../../shared/prompt.ts'
|
||||
import type { ContextMode, DaemonConfig, ModelProfile } from './config.ts'
|
||||
|
||||
export interface FimRequest {
|
||||
requestId?: string
|
||||
/** Workspace folder name; Stage C was trained with the repo directory name here. */
|
||||
repo: string
|
||||
/** Path relative to the workspace folder, '/'-separated. */
|
||||
path: string
|
||||
/** Absolute workspace folder on the daemon's machine; enables symbols mode and the this.-repair. */
|
||||
workspaceRoot?: string
|
||||
languageId?: string
|
||||
prefix: string
|
||||
suffix: string
|
||||
/** Absolute line number of prefix's first line, if the client sent a truncated document. */
|
||||
prefixLineOffset?: number
|
||||
mode?: ContextMode
|
||||
/** Override automatic single-line detection. */
|
||||
singleLine?: boolean
|
||||
maxLines?: number
|
||||
}
|
||||
|
||||
export interface BuiltPrompt {
|
||||
ids: number[]
|
||||
mode: ContextMode
|
||||
singleLine: boolean
|
||||
prefixStartLine: number
|
||||
tokenizeMs: number
|
||||
}
|
||||
|
||||
export const STOP_STRINGS = ['<|endoftext|>', '<|fim_prefix|>', '<|fim_suffix|>', '<|fim_middle|>', '<|fim_pad|>', '<|file_sep|>', '<|repo_name|>', '<|im_start|>', '<|im_end|>']
|
||||
|
||||
export class PromptBuilder {
|
||||
private encodeFn: (s: string) => number[]
|
||||
private fim: { prefix: number; suffix: number; middle: number }
|
||||
private header: { repoName: number; fileSep: number }
|
||||
private config: DaemonConfig
|
||||
|
||||
constructor(config: DaemonConfig) {
|
||||
this.config = config
|
||||
const tokenizer = new Tokenizer(
|
||||
JSON.parse(fs.readFileSync(config.tokenizer.json, 'utf8')),
|
||||
JSON.parse(fs.readFileSync(config.tokenizer.config, 'utf8')),
|
||||
)
|
||||
this.encodeFn = s => tokenizer.encode(s, { add_special_tokens: false }).ids
|
||||
const id = (t: string) => {
|
||||
const v = tokenizer.token_to_id(t)
|
||||
if (v === undefined) throw new Error(`tokenizer is missing ${t}`)
|
||||
return v
|
||||
}
|
||||
this.fim = { prefix: id('<|fim_prefix|>'), suffix: id('<|fim_suffix|>'), middle: id('<|fim_middle|>') }
|
||||
this.header = { repoName: id('<|repo_name|>'), fileSep: id('<|file_sep|>') }
|
||||
}
|
||||
|
||||
encode(s: string) { return this.encodeFn(s) }
|
||||
|
||||
/**
|
||||
* Builds the prompt in the model's trained layout. `symbols` context is passed
|
||||
* in by the caller (empty until the symbol provider exists, which degrades to `simple`).
|
||||
*/
|
||||
build(req: FimRequest, profile: ModelProfile, mode: ContextMode, symbols?: string): BuiltPrompt {
|
||||
const t0 = performance.now()
|
||||
const p = this.config.prompt
|
||||
const { tokens: prefix, startLine } = chunkedPrefixTokens(req.prefix, p.prefixTokens, this.encodeFn, p.prefixChunkLines, req.prefixLineOffset ?? 0)
|
||||
const suffix = headTokens(req.suffix, p.suffixTokens, this.encodeFn)
|
||||
const header = mode === 'raw' ? [] : repoHeader(req.repo, req.path, mode === 'symbols' ? symbols : undefined, this.encodeFn, this.header)
|
||||
return {
|
||||
ids: [...header, ...fimPrompt(profile.order, prefix, suffix, this.fim)],
|
||||
mode,
|
||||
singleLine: req.singleLine ?? wantsSingleLine(req.suffix),
|
||||
prefixStartLine: startLine,
|
||||
tokenizeMs: performance.now() - t0,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
/**
|
||||
* Worker thread that owns the TypeScript language services (one per workspace
|
||||
* root). Building a program for a large repo takes seconds, which must never
|
||||
* block the request loop, so all symbol work happens here.
|
||||
*/
|
||||
import fs from 'node:fs'
|
||||
import { parentPort, workerData } from 'node:worker_threads'
|
||||
import { Tokenizer } from '@huggingface/tokenizers'
|
||||
import { WorkspaceSymbols, type SymbolLang } from '../../shared/symbols.ts'
|
||||
|
||||
export type WorkerRequest =
|
||||
| { id: number; kind: 'warm'; root: string }
|
||||
| { id: number; kind: 'context'; root: string; filePath: string; lang: SymbolLang; visible: string; cursor: number; budget: number; members: boolean }
|
||||
| { id: number; kind: 'thisMember'; root: string; filePath: string; lang: SymbolLang; visible: string; cursor: number }
|
||||
|
||||
export interface WorkerResponse { id: number; result?: unknown; error?: string; ms: number }
|
||||
|
||||
const { tokenizerJson, tokenizerConfig } = workerData as { tokenizerJson: string; tokenizerConfig: string }
|
||||
const tokenizer = new Tokenizer(JSON.parse(fs.readFileSync(tokenizerJson, 'utf8')), JSON.parse(fs.readFileSync(tokenizerConfig, 'utf8')))
|
||||
const countTokens = (s: string) => tokenizer.encode(s, { add_special_tokens: false }).ids.length
|
||||
|
||||
const workspaces = new Map<string, WorkspaceSymbols>()
|
||||
const workspace = (root: string) => {
|
||||
let ws = workspaces.get(root)
|
||||
if (!ws) { ws = new WorkspaceSymbols(root); workspaces.set(root, ws) }
|
||||
return ws
|
||||
}
|
||||
|
||||
parentPort!.on('message', (req: WorkerRequest) => {
|
||||
const t0 = performance.now()
|
||||
try {
|
||||
let result: unknown = null
|
||||
if (req.kind === 'warm') workspace(req.root)
|
||||
if (req.kind === 'context') {
|
||||
result = workspace(req.root).context({ filePath: req.filePath, lang: req.lang, visible: req.visible, cursor: req.cursor, budget: req.budget, countTokens, members: req.members })
|
||||
}
|
||||
if (req.kind === 'thisMember') result = workspace(req.root).thisMember(req.filePath, req.lang, req.visible, req.cursor)
|
||||
parentPort!.postMessage({ id: req.id, result, ms: performance.now() - t0 } satisfies WorkerResponse)
|
||||
} catch (e) {
|
||||
parentPort!.postMessage({ id: req.id, error: (e as Error).message, ms: performance.now() - t0 } satisfies WorkerResponse)
|
||||
}
|
||||
})
|
||||
@@ -0,0 +1,110 @@
|
||||
import { Worker } from 'node:worker_threads'
|
||||
import crypto from 'node:crypto'
|
||||
import type { SymbolsResult } from '../../shared/symbols.ts'
|
||||
import type { SymbolLang } from '../../shared/symbol-lang.ts'
|
||||
import type { WorkerRequest, WorkerResponse } from './symbols-worker.ts'
|
||||
|
||||
type Payload<K extends WorkerRequest['kind']> = Omit<Extract<WorkerRequest, { kind: K }>, 'id' | 'kind'>
|
||||
|
||||
/**
|
||||
* Symbol context for `symbols` mode, with caching that keeps prompts stable.
|
||||
*
|
||||
* Context is cached per anchor: file + enclosing top-level declaration + the
|
||||
* file's import block. While you type inside one function the header is
|
||||
* identical, so llama-server's KV cache keeps hitting. When the anchor changes
|
||||
* we wait briefly for fresh context; if the language service is slower (e.g.
|
||||
* still building the program), the previous context for the file is used
|
||||
* (stale-while-revalidate) and the fresh one lands in the cache for the next keystroke.
|
||||
*/
|
||||
export class SymbolService {
|
||||
private worker: Worker
|
||||
private nextId = 1
|
||||
private pending = new Map<number, (r: WorkerResponse) => void>()
|
||||
private cache = new Map<string, SymbolsResult | null>()
|
||||
private lastForFile = new Map<string, SymbolsResult | null>()
|
||||
private inflight = new Set<string>()
|
||||
private warmed = new Set<string>()
|
||||
|
||||
constructor(tokenizer: { json: string; config: string }) {
|
||||
// Source runs load the .ts worker (Node type stripping); the bundled extension build ships symbols-worker.mjs.
|
||||
const workerFile = import.meta.url.endsWith('.ts') ? './symbols-worker.ts' : './symbols-worker.mjs'
|
||||
this.worker = new Worker(new URL(workerFile, import.meta.url), { workerData: { tokenizerJson: tokenizer.json, tokenizerConfig: tokenizer.config } })
|
||||
this.worker.on('message', (r: WorkerResponse) => { this.pending.get(r.id)?.(r); this.pending.delete(r.id) })
|
||||
this.worker.on('error', e => console.error('symbols worker error:', e))
|
||||
this.worker.unref()
|
||||
}
|
||||
|
||||
private call<K extends WorkerRequest['kind']>(kind: K, payload: Payload<K>): Promise<WorkerResponse> {
|
||||
const id = this.nextId++
|
||||
return new Promise(resolve => {
|
||||
this.pending.set(id, resolve)
|
||||
this.worker.postMessage({ id, kind, ...payload })
|
||||
})
|
||||
}
|
||||
|
||||
/** Start building the language service for a workspace before it's needed. */
|
||||
warm(root: string) {
|
||||
if (this.warmed.has(root)) return
|
||||
this.warmed.add(root)
|
||||
void this.call('warm', { root })
|
||||
}
|
||||
|
||||
async context(opts: { root: string; filePath: string; lang: SymbolLang; prefix: string; suffix: string; budget: number; members: boolean; waitMs: number }):
|
||||
Promise<{ result: SymbolsResult | null; status: 'hit' | 'fresh' | 'stale' | 'none' }> {
|
||||
const key = anchorKey(opts.filePath, opts.prefix, opts.members)
|
||||
if (this.cache.has(key)) {
|
||||
const result = this.cache.get(key)!
|
||||
this.lastForFile.set(opts.filePath, result)
|
||||
return { result, status: 'hit' }
|
||||
}
|
||||
const request = this.inflight.has(key) ? null : this.call('context', {
|
||||
root: opts.root, filePath: opts.filePath, lang: opts.lang, visible: opts.prefix + opts.suffix, cursor: opts.prefix.length, budget: opts.budget, members: opts.members,
|
||||
})
|
||||
if (request) {
|
||||
this.inflight.add(key)
|
||||
void request.then(r => {
|
||||
this.inflight.delete(key)
|
||||
if (r.error) console.error('symbols:', r.error)
|
||||
this.remember(key, (r.result as SymbolsResult | null) ?? null)
|
||||
})
|
||||
const fresh = await Promise.race([request, new Promise<null>(r => setTimeout(() => r(null), opts.waitMs))])
|
||||
if (fresh && !fresh.error) {
|
||||
const result = (fresh.result as SymbolsResult | null) ?? null
|
||||
this.lastForFile.set(opts.filePath, result)
|
||||
return { result, status: 'fresh' }
|
||||
}
|
||||
}
|
||||
const stale = this.lastForFile.get(opts.filePath)
|
||||
return stale !== undefined ? { result: stale, status: 'stale' } : { result: null, status: 'none' }
|
||||
}
|
||||
|
||||
private remember(key: string, result: SymbolsResult | null) {
|
||||
this.cache.set(key, result)
|
||||
if (this.cache.size > 500) this.cache.delete(this.cache.keys().next().value!)
|
||||
}
|
||||
|
||||
async thisMember(opts: { root: string; filePath: string; lang: SymbolLang; prefix: string; suffix: string; waitMs: number }) {
|
||||
const r = await Promise.race([
|
||||
this.call('thisMember', { root: opts.root, filePath: opts.filePath, lang: opts.lang, visible: opts.prefix + opts.suffix, cursor: opts.prefix.length }),
|
||||
new Promise<null>(r => setTimeout(() => r(null), opts.waitMs)),
|
||||
])
|
||||
return (r?.result as { name: string; identLength: number } | null) ?? null
|
||||
}
|
||||
}
|
||||
|
||||
const TOP_LEVEL = /^(export\s|async\s|function[\s*]|class\s|const\s|let\s|var\s|interface\s|type\s|enum\s|abstract\s|declare\s|<script|<template|<style|@)/
|
||||
|
||||
/** file + enclosing top-level declaration (line index) + hash of the import lines (+ receiver when member context is on). */
|
||||
export function anchorKey(filePath: string, prefix: string, members: boolean): string {
|
||||
const lines = prefix.split('\n')
|
||||
let anchor = 0
|
||||
for (let i = lines.length - 1; i >= 0; i--) if (TOP_LEVEL.test(lines[i])) { anchor = i; break }
|
||||
const imports = crypto.createHash('sha1').update(lines.filter(l => /^\s*import\s/.test(l)).join('\n')).digest('hex').slice(0, 12)
|
||||
const receiver = members ? (/([\w$.()[\]'"]+)\.[\w$]*$/.exec(prefix.slice(-200))?.[1] ?? '') : ''
|
||||
return `${filePath}|${anchor}|${imports}|${receiver}`
|
||||
}
|
||||
|
||||
/** Cheap pre-check before asking the language service whether `this.` is missing. */
|
||||
export function mightNeedThis(prefix: string): boolean {
|
||||
return /(^|[^\w$.])[A-Za-z_$][\w$]+$/.test(prefix.slice(-100)) && !/\b(function|const|let|var|class|interface|type|enum|import|export|as|new)\s+[A-Za-z_$][\w$]*$/.test(prefix.slice(-100))
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import type { FimRequest } from './prompt-builder.ts'
|
||||
|
||||
/**
|
||||
* Local-only, opt-in telemetry (config.telemetry.enabled). One JSONL file per day.
|
||||
*
|
||||
* Besides explicit feedback events from the editor, acceptance is inferred:
|
||||
* if the next request for the same file has a prefix that ends with the
|
||||
* previous prefix + the suggested text, the suggestion was typed or accepted.
|
||||
* That is the only acceptance signal Zed can give us, and a cross-check for VS Code.
|
||||
*/
|
||||
export class Telemetry {
|
||||
private enabled: boolean
|
||||
private dir: string
|
||||
private last = new Map<string, { requestId?: string; prefixTail: string; text: string }>()
|
||||
|
||||
constructor(opts: { enabled: boolean; dir: string }) {
|
||||
this.enabled = opts.enabled
|
||||
this.dir = opts.dir
|
||||
}
|
||||
|
||||
private write(record: Record<string, unknown>) {
|
||||
if (!this.enabled) return
|
||||
fs.mkdirSync(this.dir, { recursive: true })
|
||||
const file = path.join(this.dir, `${new Date().toISOString().slice(0, 10)}.jsonl`)
|
||||
fs.appendFileSync(file, JSON.stringify({ t: new Date().toISOString(), ...record }) + '\n')
|
||||
}
|
||||
|
||||
request(req: FimRequest, res: { requestId?: string; text: string; mode: string; model: string; timings: unknown }) {
|
||||
if (!this.enabled) return
|
||||
const key = `${req.repo}/${req.path}`
|
||||
const prev = this.last.get(key)
|
||||
if (prev?.text) {
|
||||
const accepted = req.prefix.endsWith(prev.prefixTail + prev.text)
|
||||
if (accepted) this.write({ event: 'inferred-accept', requestId: prev.requestId, path: key, chars: prev.text.length })
|
||||
}
|
||||
this.last.set(key, { requestId: req.requestId, prefixTail: req.prefix.slice(-200), text: res.text })
|
||||
this.write({ event: 'completion', requestId: req.requestId, path: key, languageId: req.languageId, model: res.model, mode: res.mode, chars: res.text.length, lines: res.text ? res.text.split('\n').length : 0, timings: res.timings })
|
||||
}
|
||||
|
||||
/** Editor edit events (opt-in in the editor). Always local, one JSONL file per day. */
|
||||
static recordEdits(dir: string, events: unknown[]) {
|
||||
if (!Array.isArray(events) || !events.length) return 0
|
||||
fs.mkdirSync(dir, { recursive: true })
|
||||
const file = path.join(dir, `edits-${new Date().toISOString().slice(0, 10)}.jsonl`)
|
||||
fs.appendFileSync(file, events.map(e => JSON.stringify(e)).join('\n') + '\n')
|
||||
return events.length
|
||||
}
|
||||
|
||||
feedback(body: unknown) {
|
||||
this.write({ event: 'feedback', ...(body as Record<string, unknown>) })
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,147 @@
|
||||
/**
|
||||
* End-to-end smoke test: starts acd, then exercises it the way an editor would.
|
||||
*
|
||||
* node test/smoke.ts [--config acd.config.json] [--typing-samples 15]
|
||||
*
|
||||
* Symbols/this.-repair checks and the typing replay use the private training workspace's ../data
|
||||
* (held-out samples, installed repos) when present, and are skipped otherwise.
|
||||
*/
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import crypto from 'node:crypto'
|
||||
import { spawn } from 'node:child_process'
|
||||
import { parseArgs } from 'node:util'
|
||||
|
||||
const { values: args } = parseArgs({ options: { config: { type: 'string', default: 'acd.config.json' }, 'typing-samples': { type: 'string', default: '15' }, entry: { type: 'string', default: 'src/main.ts' } } })
|
||||
const config = JSON.parse(fs.readFileSync(args.config!, 'utf8')) as { port: number; stateFile: string }
|
||||
const base = `http://127.0.0.1:${config.port}`
|
||||
let failures = 0
|
||||
const check = (name: string, ok: boolean, detail = '') => {
|
||||
console.log(`${ok ? 'PASS' : 'FAIL'} ${name}${detail ? ` (${detail})` : ''}`)
|
||||
if (!ok) failures++
|
||||
}
|
||||
|
||||
// --entry ../extensions/vscode/dist/daemon/acd.mjs tests the bundled build the extension ships.
|
||||
const daemon = spawn(process.execPath, [args.entry!, '--config', args.config!], { stdio: ['ignore', 'pipe', 'pipe'] })
|
||||
let daemonLog = ''
|
||||
daemon.stdout.on('data', d => { daemonLog += d })
|
||||
daemon.stderr.on('data', d => { daemonLog += d })
|
||||
const cleanup = () => { daemon.kill('SIGTERM') }
|
||||
process.on('exit', cleanup)
|
||||
|
||||
async function api(method: string, route: string, body?: unknown, signal?: AbortSignal) {
|
||||
const res = await fetch(base + route, { method, headers: { 'content-type': 'application/json' }, body: body ? JSON.stringify(body) : undefined, signal })
|
||||
return { status: res.status, body: await res.json() as Record<string, any> }
|
||||
}
|
||||
|
||||
for (let i = 0; ; i++) {
|
||||
await new Promise(r => setTimeout(r, 500))
|
||||
try { if ((await api('GET', '/health')).body.ok) break } catch { /* starting */ }
|
||||
if (i > 120) { console.error(daemonLog); throw new Error('daemon did not become healthy') }
|
||||
}
|
||||
|
||||
// ---------- config / toggle
|
||||
await api('PUT', '/v1/config', { model: 'stage-c', mode: 'simple', enabled: true })
|
||||
const cfg = (await api('GET', '/v1/config')).body
|
||||
check('config lists model profiles', !!cfg.models?.['stage-c']?.modes?.includes('symbols'), JSON.stringify(cfg.models?.['stage-c']))
|
||||
check('invalid mode for model is rejected', (await api('PUT', '/v1/config', { mode: 'raw' })).status === 400)
|
||||
const sym = (await api('PUT', '/v1/config', { mode: 'symbols' })).body
|
||||
check('symbols mode accepted', sym.mode === 'symbols' && sym.effectiveMode === 'symbols')
|
||||
await api('PUT', '/v1/config', { mode: 'simple' })
|
||||
check('state persisted to disk', JSON.parse(fs.readFileSync(path.resolve(path.dirname(args.config!), config.stateFile), 'utf8')).mode === 'simple')
|
||||
|
||||
// ---------- correctness on hand-written cases
|
||||
const fim = (prefix: string, suffix: string, extra: Record<string, unknown> = {}, signal?: AbortSignal) =>
|
||||
api('POST', '/v1/fim', { repo: 'smoke', path: 'src/example.ts', languageId: 'typescript', prefix, suffix, ...extra }, signal)
|
||||
|
||||
const zod = await fim("import { z } from 'zod'\n\nexport const configSchema = z.object({\n port: z.coerce.number().default(3000),\n})\n\nexport type Config = ", '\n')
|
||||
check('zod type completion', zod.body.text.includes('z.infer<typeof configSchema>'), JSON.stringify(zod.body.text))
|
||||
|
||||
const obj = await fim('export interface Invoice {\n id: string\n memo: string\n paid: boolean\n}\n\nexport function createInvoice(memo = \'\'): Invoice {\n return {\n id: crypto.randomUUID(),\n ', '\n }\n}\n')
|
||||
check('multi-line object completion stops before the suffix', obj.body.text.includes('paid') && !obj.body.text.includes('}\n}'), JSON.stringify(obj.body.text))
|
||||
|
||||
const single = await fim('const total = items.reduce((sum, item) => sum + item.', ', 0)\nconsole.log(total)\n')
|
||||
check('single-line mode when code follows the cursor', single.body.singleLine === true && !single.body.text.includes('\n'), JSON.stringify(single.body.text))
|
||||
|
||||
// ---------- cancellation: a newer request cancels the in-flight one
|
||||
const slow = fim('export async function retry<T>(fn: () => Promise<T>, attempts = 3): Promise<T> {\n ', '\n}\n', { maxLines: 50 })
|
||||
await new Promise(r => setTimeout(r, 30))
|
||||
const fast = fim('const answer = ', '\n')
|
||||
const [slowRes, fastRes] = await Promise.all([slow, fast])
|
||||
check('newer request cancels in-flight one', slowRes.body.cancelled === true && fastRes.status === 200, `first=${JSON.stringify(slowRes.body).slice(0, 80)}`)
|
||||
|
||||
const aborter = new AbortController()
|
||||
const aborted = fim('function fib(n: number): number {\n ', '\n}\n', {}, aborter.signal).catch(e => ({ status: -1, body: { error: (e as Error).name } }))
|
||||
setTimeout(() => aborter.abort(), 20)
|
||||
check('client abort is handled', (await aborted).body.error === 'AbortError')
|
||||
check('daemon still healthy after aborts', (await api('GET', '/health')).body.ok === true)
|
||||
|
||||
// ---------- symbols mode + this.-repair
|
||||
// Needs a TS project with node_modules installed: SMOKE_TS_ROOT (project dir), SMOKE_TS_FILE (a .ts file in it,
|
||||
// relative) and SMOKE_TS_ANCHOR (text in that file to put the cursor before). Skipped when not set.
|
||||
const envRoot = process.env.SMOKE_TS_ROOT ? path.resolve(process.env.SMOKE_TS_ROOT) : ''
|
||||
if (envRoot && fs.existsSync(envRoot) && process.env.SMOKE_TS_FILE && process.env.SMOKE_TS_ANCHOR) {
|
||||
const file = process.env.SMOKE_TS_FILE
|
||||
const content = fs.readFileSync(path.join(envRoot, file), 'utf8')
|
||||
const at = content.indexOf(process.env.SMOKE_TS_ANCHOR)
|
||||
const req = { repo: path.basename(envRoot), path: file, workspaceRoot: envRoot, languageId: 'typescript', prefix: content.slice(0, at), suffix: content.slice(at), mode: 'symbols' }
|
||||
let first = (await api('POST', '/v1/fim', req)).body
|
||||
for (let i = 0; i < 20 && first.symbols?.status !== 'hit' && !(first.symbols?.items > 0); i++) {
|
||||
await new Promise(r => setTimeout(r, 250)) // language service still building the program
|
||||
first = (await api('POST', '/v1/fim', req)).body
|
||||
}
|
||||
check('symbols mode attaches language-service context', first.mode === 'symbols' && first.symbols?.items > 0, JSON.stringify(first.symbols))
|
||||
const again = (await api('POST', '/v1/fim', { ...req, prefix: req.prefix + 'x' })).body
|
||||
check('same anchor reuses cached context (KV-stable header)', again.symbols?.status === 'hit', JSON.stringify(again.symbols))
|
||||
const outside = (await api('POST', '/v1/fim', { ...req, workspaceRoot: undefined })).body
|
||||
check('symbols without a workspace falls back to simple', outside.mode === 'simple')
|
||||
|
||||
const cls = 'export class UserService {\n private users = new Map<string, string>()\n\n getUser(id: string) {\n return this.users.get(id)\n }\n\n printUser(id: string) {\n const user = getU'
|
||||
const repaired = (await api('POST', '/v1/fim', { repo: req.repo, path: 'src/probe-user-service.ts', workspaceRoot: envRoot, languageId: 'typescript', prefix: cls, suffix: '\n }\n}\n' })).body
|
||||
check('missing this. is repaired as a replacement', repaired.replaceBefore === 4 && repaired.text.startsWith('this.getU') && repaired.filterText?.startsWith('getU'), JSON.stringify({ text: repaired.text, replaceBefore: repaired.replaceBefore, filterText: repaired.filterText }))
|
||||
const local = (await api('POST', '/v1/fim', { repo: req.repo, path: 'src/probe-user-service.ts', workspaceRoot: envRoot, languageId: 'typescript', prefix: cls.replace('const user = getU', 'const getUsername = () => id\n const user = getU'), suffix: '\n }\n}\n' })).body
|
||||
check('no repair when a local matches', local.replaceBefore === 0, JSON.stringify(local.text))
|
||||
} else {
|
||||
console.log('SKIP symbols / this.-repair checks (set SMOKE_TS_ROOT, SMOKE_TS_FILE, SMOKE_TS_ANCHOR)')
|
||||
}
|
||||
|
||||
// ---------- edit recording endpoint
|
||||
const recorded = await api('POST', '/v1/edits', { events: [{ t: Date.now(), repo: 'smoke', path: 'src/a.ts', version: 2, changes: [{ offset: 0, length: 0, text: 'x' }], snapshot: 'x' }] })
|
||||
check('edit recordings are accepted', recorded.body.recorded === 1)
|
||||
|
||||
// ---------- warm typing latency on held-out code
|
||||
// Optional typing replay over held-out FIM samples from the (private) training workspace.
|
||||
const dataDir = path.resolve(process.env.SMOKE_DATA_DIR ?? '../../data')
|
||||
const testFim = path.join(dataDir, 'fim/test-repo.jsonl')
|
||||
if (fs.existsSync(testFim)) {
|
||||
const files = new Map(fs.readFileSync(path.join(dataDir, 'corpus/files.jsonl'), 'utf8').trim().split('\n').map(l => JSON.parse(l)).map(r => [r.id, r]))
|
||||
const rank = (id: string) => crypto.createHash('sha256').update(`smoke:${id}`).digest('hex')
|
||||
const samples = fs.readFileSync(testFim, 'utf8').trim().split('\n').map(l => JSON.parse(l))
|
||||
.filter(s => s.suffix_mode === 'full' && ['line-rest', 'statement', 'call-args'].includes(s.type) && s.middle.length >= 12 && !s.middle.includes('\n'))
|
||||
.sort((a, b) => rank(a.id).localeCompare(rank(b.id)))
|
||||
.slice(0, Number(args['typing-samples']))
|
||||
for (const mode of ['simple', 'symbols']) {
|
||||
await api('PUT', '/v1/config', { mode })
|
||||
const steps: { total: number; processed: number }[] = []
|
||||
let firstAtLocation: number[] = []
|
||||
for (const s of samples) {
|
||||
const content = files.get(s.file_id).content as string
|
||||
for (let typed = 0; typed <= s.middle.length; typed += 3) {
|
||||
const workspaceRoot = path.join(dataDir, 'ts-env', s.repo)
|
||||
const r = await api('POST', '/v1/fim', { repo: s.repo, path: s.path, workspaceRoot: fs.existsSync(workspaceRoot) ? workspaceRoot : undefined, languageId: s.lang, prefix: content.slice(0, s.start) + s.middle.slice(0, typed), suffix: content.slice(s.end) })
|
||||
if (typed === 0) { firstAtLocation.push(r.body.timings.totalMs); continue }
|
||||
steps.push({ total: r.body.timings.totalMs, processed: r.body.timings.processedTokens })
|
||||
}
|
||||
}
|
||||
const pct = (xs: number[], p: number) => [...xs].sort((a, b) => a - b)[Math.min(xs.length - 1, Math.floor(p * xs.length))]
|
||||
console.log(`\nwarm typing, mode=${mode} (${samples.length} completions, ${steps.length} keystroke requests, end-to-end through acd):`)
|
||||
console.log(` first request at a new location: p50 ${pct(firstAtLocation, 0.5)} ms, p90 ${pct(firstAtLocation, 0.9)} ms`)
|
||||
console.log(` per keystroke: total p50 ${pct(steps.map(s => s.total), 0.5)} ms, p90 ${pct(steps.map(s => s.total), 0.9)} ms; tokens re-processed p50 ${pct(steps.map(s => s.processed), 0.5)}, p90 ${pct(steps.map(s => s.processed), 0.9)}`)
|
||||
check(`${mode}: warm keystrokes reuse the KV cache (p50 re-processed tokens < 50)`, pct(steps.map(s => s.processed), 0.5) < 50)
|
||||
}
|
||||
await api('PUT', '/v1/config', { mode: 'simple' })
|
||||
}
|
||||
|
||||
console.log(failures ? `\n${failures} FAILED` : '\nall passed')
|
||||
cleanup()
|
||||
process.exit(failures ? 1 : 0)
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "es2023",
|
||||
"module": "nodenext",
|
||||
"moduleResolution": "nodenext",
|
||||
"strict": true,
|
||||
"noEmit": true,
|
||||
"allowImportingTsExtensions": true,
|
||||
"erasableSyntaxOnly": true,
|
||||
"verbatimModuleSyntax": true,
|
||||
"skipLibCheck": true,
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["src", "test", "../shared"]
|
||||
}
|
||||
Reference in New Issue
Block a user