M9: benchmarks, chunked sorted map and JSON clone for the in-memory driver
Profiling showed document updates dominated by O(n) sorted-array splices; the chunked map makes inserts O(log n + chunk). Baselines recorded in DESIGN.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
This commit is contained in:
@@ -589,8 +589,58 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
|
||||
socket, and against a minimal server driver with discovered capabilities. A smoke test runs KV
|
||||
indexes, events and unique-indexed documents over HTTP.
|
||||
|
||||
### M9 — Benchmarks and in-memory driver performance
|
||||
|
||||
- **The in-memory driver's sorted array was replaced by `ChunkedSortedMap`** (sorted chunks of ≤1024
|
||||
keys, split in half on overflow; values in a `Map`). Profiling showed 59% of document-update time
|
||||
in the sorted array's O(n) `splice` once the store held ~70k keys (each write adds commit, index
|
||||
and claim keys). Inserts and deletes are now O(log n + 1024). Iteration is O(1) per step and
|
||||
re-seeks after a mutation, which replaces the old version-counter revalidation in the driver.
|
||||
The structure stays behind the `SortedMap` interface; a B-tree remains possible.
|
||||
- **Value cloning uses a hand-written JSON deep copy** instead of `structuredClone` (about 10× cheaper
|
||||
per value), in the in-memory driver and the document layer. Values are JSON by contract (§10).
|
||||
|
||||
<!-- decisions appended per milestone -->
|
||||
|
||||
## 13. Benchmark baselines
|
||||
|
||||
_Filled in at milestone 9._
|
||||
`npm run bench` (Vitest bench), in-memory driver with default cloning, Node 22.22, Linux x64 cloud
|
||||
container (shared CPU, so expect ±10–20% run-to-run). Mean time per operation.
|
||||
|
||||
**KV** (`bench/kv.bench.ts`): values are small objects.
|
||||
|
||||
| Operation | 10k keys | 100k keys |
|
||||
|---|---|---|
|
||||
| `get` | 1.2 µs | 1.6 µs |
|
||||
| `set` (overwrite) | 1.7 µs | 2.0 µs |
|
||||
| `getMany` × 100 keys | 21 µs | 22 µs |
|
||||
|
||||
**`keys()`** (`bench/keys.bench.ts`): node with 1 000 children, each with 100 descendants (101k keys).
|
||||
|
||||
| Operation | Mean |
|
||||
|---|---|
|
||||
| `keys()` skip-scan (1 000 seeks) | 2.9 ms |
|
||||
| Reference: full subtree scan | 36 ms |
|
||||
|
||||
**Events** (`bench/events.bench.ts`): single-event commits.
|
||||
|
||||
| Operation | Mean |
|
||||
|---|---|
|
||||
| `append` 1 event (warm head cache) | 6.3 µs |
|
||||
| `read` 100 events from version 5 000 of 10 000 | 174 µs |
|
||||
| `read` full 10 000-event stream | 21 ms |
|
||||
| `version()` (reverse scan, `limit: 1`) | 3.9 µs |
|
||||
|
||||
**Documents** (`bench/documents.bench.ts`): 10 000 docs, indexes on `age` and unique `email`.
|
||||
|
||||
| Operation | Mean |
|
||||
|---|---|
|
||||
| `get` | 2.6 µs |
|
||||
| `update` (`$inc`, indexed node: projection read + append + batch) | 22 µs |
|
||||
| `findOne` by unique index (`email`) | 16 µs |
|
||||
| `find` indexed (`age = k`, ~100 hits) | 0.57 ms |
|
||||
| `find` full scan (`city = k`, ~200 hits of 10k) | 28 ms |
|
||||
|
||||
For comparison, the first run with the sorted-array map and `structuredClone` measured: KV `get`
|
||||
3.7 µs, `getMany`×100 186 µs, `append` 34 µs, document `update` 110 µs, full scan 57 ms, and
|
||||
full stream read 62 ms.
|
||||
|
||||
Reference in New Issue
Block a user