Add Prettier formatting and Gitea Actions CI (format, typecheck, test, build)
CI / Node 20 (push) Successful in 1m49s
CI / Node 22 (push) Successful in 1m52s

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
This commit is contained in:
Claude
2026-10-23 07:29:38 +00:00
parent 5a370f8c78
commit e27541c46a
24 changed files with 341 additions and 209 deletions
+83 -74
View File
@@ -18,7 +18,7 @@ implementation decisions made where the brief was silent (section 12) and benchm
## 1. Principles
1. **Three layers**, each usable on its own: **KV** → **Event** (append-only, built on KV) → **Document** (Mongo-like, built on Events or native).
2. **Drivers vs stores.** Backends implement a tiny, flat, string-keyed *driver*. All path, child, index, escaping and typing logic lives once in the user-facing *node* classes, so every backend gets identical behaviour.
2. **Drivers vs stores.** Backends implement a tiny, flat, string-keyed _driver_. All path, child, index, escaping and typing logic lives once in the user-facing _node_ classes, so every backend gets identical behaviour.
3. **One navigation primitive: `child(key)`.** There are no collection/document classes. Every node is a store over its direct children, and `child(key)` re-roots to that key. Navigation is synchronous and does no I/O.
4. **No methods on data.** Reads return plain values.
5. **Performance is a requirement.** Batch operations are the primitive; scans are ordered and streamed; nothing replays or scans more than it must (section 8).
@@ -59,24 +59,22 @@ export interface ScanOptions {
lt?: string
limit?: number
reverse?: boolean
values?: boolean // when false/omitted, yield value as undefined
values?: boolean // when false/omitted, yield value as undefined
}
export interface KVDriver {
getMany(keys: string[]): Promise<(unknown | undefined)[]> // same order as keys
getMany(keys: string[]): Promise<(unknown | undefined)[]> // same order as keys
setMany(entries: [key: string, value: unknown][]): Promise<void>
deleteMany(keys: string[]): Promise<void>
scan(opts: ScanOptions): AsyncIterable<[key: string, value: unknown]> // MUST be key-ordered
scan(opts: ScanOptions): AsyncIterable<[key: string, value: unknown]> // MUST be key-ordered
// Optional capabilities. Stores feature-detect them and fall back where possible.
setIfAbsent?(entries: [string, unknown][]): Promise<boolean[]> // true = written
batch?(ops: BatchOp[]): Promise<void> // atomic multi-op write
setIfAbsent?(entries: [string, unknown][]): Promise<boolean[]> // true = written
batch?(ops: BatchOp[]): Promise<void> // atomic multi-op write
deleteRange?(gte: string, lt: string): Promise<void>
}
export type BatchOp =
| { type: 'set'; key: string; value: unknown }
| { type: 'delete'; key: string }
export type BatchOp = { type: 'set'; key: string; value: unknown } | { type: 'delete'; key: string }
```
- Values are passed through as JSON-serialisable values. Drivers that need bytes (HTTP, Redis, files) serialise internally via a pluggable `Codec` (default JSON).
@@ -94,29 +92,31 @@ const user = users.child('123')
API of every `KVNode` (types in section 7):
| Method | Meaning |
|---|---|
| `child(key)` | Sync. Returns a node at `path + key`. |
| `as<S>()` | Type assertion for untyped use; returns the same node retyped. |
| `get()` / `get(key)` | Own value at `selfKey` / value at child key. |
| `getMany(keys)` | Batch child read, same order, `undefined` for missing. |
| `set(value)` / `set(key, value)` | Write own value / child value. |
| `setMany(entries)` | Batch child write. |
| `delete()` / `delete(key)` | Delete own value / child value only (not descendants). |
| `deleteMany(keys)` | Batch child delete. |
| `deleteTree()` / `deleteTree(key)` | Delete own value **and all descendants** (uses `deleteRange` if available). |
| `keys(opts?)` | `AsyncIterable<string>` of **direct** child keys (unescaped). Options: `limit`, `after`, `reverse`. |
| `entries(opts?)` | `AsyncIterable<[key, value]>` of direct children that have an own value. |
| `tree({ depth })` | Assemble an object from descendant keys up to `depth` (for field-per-key style). Bounded and explicit; never the default. |
| `findBy(index, value \| { gte?, lt? }, opts?)` | Index lookup over direct children (section 3.3). |
| Method | Meaning |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `child(key)` | Sync. Returns a node at `path + key`. |
| `as<S>()` | Type assertion for untyped use; returns the same node retyped. |
| `get()` / `get(key)` | Own value at `selfKey` / value at child key. |
| `getMany(keys)` | Batch child read, same order, `undefined` for missing. |
| `set(value)` / `set(key, value)` | Write own value / child value. |
| `setMany(entries)` | Batch child write. |
| `delete()` / `delete(key)` | Delete own value / child value only (not descendants). |
| `deleteMany(keys)` | Batch child delete. |
| `deleteTree()` / `deleteTree(key)` | Delete own value **and all descendants** (uses `deleteRange` if available). |
| `keys(opts?)` | `AsyncIterable<string>` of **direct** child keys (unescaped). Options: `limit`, `after`, `reverse`. |
| `entries(opts?)` | `AsyncIterable<[key, value]>` of direct children that have an own value. |
| `tree({ depth })` | Assemble an object from descendant keys up to `depth` (for field-per-key style). Bounded and explicit; never the default. |
| `findBy(index, value \| { gte?, lt? }, opts?)` | Index lookup over direct children (section 3.3). |
**Arity dispatch (important):**
- Self vs child is decided by `arguments.length`, never by checking for `undefined`.
- If a key argument is passed but is `undefined` or `null`, throw a `TypeError` (this prevents `users.get(maybeId)` silently reading self).
- The root node (`path = []`) has no own value: `get()`, `set(v)` and `delete()` on root throw.
- Implement one private `#resolve(args)` that every method uses.
**Direct-children listing (`keys`)** uses a skip-scan:
1. Scan `gte: prefix`, `lt: prefix + succ(sep)`.
2. For each key, take the first segment after `prefix`. Skip reserved (`~…`) segments. Yield the segment (unescaped) if it differs from the previous one.
3. When the next key is a descendant of the segment just yielded, re-seek the driver to `gte: prefix + escSeg + succ(sep)` (where `succ(c)` is the next char code) instead of iterating through the subtree. Keep using the same iterator while keys are siblings.
@@ -130,8 +130,8 @@ Indexes are declared on **path patterns** in the `KVStorage` constructor. A patt
```ts
new KVStorage(driver, {
indexes: {
'users': { byEmail: (u: User) => u.email },
'users/*/posts': { byDate: (p: Post) => p.createdAt, byTag: (p: Post) => p.tags },
users: { byEmail: (u: User) => u.email },
'users/*/posts': { byDate: (p: Post) => p.createdAt, byTag: (p: Post) => p.tags },
},
})
```
@@ -155,14 +155,17 @@ new KVStorage(driver, {
An append-only log per stream. A stream is identified by a node path. Same `child()` navigation as KV.
```ts
const events = new EventStorage(kv) // takes a KVStorage (or a KVNode as root)
const events = new EventStorage(kv) // takes a KVStorage (or a KVNode as root)
const orders = events.child('orders')
await orders.append('o1', [{ type: 'created', total: 40 }], { expectedVersion: 0 })
await orders.child('o1').append([{ type: 'paid' }]) // self form
for await (const e of orders.read('o1', { from: 1 })) { /* { version, commit, at, data } */ }
await orders.child('o1').append([{ type: 'paid' }]) // self form
for await (const e of orders.read('o1', { from: 1 })) {
/* { version, commit, at, data } */
}
await orders.version('o1')
for await (const id of orders.keys()) { } // child streams
for await (const id of orders.keys()) {
} // child streams
orders.child('o1').child('shipments')
```
@@ -186,23 +189,25 @@ orders.child('o1').child('shipments')
## 5. Layer 3: Documents (`src/documents/`)
`DocumentStorage` is the interface. There are two implementations:
- `EventSourcedDocumentStorage(events, { projections?: KVNode })` is the v1 target. Projections default to the `~p` root on the same KV.
- `MongoDocumentStorage(client)` is a later milestone (section 9).
```ts
const docs = new EventSourcedDocumentStorage(events, { indexes: { 'users': [{ fields: ['email'], unique: true }, { fields: ['age'] }] } })
const docs = new EventSourcedDocumentStorage(events, { indexes: { users: [{ fields: ['email'], unique: true }, { fields: ['age'] }] } })
const users = docs.child('users')
const id = await users.insert({ name: 'Ada', age: 36 }) // generates a time-sortable id
const id = await users.insert({ name: 'Ada', age: 36 }) // generates a time-sortable id
await users.set('123', { name: 'Ada', age: 36 })
await users.update('123', { $set: { name: 'Ada L.' }, $inc: { logins: 1 } })
await users.child('123').update({ $unset: { nickname: true } }) // self form
await users.child('123').update({ $unset: { nickname: true } }) // self form
await users.get('123')
await users.delete('123')
for await (const { key, data } of users.find({ age: { $gte: 18 } }, { sort: { age: 1 }, limit: 20 })) { }
for await (const { key, data } of users.find({ age: { $gte: 18 } }, { sort: { age: 1 }, limit: 20 })) {
}
await users.findOne({ email: 'ada@x.io' })
await users.history('123') // event-sourced only
await users.getAt('123', 17) // state as of version 17
await users.history('123') // event-sourced only
await users.getAt('123', 17) // state as of version 17
users.child('123').child('posts') // nested docs, same as KV
```
@@ -214,6 +219,7 @@ users.child('123').child('posts') // nested docs, same as KV
- Supported update operators in v1: `$set`, `$unset`, `$inc`, `$push`, `$pull`, `$addToSet`, with dot-path support.
**Write path:**
1. Read the projection.
2. Compute the new state and patch.
3. `append` with `expectedVersion = projection.version`.
@@ -223,6 +229,7 @@ users.child('123').child('posts') // nested docs, same as KV
**Reads** are a single projection `get`. Never replay on read.
**Crash between append and projection write:**
- The projection lags the stream. The next write's `expectedVersion` fails, which triggers catch-up, so the system self-heals.
- Provide `get(key, { consistent: true })` that checks the stream head and catches up first.
- Provide a `catchUp(node?)` maintenance method to repair a subtree.
@@ -257,16 +264,15 @@ Each node is typed by a single **Spec** that describes its own value and its chi
```ts
export type Spec = { value?: unknown; children?: Record<string, Spec> }
export type AnySpec = { value: unknown; children: Record<string, AnySpec> } // untyped default
export type AnySpec = { value: unknown; children: Record<string, AnySpec> } // untyped default
export type ValueOf<S> = S extends { value: infer V } ? V : never
export type ValueOf<S> = S extends { value: infer V } ? V : never
export type ChildrenOf<S> = S extends { children: infer C } ? C : {}
export type KeyOf<S> = Extract<keyof ChildrenOf<S>, string | number>
export type KeyOf<S> = Extract<keyof ChildrenOf<S>, string | number>
export type ChildOf<S, K extends KeyOf<S>> = ChildrenOf<S>[K]
// Helpers
export type Many<V, C extends Record<string, Spec> = {}> =
{ children: { [id: string]: { value: V; children: C } } }
export type Many<V, C extends Record<string, Spec> = {}> = { children: { [id: string]: { value: V; children: C } } }
export type Fields<T> = { [K in keyof T & string]: { value: T[K] } }
```
@@ -286,21 +292,24 @@ Example to cover with `expectTypeOf` tests:
```ts
type User = { username: string; age: number }
type Post = { title: string; createdAt: string }
type Schema = { children: {
users: Many<User, { posts: Many<Post> }>
profile: { children: { [id: string]: { children: Fields<User> } } } // field-per-key style
} }
type Schema = {
children: {
users: Many<User, { posts: Many<Post> }>
profile: { children: { [id: string]: { children: Fields<User> } } } // field-per-key style
}
}
const db = new KVStorage<Schema>(driver)
db.child('users').get('123') // User | undefined
db.child('users').child('123').get() // User | undefined (same key)
db.child('users').child('123').child('posts') // KVNode<Many<Post>>
db.child('users').child('123').child('postz') // ✗ type error
db.child('profile').child('9').get('age') // number | undefined
db.child('profile').child('9').get('agee') // ✗ type error
db.child('users').get('123') // User | undefined
db.child('users').child('123').get() // User | undefined (same key)
db.child('users').child('123').child('posts') // KVNode<Many<Post>>
db.child('users').child('123').child('postz') // ✗ type error
db.child('profile').child('9').get('age') // number | undefined
db.child('profile').child('9').get('agee') // ✗ type error
```
Notes:
- **Named levels** check keys at compile time; **dynamic `[id: string]` levels** accept any string.
- One node's children can't mix an index signature with named keys (a TS restriction), so schemas alternate between dynamic and named levels. That alternation exists only in types; the runtime API is just `child()`.
- `EventStorage` and `DocumentStorage` nodes use the same Spec machinery (the value type is the event type or the document type respectively).
@@ -383,7 +392,7 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
### M2 — KV driver
- **Conformance suite location**: `src/testing/kv-conformance.ts`, exported as `potential-storage/testing`.
It imports `vitest`, which is an *optional peer dependency*; the core entry point never imports it.
It imports `vitest`, which is an _optional peer dependency_; the core entry point never imports it.
- **`InMemoryKVDriver` clones values** on write and read (`structuredClone`, skipped for primitives) so
stored state can't be mutated through shared references, matching serialising drivers. Opt out
with `{ clone: false }`.
@@ -402,7 +411,7 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
- **Child order is "subtree order", not plain segment order.** The brief's skip-scan assumes a
child's keys are contiguous, but they aren't when a sibling extends a segment with a character
that sorts below the separator (for `/`: space and `!"#$%&'()*+,-.`). With children `a` and `a-1`
the key space is `p/a`, `p/a-1`, `p/a-1/x`, `p/a/x`, so `a`'s descendants sort *after* `a-1`, and
the key space is `p/a`, `p/a-1`, `p/a-1/x`, `p/a/x`, so `a`'s descendants sort _after_ `a-1`, and
"yield if different from the previous segment" would emit `a` twice. `keys()`/`entries()` therefore
order children by **escaped segment followed by the separator** (`a-1/` < `a/`), which is the order
of their subtrees in key space. For keys without such characters this is plain lexicographic order.
@@ -541,7 +550,7 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
field, both for index-ordered and in-memory sorts.
- **In-memory sort** keeps a bounded top `skip + limit` buffer when a limit is given. Without a limit,
it buffers all matches; this is the one place results are materialised.
- **Full-scan warning**: `console.warn` once per filter *shape* (values replaced by `?`) per storage
- **Full-scan warning**: `console.warn` once per filter _shape_ (values replaced by `?`) per storage
instance, only for non-empty filters. On by default unless `NODE_ENV === 'production'`; set it with
`warnOnFullScan`.
- **Catch-up** re-asserts unique claims for the caught-up state with plain writes, since the log is
@@ -609,37 +618,37 @@ container (shared CPU, so expect ±10–20% run-to-run). Mean time per operation
**KV** (`bench/kv.bench.ts`): values are small objects.
| Operation | 10k keys | 100k keys |
|---|---|---|
| `get` | 1.2 µs | 1.6 µs |
| `set` (overwrite) | 1.7 µs | 2.0 µs |
| `getMany` × 100 keys | 21 µs | 22 µs |
| Operation | 10k keys | 100k keys |
| -------------------- | -------- | --------- |
| `get` | 1.2 µs | 1.6 µs |
| `set` (overwrite) | 1.7 µs | 2.0 µs |
| `getMany` × 100 keys | 21 µs | 22 µs |
**`keys()`** (`bench/keys.bench.ts`): node with 1 000 children, each with 100 descendants (101k keys).
| Operation | Mean |
|---|---|
| Operation | Mean |
| -------------------------------- | ------ |
| `keys()` skip-scan (1 000 seeks) | 2.9 ms |
| Reference: full subtree scan | 36 ms |
| Reference: full subtree scan | 36 ms |
**Events** (`bench/events.bench.ts`): single-event commits.
| Operation | Mean |
|---|---|
| `append` 1 event (warm head cache) | 6.3 µs |
| Operation | Mean |
| ---------------------------------------------- | ------ |
| `append` 1 event (warm head cache) | 6.3 µs |
| `read` 100 events from version 5 000 of 10 000 | 174 µs |
| `read` full 10 000-event stream | 21 ms |
| `version()` (reverse scan, `limit: 1`) | 3.9 µs |
| `read` full 10 000-event stream | 21 ms |
| `version()` (reverse scan, `limit: 1`) | 3.9 µs |
**Documents** (`bench/documents.bench.ts`): 10 000 docs, indexes on `age` and unique `email`.
| Operation | Mean |
|---|---|
| `get` | 2.6 µs |
| `update` (`$inc`, indexed node: projection read + append + batch) | 22 µs |
| `findOne` by unique index (`email`) | 16 µs |
| `find` indexed (`age = k`, ~100 hits) | 0.57 ms |
| `find` full scan (`city = k`, ~200 hits of 10k) | 28 ms |
| Operation | Mean |
| ----------------------------------------------------------------- | ------- |
| `get` | 2.6 µs |
| `update` (`$inc`, indexed node: projection read + append + batch) | 22 µs |
| `findOne` by unique index (`email`) | 16 µs |
| `find` indexed (`age = k`, ~100 hits) | 0.57 ms |
| `find` full scan (`city = k`, ~200 hits of 10k) | 28 ms |
For comparison, the first run with the sorted-array map and `structuredClone` measured: KV `get`
3.7 µs, `getMany`×100 186 µs, `append` 34 µs, document `update` 110 µs, full scan 57 ms, and