Add Prettier formatting and Gitea Actions CI (format, typecheck, test, build)
CI / Node 20 (push) Successful in 1m49s
CI / Node 22 (push) Successful in 1m52s

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
This commit is contained in:
Claude
2026-10-23 07:29:38 +00:00
parent 5a370f8c78
commit e27541c46a
24 changed files with 341 additions and 209 deletions
+30
View File
@@ -0,0 +1,30 @@
name: CI
on:
push:
branches: [main]
pull_request:
jobs:
check:
name: Node ${{ matrix.node }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node: [20, 22]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: npm
- run: npm ci
- name: Format
run: npm run format:check
- name: Typecheck
run: npm run typecheck
- name: Test
run: npm run test:only
- name: Build
run: npm run build
+4
View File
@@ -0,0 +1,4 @@
node_modules
dist
coverage
package-lock.json
+6
View File
@@ -0,0 +1,6 @@
{
"semi": false,
"singleQuote": true,
"printWidth": 160,
"trailingComma": "all"
}
+3
View File
@@ -11,6 +11,9 @@ Record new decisions in DESIGN.md section 12.
- `npm run typecheck` — `tsc --noEmit` over src, test and bench - `npm run typecheck` — `tsc --noEmit` over src, test and bench
- `npm run bench` — `vitest bench` - `npm run bench` — `vitest bench`
- `npm run build` — emit `dist/` - `npm run build` — emit `dist/`
- `npm run format` / `npm run format:check` — Prettier
CI (`.gitea/workflows/ci.yml`, Gitea Actions) runs format check, typecheck, tests and build on Node 20 and 22.
## Rules ## Rules
+30 -21
View File
@@ -18,7 +18,7 @@ implementation decisions made where the brief was silent (section 12) and benchm
## 1. Principles ## 1. Principles
1. **Three layers**, each usable on its own: **KV** → **Event** (append-only, built on KV) → **Document** (Mongo-like, built on Events or native). 1. **Three layers**, each usable on its own: **KV** → **Event** (append-only, built on KV) → **Document** (Mongo-like, built on Events or native).
2. **Drivers vs stores.** Backends implement a tiny, flat, string-keyed *driver*. All path, child, index, escaping and typing logic lives once in the user-facing *node* classes, so every backend gets identical behaviour. 2. **Drivers vs stores.** Backends implement a tiny, flat, string-keyed _driver_. All path, child, index, escaping and typing logic lives once in the user-facing _node_ classes, so every backend gets identical behaviour.
3. **One navigation primitive: `child(key)`.** There are no collection/document classes. Every node is a store over its direct children, and `child(key)` re-roots to that key. Navigation is synchronous and does no I/O. 3. **One navigation primitive: `child(key)`.** There are no collection/document classes. Every node is a store over its direct children, and `child(key)` re-roots to that key. Navigation is synchronous and does no I/O.
4. **No methods on data.** Reads return plain values. 4. **No methods on data.** Reads return plain values.
5. **Performance is a requirement.** Batch operations are the primitive; scans are ordered and streamed; nothing replays or scans more than it must (section 8). 5. **Performance is a requirement.** Batch operations are the primitive; scans are ordered and streamed; nothing replays or scans more than it must (section 8).
@@ -74,9 +74,7 @@ export interface KVDriver {
deleteRange?(gte: string, lt: string): Promise<void> deleteRange?(gte: string, lt: string): Promise<void>
} }
export type BatchOp = export type BatchOp = { type: 'set'; key: string; value: unknown } | { type: 'delete'; key: string }
| { type: 'set'; key: string; value: unknown }
| { type: 'delete'; key: string }
``` ```
- Values are passed through as JSON-serialisable values. Drivers that need bytes (HTTP, Redis, files) serialise internally via a pluggable `Codec` (default JSON). - Values are passed through as JSON-serialisable values. Drivers that need bytes (HTTP, Redis, files) serialise internally via a pluggable `Codec` (default JSON).
@@ -95,7 +93,7 @@ const user = users.child('123')
API of every `KVNode` (types in section 7): API of every `KVNode` (types in section 7):
| Method | Meaning | | Method | Meaning |
|---|---| | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `child(key)` | Sync. Returns a node at `path + key`. | | `child(key)` | Sync. Returns a node at `path + key`. |
| `as<S>()` | Type assertion for untyped use; returns the same node retyped. | | `as<S>()` | Type assertion for untyped use; returns the same node retyped. |
| `get()` / `get(key)` | Own value at `selfKey` / value at child key. | | `get()` / `get(key)` | Own value at `selfKey` / value at child key. |
@@ -111,12 +109,14 @@ API of every `KVNode` (types in section 7):
| `findBy(index, value \| { gte?, lt? }, opts?)` | Index lookup over direct children (section 3.3). | | `findBy(index, value \| { gte?, lt? }, opts?)` | Index lookup over direct children (section 3.3). |
**Arity dispatch (important):** **Arity dispatch (important):**
- Self vs child is decided by `arguments.length`, never by checking for `undefined`. - Self vs child is decided by `arguments.length`, never by checking for `undefined`.
- If a key argument is passed but is `undefined` or `null`, throw a `TypeError` (this prevents `users.get(maybeId)` silently reading self). - If a key argument is passed but is `undefined` or `null`, throw a `TypeError` (this prevents `users.get(maybeId)` silently reading self).
- The root node (`path = []`) has no own value: `get()`, `set(v)` and `delete()` on root throw. - The root node (`path = []`) has no own value: `get()`, `set(v)` and `delete()` on root throw.
- Implement one private `#resolve(args)` that every method uses. - Implement one private `#resolve(args)` that every method uses.
**Direct-children listing (`keys`)** uses a skip-scan: **Direct-children listing (`keys`)** uses a skip-scan:
1. Scan `gte: prefix`, `lt: prefix + succ(sep)`. 1. Scan `gte: prefix`, `lt: prefix + succ(sep)`.
2. For each key, take the first segment after `prefix`. Skip reserved (`~…`) segments. Yield the segment (unescaped) if it differs from the previous one. 2. For each key, take the first segment after `prefix`. Skip reserved (`~…`) segments. Yield the segment (unescaped) if it differs from the previous one.
3. When the next key is a descendant of the segment just yielded, re-seek the driver to `gte: prefix + escSeg + succ(sep)` (where `succ(c)` is the next char code) instead of iterating through the subtree. Keep using the same iterator while keys are siblings. 3. When the next key is a descendant of the segment just yielded, re-seek the driver to `gte: prefix + escSeg + succ(sep)` (where `succ(c)` is the next char code) instead of iterating through the subtree. Keep using the same iterator while keys are siblings.
@@ -130,7 +130,7 @@ Indexes are declared on **path patterns** in the `KVStorage` constructor. A patt
```ts ```ts
new KVStorage(driver, { new KVStorage(driver, {
indexes: { indexes: {
'users': { byEmail: (u: User) => u.email }, users: { byEmail: (u: User) => u.email },
'users/*/posts': { byDate: (p: Post) => p.createdAt, byTag: (p: Post) => p.tags }, 'users/*/posts': { byDate: (p: Post) => p.createdAt, byTag: (p: Post) => p.tags },
}, },
}) })
@@ -160,9 +160,12 @@ const orders = events.child('orders')
await orders.append('o1', [{ type: 'created', total: 40 }], { expectedVersion: 0 }) await orders.append('o1', [{ type: 'created', total: 40 }], { expectedVersion: 0 })
await orders.child('o1').append([{ type: 'paid' }]) // self form await orders.child('o1').append([{ type: 'paid' }]) // self form
for await (const e of orders.read('o1', { from: 1 })) { /* { version, commit, at, data } */ } for await (const e of orders.read('o1', { from: 1 })) {
/* { version, commit, at, data } */
}
await orders.version('o1') await orders.version('o1')
for await (const id of orders.keys()) { } // child streams for await (const id of orders.keys()) {
} // child streams
orders.child('o1').child('shipments') orders.child('o1').child('shipments')
``` ```
@@ -186,11 +189,12 @@ orders.child('o1').child('shipments')
## 5. Layer 3: Documents (`src/documents/`) ## 5. Layer 3: Documents (`src/documents/`)
`DocumentStorage` is the interface. There are two implementations: `DocumentStorage` is the interface. There are two implementations:
- `EventSourcedDocumentStorage(events, { projections?: KVNode })` is the v1 target. Projections default to the `~p` root on the same KV. - `EventSourcedDocumentStorage(events, { projections?: KVNode })` is the v1 target. Projections default to the `~p` root on the same KV.
- `MongoDocumentStorage(client)` is a later milestone (section 9). - `MongoDocumentStorage(client)` is a later milestone (section 9).
```ts ```ts
const docs = new EventSourcedDocumentStorage(events, { indexes: { 'users': [{ fields: ['email'], unique: true }, { fields: ['age'] }] } }) const docs = new EventSourcedDocumentStorage(events, { indexes: { users: [{ fields: ['email'], unique: true }, { fields: ['age'] }] } })
const users = docs.child('users') const users = docs.child('users')
const id = await users.insert({ name: 'Ada', age: 36 }) // generates a time-sortable id const id = await users.insert({ name: 'Ada', age: 36 }) // generates a time-sortable id
@@ -199,7 +203,8 @@ await users.update('123', { $set: { name: 'Ada L.' }, $inc: { logins: 1 } })
await users.child('123').update({ $unset: { nickname: true } }) // self form await users.child('123').update({ $unset: { nickname: true } }) // self form
await users.get('123') await users.get('123')
await users.delete('123') await users.delete('123')
for await (const { key, data } of users.find({ age: { $gte: 18 } }, { sort: { age: 1 }, limit: 20 })) { } for await (const { key, data } of users.find({ age: { $gte: 18 } }, { sort: { age: 1 }, limit: 20 })) {
}
await users.findOne({ email: 'ada@x.io' }) await users.findOne({ email: 'ada@x.io' })
await users.history('123') // event-sourced only await users.history('123') // event-sourced only
await users.getAt('123', 17) // state as of version 17 await users.getAt('123', 17) // state as of version 17
@@ -214,6 +219,7 @@ users.child('123').child('posts') // nested docs, same as KV
- Supported update operators in v1: `$set`, `$unset`, `$inc`, `$push`, `$pull`, `$addToSet`, with dot-path support. - Supported update operators in v1: `$set`, `$unset`, `$inc`, `$push`, `$pull`, `$addToSet`, with dot-path support.
**Write path:** **Write path:**
1. Read the projection. 1. Read the projection.
2. Compute the new state and patch. 2. Compute the new state and patch.
3. `append` with `expectedVersion = projection.version`. 3. `append` with `expectedVersion = projection.version`.
@@ -223,6 +229,7 @@ users.child('123').child('posts') // nested docs, same as KV
**Reads** are a single projection `get`. Never replay on read. **Reads** are a single projection `get`. Never replay on read.
**Crash between append and projection write:** **Crash between append and projection write:**
- The projection lags the stream. The next write's `expectedVersion` fails, which triggers catch-up, so the system self-heals. - The projection lags the stream. The next write's `expectedVersion` fails, which triggers catch-up, so the system self-heals.
- Provide `get(key, { consistent: true })` that checks the stream head and catches up first. - Provide `get(key, { consistent: true })` that checks the stream head and catches up first.
- Provide a `catchUp(node?)` maintenance method to repair a subtree. - Provide a `catchUp(node?)` maintenance method to repair a subtree.
@@ -265,8 +272,7 @@ export type KeyOf<S> = Extract<keyof ChildrenOf<S>, string | number>
export type ChildOf<S, K extends KeyOf<S>> = ChildrenOf<S>[K] export type ChildOf<S, K extends KeyOf<S>> = ChildrenOf<S>[K]
// Helpers // Helpers
export type Many<V, C extends Record<string, Spec> = {}> = export type Many<V, C extends Record<string, Spec> = {}> = { children: { [id: string]: { value: V; children: C } } }
{ children: { [id: string]: { value: V; children: C } } }
export type Fields<T> = { [K in keyof T & string]: { value: T[K] } } export type Fields<T> = { [K in keyof T & string]: { value: T[K] } }
``` ```
@@ -286,10 +292,12 @@ Example to cover with `expectTypeOf` tests:
```ts ```ts
type User = { username: string; age: number } type User = { username: string; age: number }
type Post = { title: string; createdAt: string } type Post = { title: string; createdAt: string }
type Schema = { children: { type Schema = {
children: {
users: Many<User, { posts: Many<Post> }> users: Many<User, { posts: Many<Post> }>
profile: { children: { [id: string]: { children: Fields<User> } } } // field-per-key style profile: { children: { [id: string]: { children: Fields<User> } } } // field-per-key style
} } }
}
const db = new KVStorage<Schema>(driver) const db = new KVStorage<Schema>(driver)
db.child('users').get('123') // User | undefined db.child('users').get('123') // User | undefined
@@ -301,6 +309,7 @@ db.child('profile').child('9').get('agee') // ✗ type error
``` ```
Notes: Notes:
- **Named levels** check keys at compile time; **dynamic `[id: string]` levels** accept any string. - **Named levels** check keys at compile time; **dynamic `[id: string]` levels** accept any string.
- One node's children can't mix an index signature with named keys (a TS restriction), so schemas alternate between dynamic and named levels. That alternation exists only in types; the runtime API is just `child()`. - One node's children can't mix an index signature with named keys (a TS restriction), so schemas alternate between dynamic and named levels. That alternation exists only in types; the runtime API is just `child()`.
- `EventStorage` and `DocumentStorage` nodes use the same Spec machinery (the value type is the event type or the document type respectively). - `EventStorage` and `DocumentStorage` nodes use the same Spec machinery (the value type is the event type or the document type respectively).
@@ -383,7 +392,7 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
### M2 — KV driver ### M2 — KV driver
- **Conformance suite location**: `src/testing/kv-conformance.ts`, exported as `potential-storage/testing`. - **Conformance suite location**: `src/testing/kv-conformance.ts`, exported as `potential-storage/testing`.
It imports `vitest`, which is an *optional peer dependency*; the core entry point never imports it. It imports `vitest`, which is an _optional peer dependency_; the core entry point never imports it.
- **`InMemoryKVDriver` clones values** on write and read (`structuredClone`, skipped for primitives) so - **`InMemoryKVDriver` clones values** on write and read (`structuredClone`, skipped for primitives) so
stored state can't be mutated through shared references, matching serialising drivers. Opt out stored state can't be mutated through shared references, matching serialising drivers. Opt out
with `{ clone: false }`. with `{ clone: false }`.
@@ -402,7 +411,7 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
- **Child order is "subtree order", not plain segment order.** The brief's skip-scan assumes a - **Child order is "subtree order", not plain segment order.** The brief's skip-scan assumes a
child's keys are contiguous, but they aren't when a sibling extends a segment with a character child's keys are contiguous, but they aren't when a sibling extends a segment with a character
that sorts below the separator (for `/`: space and `!"#$%&'()*+,-.`). With children `a` and `a-1` that sorts below the separator (for `/`: space and `!"#$%&'()*+,-.`). With children `a` and `a-1`
the key space is `p/a`, `p/a-1`, `p/a-1/x`, `p/a/x`, so `a`'s descendants sort *after* `a-1`, and the key space is `p/a`, `p/a-1`, `p/a-1/x`, `p/a/x`, so `a`'s descendants sort _after_ `a-1`, and
"yield if different from the previous segment" would emit `a` twice. `keys()`/`entries()` therefore "yield if different from the previous segment" would emit `a` twice. `keys()`/`entries()` therefore
order children by **escaped segment followed by the separator** (`a-1/` < `a/`), which is the order order children by **escaped segment followed by the separator** (`a-1/` < `a/`), which is the order
of their subtrees in key space. For keys without such characters this is plain lexicographic order. of their subtrees in key space. For keys without such characters this is plain lexicographic order.
@@ -541,7 +550,7 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
field, both for index-ordered and in-memory sorts. field, both for index-ordered and in-memory sorts.
- **In-memory sort** keeps a bounded top `skip + limit` buffer when a limit is given. Without a limit, - **In-memory sort** keeps a bounded top `skip + limit` buffer when a limit is given. Without a limit,
it buffers all matches; this is the one place results are materialised. it buffers all matches; this is the one place results are materialised.
- **Full-scan warning**: `console.warn` once per filter *shape* (values replaced by `?`) per storage - **Full-scan warning**: `console.warn` once per filter _shape_ (values replaced by `?`) per storage
instance, only for non-empty filters. On by default unless `NODE_ENV === 'production'`; set it with instance, only for non-empty filters. On by default unless `NODE_ENV === 'production'`; set it with
`warnOnFullScan`. `warnOnFullScan`.
- **Catch-up** re-asserts unique claims for the caught-up state with plain writes, since the log is - **Catch-up** re-asserts unique claims for the caught-up state with plain writes, since the log is
@@ -610,7 +619,7 @@ container (shared CPU, so expect ±10–20% run-to-run). Mean time per operation
**KV** (`bench/kv.bench.ts`): values are small objects. **KV** (`bench/kv.bench.ts`): values are small objects.
| Operation | 10k keys | 100k keys | | Operation | 10k keys | 100k keys |
|---|---|---| | -------------------- | -------- | --------- |
| `get` | 1.2 µs | 1.6 µs | | `get` | 1.2 µs | 1.6 µs |
| `set` (overwrite) | 1.7 µs | 2.0 µs | | `set` (overwrite) | 1.7 µs | 2.0 µs |
| `getMany` × 100 keys | 21 µs | 22 µs | | `getMany` × 100 keys | 21 µs | 22 µs |
@@ -618,14 +627,14 @@ container (shared CPU, so expect ±10–20% run-to-run). Mean time per operation
**`keys()`** (`bench/keys.bench.ts`): node with 1 000 children, each with 100 descendants (101k keys). **`keys()`** (`bench/keys.bench.ts`): node with 1 000 children, each with 100 descendants (101k keys).
| Operation | Mean | | Operation | Mean |
|---|---| | -------------------------------- | ------ |
| `keys()` skip-scan (1 000 seeks) | 2.9 ms | | `keys()` skip-scan (1 000 seeks) | 2.9 ms |
| Reference: full subtree scan | 36 ms | | Reference: full subtree scan | 36 ms |
**Events** (`bench/events.bench.ts`): single-event commits. **Events** (`bench/events.bench.ts`): single-event commits.
| Operation | Mean | | Operation | Mean |
|---|---| | ---------------------------------------------- | ------ |
| `append` 1 event (warm head cache) | 6.3 µs | | `append` 1 event (warm head cache) | 6.3 µs |
| `read` 100 events from version 5 000 of 10 000 | 174 µs | | `read` 100 events from version 5 000 of 10 000 | 174 µs |
| `read` full 10 000-event stream | 21 ms | | `read` full 10 000-event stream | 21 ms |
@@ -634,7 +643,7 @@ container (shared CPU, so expect ±10–20% run-to-run). Mean time per operation
**Documents** (`bench/documents.bench.ts`): 10 000 docs, indexes on `age` and unique `email`. **Documents** (`bench/documents.bench.ts`): 10 000 docs, indexes on `age` and unique `email`.
| Operation | Mean | | Operation | Mean |
|---|---| | ----------------------------------------------------------------- | ------- |
| `get` | 2.6 µs | | `get` | 2.6 µs |
| `update` (`$inc`, indexed node: projection read + append + batch) | 22 µs | | `update` (`$inc`, indexed node: projection read + append + batch) | 22 µs |
| `findOne` by unique index (`email`) | 16 µs | | `findOne` by unique index (`email`) | 16 µs |
+25
View File
@@ -10,11 +10,20 @@
"devDependencies": { "devDependencies": {
"@types/node": "^22.20.5", "@types/node": "^22.20.5",
"fast-check": "^4.10.2", "fast-check": "^4.10.2",
"prettier": "^3.9.10",
"typescript": "^5.9.3", "typescript": "^5.9.3",
"vitest": "^3.2.7" "vitest": "^3.2.7"
}, },
"engines": { "engines": {
"node": ">=20" "node": ">=20"
},
"peerDependencies": {
"vitest": ">=3"
},
"peerDependenciesMeta": {
"vitest": {
"optional": true
}
} }
}, },
"node_modules/@esbuild/aix-ppc64": { "node_modules/@esbuild/aix-ppc64": {
@@ -1299,6 +1308,22 @@
"node": "^10 || ^12 || >=14" "node": "^10 || ^12 || >=14"
} }
}, },
"node_modules/prettier": {
"version": "3.9.10",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.10.tgz",
"integrity": "sha512-skGHTH1Yb9QvaTC4hWnRTzEYtCpApF0UMBPz+t/0ZPzP8ihefo/btKd+QA7pUE0fEwA1SbS/NmhyEcstdXOIdA==",
"dev": true,
"license": "MIT",
"bin": {
"prettier": "bin/prettier.cjs"
},
"engines": {
"node": ">=14"
},
"funding": {
"url": "https://github.com/prettier/prettier?sponsor=1"
}
},
"node_modules/pure-rand": { "node_modules/pure-rand": {
"version": "8.4.2", "version": "8.4.2",
"resolved": "https://registry.npmjs.org/pure-rand/-/pure-rand-8.4.2.tgz", "resolved": "https://registry.npmjs.org/pure-rand/-/pure-rand-8.4.2.tgz",
+4 -1
View File
@@ -43,11 +43,14 @@
"test": "tsc --noEmit && vitest run", "test": "tsc --noEmit && vitest run",
"test:watch": "vitest", "test:watch": "vitest",
"bench": "vitest bench --run", "bench": "vitest bench --run",
"test:only": "vitest run" "test:only": "vitest run",
"format": "prettier --write .",
"format:check": "prettier --check ."
}, },
"devDependencies": { "devDependencies": {
"@types/node": "^22.20.5", "@types/node": "^22.20.5",
"fast-check": "^4.10.2", "fast-check": "^4.10.2",
"prettier": "^3.9.10",
"typescript": "^5.9.3", "typescript": "^5.9.3",
"vitest": "^3.2.7" "vitest": "^3.2.7"
}, },
+1 -2
View File
@@ -62,8 +62,7 @@ export function isIndexValue(v: unknown): v is IndexValue {
function makeCodec(sep: string): IndexValueCodec { function makeCodec(sep: string): IndexValueCodec {
const sepCode = sep.charCodeAt(0) const sepCode = sep.charCodeAt(0)
const digits = const digits = sepCode < 0x30 ? '0123456789abcdef' : Array.from({ length: 16 }, (_, i) => String.fromCharCode(sepCode + 1 + i)).join('')
sepCode < 0x30 ? '0123456789abcdef' : Array.from({ length: 16 }, (_, i) => String.fromCharCode(sepCode + 1 + i)).join('')
const digitValue = new Map<string, number>([...digits].map((d, i) => [d, i])) const digitValue = new Map<string, number>([...digits].map((d, i) => [d, i]))
const TAG_NULL = digits[0]! const TAG_NULL = digits[0]!
const TAG_FALSE = digits[1]! const TAG_FALSE = digits[1]!
+22 -4
View File
@@ -39,7 +39,8 @@ export interface EventSourcedDocumentStorageOptions {
} }
/** Stored projection of one document. */ /** Stored projection of one document. */
export type Projection = { state: Doc; version: number; commit: number; deleted?: undefined } | { deleted: true; version: number; commit: number; state?: undefined } export type Projection =
{ state: Doc; version: number; commit: number; deleted?: undefined } | { deleted: true; version: number; commit: number; state?: undefined }
interface DocContext { interface DocContext {
readonly driver: KVDriver readonly driver: KVDriver
@@ -120,7 +121,16 @@ export class EventSourcedDocumentNodeImpl<S extends Spec = AnySpec> implements E
child<K extends KeyOf<S>>(key: K): EventSourcedDocumentNodeImpl<ChildOf<S, K>> { child<K extends KeyOf<S>>(key: K): EventSourcedDocumentNodeImpl<ChildOf<S, K>> {
const seg = toSegment(key) const seg = toSegment(key)
const esc = escapeSegment(seg, this.#ctx.sep) const esc = escapeSegment(seg, this.#ctx.sep)
return new EventSourcedDocumentNodeImpl(this.#ctx, this.#ev.child(seg), this.#ev, [...this.path, seg], this.#prefix + esc, esc, this.#ctx.registry.trie.step(this.#state, seg), this.#childIdx) return new EventSourcedDocumentNodeImpl(
this.#ctx,
this.#ev.child(seg),
this.#ev,
[...this.path, seg],
this.#prefix + esc,
esc,
this.#ctx.registry.trie.step(this.#state, seg),
this.#childIdx,
)
} }
as<S2 extends Spec>(): EventSourcedDocumentNodeImpl<S2> { as<S2 extends Spec>(): EventSourcedDocumentNodeImpl<S2> {
@@ -356,7 +366,9 @@ export class EventSourcedDocumentNodeImpl<S extends Spec = AnySpec> implements E
throw err throw err
} }
const releases = oldClaims ? [...oldClaims.keys()].filter((k) => !newClaims!.has(k)) : [] const releases = oldClaims ? [...oldClaims.keys()].filter((k) => !newClaims!.has(k)) : []
const projection: Projection = next ? { state: next, version: result.version, commit: result.commit } : { deleted: true, version: result.version, commit: result.commit } const projection: Projection = next
? { state: next, version: result.version, commit: result.commit }
: { deleted: true, version: result.version, commit: result.commit }
await this.#writeProjection(t, cur, next, projection, releases) await this.#writeProjection(t, cur, next, projection, releases)
return next return next
} }
@@ -515,7 +527,13 @@ export class EventSourcedDocumentNodeImpl<S extends Spec = AnySpec> implements E
/** Filter with values replaced by `?`, for de-duplicating warnings. */ /** Filter with values replaced by `?`, for de-duplicating warnings. */
function filterShape(f: unknown): unknown { function filterShape(f: unknown): unknown {
if (Array.isArray(f)) return f.map(filterShape) if (Array.isArray(f)) return f.map(filterShape)
if (isPlainObject(f)) return Object.fromEntries(Object.entries(f).map(([k, v]) => [k, k.startsWith('$') && (k === '$and' || k === '$or' || k === '$not') ? filterShape(v) : isPlainObject(v) ? filterShape(v) : '?'])) if (isPlainObject(f))
return Object.fromEntries(
Object.entries(f).map(([k, v]) => [
k,
k.startsWith('$') && (k === '$and' || k === '$or' || k === '$not') ? filterShape(v) : isPlainObject(v) ? filterShape(v) : '?',
]),
)
return '?' return '?'
} }
+2 -6
View File
@@ -12,10 +12,7 @@ export interface UpdateSpec {
} }
/** Stored document events: resolved deltas, never operators, so replay is deterministic. */ /** Stored document events: resolved deltas, never operators, so replay is deterministic. */
export type DocEvent = export type DocEvent = { op: 'set'; doc: Doc } | { op: 'patch'; set?: Record<string, unknown>; unset?: string[] } | { op: 'delete' }
| { op: 'set'; doc: Doc }
| { op: 'patch'; set?: Record<string, unknown>; unset?: string[] }
| { op: 'delete' }
export interface Patch { export interface Patch {
set: Record<string, unknown> set: Record<string, unknown>
@@ -44,8 +41,7 @@ export function resolveUpdate(state: Doc, spec: UpdateSpec): Patch {
if (!Array.isArray(cur)) throw new TypeError(`${op} on non-array field "${path}"`) if (!Array.isArray(cur)) throw new TypeError(`${op} on non-array field "${path}"`)
return cur return cur
} }
const eachOf = (v: unknown): unknown[] => const eachOf = (v: unknown): unknown[] => (isPlainObject(v) && Object.keys(v).length === 1 && Array.isArray(v.$each) ? (v.$each as unknown[]) : [v])
isPlainObject(v) && Object.keys(v).length === 1 && Array.isArray(v.$each) ? (v.$each as unknown[]) : [v]
for (const [path, v] of Object.entries(spec.$set ?? {})) { for (const [path, v] of Object.entries(spec.$set ?? {})) {
touch(path) touch(path)
+2 -1
View File
@@ -81,7 +81,8 @@ export function unsetPath(doc: Doc, path: string): void {
if (typeof v !== 'object' || v === null) return if (typeof v !== 'object' || v === null) return
} }
const last = segs.at(-1)! const last = segs.at(-1)!
if (Array.isArray(v) && /^\d+$/.test(last)) v[Number(last)] = null // Mongo semantics: keep positions if (Array.isArray(v) && /^\d+$/.test(last))
v[Number(last)] = null // Mongo semantics: keep positions
else if (isPlainObject(v)) delete v[last] else if (isPlainObject(v)) delete v[last]
} }
+4 -1
View File
@@ -30,7 +30,10 @@ export class UniqueConstraintError extends PotentialStorageError {
/** The driver lacks an optional capability required by the requested feature. */ /** The driver lacks an optional capability required by the requested feature. */
export class CapabilityError extends PotentialStorageError { export class CapabilityError extends PotentialStorageError {
constructor(readonly capability: string, detail?: string) { constructor(
readonly capability: string,
detail?: string,
) {
super(`Driver lacks capability "${capability}"${detail ? `: ${detail}` : ''}`) super(`Driver lacks capability "${capability}"${detail ? `: ${detail}` : ''}`)
} }
} }
-1
View File
@@ -400,4 +400,3 @@ export class KVStorage<S extends Spec = AnySpec> extends KVNode<S> {
super({ driver, sep, registry, readRepair: opts.readRepair ?? true }, [], '', '', registry.trie.rootState, undefined) super({ driver, sep, registry, readRepair: opts.readRepair ?? true }, [], '', '', registry.trie.rootState, undefined)
} }
} }
-1
View File
@@ -17,7 +17,6 @@ export interface SortedMap<V> {
keys(range: { gte?: string; lt?: string; reverse?: boolean }): IterableIterator<string> keys(range: { gte?: string; lt?: string; reverse?: boolean }): IterableIterator<string>
} }
/** /**
* Sorted chunks of at most `maxChunk` keys (split in half on overflow) plus a `Map` for values. * Sorted chunks of at most `maxChunk` keys (split in half on overflow) plus a `Map` for values.
* Inserts and deletes cost O(log n + maxChunk); iteration is O(1) per step while the map * Inserts and deletes cost O(log n + maxChunk); iteration is O(1) per step while the map
+2 -3
View File
@@ -22,9 +22,8 @@ export type KeyString<S> = `${KeyOf<S>}`
export type EntryOf<S> = { [K in KeyOf<S>]: [K, ValueOf<ChildOf<S, K>>] }[KeyOf<S>] export type EntryOf<S> = { [K in KeyOf<S>]: [K, ValueOf<ChildOf<S, K>>] }[KeyOf<S>]
/** `[key, value]` as yielded at runtime (string keys). Dynamic levels collapse to `[string, V]`. */ /** `[key, value]` as yielded at runtime (string keys). Dynamic levels collapse to `[string, V]`. */
export type StringEntryOf<S> = string extends KeyOf<S> export type StringEntryOf<S> =
? [string, ValueOf<ChildOf<S, KeyOf<S> & string>>] string extends KeyOf<S> ? [string, ValueOf<ChildOf<S, KeyOf<S> & string>>] : { [K in KeyOf<S>]: [`${K}`, ValueOf<ChildOf<S, K>>] }[KeyOf<S>]
: { [K in KeyOf<S>]: [`${K}`, ValueOf<ChildOf<S, K>>] }[KeyOf<S>]
/** A dynamic level of children with values `V`, each with children `C`. */ /** A dynamic level of children with values `V`, each with children `C`. */
export type Many<V, C extends Record<string, Spec> = {}> = { children: { [id: string]: { value: V; children: C } } } export type Many<V, C extends Record<string, Spec> = {}> = { children: { [id: string]: { value: V; children: C } } }
+2 -1
View File
@@ -32,7 +32,8 @@ describe('encodeIndexValue', () => {
fc.assert( fc.assert(
fc.property(indexValue, separators, (v, sep) => { fc.property(indexValue, separators, (v, sep) => {
const decoded = decodeIndexValue(encodeIndexValue(v, sep), sep) const decoded = decodeIndexValue(encodeIndexValue(v, sep), sep)
if (typeof v === 'number' && v === 0) expect(decoded).toBe(0) // -0 normalised if (typeof v === 'number' && v === 0)
expect(decoded).toBe(0) // -0 normalised
else expect(decoded).toEqual(v) else expect(decoded).toEqual(v)
}), }),
) )
+1 -10
View File
@@ -1,15 +1,6 @@
import { describe, expect, it } from 'vitest' import { describe, expect, it } from 'vitest'
import fc from 'fast-check' import fc from 'fast-check'
import { import { escapeSegment, joinPath, pathKeys, splitPath, succ, toSegment, unescapeSegment, validateSeparator } from '../../src/core/path.js'
escapeSegment,
joinPath,
pathKeys,
splitPath,
succ,
toSegment,
unescapeSegment,
validateSeparator,
} from '../../src/core/path.js'
import { InvalidKeyError } from '../../src/errors.js' import { InvalidKeyError } from '../../src/errors.js'
const separators = fc.constantFrom('/', ':', '.', '|', '#', '\\', '!') const separators = fc.constantFrom('/', ':', '.', '|', '#', '\\', '!')
+16 -4
View File
@@ -186,7 +186,9 @@ describe.each([
expect(await keysOf(users.find({}, { sort: { name: -1 }, skip: 1, limit: 2 }))).toEqual(['d', 'c']) expect(await keysOf(users.find({}, { sort: { name: -1 }, skip: 1, limit: 2 }))).toEqual(['d', 'c'])
expect(await keysOf(users.find({ age: { $gte: 20 } }, { sort: { age: -1 }, limit: 2 }))).toEqual(['c', 'a']) expect(await keysOf(users.find({ age: { $gte: 20 } }, { sort: { age: -1 }, limit: 2 }))).toEqual(['c', 'a'])
expect(await keysOf(users.find({ 'address.city': 'London' }, { sort: { age: 1 } }))).toEqual(['e', 'a', 'c']) expect(await keysOf(users.find({ 'address.city': 'London' }, { sort: { age: 1 } }))).toEqual(['e', 'a', 'c'])
expect(await collect(users.find({ age: 36 }, { projection: { name: 1, 'address.city': 1 } }))).toEqual([{ key: 'a', data: { name: 'Ada', address: { city: 'London' } } }]) expect(await collect(users.find({ age: 36 }, { projection: { name: 1, 'address.city': 1 } }))).toEqual([
{ key: 'a', data: { name: 'Ada', address: { city: 'London' } } },
])
expect(await collect(users.find({ age: 36 }, { projection: { tags: 0, address: 0, email: 0 } }))).toEqual([{ key: 'a', data: { name: 'Ada', age: 36 } }]) expect(await collect(users.find({ age: 36 }, { projection: { tags: 0, address: 0, email: 0 } }))).toEqual([{ key: 'a', data: { name: 'Ada', age: 36 } }])
expect(await users.findOne({ email: 'bob@x' })).toEqual({ key: 'b', data: expect.objectContaining({ name: 'Bob' }) }) expect(await users.findOne({ email: 'bob@x' })).toEqual({ key: 'b', data: expect.objectContaining({ name: 'Bob' }) })
expect(await users.findOne({ email: 'nobody' })).toBeUndefined() expect(await users.findOne({ email: 'nobody' })).toBeUndefined()
@@ -278,7 +280,11 @@ describe.each([
it('catchUp() repairs a subtree', async () => { it('catchUp() repairs a subtree', async () => {
const { users, docs, events } = await lagged() const { users, docs, events } = await lagged()
await events.child('users').child('9').child('posts').append('p', [{ op: 'set', doc: { createdAt: 'x' } }]) await events
.child('users')
.child('9')
.child('posts')
.append('p', [{ op: 'set', doc: { createdAt: 'x' } }])
expect(await docs.catchUp()).toBe(2) expect(await docs.catchUp()).toBe(2)
expect(await users.get('1')).toEqual({ n: 2, email: 'b@x' }) expect(await users.get('1')).toEqual({ n: 2, email: 'b@x' })
expect(await users.child('9').child('posts').get('p')).toEqual({ createdAt: 'x' }) expect(await users.child('9').child('posts').get('p')).toEqual({ createdAt: 'x' })
@@ -339,7 +345,12 @@ describe('property: indexed find equals unindexed find', () => {
it('returns the same results in the same order', async () => { it('returns the same results in the same order', async () => {
await fc.assert( await fc.assert(
fc.asyncProperty(fc.array(fc.tuple(fc.constantFrom('1', '2', '3', '4', '5'), doc), { maxLength: 12 }), filter, sort, fc.option(fc.integer({ min: 1, max: 4 }), { nil: undefined }), async (writes, f, s, limit) => { fc.asyncProperty(
fc.array(fc.tuple(fc.constantFrom('1', '2', '3', '4', '5'), doc), { maxLength: 12 }),
filter,
sort,
fc.option(fc.integer({ min: 1, max: 4 }), { nil: undefined }),
async (writes, f, s, limit) => {
const plain = new EventSourcedDocumentStorage(new KVEventStorage(new KVStorage(new InMemoryKVDriver())), { warnOnFullScan: false }).child('users') const plain = new EventSourcedDocumentStorage(new KVEventStorage(new KVStorage(new InMemoryKVDriver())), { warnOnFullScan: false }).child('users')
const indexed = new EventSourcedDocumentStorage(new KVEventStorage(new KVStorage(new InMemoryKVDriver())), { const indexed = new EventSourcedDocumentStorage(new KVEventStorage(new KVStorage(new InMemoryKVDriver())), {
warnOnFullScan: false, warnOnFullScan: false,
@@ -360,7 +371,8 @@ describe('property: indexed find equals unindexed find', () => {
const all = new Set((await collect(plain.find(f))).map((r) => r.key)) const all = new Set((await collect(plain.find(f))).map((r) => r.key))
expect(b).toHaveLength(a.length) expect(b).toHaveLength(a.length)
for (const r of b) expect(all.has(r.key)).toBe(true) for (const r of b) expect(all.has(r.key)).toBe(true)
}), },
),
{ numRuns: 300 }, { numRuns: 300 },
) )
}) })
+7 -1
View File
@@ -45,7 +45,13 @@ describe('filter semantics', () => {
describe('update resolution', () => { describe('update resolution', () => {
it('$pull by sub-document filter and by condition', () => { it('$pull by sub-document filter and by condition', () => {
const state = { items: [{ k: 1, v: 'a' }, { k: 2, v: 'b' }], n: [1, 5, 9] } const state = {
items: [
{ k: 1, v: 'a' },
{ k: 2, v: 'b' },
],
n: [1, 5, 9],
}
const patch = resolveUpdate(state, { $pull: { items: { k: 2 }, n: { $gte: 5 } } }) const patch = resolveUpdate(state, { $pull: { items: { k: 2 }, n: { $gte: 5 } } })
expect(applyPatch(state, patch)).toEqual({ items: [{ k: 1, v: 'a' }], n: [1] }) expect(applyPatch(state, patch)).toEqual({ items: [{ k: 1, v: 'a' }], n: [1] })
}) })
+1 -2
View File
@@ -40,8 +40,7 @@ export class CountingDriver implements KVDriver {
readonly inner: KVDriver = new InMemoryKVDriver(), readonly inner: KVDriver = new InMemoryKVDriver(),
hide: Capability[] = [], hide: Capability[] = [],
) { ) {
if (inner.setIfAbsent && !hide.includes('setIfAbsent')) if (inner.setIfAbsent && !hide.includes('setIfAbsent')) this.setIfAbsent = (e: [string, unknown][]) => (this.#count('setIfAbsent'), inner.setIfAbsent!(e))
this.setIfAbsent = (e: [string, unknown][]) => (this.#count('setIfAbsent'), inner.setIfAbsent!(e))
if (inner.batch && !hide.includes('batch')) this.batch = (ops: BatchOp[]) => (this.#count('batch'), inner.batch!(ops)) if (inner.batch && !hide.includes('batch')) this.batch = (ops: BatchOp[]) => (this.#count('batch'), inner.batch!(ops))
if (inner.deleteRange && !hide.includes('deleteRange')) if (inner.deleteRange && !hide.includes('deleteRange'))
this.deleteRange = (gte: string, lt: string) => (this.#count('deleteRange'), inner.deleteRange!(gte, lt)) this.deleteRange = (gte: string, lt: string) => (this.#count('deleteRange'), inner.deleteRange!(gte, lt))
+14 -3
View File
@@ -204,7 +204,10 @@ describe('index declarations', () => {
}) })
describe('property: findBy equals brute-force filter', () => { describe('property: findBy equals brute-force filter', () => {
const val = fc.record({ age: fc.option(fc.integer({ min: -3, max: 3 }), { nil: undefined }), tags: fc.array(fc.constantFrom('a', 'b', 'c/d'), { maxLength: 3 }) }) const val = fc.record({
age: fc.option(fc.integer({ min: -3, max: 3 }), { nil: undefined }),
tags: fc.array(fc.constantFrom('a', 'b', 'c/d'), { maxLength: 3 }),
})
const op = fc.oneof( const op = fc.oneof(
fc.record({ t: fc.constant('set' as const), k: fc.constantFrom('1', '2', '3', 'x/y'), v: val }), fc.record({ t: fc.constant('set' as const), k: fc.constantFrom('1', '2', '3', 'x/y'), v: val }),
fc.record({ t: fc.constant('del' as const), k: fc.constantFrom('1', '2', '3', 'x/y') }), fc.record({ t: fc.constant('del' as const), k: fc.constantFrom('1', '2', '3', 'x/y') }),
@@ -226,11 +229,19 @@ describe('property: findBy equals brute-force filter', () => {
} }
} }
const got = (await collect(users.findBy('byAge', { gte: lo, lt: hi }))).map(([k]) => k).sort() const got = (await collect(users.findBy('byAge', { gte: lo, lt: hi }))).map(([k]) => k).sort()
const exp = [...model].filter(([, v]) => v.age !== undefined && v.age >= lo && v.age < hi).map(([k]) => k).sort() const exp = [...model]
.filter(([, v]) => v.age !== undefined && v.age >= lo && v.age < hi)
.map(([k]) => k)
.sort()
expect(got).toEqual(exp) expect(got).toEqual(exp)
for (const tag of ['a', 'b', 'c/d']) { for (const tag of ['a', 'b', 'c/d']) {
const g = (await collect(users.findBy('byTag', tag))).map(([k]) => k).sort() const g = (await collect(users.findBy('byTag', tag))).map(([k]) => k).sort()
expect(g).toEqual([...model].filter(([, v]) => v.tags?.includes(tag)).map(([k]) => k).sort()) expect(g).toEqual(
[...model]
.filter(([, v]) => v.tags?.includes(tag))
.map(([k]) => k)
.sort(),
)
} }
// No stale entries are ever left behind by clean writes. // No stale entries are ever left behind by clean writes.
let entryCount = 0 let entryCount = 0
+6 -2
View File
@@ -293,7 +293,10 @@ describe('model-based (random ops vs reference Map)', () => {
] as const) { ] as const) {
it(`matches the model (${name})`, async () => { it(`matches the model (${name})`, async () => {
await fc.assert( await fc.assert(
fc.asyncProperty(fc.array(opArb, { maxLength: 30 }), fc.array(fc.array(segArb, { maxLength: 2 }), { minLength: 1, maxLength: 3 }), async (ops, probes) => { fc.asyncProperty(
fc.array(opArb, { maxLength: 30 }),
fc.array(fc.array(segArb, { maxLength: 2 }), { minLength: 1, maxLength: 3 }),
async (ops, probes) => {
const db = new KVStorage(make()) const db = new KVStorage(make())
const model = new Map<string, unknown>() const model = new Map<string, unknown>()
for (const op of ops) { for (const op of ops) {
@@ -342,7 +345,8 @@ describe('model-based (random ops vs reference Map)', () => {
expect(await collect(node.keys({ limit: i + 1 }))).toEqual(keys.slice(0, i + 1)) expect(await collect(node.keys({ limit: i + 1 }))).toEqual(keys.slice(0, i + 1))
} }
} }
}), },
),
{ numRuns: 300 }, { numRuns: 300 },
) )
}, 60_000) }, 60_000)
+7 -2
View File
@@ -36,7 +36,11 @@ describe('ChunkedSortedMap (model-based, tiny chunks)', () => {
it('iterators resume correctly after mutations between steps', () => { it('iterators resume correctly after mutations between steps', () => {
fc.assert( fc.assert(
fc.property(fc.uniqueArray(key, { minLength: 1, maxLength: 60 }), fc.array(fc.tuple(fc.boolean(), key), { maxLength: 40 }), fc.boolean(), (initial, muts, reverse) => { fc.property(
fc.uniqueArray(key, { minLength: 1, maxLength: 60 }),
fc.array(fc.tuple(fc.boolean(), key), { maxLength: 40 }),
fc.boolean(),
(initial, muts, reverse) => {
const m = new ChunkedSortedMap<number>(3) const m = new ChunkedSortedMap<number>(3)
initial.forEach((k) => m.set(k, 0)) initial.forEach((k) => m.set(k, 0))
const live = new Set(initial) const live = new Set(initial)
@@ -55,7 +59,8 @@ describe('ChunkedSortedMap (model-based, tiny chunks)', () => {
for (let i = 1; i < seen.length; i++) expect(reverse ? seen[i]! < seen[i - 1]! : seen[i]! > seen[i - 1]!).toBe(true) for (let i = 1; i < seen.length; i++) expect(reverse ? seen[i]! < seen[i - 1]! : seen[i]! > seen[i - 1]!).toBe(true)
const stable = initial.filter((k) => !muts.some(([, mk]) => mk === k)) const stable = initial.filter((k) => !muts.some(([, mk]) => mk === k))
for (const k of stable) expect(seen).toContain(k) for (const k of stable) expect(seen).toContain(k)
}), },
),
{ numRuns: 500 }, { numRuns: 500 },
) )
}) })
+12 -3
View File
@@ -47,12 +47,18 @@ describe('KV types (section 7)', () => {
db.child('users').child('1').set({ username: 'a', age: 1 }) db.child('users').child('1').set({ username: 'a', age: 1 })
// @ts-expect-error wrong value type // @ts-expect-error wrong value type
db.child('users').set('1', { username: 'a' }) db.child('users').set('1', { username: 'a' })
db.child('profile').child('9').setMany([ db.child('profile')
.child('9')
.setMany([
['age', 3], ['age', 3],
['username', 'x'], ['username', 'x'],
]) ])
db.child('profile')
.child('9')
.setMany([
// @ts-expect-error value doesn't match its key // @ts-expect-error value doesn't match its key
db.child('profile').child('9').setMany([['age', 'x']]) ['age', 'x'],
])
}) })
it('types listings', () => { it('types listings', () => {
@@ -82,7 +88,10 @@ describe('event types', () => {
orders.child('o1').append([{ type: 'paid' }]) orders.child('o1').append([{ type: 'paid' }])
// @ts-expect-error not an OrderEvent // @ts-expect-error not an OrderEvent
orders.append('o1', [{ type: 'refunded' }]) orders.append('o1', [{ type: 'refunded' }])
orders.child('o1').child('shipments').append('s1', [{ carrier: 'x' }]) orders
.child('o1')
.child('shipments')
.append('s1', [{ carrier: 'x' }])
}) })
it('types read results', () => { it('types read results', () => {