Add Prettier formatting and Gitea Actions CI (format, typecheck, test, build)
CI / Node 20 (push) Successful in 1m49s
CI / Node 22 (push) Successful in 1m52s

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
This commit is contained in:
Claude
2026-10-23 07:29:38 +00:00
parent 5a370f8c78
commit e27541c46a
24 changed files with 341 additions and 209 deletions
+30
View File
@@ -0,0 +1,30 @@
name: CI
on:
push:
branches: [main]
pull_request:
jobs:
check:
name: Node ${{ matrix.node }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node: [20, 22]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: npm
- run: npm ci
- name: Format
run: npm run format:check
- name: Typecheck
run: npm run typecheck
- name: Test
run: npm run test:only
- name: Build
run: npm run build
+4
View File
@@ -0,0 +1,4 @@
node_modules
dist
coverage
package-lock.json
+6
View File
@@ -0,0 +1,6 @@
{
"semi": false,
"singleQuote": true,
"printWidth": 160,
"trailingComma": "all"
}
+3
View File
@@ -11,6 +11,9 @@ Record new decisions in DESIGN.md section 12.
- `npm run typecheck` — `tsc --noEmit` over src, test and bench - `npm run typecheck` — `tsc --noEmit` over src, test and bench
- `npm run bench` — `vitest bench` - `npm run bench` — `vitest bench`
- `npm run build` — emit `dist/` - `npm run build` — emit `dist/`
- `npm run format` / `npm run format:check` — Prettier
CI (`.gitea/workflows/ci.yml`, Gitea Actions) runs format check, typecheck, tests and build on Node 20 and 22.
## Rules ## Rules
+83 -74
View File
@@ -18,7 +18,7 @@ implementation decisions made where the brief was silent (section 12) and benchm
## 1. Principles ## 1. Principles
1. **Three layers**, each usable on its own: **KV** → **Event** (append-only, built on KV) → **Document** (Mongo-like, built on Events or native). 1. **Three layers**, each usable on its own: **KV** → **Event** (append-only, built on KV) → **Document** (Mongo-like, built on Events or native).
2. **Drivers vs stores.** Backends implement a tiny, flat, string-keyed *driver*. All path, child, index, escaping and typing logic lives once in the user-facing *node* classes, so every backend gets identical behaviour. 2. **Drivers vs stores.** Backends implement a tiny, flat, string-keyed _driver_. All path, child, index, escaping and typing logic lives once in the user-facing _node_ classes, so every backend gets identical behaviour.
3. **One navigation primitive: `child(key)`.** There are no collection/document classes. Every node is a store over its direct children, and `child(key)` re-roots to that key. Navigation is synchronous and does no I/O. 3. **One navigation primitive: `child(key)`.** There are no collection/document classes. Every node is a store over its direct children, and `child(key)` re-roots to that key. Navigation is synchronous and does no I/O.
4. **No methods on data.** Reads return plain values. 4. **No methods on data.** Reads return plain values.
5. **Performance is a requirement.** Batch operations are the primitive; scans are ordered and streamed; nothing replays or scans more than it must (section 8). 5. **Performance is a requirement.** Batch operations are the primitive; scans are ordered and streamed; nothing replays or scans more than it must (section 8).
@@ -59,24 +59,22 @@ export interface ScanOptions {
lt?: string lt?: string
limit?: number limit?: number
reverse?: boolean reverse?: boolean
values?: boolean // when false/omitted, yield value as undefined values?: boolean // when false/omitted, yield value as undefined
} }
export interface KVDriver { export interface KVDriver {
getMany(keys: string[]): Promise<(unknown | undefined)[]> // same order as keys getMany(keys: string[]): Promise<(unknown | undefined)[]> // same order as keys
setMany(entries: [key: string, value: unknown][]): Promise<void> setMany(entries: [key: string, value: unknown][]): Promise<void>
deleteMany(keys: string[]): Promise<void> deleteMany(keys: string[]): Promise<void>
scan(opts: ScanOptions): AsyncIterable<[key: string, value: unknown]> // MUST be key-ordered scan(opts: ScanOptions): AsyncIterable<[key: string, value: unknown]> // MUST be key-ordered
// Optional capabilities. Stores feature-detect them and fall back where possible. // Optional capabilities. Stores feature-detect them and fall back where possible.
setIfAbsent?(entries: [string, unknown][]): Promise<boolean[]> // true = written setIfAbsent?(entries: [string, unknown][]): Promise<boolean[]> // true = written
batch?(ops: BatchOp[]): Promise<void> // atomic multi-op write batch?(ops: BatchOp[]): Promise<void> // atomic multi-op write
deleteRange?(gte: string, lt: string): Promise<void> deleteRange?(gte: string, lt: string): Promise<void>
} }
export type BatchOp = export type BatchOp = { type: 'set'; key: string; value: unknown } | { type: 'delete'; key: string }
| { type: 'set'; key: string; value: unknown }
| { type: 'delete'; key: string }
``` ```
- Values are passed through as JSON-serialisable values. Drivers that need bytes (HTTP, Redis, files) serialise internally via a pluggable `Codec` (default JSON). - Values are passed through as JSON-serialisable values. Drivers that need bytes (HTTP, Redis, files) serialise internally via a pluggable `Codec` (default JSON).
@@ -94,29 +92,31 @@ const user = users.child('123')
API of every `KVNode` (types in section 7): API of every `KVNode` (types in section 7):
| Method | Meaning | | Method | Meaning |
|---|---| | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `child(key)` | Sync. Returns a node at `path + key`. | | `child(key)` | Sync. Returns a node at `path + key`. |
| `as<S>()` | Type assertion for untyped use; returns the same node retyped. | | `as<S>()` | Type assertion for untyped use; returns the same node retyped. |
| `get()` / `get(key)` | Own value at `selfKey` / value at child key. | | `get()` / `get(key)` | Own value at `selfKey` / value at child key. |
| `getMany(keys)` | Batch child read, same order, `undefined` for missing. | | `getMany(keys)` | Batch child read, same order, `undefined` for missing. |
| `set(value)` / `set(key, value)` | Write own value / child value. | | `set(value)` / `set(key, value)` | Write own value / child value. |
| `setMany(entries)` | Batch child write. | | `setMany(entries)` | Batch child write. |
| `delete()` / `delete(key)` | Delete own value / child value only (not descendants). | | `delete()` / `delete(key)` | Delete own value / child value only (not descendants). |
| `deleteMany(keys)` | Batch child delete. | | `deleteMany(keys)` | Batch child delete. |
| `deleteTree()` / `deleteTree(key)` | Delete own value **and all descendants** (uses `deleteRange` if available). | | `deleteTree()` / `deleteTree(key)` | Delete own value **and all descendants** (uses `deleteRange` if available). |
| `keys(opts?)` | `AsyncIterable<string>` of **direct** child keys (unescaped). Options: `limit`, `after`, `reverse`. | | `keys(opts?)` | `AsyncIterable<string>` of **direct** child keys (unescaped). Options: `limit`, `after`, `reverse`. |
| `entries(opts?)` | `AsyncIterable<[key, value]>` of direct children that have an own value. | | `entries(opts?)` | `AsyncIterable<[key, value]>` of direct children that have an own value. |
| `tree({ depth })` | Assemble an object from descendant keys up to `depth` (for field-per-key style). Bounded and explicit; never the default. | | `tree({ depth })` | Assemble an object from descendant keys up to `depth` (for field-per-key style). Bounded and explicit; never the default. |
| `findBy(index, value \| { gte?, lt? }, opts?)` | Index lookup over direct children (section 3.3). | | `findBy(index, value \| { gte?, lt? }, opts?)` | Index lookup over direct children (section 3.3). |
**Arity dispatch (important):** **Arity dispatch (important):**
- Self vs child is decided by `arguments.length`, never by checking for `undefined`. - Self vs child is decided by `arguments.length`, never by checking for `undefined`.
- If a key argument is passed but is `undefined` or `null`, throw a `TypeError` (this prevents `users.get(maybeId)` silently reading self). - If a key argument is passed but is `undefined` or `null`, throw a `TypeError` (this prevents `users.get(maybeId)` silently reading self).
- The root node (`path = []`) has no own value: `get()`, `set(v)` and `delete()` on root throw. - The root node (`path = []`) has no own value: `get()`, `set(v)` and `delete()` on root throw.
- Implement one private `#resolve(args)` that every method uses. - Implement one private `#resolve(args)` that every method uses.
**Direct-children listing (`keys`)** uses a skip-scan: **Direct-children listing (`keys`)** uses a skip-scan:
1. Scan `gte: prefix`, `lt: prefix + succ(sep)`. 1. Scan `gte: prefix`, `lt: prefix + succ(sep)`.
2. For each key, take the first segment after `prefix`. Skip reserved (`~…`) segments. Yield the segment (unescaped) if it differs from the previous one. 2. For each key, take the first segment after `prefix`. Skip reserved (`~…`) segments. Yield the segment (unescaped) if it differs from the previous one.
3. When the next key is a descendant of the segment just yielded, re-seek the driver to `gte: prefix + escSeg + succ(sep)` (where `succ(c)` is the next char code) instead of iterating through the subtree. Keep using the same iterator while keys are siblings. 3. When the next key is a descendant of the segment just yielded, re-seek the driver to `gte: prefix + escSeg + succ(sep)` (where `succ(c)` is the next char code) instead of iterating through the subtree. Keep using the same iterator while keys are siblings.
@@ -130,8 +130,8 @@ Indexes are declared on **path patterns** in the `KVStorage` constructor. A patt
```ts ```ts
new KVStorage(driver, { new KVStorage(driver, {
indexes: { indexes: {
'users': { byEmail: (u: User) => u.email }, users: { byEmail: (u: User) => u.email },
'users/*/posts': { byDate: (p: Post) => p.createdAt, byTag: (p: Post) => p.tags }, 'users/*/posts': { byDate: (p: Post) => p.createdAt, byTag: (p: Post) => p.tags },
}, },
}) })
``` ```
@@ -155,14 +155,17 @@ new KVStorage(driver, {
An append-only log per stream. A stream is identified by a node path. Same `child()` navigation as KV. An append-only log per stream. A stream is identified by a node path. Same `child()` navigation as KV.
```ts ```ts
const events = new EventStorage(kv) // takes a KVStorage (or a KVNode as root) const events = new EventStorage(kv) // takes a KVStorage (or a KVNode as root)
const orders = events.child('orders') const orders = events.child('orders')
await orders.append('o1', [{ type: 'created', total: 40 }], { expectedVersion: 0 }) await orders.append('o1', [{ type: 'created', total: 40 }], { expectedVersion: 0 })
await orders.child('o1').append([{ type: 'paid' }]) // self form await orders.child('o1').append([{ type: 'paid' }]) // self form
for await (const e of orders.read('o1', { from: 1 })) { /* { version, commit, at, data } */ } for await (const e of orders.read('o1', { from: 1 })) {
/* { version, commit, at, data } */
}
await orders.version('o1') await orders.version('o1')
for await (const id of orders.keys()) { } // child streams for await (const id of orders.keys()) {
} // child streams
orders.child('o1').child('shipments') orders.child('o1').child('shipments')
``` ```
@@ -186,23 +189,25 @@ orders.child('o1').child('shipments')
## 5. Layer 3: Documents (`src/documents/`) ## 5. Layer 3: Documents (`src/documents/`)
`DocumentStorage` is the interface. There are two implementations: `DocumentStorage` is the interface. There are two implementations:
- `EventSourcedDocumentStorage(events, { projections?: KVNode })` is the v1 target. Projections default to the `~p` root on the same KV. - `EventSourcedDocumentStorage(events, { projections?: KVNode })` is the v1 target. Projections default to the `~p` root on the same KV.
- `MongoDocumentStorage(client)` is a later milestone (section 9). - `MongoDocumentStorage(client)` is a later milestone (section 9).
```ts ```ts
const docs = new EventSourcedDocumentStorage(events, { indexes: { 'users': [{ fields: ['email'], unique: true }, { fields: ['age'] }] } }) const docs = new EventSourcedDocumentStorage(events, { indexes: { users: [{ fields: ['email'], unique: true }, { fields: ['age'] }] } })
const users = docs.child('users') const users = docs.child('users')
const id = await users.insert({ name: 'Ada', age: 36 }) // generates a time-sortable id const id = await users.insert({ name: 'Ada', age: 36 }) // generates a time-sortable id
await users.set('123', { name: 'Ada', age: 36 }) await users.set('123', { name: 'Ada', age: 36 })
await users.update('123', { $set: { name: 'Ada L.' }, $inc: { logins: 1 } }) await users.update('123', { $set: { name: 'Ada L.' }, $inc: { logins: 1 } })
await users.child('123').update({ $unset: { nickname: true } }) // self form await users.child('123').update({ $unset: { nickname: true } }) // self form
await users.get('123') await users.get('123')
await users.delete('123') await users.delete('123')
for await (const { key, data } of users.find({ age: { $gte: 18 } }, { sort: { age: 1 }, limit: 20 })) { } for await (const { key, data } of users.find({ age: { $gte: 18 } }, { sort: { age: 1 }, limit: 20 })) {
}
await users.findOne({ email: 'ada@x.io' }) await users.findOne({ email: 'ada@x.io' })
await users.history('123') // event-sourced only await users.history('123') // event-sourced only
await users.getAt('123', 17) // state as of version 17 await users.getAt('123', 17) // state as of version 17
users.child('123').child('posts') // nested docs, same as KV users.child('123').child('posts') // nested docs, same as KV
``` ```
@@ -214,6 +219,7 @@ users.child('123').child('posts') // nested docs, same as KV
- Supported update operators in v1: `$set`, `$unset`, `$inc`, `$push`, `$pull`, `$addToSet`, with dot-path support. - Supported update operators in v1: `$set`, `$unset`, `$inc`, `$push`, `$pull`, `$addToSet`, with dot-path support.
**Write path:** **Write path:**
1. Read the projection. 1. Read the projection.
2. Compute the new state and patch. 2. Compute the new state and patch.
3. `append` with `expectedVersion = projection.version`. 3. `append` with `expectedVersion = projection.version`.
@@ -223,6 +229,7 @@ users.child('123').child('posts') // nested docs, same as KV
**Reads** are a single projection `get`. Never replay on read. **Reads** are a single projection `get`. Never replay on read.
**Crash between append and projection write:** **Crash between append and projection write:**
- The projection lags the stream. The next write's `expectedVersion` fails, which triggers catch-up, so the system self-heals. - The projection lags the stream. The next write's `expectedVersion` fails, which triggers catch-up, so the system self-heals.
- Provide `get(key, { consistent: true })` that checks the stream head and catches up first. - Provide `get(key, { consistent: true })` that checks the stream head and catches up first.
- Provide a `catchUp(node?)` maintenance method to repair a subtree. - Provide a `catchUp(node?)` maintenance method to repair a subtree.
@@ -257,16 +264,15 @@ Each node is typed by a single **Spec** that describes its own value and its chi
```ts ```ts
export type Spec = { value?: unknown; children?: Record<string, Spec> } export type Spec = { value?: unknown; children?: Record<string, Spec> }
export type AnySpec = { value: unknown; children: Record<string, AnySpec> } // untyped default export type AnySpec = { value: unknown; children: Record<string, AnySpec> } // untyped default
export type ValueOf<S> = S extends { value: infer V } ? V : never export type ValueOf<S> = S extends { value: infer V } ? V : never
export type ChildrenOf<S> = S extends { children: infer C } ? C : {} export type ChildrenOf<S> = S extends { children: infer C } ? C : {}
export type KeyOf<S> = Extract<keyof ChildrenOf<S>, string | number> export type KeyOf<S> = Extract<keyof ChildrenOf<S>, string | number>
export type ChildOf<S, K extends KeyOf<S>> = ChildrenOf<S>[K] export type ChildOf<S, K extends KeyOf<S>> = ChildrenOf<S>[K]
// Helpers // Helpers
export type Many<V, C extends Record<string, Spec> = {}> = export type Many<V, C extends Record<string, Spec> = {}> = { children: { [id: string]: { value: V; children: C } } }
{ children: { [id: string]: { value: V; children: C } } }
export type Fields<T> = { [K in keyof T & string]: { value: T[K] } } export type Fields<T> = { [K in keyof T & string]: { value: T[K] } }
``` ```
@@ -286,21 +292,24 @@ Example to cover with `expectTypeOf` tests:
```ts ```ts
type User = { username: string; age: number } type User = { username: string; age: number }
type Post = { title: string; createdAt: string } type Post = { title: string; createdAt: string }
type Schema = { children: { type Schema = {
users: Many<User, { posts: Many<Post> }> children: {
profile: { children: { [id: string]: { children: Fields<User> } } } // field-per-key style users: Many<User, { posts: Many<Post> }>
} } profile: { children: { [id: string]: { children: Fields<User> } } } // field-per-key style
}
}
const db = new KVStorage<Schema>(driver) const db = new KVStorage<Schema>(driver)
db.child('users').get('123') // User | undefined db.child('users').get('123') // User | undefined
db.child('users').child('123').get() // User | undefined (same key) db.child('users').child('123').get() // User | undefined (same key)
db.child('users').child('123').child('posts') // KVNode<Many<Post>> db.child('users').child('123').child('posts') // KVNode<Many<Post>>
db.child('users').child('123').child('postz') // ✗ type error db.child('users').child('123').child('postz') // ✗ type error
db.child('profile').child('9').get('age') // number | undefined db.child('profile').child('9').get('age') // number | undefined
db.child('profile').child('9').get('agee') // ✗ type error db.child('profile').child('9').get('agee') // ✗ type error
``` ```
Notes: Notes:
- **Named levels** check keys at compile time; **dynamic `[id: string]` levels** accept any string. - **Named levels** check keys at compile time; **dynamic `[id: string]` levels** accept any string.
- One node's children can't mix an index signature with named keys (a TS restriction), so schemas alternate between dynamic and named levels. That alternation exists only in types; the runtime API is just `child()`. - One node's children can't mix an index signature with named keys (a TS restriction), so schemas alternate between dynamic and named levels. That alternation exists only in types; the runtime API is just `child()`.
- `EventStorage` and `DocumentStorage` nodes use the same Spec machinery (the value type is the event type or the document type respectively). - `EventStorage` and `DocumentStorage` nodes use the same Spec machinery (the value type is the event type or the document type respectively).
@@ -383,7 +392,7 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
### M2 — KV driver ### M2 — KV driver
- **Conformance suite location**: `src/testing/kv-conformance.ts`, exported as `potential-storage/testing`. - **Conformance suite location**: `src/testing/kv-conformance.ts`, exported as `potential-storage/testing`.
It imports `vitest`, which is an *optional peer dependency*; the core entry point never imports it. It imports `vitest`, which is an _optional peer dependency_; the core entry point never imports it.
- **`InMemoryKVDriver` clones values** on write and read (`structuredClone`, skipped for primitives) so - **`InMemoryKVDriver` clones values** on write and read (`structuredClone`, skipped for primitives) so
stored state can't be mutated through shared references, matching serialising drivers. Opt out stored state can't be mutated through shared references, matching serialising drivers. Opt out
with `{ clone: false }`. with `{ clone: false }`.
@@ -402,7 +411,7 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
- **Child order is "subtree order", not plain segment order.** The brief's skip-scan assumes a - **Child order is "subtree order", not plain segment order.** The brief's skip-scan assumes a
child's keys are contiguous, but they aren't when a sibling extends a segment with a character child's keys are contiguous, but they aren't when a sibling extends a segment with a character
that sorts below the separator (for `/`: space and `!"#$%&'()*+,-.`). With children `a` and `a-1` that sorts below the separator (for `/`: space and `!"#$%&'()*+,-.`). With children `a` and `a-1`
the key space is `p/a`, `p/a-1`, `p/a-1/x`, `p/a/x`, so `a`'s descendants sort *after* `a-1`, and the key space is `p/a`, `p/a-1`, `p/a-1/x`, `p/a/x`, so `a`'s descendants sort _after_ `a-1`, and
"yield if different from the previous segment" would emit `a` twice. `keys()`/`entries()` therefore "yield if different from the previous segment" would emit `a` twice. `keys()`/`entries()` therefore
order children by **escaped segment followed by the separator** (`a-1/` < `a/`), which is the order order children by **escaped segment followed by the separator** (`a-1/` < `a/`), which is the order
of their subtrees in key space. For keys without such characters this is plain lexicographic order. of their subtrees in key space. For keys without such characters this is plain lexicographic order.
@@ -541,7 +550,7 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
field, both for index-ordered and in-memory sorts. field, both for index-ordered and in-memory sorts.
- **In-memory sort** keeps a bounded top `skip + limit` buffer when a limit is given. Without a limit, - **In-memory sort** keeps a bounded top `skip + limit` buffer when a limit is given. Without a limit,
it buffers all matches; this is the one place results are materialised. it buffers all matches; this is the one place results are materialised.
- **Full-scan warning**: `console.warn` once per filter *shape* (values replaced by `?`) per storage - **Full-scan warning**: `console.warn` once per filter _shape_ (values replaced by `?`) per storage
instance, only for non-empty filters. On by default unless `NODE_ENV === 'production'`; set it with instance, only for non-empty filters. On by default unless `NODE_ENV === 'production'`; set it with
`warnOnFullScan`. `warnOnFullScan`.
- **Catch-up** re-asserts unique claims for the caught-up state with plain writes, since the log is - **Catch-up** re-asserts unique claims for the caught-up state with plain writes, since the log is
@@ -609,37 +618,37 @@ container (shared CPU, so expect ±10–20% run-to-run). Mean time per operation
**KV** (`bench/kv.bench.ts`): values are small objects. **KV** (`bench/kv.bench.ts`): values are small objects.
| Operation | 10k keys | 100k keys | | Operation | 10k keys | 100k keys |
|---|---|---| | -------------------- | -------- | --------- |
| `get` | 1.2 µs | 1.6 µs | | `get` | 1.2 µs | 1.6 µs |
| `set` (overwrite) | 1.7 µs | 2.0 µs | | `set` (overwrite) | 1.7 µs | 2.0 µs |
| `getMany` × 100 keys | 21 µs | 22 µs | | `getMany` × 100 keys | 21 µs | 22 µs |
**`keys()`** (`bench/keys.bench.ts`): node with 1 000 children, each with 100 descendants (101k keys). **`keys()`** (`bench/keys.bench.ts`): node with 1 000 children, each with 100 descendants (101k keys).
| Operation | Mean | | Operation | Mean |
|---|---| | -------------------------------- | ------ |
| `keys()` skip-scan (1 000 seeks) | 2.9 ms | | `keys()` skip-scan (1 000 seeks) | 2.9 ms |
| Reference: full subtree scan | 36 ms | | Reference: full subtree scan | 36 ms |
**Events** (`bench/events.bench.ts`): single-event commits. **Events** (`bench/events.bench.ts`): single-event commits.
| Operation | Mean | | Operation | Mean |
|---|---| | ---------------------------------------------- | ------ |
| `append` 1 event (warm head cache) | 6.3 µs | | `append` 1 event (warm head cache) | 6.3 µs |
| `read` 100 events from version 5 000 of 10 000 | 174 µs | | `read` 100 events from version 5 000 of 10 000 | 174 µs |
| `read` full 10 000-event stream | 21 ms | | `read` full 10 000-event stream | 21 ms |
| `version()` (reverse scan, `limit: 1`) | 3.9 µs | | `version()` (reverse scan, `limit: 1`) | 3.9 µs |
**Documents** (`bench/documents.bench.ts`): 10 000 docs, indexes on `age` and unique `email`. **Documents** (`bench/documents.bench.ts`): 10 000 docs, indexes on `age` and unique `email`.
| Operation | Mean | | Operation | Mean |
|---|---| | ----------------------------------------------------------------- | ------- |
| `get` | 2.6 µs | | `get` | 2.6 µs |
| `update` (`$inc`, indexed node: projection read + append + batch) | 22 µs | | `update` (`$inc`, indexed node: projection read + append + batch) | 22 µs |
| `findOne` by unique index (`email`) | 16 µs | | `findOne` by unique index (`email`) | 16 µs |
| `find` indexed (`age = k`, ~100 hits) | 0.57 ms | | `find` indexed (`age = k`, ~100 hits) | 0.57 ms |
| `find` full scan (`city = k`, ~200 hits of 10k) | 28 ms | | `find` full scan (`city = k`, ~200 hits of 10k) | 28 ms |
For comparison, the first run with the sorted-array map and `structuredClone` measured: KV `get` For comparison, the first run with the sorted-array map and `structuredClone` measured: KV `get`
3.7 µs, `getMany`×100 186 µs, `append` 34 µs, document `update` 110 µs, full scan 57 ms, and 3.7 µs, `getMany`×100 186 µs, `append` 34 µs, document `update` 110 µs, full scan 57 ms, and
+25
View File
@@ -10,11 +10,20 @@
"devDependencies": { "devDependencies": {
"@types/node": "^22.20.5", "@types/node": "^22.20.5",
"fast-check": "^4.10.2", "fast-check": "^4.10.2",
"prettier": "^3.9.10",
"typescript": "^5.9.3", "typescript": "^5.9.3",
"vitest": "^3.2.7" "vitest": "^3.2.7"
}, },
"engines": { "engines": {
"node": ">=20" "node": ">=20"
},
"peerDependencies": {
"vitest": ">=3"
},
"peerDependenciesMeta": {
"vitest": {
"optional": true
}
} }
}, },
"node_modules/@esbuild/aix-ppc64": { "node_modules/@esbuild/aix-ppc64": {
@@ -1299,6 +1308,22 @@
"node": "^10 || ^12 || >=14" "node": "^10 || ^12 || >=14"
} }
}, },
"node_modules/prettier": {
"version": "3.9.10",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.10.tgz",
"integrity": "sha512-skGHTH1Yb9QvaTC4hWnRTzEYtCpApF0UMBPz+t/0ZPzP8ihefo/btKd+QA7pUE0fEwA1SbS/NmhyEcstdXOIdA==",
"dev": true,
"license": "MIT",
"bin": {
"prettier": "bin/prettier.cjs"
},
"engines": {
"node": ">=14"
},
"funding": {
"url": "https://github.com/prettier/prettier?sponsor=1"
}
},
"node_modules/pure-rand": { "node_modules/pure-rand": {
"version": "8.4.2", "version": "8.4.2",
"resolved": "https://registry.npmjs.org/pure-rand/-/pure-rand-8.4.2.tgz", "resolved": "https://registry.npmjs.org/pure-rand/-/pure-rand-8.4.2.tgz",
+4 -1
View File
@@ -43,11 +43,14 @@
"test": "tsc --noEmit && vitest run", "test": "tsc --noEmit && vitest run",
"test:watch": "vitest", "test:watch": "vitest",
"bench": "vitest bench --run", "bench": "vitest bench --run",
"test:only": "vitest run" "test:only": "vitest run",
"format": "prettier --write .",
"format:check": "prettier --check ."
}, },
"devDependencies": { "devDependencies": {
"@types/node": "^22.20.5", "@types/node": "^22.20.5",
"fast-check": "^4.10.2", "fast-check": "^4.10.2",
"prettier": "^3.9.10",
"typescript": "^5.9.3", "typescript": "^5.9.3",
"vitest": "^3.2.7" "vitest": "^3.2.7"
}, },
+1 -2
View File
@@ -62,8 +62,7 @@ export function isIndexValue(v: unknown): v is IndexValue {
function makeCodec(sep: string): IndexValueCodec { function makeCodec(sep: string): IndexValueCodec {
const sepCode = sep.charCodeAt(0) const sepCode = sep.charCodeAt(0)
const digits = const digits = sepCode < 0x30 ? '0123456789abcdef' : Array.from({ length: 16 }, (_, i) => String.fromCharCode(sepCode + 1 + i)).join('')
sepCode < 0x30 ? '0123456789abcdef' : Array.from({ length: 16 }, (_, i) => String.fromCharCode(sepCode + 1 + i)).join('')
const digitValue = new Map<string, number>([...digits].map((d, i) => [d, i])) const digitValue = new Map<string, number>([...digits].map((d, i) => [d, i]))
const TAG_NULL = digits[0]! const TAG_NULL = digits[0]!
const TAG_FALSE = digits[1]! const TAG_FALSE = digits[1]!
+22 -4
View File
@@ -39,7 +39,8 @@ export interface EventSourcedDocumentStorageOptions {
} }
/** Stored projection of one document. */ /** Stored projection of one document. */
export type Projection = { state: Doc; version: number; commit: number; deleted?: undefined } | { deleted: true; version: number; commit: number; state?: undefined } export type Projection =
{ state: Doc; version: number; commit: number; deleted?: undefined } | { deleted: true; version: number; commit: number; state?: undefined }
interface DocContext { interface DocContext {
readonly driver: KVDriver readonly driver: KVDriver
@@ -120,7 +121,16 @@ export class EventSourcedDocumentNodeImpl<S extends Spec = AnySpec> implements E
child<K extends KeyOf<S>>(key: K): EventSourcedDocumentNodeImpl<ChildOf<S, K>> { child<K extends KeyOf<S>>(key: K): EventSourcedDocumentNodeImpl<ChildOf<S, K>> {
const seg = toSegment(key) const seg = toSegment(key)
const esc = escapeSegment(seg, this.#ctx.sep) const esc = escapeSegment(seg, this.#ctx.sep)
return new EventSourcedDocumentNodeImpl(this.#ctx, this.#ev.child(seg), this.#ev, [...this.path, seg], this.#prefix + esc, esc, this.#ctx.registry.trie.step(this.#state, seg), this.#childIdx) return new EventSourcedDocumentNodeImpl(
this.#ctx,
this.#ev.child(seg),
this.#ev,
[...this.path, seg],
this.#prefix + esc,
esc,
this.#ctx.registry.trie.step(this.#state, seg),
this.#childIdx,
)
} }
as<S2 extends Spec>(): EventSourcedDocumentNodeImpl<S2> { as<S2 extends Spec>(): EventSourcedDocumentNodeImpl<S2> {
@@ -356,7 +366,9 @@ export class EventSourcedDocumentNodeImpl<S extends Spec = AnySpec> implements E
throw err throw err
} }
const releases = oldClaims ? [...oldClaims.keys()].filter((k) => !newClaims!.has(k)) : [] const releases = oldClaims ? [...oldClaims.keys()].filter((k) => !newClaims!.has(k)) : []
const projection: Projection = next ? { state: next, version: result.version, commit: result.commit } : { deleted: true, version: result.version, commit: result.commit } const projection: Projection = next
? { state: next, version: result.version, commit: result.commit }
: { deleted: true, version: result.version, commit: result.commit }
await this.#writeProjection(t, cur, next, projection, releases) await this.#writeProjection(t, cur, next, projection, releases)
return next return next
} }
@@ -515,7 +527,13 @@ export class EventSourcedDocumentNodeImpl<S extends Spec = AnySpec> implements E
/** Filter with values replaced by `?`, for de-duplicating warnings. */ /** Filter with values replaced by `?`, for de-duplicating warnings. */
function filterShape(f: unknown): unknown { function filterShape(f: unknown): unknown {
if (Array.isArray(f)) return f.map(filterShape) if (Array.isArray(f)) return f.map(filterShape)
if (isPlainObject(f)) return Object.fromEntries(Object.entries(f).map(([k, v]) => [k, k.startsWith('$') && (k === '$and' || k === '$or' || k === '$not') ? filterShape(v) : isPlainObject(v) ? filterShape(v) : '?'])) if (isPlainObject(f))
return Object.fromEntries(
Object.entries(f).map(([k, v]) => [
k,
k.startsWith('$') && (k === '$and' || k === '$or' || k === '$not') ? filterShape(v) : isPlainObject(v) ? filterShape(v) : '?',
]),
)
return '?' return '?'
} }
+2 -6
View File
@@ -12,10 +12,7 @@ export interface UpdateSpec {
} }
/** Stored document events: resolved deltas, never operators, so replay is deterministic. */ /** Stored document events: resolved deltas, never operators, so replay is deterministic. */
export type DocEvent = export type DocEvent = { op: 'set'; doc: Doc } | { op: 'patch'; set?: Record<string, unknown>; unset?: string[] } | { op: 'delete' }
| { op: 'set'; doc: Doc }
| { op: 'patch'; set?: Record<string, unknown>; unset?: string[] }
| { op: 'delete' }
export interface Patch { export interface Patch {
set: Record<string, unknown> set: Record<string, unknown>
@@ -44,8 +41,7 @@ export function resolveUpdate(state: Doc, spec: UpdateSpec): Patch {
if (!Array.isArray(cur)) throw new TypeError(`${op} on non-array field "${path}"`) if (!Array.isArray(cur)) throw new TypeError(`${op} on non-array field "${path}"`)
return cur return cur
} }
const eachOf = (v: unknown): unknown[] => const eachOf = (v: unknown): unknown[] => (isPlainObject(v) && Object.keys(v).length === 1 && Array.isArray(v.$each) ? (v.$each as unknown[]) : [v])
isPlainObject(v) && Object.keys(v).length === 1 && Array.isArray(v.$each) ? (v.$each as unknown[]) : [v]
for (const [path, v] of Object.entries(spec.$set ?? {})) { for (const [path, v] of Object.entries(spec.$set ?? {})) {
touch(path) touch(path)
+2 -1
View File
@@ -81,7 +81,8 @@ export function unsetPath(doc: Doc, path: string): void {
if (typeof v !== 'object' || v === null) return if (typeof v !== 'object' || v === null) return
} }
const last = segs.at(-1)! const last = segs.at(-1)!
if (Array.isArray(v) && /^\d+$/.test(last)) v[Number(last)] = null // Mongo semantics: keep positions if (Array.isArray(v) && /^\d+$/.test(last))
v[Number(last)] = null // Mongo semantics: keep positions
else if (isPlainObject(v)) delete v[last] else if (isPlainObject(v)) delete v[last]
} }
+4 -1
View File
@@ -30,7 +30,10 @@ export class UniqueConstraintError extends PotentialStorageError {
/** The driver lacks an optional capability required by the requested feature. */ /** The driver lacks an optional capability required by the requested feature. */
export class CapabilityError extends PotentialStorageError { export class CapabilityError extends PotentialStorageError {
constructor(readonly capability: string, detail?: string) { constructor(
readonly capability: string,
detail?: string,
) {
super(`Driver lacks capability "${capability}"${detail ? `: ${detail}` : ''}`) super(`Driver lacks capability "${capability}"${detail ? `: ${detail}` : ''}`)
} }
} }
-1
View File
@@ -400,4 +400,3 @@ export class KVStorage<S extends Spec = AnySpec> extends KVNode<S> {
super({ driver, sep, registry, readRepair: opts.readRepair ?? true }, [], '', '', registry.trie.rootState, undefined) super({ driver, sep, registry, readRepair: opts.readRepair ?? true }, [], '', '', registry.trie.rootState, undefined)
} }
} }
-1
View File
@@ -17,7 +17,6 @@ export interface SortedMap<V> {
keys(range: { gte?: string; lt?: string; reverse?: boolean }): IterableIterator<string> keys(range: { gte?: string; lt?: string; reverse?: boolean }): IterableIterator<string>
} }
/** /**
* Sorted chunks of at most `maxChunk` keys (split in half on overflow) plus a `Map` for values. * Sorted chunks of at most `maxChunk` keys (split in half on overflow) plus a `Map` for values.
* Inserts and deletes cost O(log n + maxChunk); iteration is O(1) per step while the map * Inserts and deletes cost O(log n + maxChunk); iteration is O(1) per step while the map
+2 -3
View File
@@ -22,9 +22,8 @@ export type KeyString<S> = `${KeyOf<S>}`
export type EntryOf<S> = { [K in KeyOf<S>]: [K, ValueOf<ChildOf<S, K>>] }[KeyOf<S>] export type EntryOf<S> = { [K in KeyOf<S>]: [K, ValueOf<ChildOf<S, K>>] }[KeyOf<S>]
/** `[key, value]` as yielded at runtime (string keys). Dynamic levels collapse to `[string, V]`. */ /** `[key, value]` as yielded at runtime (string keys). Dynamic levels collapse to `[string, V]`. */
export type StringEntryOf<S> = string extends KeyOf<S> export type StringEntryOf<S> =
? [string, ValueOf<ChildOf<S, KeyOf<S> & string>>] string extends KeyOf<S> ? [string, ValueOf<ChildOf<S, KeyOf<S> & string>>] : { [K in KeyOf<S>]: [`${K}`, ValueOf<ChildOf<S, K>>] }[KeyOf<S>]
: { [K in KeyOf<S>]: [`${K}`, ValueOf<ChildOf<S, K>>] }[KeyOf<S>]
/** A dynamic level of children with values `V`, each with children `C`. */ /** A dynamic level of children with values `V`, each with children `C`. */
export type Many<V, C extends Record<string, Spec> = {}> = { children: { [id: string]: { value: V; children: C } } } export type Many<V, C extends Record<string, Spec> = {}> = { children: { [id: string]: { value: V; children: C } } }
+2 -1
View File
@@ -32,7 +32,8 @@ describe('encodeIndexValue', () => {
fc.assert( fc.assert(
fc.property(indexValue, separators, (v, sep) => { fc.property(indexValue, separators, (v, sep) => {
const decoded = decodeIndexValue(encodeIndexValue(v, sep), sep) const decoded = decodeIndexValue(encodeIndexValue(v, sep), sep)
if (typeof v === 'number' && v === 0) expect(decoded).toBe(0) // -0 normalised if (typeof v === 'number' && v === 0)
expect(decoded).toBe(0) // -0 normalised
else expect(decoded).toEqual(v) else expect(decoded).toEqual(v)
}), }),
) )
+1 -10
View File
@@ -1,15 +1,6 @@
import { describe, expect, it } from 'vitest' import { describe, expect, it } from 'vitest'
import fc from 'fast-check' import fc from 'fast-check'
import { import { escapeSegment, joinPath, pathKeys, splitPath, succ, toSegment, unescapeSegment, validateSeparator } from '../../src/core/path.js'
escapeSegment,
joinPath,
pathKeys,
splitPath,
succ,
toSegment,
unescapeSegment,
validateSeparator,
} from '../../src/core/path.js'
import { InvalidKeyError } from '../../src/errors.js' import { InvalidKeyError } from '../../src/errors.js'
const separators = fc.constantFrom('/', ':', '.', '|', '#', '\\', '!') const separators = fc.constantFrom('/', ':', '.', '|', '#', '\\', '!')
+37 -25
View File
@@ -157,7 +157,7 @@ describe.each([
it('evaluates comparison, logical and element operators', async () => { it('evaluates comparison, logical and element operators', async () => {
const { users } = await seed() const { users } = await seed()
// Unsorted results come in index or key order depending on the plan; compare as sets. // Unsorted results come in index or key order depending on the plan; compare as sets.
const q = async (f: Filter) => (await keysOf(users.find(f))).sort() const q = async (f: Filter) => (await keysOf(users.find(f))).sort()
expect(await q({})).toEqual(['a', 'b', 'c', 'd', 'e']) expect(await q({})).toEqual(['a', 'b', 'c', 'd', 'e'])
expect(await q({ age: 20 })).toEqual(['b', 'd']) expect(await q({ age: 20 })).toEqual(['b', 'd'])
expect(await q({ age: { $gte: 20, $lt: 40 } })).toEqual(['a', 'b', 'd']) expect(await q({ age: { $gte: 20, $lt: 40 } })).toEqual(['a', 'b', 'd'])
@@ -186,7 +186,9 @@ describe.each([
expect(await keysOf(users.find({}, { sort: { name: -1 }, skip: 1, limit: 2 }))).toEqual(['d', 'c']) expect(await keysOf(users.find({}, { sort: { name: -1 }, skip: 1, limit: 2 }))).toEqual(['d', 'c'])
expect(await keysOf(users.find({ age: { $gte: 20 } }, { sort: { age: -1 }, limit: 2 }))).toEqual(['c', 'a']) expect(await keysOf(users.find({ age: { $gte: 20 } }, { sort: { age: -1 }, limit: 2 }))).toEqual(['c', 'a'])
expect(await keysOf(users.find({ 'address.city': 'London' }, { sort: { age: 1 } }))).toEqual(['e', 'a', 'c']) expect(await keysOf(users.find({ 'address.city': 'London' }, { sort: { age: 1 } }))).toEqual(['e', 'a', 'c'])
expect(await collect(users.find({ age: 36 }, { projection: { name: 1, 'address.city': 1 } }))).toEqual([{ key: 'a', data: { name: 'Ada', address: { city: 'London' } } }]) expect(await collect(users.find({ age: 36 }, { projection: { name: 1, 'address.city': 1 } }))).toEqual([
{ key: 'a', data: { name: 'Ada', address: { city: 'London' } } },
])
expect(await collect(users.find({ age: 36 }, { projection: { tags: 0, address: 0, email: 0 } }))).toEqual([{ key: 'a', data: { name: 'Ada', age: 36 } }]) expect(await collect(users.find({ age: 36 }, { projection: { tags: 0, address: 0, email: 0 } }))).toEqual([{ key: 'a', data: { name: 'Ada', age: 36 } }])
expect(await users.findOne({ email: 'bob@x' })).toEqual({ key: 'b', data: expect.objectContaining({ name: 'Bob' }) }) expect(await users.findOne({ email: 'bob@x' })).toEqual({ key: 'b', data: expect.objectContaining({ name: 'Bob' }) })
expect(await users.findOne({ email: 'nobody' })).toBeUndefined() expect(await users.findOne({ email: 'nobody' })).toBeUndefined()
@@ -278,7 +280,11 @@ describe.each([
it('catchUp() repairs a subtree', async () => { it('catchUp() repairs a subtree', async () => {
const { users, docs, events } = await lagged() const { users, docs, events } = await lagged()
await events.child('users').child('9').child('posts').append('p', [{ op: 'set', doc: { createdAt: 'x' } }]) await events
.child('users')
.child('9')
.child('posts')
.append('p', [{ op: 'set', doc: { createdAt: 'x' } }])
expect(await docs.catchUp()).toBe(2) expect(await docs.catchUp()).toBe(2)
expect(await users.get('1')).toEqual({ n: 2, email: 'b@x' }) expect(await users.get('1')).toEqual({ n: 2, email: 'b@x' })
expect(await users.child('9').child('posts').get('p')).toEqual({ createdAt: 'x' }) expect(await users.child('9').child('posts').get('p')).toEqual({ createdAt: 'x' })
@@ -339,28 +345,34 @@ describe('property: indexed find equals unindexed find', () => {
it('returns the same results in the same order', async () => { it('returns the same results in the same order', async () => {
await fc.assert( await fc.assert(
fc.asyncProperty(fc.array(fc.tuple(fc.constantFrom('1', '2', '3', '4', '5'), doc), { maxLength: 12 }), filter, sort, fc.option(fc.integer({ min: 1, max: 4 }), { nil: undefined }), async (writes, f, s, limit) => { fc.asyncProperty(
const plain = new EventSourcedDocumentStorage(new KVEventStorage(new KVStorage(new InMemoryKVDriver())), { warnOnFullScan: false }).child('users') fc.array(fc.tuple(fc.constantFrom('1', '2', '3', '4', '5'), doc), { maxLength: 12 }),
const indexed = new EventSourcedDocumentStorage(new KVEventStorage(new KVStorage(new InMemoryKVDriver())), { filter,
warnOnFullScan: false, sort,
indexes: { users: userIndexes.users!.map((i) => ({ ...i, unique: false })) }, fc.option(fc.integer({ min: 1, max: 4 }), { nil: undefined }),
}).child('users') async (writes, f, s, limit) => {
for (const [k, v] of writes) { const plain = new EventSourcedDocumentStorage(new KVEventStorage(new KVStorage(new InMemoryKVDriver())), { warnOnFullScan: false }).child('users')
await plain.set(k, v) const indexed = new EventSourcedDocumentStorage(new KVEventStorage(new KVStorage(new InMemoryKVDriver())), {
await indexed.set(k, v) warnOnFullScan: false,
} indexes: { users: userIndexes.users!.map((i) => ({ ...i, unique: false })) },
const opts = { ...(s ? { sort: s } : {}), ...(limit ? { limit } : {}) } }).child('users')
const a = await collect(plain.find(f, opts)) for (const [k, v] of writes) {
const b = await collect(indexed.find(f, opts)) await plain.set(k, v)
if (s) { await indexed.set(k, v)
expect(b).toEqual(a) }
return const opts = { ...(s ? { sort: s } : {}), ...(limit ? { limit } : {}) }
} const a = await collect(plain.find(f, opts))
// Unsorted order depends on the plan: compare as sets (a limited result is any subset of that size). const b = await collect(indexed.find(f, opts))
const all = new Set((await collect(plain.find(f))).map((r) => r.key)) if (s) {
expect(b).toHaveLength(a.length) expect(b).toEqual(a)
for (const r of b) expect(all.has(r.key)).toBe(true) return
}), }
// Unsorted order depends on the plan: compare as sets (a limited result is any subset of that size).
const all = new Set((await collect(plain.find(f))).map((r) => r.key))
expect(b).toHaveLength(a.length)
for (const r of b) expect(all.has(r.key)).toBe(true)
},
),
{ numRuns: 300 }, { numRuns: 300 },
) )
}) })
+7 -1
View File
@@ -45,7 +45,13 @@ describe('filter semantics', () => {
describe('update resolution', () => { describe('update resolution', () => {
it('$pull by sub-document filter and by condition', () => { it('$pull by sub-document filter and by condition', () => {
const state = { items: [{ k: 1, v: 'a' }, { k: 2, v: 'b' }], n: [1, 5, 9] } const state = {
items: [
{ k: 1, v: 'a' },
{ k: 2, v: 'b' },
],
n: [1, 5, 9],
}
const patch = resolveUpdate(state, { $pull: { items: { k: 2 }, n: { $gte: 5 } } }) const patch = resolveUpdate(state, { $pull: { items: { k: 2 }, n: { $gte: 5 } } })
expect(applyPatch(state, patch)).toEqual({ items: [{ k: 1, v: 'a' }], n: [1] }) expect(applyPatch(state, patch)).toEqual({ items: [{ k: 1, v: 'a' }], n: [1] })
}) })
+1 -2
View File
@@ -40,8 +40,7 @@ export class CountingDriver implements KVDriver {
readonly inner: KVDriver = new InMemoryKVDriver(), readonly inner: KVDriver = new InMemoryKVDriver(),
hide: Capability[] = [], hide: Capability[] = [],
) { ) {
if (inner.setIfAbsent && !hide.includes('setIfAbsent')) if (inner.setIfAbsent && !hide.includes('setIfAbsent')) this.setIfAbsent = (e: [string, unknown][]) => (this.#count('setIfAbsent'), inner.setIfAbsent!(e))
this.setIfAbsent = (e: [string, unknown][]) => (this.#count('setIfAbsent'), inner.setIfAbsent!(e))
if (inner.batch && !hide.includes('batch')) this.batch = (ops: BatchOp[]) => (this.#count('batch'), inner.batch!(ops)) if (inner.batch && !hide.includes('batch')) this.batch = (ops: BatchOp[]) => (this.#count('batch'), inner.batch!(ops))
if (inner.deleteRange && !hide.includes('deleteRange')) if (inner.deleteRange && !hide.includes('deleteRange'))
this.deleteRange = (gte: string, lt: string) => (this.#count('deleteRange'), inner.deleteRange!(gte, lt)) this.deleteRange = (gte: string, lt: string) => (this.#count('deleteRange'), inner.deleteRange!(gte, lt))
+14 -3
View File
@@ -204,7 +204,10 @@ describe('index declarations', () => {
}) })
describe('property: findBy equals brute-force filter', () => { describe('property: findBy equals brute-force filter', () => {
const val = fc.record({ age: fc.option(fc.integer({ min: -3, max: 3 }), { nil: undefined }), tags: fc.array(fc.constantFrom('a', 'b', 'c/d'), { maxLength: 3 }) }) const val = fc.record({
age: fc.option(fc.integer({ min: -3, max: 3 }), { nil: undefined }),
tags: fc.array(fc.constantFrom('a', 'b', 'c/d'), { maxLength: 3 }),
})
const op = fc.oneof( const op = fc.oneof(
fc.record({ t: fc.constant('set' as const), k: fc.constantFrom('1', '2', '3', 'x/y'), v: val }), fc.record({ t: fc.constant('set' as const), k: fc.constantFrom('1', '2', '3', 'x/y'), v: val }),
fc.record({ t: fc.constant('del' as const), k: fc.constantFrom('1', '2', '3', 'x/y') }), fc.record({ t: fc.constant('del' as const), k: fc.constantFrom('1', '2', '3', 'x/y') }),
@@ -226,11 +229,19 @@ describe('property: findBy equals brute-force filter', () => {
} }
} }
const got = (await collect(users.findBy('byAge', { gte: lo, lt: hi }))).map(([k]) => k).sort() const got = (await collect(users.findBy('byAge', { gte: lo, lt: hi }))).map(([k]) => k).sort()
const exp = [...model].filter(([, v]) => v.age !== undefined && v.age >= lo && v.age < hi).map(([k]) => k).sort() const exp = [...model]
.filter(([, v]) => v.age !== undefined && v.age >= lo && v.age < hi)
.map(([k]) => k)
.sort()
expect(got).toEqual(exp) expect(got).toEqual(exp)
for (const tag of ['a', 'b', 'c/d']) { for (const tag of ['a', 'b', 'c/d']) {
const g = (await collect(users.findBy('byTag', tag))).map(([k]) => k).sort() const g = (await collect(users.findBy('byTag', tag))).map(([k]) => k).sort()
expect(g).toEqual([...model].filter(([, v]) => v.tags?.includes(tag)).map(([k]) => k).sort()) expect(g).toEqual(
[...model]
.filter(([, v]) => v.tags?.includes(tag))
.map(([k]) => k)
.sort(),
)
} }
// No stale entries are ever left behind by clean writes. // No stale entries are ever left behind by clean writes.
let entryCount = 0 let entryCount = 0
+51 -47
View File
@@ -293,56 +293,60 @@ describe('model-based (random ops vs reference Map)', () => {
] as const) { ] as const) {
it(`matches the model (${name})`, async () => { it(`matches the model (${name})`, async () => {
await fc.assert( await fc.assert(
fc.asyncProperty(fc.array(opArb, { maxLength: 30 }), fc.array(fc.array(segArb, { maxLength: 2 }), { minLength: 1, maxLength: 3 }), async (ops, probes) => { fc.asyncProperty(
const db = new KVStorage(make()) fc.array(opArb, { maxLength: 30 }),
const model = new Map<string, unknown>() fc.array(fc.array(segArb, { maxLength: 2 }), { minLength: 1, maxLength: 3 }),
for (const op of ops) { async (ops, probes) => {
switch (op.t) { const db = new KVStorage(make())
case 'set': { const model = new Map<string, unknown>()
if (op.selfForm) await nodeAt(db, op.path).set(op.value) for (const op of ops) {
else await nodeAt(db, op.path.slice(0, -1)).set(op.path.at(-1)!, op.value) switch (op.t) {
model.set(enc(op.path), op.value) case 'set': {
break if (op.selfForm) await nodeAt(db, op.path).set(op.value)
} else await nodeAt(db, op.path.slice(0, -1)).set(op.path.at(-1)!, op.value)
case 'delete': { model.set(enc(op.path), op.value)
if (op.selfForm) await nodeAt(db, op.path).delete() break
else await nodeAt(db, op.path.slice(0, -1)).delete(op.path.at(-1)!) }
model.delete(enc(op.path)) case 'delete': {
break if (op.selfForm) await nodeAt(db, op.path).delete()
} else await nodeAt(db, op.path.slice(0, -1)).delete(op.path.at(-1)!)
case 'deleteTree': { model.delete(enc(op.path))
await nodeAt(db, op.path).deleteTree() break
for (const k of [...model.keys()]) if (isPrefix(op.path, JSON.parse(k))) model.delete(k) }
break case 'deleteTree': {
} await nodeAt(db, op.path).deleteTree()
case 'setMany': { for (const k of [...model.keys()]) if (isPrefix(op.path, JSON.parse(k))) model.delete(k)
await nodeAt(db, op.parent).setMany(op.entries) break
for (const [k, v] of op.entries) model.set(enc([...op.parent, k]), v) }
break case 'setMany': {
await nodeAt(db, op.parent).setMany(op.entries)
for (const [k, v] of op.entries) model.set(enc([...op.parent, k]), v)
break
}
} }
} }
} for (const [k, v] of model) expect(await nodeAt(db, JSON.parse(k)).get()).toEqual(v)
for (const [k, v] of model) expect(await nodeAt(db, JSON.parse(k)).get()).toEqual(v) for (const parent of probes) {
for (const parent of probes) { const node = nodeAt(db, parent)
const node = nodeAt(db, parent) const exp = expectedChildren(model, parent)
const exp = expectedChildren(model, parent) const keys = exp.map((e) => e.key)
const keys = exp.map((e) => e.key) expect(await collect(node.keys())).toEqual(keys)
expect(await collect(node.keys())).toEqual(keys) expect(await collect(node.keys({ reverse: true }))).toEqual([...keys].reverse())
expect(await collect(node.keys({ reverse: true }))).toEqual([...keys].reverse()) expect(await collect(node.entries())).toEqual(exp.filter((e) => e.hasOwn).map((e) => [e.key, e.own]))
expect(await collect(node.entries())).toEqual(exp.filter((e) => e.hasOwn).map((e) => [e.key, e.own])) expect(await collect(node.entries({ reverse: true }))).toEqual(
expect(await collect(node.entries({ reverse: true }))).toEqual( exp
exp .filter((e) => e.hasOwn)
.filter((e) => e.hasOwn) .map((e) => [e.key, e.own])
.map((e) => [e.key, e.own]) .reverse(),
.reverse(), )
) for (let i = 0; i < keys.length; i++) {
for (let i = 0; i < keys.length; i++) { expect(await collect(node.keys({ after: keys[i]! }))).toEqual(keys.slice(i + 1))
expect(await collect(node.keys({ after: keys[i]! }))).toEqual(keys.slice(i + 1)) expect(await collect(node.keys({ after: keys[i]!, reverse: true }))).toEqual(keys.slice(0, i).reverse())
expect(await collect(node.keys({ after: keys[i]!, reverse: true }))).toEqual(keys.slice(0, i).reverse()) expect(await collect(node.keys({ limit: i + 1 }))).toEqual(keys.slice(0, i + 1))
expect(await collect(node.keys({ limit: i + 1 }))).toEqual(keys.slice(0, i + 1)) }
} }
} },
}), ),
{ numRuns: 300 }, { numRuns: 300 },
) )
}, 60_000) }, 60_000)
+24 -19
View File
@@ -36,26 +36,31 @@ describe('ChunkedSortedMap (model-based, tiny chunks)', () => {
it('iterators resume correctly after mutations between steps', () => { it('iterators resume correctly after mutations between steps', () => {
fc.assert( fc.assert(
fc.property(fc.uniqueArray(key, { minLength: 1, maxLength: 60 }), fc.array(fc.tuple(fc.boolean(), key), { maxLength: 40 }), fc.boolean(), (initial, muts, reverse) => { fc.property(
const m = new ChunkedSortedMap<number>(3) fc.uniqueArray(key, { minLength: 1, maxLength: 60 }),
initial.forEach((k) => m.set(k, 0)) fc.array(fc.tuple(fc.boolean(), key), { maxLength: 40 }),
const live = new Set(initial) fc.boolean(),
const seen: string[] = [] (initial, muts, reverse) => {
let step = 0 const m = new ChunkedSortedMap<number>(3)
for (const k of m.keys({ reverse })) { initial.forEach((k) => m.set(k, 0))
seen.push(k) const live = new Set(initial)
const mut = muts[step++] const seen: string[] = []
if (mut) { let step = 0
const [add, mk] = mut for (const k of m.keys({ reverse })) {
if (add) (m.set(mk, 1), live.add(mk)) seen.push(k)
else (m.delete(mk), live.delete(mk)) const mut = muts[step++]
if (mut) {
const [add, mk] = mut
if (add) (m.set(mk, 1), live.add(mk))
else (m.delete(mk), live.delete(mk))
}
} }
} // Strictly monotone, and every key present throughout the iteration was visited.
// Strictly monotone, and every key present throughout the iteration was visited. for (let i = 1; i < seen.length; i++) expect(reverse ? seen[i]! < seen[i - 1]! : seen[i]! > seen[i - 1]!).toBe(true)
for (let i = 1; i < seen.length; i++) expect(reverse ? seen[i]! < seen[i - 1]! : seen[i]! > seen[i - 1]!).toBe(true) const stable = initial.filter((k) => !muts.some(([, mk]) => mk === k))
const stable = initial.filter((k) => !muts.some(([, mk]) => mk === k)) for (const k of stable) expect(seen).toContain(k)
for (const k of stable) expect(seen).toContain(k) },
}), ),
{ numRuns: 500 }, { numRuns: 500 },
) )
}) })
+16 -7
View File
@@ -47,12 +47,18 @@ describe('KV types (section 7)', () => {
db.child('users').child('1').set({ username: 'a', age: 1 }) db.child('users').child('1').set({ username: 'a', age: 1 })
// @ts-expect-error wrong value type // @ts-expect-error wrong value type
db.child('users').set('1', { username: 'a' }) db.child('users').set('1', { username: 'a' })
db.child('profile').child('9').setMany([ db.child('profile')
['age', 3], .child('9')
['username', 'x'], .setMany([
]) ['age', 3],
// @ts-expect-error value doesn't match its key ['username', 'x'],
db.child('profile').child('9').setMany([['age', 'x']]) ])
db.child('profile')
.child('9')
.setMany([
// @ts-expect-error value doesn't match its key
['age', 'x'],
])
}) })
it('types listings', () => { it('types listings', () => {
@@ -82,7 +88,10 @@ describe('event types', () => {
orders.child('o1').append([{ type: 'paid' }]) orders.child('o1').append([{ type: 'paid' }])
// @ts-expect-error not an OrderEvent // @ts-expect-error not an OrderEvent
orders.append('o1', [{ type: 'refunded' }]) orders.append('o1', [{ type: 'refunded' }])
orders.child('o1').child('shipments').append('s1', [{ carrier: 'x' }]) orders
.child('o1')
.child('shipments')
.append('s1', [{ carrier: 'x' }])
}) })
it('types read results', () => { it('types read results', () => {