Plain operations: last write wins, hash-keyed events, simpler composite
- KV: one write primitive, write(changes) with { key, value? }; writeIfAbsent removed.
The server executes exactly what clients send and never reads values, so they can be
encrypted end to end. Adds setFields/getFields on the base class.
- HTTP: no server history or event ids; SSE relays every change (own writes included) and
the client emits resync when a dropped stream reconnects.
- Composite: writes apply locally, then go to the source (undone locally if refused);
pushed changes are applied and announced only where they differ from local values;
offline writes go to a queue store (any KV storage); sync() = replay, diff, replay.
Removes KVReplicaSet, KVOutbox, KeyCounter, SerialQueue and RetryTimer.
- Events: id = <time>_<sha256(time + canonical data)>, so concurrent appends never
collide and retries can't duplicate; no versions or ConcurrencyError.
- Event-sourced documents: read, resolve, append, apply; catchUp folds in foreign events;
getAt takes an event id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
This commit is contained in:
Claude
@@ -57,28 +57,32 @@ bench/
|
||||
|
||||
## 4. KV storage
|
||||
|
||||
`BaseKVStorage` primitives (absolute keys). Implementations provide the reads and the protected
|
||||
`apply*` writes; the public `writeMany` / `removeMany` / `writeIfAbsent` live in the base class and
|
||||
emit a `change` event after each write, so every store emits change events.
|
||||
`BaseKVStorage` primitives (absolute keys). Implementations provide two reads and one protected
|
||||
write; the public `writeMany(changes)` / `removeMany(keys)` live in the base class and emit a
|
||||
`change` event after each write, so every store emits change events.
|
||||
|
||||
| Primitive | |
|
||||
| ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------- |
|
||||
| `readMany(keys)` | values in order, `undefined` for missing |
|
||||
| `scan({ gte, lt, limit, reverse, values })` | key-ordered; must keep working if keys change mid-scan (keys present throughout appear exactly once) |
|
||||
| `applyWrite(entries)` → public `writeMany` | |
|
||||
| `applyRemove(keys)` → public `removeMany` | |
|
||||
| `applyWriteIfAbsent(entries)` → public `writeIfAbsent` | `true` = written; atomic per key (needed by events) |
|
||||
| Primitive | |
|
||||
| ------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
|
||||
| `readMany(keys)` | values in order, `undefined` for missing |
|
||||
| `scan({ gte, lt, limit, reverse, values })` | key-ordered; must keep working if keys change mid-scan (keys present throughout appear exactly once) |
|
||||
| `write(changes)` → public `writeMany` | `{ key, value }` sets, `{ key }` deletes, in one batch |
|
||||
|
||||
There is no conditional write. Stores execute exactly the operations they are given, and the last
|
||||
write wins. Data that can be written concurrently belongs in event storage (section 5), where
|
||||
concurrent appends never touch the same key. Data whose key is the hash of its content is
|
||||
immutable, so concurrent writes of it are identical.
|
||||
|
||||
**Values** are JSON plus `bigint` and `Uint8Array` (as in JSON, `-0` becomes `0` and `undefined` properties are dropped). Anything crossing the network is encoded as
|
||||
Extended JSON (`<bigint: 123n>`, `<uint8array: abcd>`); IndexedDB stores them natively.
|
||||
|
||||
**Events** (`EventEmitter`, shared by all nodes of a storage): `change` →
|
||||
`{ changes: { key, value? }[], origin: 'local' | 'remote', id? }` (no `value` = removed), and
|
||||
`resync` → changes may have been missed, so reload.
|
||||
`{ changes: { key, value? }[], origin: 'local' | 'remote' }` (no `value` = removed), and
|
||||
`resync` → changes may have been missed, so re-read.
|
||||
|
||||
Node API (defined once): `child`, `as`, `get(key?)`, `getMany`, `set(value)` / `set(key, value)`,
|
||||
`setMany`, `delete(key?)`, `deleteMany`, `deleteTree(key?)`, `keys({ limit, after, reverse })`,
|
||||
`entries(...)`. The root has no own value. A node's own value and its children are independent keys.
|
||||
`setMany`, `setFields({ a, b })` / `getFields()` (direct children as an object), `delete(key?)`,
|
||||
`deleteMany`, `deleteTree(key?)`, `keys({ limit, after, reverse })`, `entries(...)`. The root has
|
||||
no own value. A node's own value and its children are independent keys.
|
||||
|
||||
- **`keys()`** is a skip-scan: one seek per child that has descendants; siblings without
|
||||
descendants share one iterator. It lists children that only have descendants, and skips
|
||||
@@ -89,23 +93,22 @@ Node API (defined once): `child`, `as`, `get(key?)`, `getMany`, `set(value)` / `
|
||||
- `KVStorageInMemory`: `SortedMap` (sorted chunks of ≤1024 keys); values deep-copied.
|
||||
- `KVStorageIndexedDB`: one object store with string keys. Scans read in batches, each in its own
|
||||
transaction, resuming after the last key (IndexedDB transactions close when the code awaits
|
||||
anything else). `writeIfAbsent` runs in one readwrite transaction.
|
||||
anything else). A write is one readwrite transaction.
|
||||
- `KVStorageHttp` (client) and `KVStorageHttpServer` (wraps any KV storage;
|
||||
`handle(Request) → Response`): `POST /<primitive>` with Extended JSON bodies; `scan` streams
|
||||
NDJSON. Unreachable server (fetch failure, 502/503/504) → `StorageUnavailableError`. The server
|
||||
exposes the whole underlying store (primitives use absolute keys).
|
||||
`handle(Request) → Response`): `POST /readMany`, `/write`, `/scan` with Extended JSON bodies;
|
||||
`scan` streams NDJSON. Unreachable server (fetch failure, 502/503/504) →
|
||||
`StorageUnavailableError`. The server executes exactly what clients send and never reads values
|
||||
for decisions, so values can be encrypted end to end.
|
||||
- `KVStorageComposite`: see 4.2.
|
||||
- **Change stream (SSE).** `POST /events` with `{ paths }` streams changes to those keys and their
|
||||
subtrees as `event: change` with id `<epoch>-<seq>` (epoch = server instance). The server keeps the last
|
||||
`historySize` (10 000) change batches.
|
||||
- A client reconnecting with `Last-Event-ID` gets the batches it missed, or `event: resync` if it
|
||||
can't be resumed (another epoch, or too old).
|
||||
- Every catch-up ends with `event: ready`.
|
||||
- Keep-alive comments are sent every 15 s.
|
||||
- `KVStorageHttp.subscribe(path = this.key)` / `unsubscribe(path)` manage the subscribed paths.
|
||||
Each change reconnects the stream (vendored `SSESession`, backoff) with the new list, resuming
|
||||
from `lastEventId`. The client emits `change` (`origin: 'remote'`) and `resync`, and `subscribe`
|
||||
resolves once caught up. A 401/403 rejects `subscribe` instead of retrying.
|
||||
- **Change stream (SSE).** `POST /events` with `{ paths }` streams every change to those keys and
|
||||
their subtrees as `event: change` (data: the changes), including the subscriber's own writes.
|
||||
`event: ready` follows each connection; keep-alive comments are sent every 15 s. The server keeps
|
||||
no history: nothing is replayed after a reconnect.
|
||||
- `KVStorageHttp.subscribe(path = this.key)` / `unsubscribe(path)` / `close()` manage the
|
||||
subscribed paths. Each change reconnects the stream (vendored `SSESession`, backoff) with the
|
||||
new list; `subscribe` resolves once connected. A 401/403 rejects instead of retrying.
|
||||
- The client emits each pushed batch as `change` (`origin: 'remote'`), and `resync` whenever the
|
||||
stream reconnects after a drop, since changes may have been missed meanwhile.
|
||||
- Limitation: a server only sees writes made through its own storage instance, so several server
|
||||
processes over one database need a shared pub/sub (not built).
|
||||
- **Wrappers** (e.g. a future encryption layer) extend `BaseKVStorage` and delegate the primitives.
|
||||
@@ -130,83 +133,67 @@ development and tests). It can't be left open by omission.
|
||||
dependencies; Node 22+ and current browsers; keys persist as PKCS #8). Others, e.g. secp256k1 via
|
||||
libauth, implement `RequestSigner` (`algorithm`, `publicKey`, `sign`) and `SignatureVerifier`
|
||||
(`algorithm`, `verify`). The server can accept several algorithms at once.
|
||||
- **Authorization (403): the public key is the user id.** Every key in `readMany`, `writeMany`,
|
||||
`removeMany` and `writeIfAbsent`, and every subscription path, must be `<publicKey>` or under
|
||||
- **Authorization (403): the public key is the user id.** Every key in `readMany` and `write`, and
|
||||
every subscription path, must be `<publicKey>` or under
|
||||
`<publicKey>/`. A scan must stay within `[<publicKey>, <publicKey>0)`, i.e. the key and its
|
||||
subtree (escaping guarantees no other key falls in that range, including lookalikes such as
|
||||
`<publicKey>x`).
|
||||
- **Scoped events.** Each subscriber only receives changes under its own (authorized) paths, live
|
||||
and when resuming. Event ids are a global sequence, so a client can infer the server's overall
|
||||
write volume, though nothing about the contents.
|
||||
- **Scoped events.** Each subscriber only receives changes under its own (authorized) paths.
|
||||
- Proxies must preserve the URL path, because the signature covers it.
|
||||
|
||||
### 4.2 `KVStorageComposite({ source, prefix?, replicas, retry? })`
|
||||
### 4.2 `KVStorageComposite({ source, replicas, queue?, prefix?, retryMs? })`
|
||||
|
||||
Keeps local copies of a source (e.g. `KVStorageHttp`) so reads are local and the app works
|
||||
offline. With `prefix` (e.g. the user's public key) only that key and its subtree are synced, cached
|
||||
and subscribed to; keys outside it throw `RangeError`. Without it, the whole source is copied, which
|
||||
is only appropriate when the client may read all of it. Replicas are listed fastest first (e.g. `[inMemory, indexedDb]`). The last replica is the
|
||||
persistent one: it seeds the others on start and stores the composite's internal keys under `~q/`
|
||||
(hidden from reads): `~q/meta` (`{ lastEventId }`, present once a sync has completed) and the
|
||||
outbox `~q/o/<seq:12>`.
|
||||
is only appropriate when the client may read all of it.
|
||||
|
||||
- **Start** (lazy, on first use):
|
||||
1. Load the outbox and copy the persistent replica into the faster ones.
|
||||
2. If a previous sync exists, serve reads locally straight away.
|
||||
3. Watch the source, resuming from `lastEventId`.
|
||||
4. Run a full sync only when there's no resume point, when the source sends `resync`, or when the
|
||||
source can't be watched.
|
||||
5. Before the first sync completes, reads go to the source.
|
||||
- **Remote changes** (and full syncs) are applied in order through one queue. A change that arrives
|
||||
during a full sync waits behind it, so the end state always matches the source. `lastEventId` is
|
||||
saved after each applied batch. Changes the replicas already hold are skipped.
|
||||
- **Full sync** merge-scans the source against the persistent replica (both key-ordered) and applies
|
||||
the difference to every replica in batches of 500.
|
||||
- **Writes** go to the source, then every replica.
|
||||
- On `StorageUnavailableError` (or while anything is queued, to keep order), the write is applied
|
||||
locally and appended to the outbox. The outbox is replayed in order with exponential backoff,
|
||||
and also whenever the source shows signs of life. Last writer wins on conflicts.
|
||||
- Remote changes to keys with queued writes are ignored until those writes are replayed.
|
||||
- `writeIfAbsent` always needs the source.
|
||||
- **Echo tracking.** Each write sent to a watched source expects one echo per key. A remote change
|
||||
to a key with an outstanding echo is consumed instead of applied. The stream is in server order,
|
||||
so that change is either the echo or older than it, and this avoids briefly showing a stale value.
|
||||
A full sync clears the counts.
|
||||
- Write through the composite, not directly through the same source instance (its `local`-origin
|
||||
changes are not mirrored).
|
||||
- `start()`, `synced()` (complete copy, all received changes applied), `queued` (outbox size),
|
||||
`close()`.
|
||||
|
||||
**Structure.** The composite only orchestrates (what to read from, when to sync, whether to queue a
|
||||
write). The work is done by small classes:
|
||||
|
||||
| Class | Job |
|
||||
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `KVReplicaSet` | the replicas of one `KeyRange`: seed the faster ones, apply changes, scan locally (hiding `~q/` bookkeeping), diff against the source, sync metadata |
|
||||
| `KVOutbox` | the persisted offline queue (`~q/o/<seq>`), its pending-key counts, and in-order replay with backoff |
|
||||
| `KeyRange` (utils) | a key plus its subtree: `contains`, `clamp` scan bounds, `covers` a scan; also used by the server's authorization |
|
||||
| `KeyCounter` (utils) | per-key counts: pending outbox writes and expected echoes |
|
||||
| `SerialQueue` (utils) | one-at-a-time tasks, optionally per key: applying remote changes, subscription updates, event-sourced writes |
|
||||
| `RetryTimer` (utils) | backoff retries, one timer per job (full sync and outbox replay retry independently) |
|
||||
- **Replicas** are listed fastest first (e.g. `[inMemory, indexedDb]`); the last one seeds the
|
||||
others on start. Reads come from the first.
|
||||
- **The queue** is any KV store (default: in memory). It holds writes the source couldn't take
|
||||
(`writes/<ulid>`) and whether a sync has completed (`synced`). Use a persistent node outside the
|
||||
synced range, e.g. `indexedDb.child('~sync')` with `prefix: userKey`; a queue inside the range is
|
||||
refused, since a sync would delete it.
|
||||
- **Writes** are applied to every replica, then sent to the source, exactly as given.
|
||||
- Applying locally first means the source relaying the write back finds nothing new.
|
||||
- A write the source refuses (e.g. 403) is undone locally and rethrown.
|
||||
- On `StorageUnavailableError`, or while a sync runs or writes are queued (to keep order), the
|
||||
write goes to the queue and is sent later. The last write to reach the source wins.
|
||||
- **Remote changes** pushed by the source are applied and announced (`change`, `origin: 'remote'`)
|
||||
only where they differ from what the first replica holds. No write tagging or echo tracking is
|
||||
needed: echoes of this client's writes change nothing.
|
||||
- **`sync()`**: replay the queue in order (send, then re-apply locally) → merge-scan the source
|
||||
against the last replica (both key-ordered), applying the difference in batches of 500 →
|
||||
replay again (writes made meanwhile) → mark `synced`. It runs on start, on `resync` (the stream
|
||||
reconnected), and `retryMs` after a failure while offline.
|
||||
- **Reads** wait for the first sync. Until one has completed, they go to the source. With a
|
||||
persistent queue, a restart reads locally straight away (offline too).
|
||||
- `start()` (also lazy on first use), `sync()`, `queued`, `close()`.
|
||||
- Limitations: catching up re-scans the whole range (no server history); a remote change can
|
||||
overwrite a value whose write is still queued until that write is replayed (last write wins);
|
||||
replicas, source and queue must be root stores (the primitives use absolute keys).
|
||||
|
||||
## 5. Event storage
|
||||
|
||||
`BaseEventStorage` primitives over stream paths: `appendTo`, `readFrom`, `versionOf`, `streamsOf`.
|
||||
Node API: `append(events, opts?)` / `append(key, events, opts?)`, `read(opts?)` / `read(key, opts?)`,
|
||||
`version(key?)`, `keys()`. The root is not a stream. An empty stream is version 0; the first event is 1.
|
||||
`expectedVersion: number | 'any'` (default `'any'`).
|
||||
`BaseEventStorage` primitives over stream paths: `appendTo(stream, events)` → ids,
|
||||
`readFrom(stream, { after, limit, reverse })`, `streamsOf`. Node API: `append(events)` /
|
||||
`append(key, events)` → ids, `read(opts?)` / `read(key, opts?)` → `{ id, at, data }`, `keys()`.
|
||||
The root is not a stream.
|
||||
|
||||
**`EventStorageKV`** (on any KV storage):
|
||||
**Appends never conflict.** Concurrent writers each add their own events, and every reader sees all
|
||||
of them in the same order. There are no versions and no `expectedVersion`; folding events into
|
||||
state is where concurrent changes meet (see event-sourced documents).
|
||||
|
||||
- One key per append, densely numbered: `<stream>/~e/<commit:12>` → `{ firstVersion, at, events }`.
|
||||
Keying by commit number (not version) means any writer with a stale view collides with the
|
||||
existing key.
|
||||
- Append: head from a bounded cache (else one reverse scan, `limit: 1`) → check `expectedVersion`
|
||||
(re-reading the head before rejecting, in case the cache is stale) → `writeIfAbsent`. On a lost race:
|
||||
`ConcurrencyError` if a version was expected, otherwise retry (default 5).
|
||||
- `read({ from })`: commit `n` starts at a version ≥ n, so try commit `from` first (exact for
|
||||
single-event commits), else binary-search; then forward-scan.
|
||||
- Multi-writer safe on any store, because `writeIfAbsent` is a required primitive.
|
||||
**`EventStorageKV`** (on any KV storage): each event is one plain KV write,
|
||||
`<stream>/~e/<time:15>_<sha256(time + canonical Extended JSON of the event)>` → the event.
|
||||
|
||||
- Keys sort by time, so a stream reads in time order with one scan; `after` is an exclusive id.
|
||||
- Events from different writers in the same millisecond order by hash: arbitrary, but the same for
|
||||
every reader. One instance never reuses a time, so its own appends keep their order.
|
||||
- The id can be recomputed from the content, and re-sending an event (e.g. a retried request)
|
||||
writes the same key, so it can't be duplicated.
|
||||
- An append is one `writeMany`, so it works through `KVStorageComposite` offline and syncs like any
|
||||
other data.
|
||||
|
||||
## 6. Document storage
|
||||
|
||||
@@ -240,14 +227,14 @@ Node API: `get(key?)`, `getMany`, `set(doc)` / `set(key, doc)`, `insert(doc)`
|
||||
- `EventSourcedDocumentStorage(events, projections)`: each document's `DocChange`s are appended to
|
||||
its own stream. Current state lives in **any** `BaseDocumentStorage` used as the projection store,
|
||||
which serves all reads, queries and indexes.
|
||||
- Projections carry `_version` (reserved; stripped from results), used as `expectedVersion` on the
|
||||
next append.
|
||||
- If a write crashes between append and projection update, the next write conflicts, rebuilds the
|
||||
projection from the stream, and retries. `catchUp(key?)` rebuilds on demand.
|
||||
- Missing projections with stream history (deleted documents, crashed first writes) are rebuilt
|
||||
before writing.
|
||||
- Writes to one document are serialised in-process, so retries only happen across processes.
|
||||
- Extras: `history(key?)`, `getAt(version)` / `getAt(key, version)`. Children keep these methods
|
||||
- A write reads the projection, resolves the change (`$inc` → `set: { n: 6 }`), appends it, and
|
||||
applies it to the projection.
|
||||
- Concurrent edits to different fields both survive; the same field ends with the later event's
|
||||
value. (Increments from two devices don't add up: each resolves to a final value.)
|
||||
- Events written elsewhere (another device, arriving through sync) are folded in by
|
||||
`catchUp(key?)`, which rebuilds the projection from the whole stream. It also repairs a
|
||||
projection left behind by a crash between append and projection write.
|
||||
- Extras: `history(key?)`, `getAt(eventId)` / `getAt(key, eventId)`. Children keep these methods
|
||||
in their types through interface declaration merging (types only).
|
||||
|
||||
## 7. Changes from the original brief
|
||||
@@ -260,14 +247,16 @@ Node API: `get(key?)`, `getMany`, `set(doc)` / `set(key, doc)`, `insert(doc)`
|
||||
- Separator fixed to `/`, and every character ≤ `/` is escaped so child keys stay contiguous. This
|
||||
removed the subtree-order workaround in child listing.
|
||||
- Event-sourced documents are events + a projection store instead of a bespoke projection format.
|
||||
- `writeIfAbsent` is a required primitive (no single-writer mutex fallback; no optional `batch` /
|
||||
`deleteRange`).
|
||||
- No conditional writes and no versions: KV stores execute exactly what they're given (so a server
|
||||
never needs to read values, which may be encrypted), and events get time + content-hash ids so
|
||||
concurrent appends never collide. Optimistic concurrency (`expectedVersion`) was dropped.
|
||||
- `tree()` removed (use `entries()` for field-per-key reads).
|
||||
|
||||
## 8. Open questions
|
||||
|
||||
- Composite: lazily syncing parts of a large subtree on first read; conflict policies beyond last
|
||||
writer wins for offline writes.
|
||||
- Composite: lazily syncing parts of a large subtree on first read; catching up without a full
|
||||
re-scan (would need server-side history or a change log in event storage).
|
||||
- Event-sourced documents: subscribing to event streams to run `catchUp` automatically.
|
||||
- Authorization beyond "public key = user id" (sharing a subtree with other keys).
|
||||
- Multi-process servers need a shared change feed (pub/sub) for SSE.
|
||||
|
||||
@@ -285,9 +274,10 @@ Node API: `get(key?)`, `getMany`, `set(doc)` / `set(key, doc)`, `insert(doc)`
|
||||
| KV `get` / `set` @ 100k keys | 2.0 / 2.3 µs |
|
||||
| KV `getMany` × 100 | 25–35 µs |
|
||||
| `keys()` on 1k children × 100 descendants (skip-scan vs full scan) | 2.8 ms vs 34 ms |
|
||||
| Event `append` (warm cache) | 7.8 µs |
|
||||
| Event `read` 100 from version 5 000 of 10k | 131 µs |
|
||||
| Event `read` full 10k stream | 18 ms |
|
||||
| Event `append` (one SHA-256 via WebCrypto + one write) | 72 µs |
|
||||
| Event `read` 100 from the middle of 10k | 119 µs |
|
||||
| Event `read` full 10k stream | 11 ms |
|
||||
| Event-sourced document `update ($inc)` | 86 µs |
|
||||
| Documents in-memory @ 10k: `get` / `update` | 1.8 / 6.6 µs |
|
||||
| Documents in-memory: `find` indexed (~100 hits) / scan (~200 of 10k) | 0.20 / 11 ms |
|
||||
| Event-sourced @ 10k: `get` / `update` | 1.6 / 17 µs |
|
||||
|
||||
Reference in New Issue
Block a user