M6: EventSourcedDocumentStorage with projections, update operators, queries, planner, unique indexes, catch-up

Writes append resolved deltas with optimistic concurrency and self-heal lagging
projections. Reads hit projections only. Index-backed find is property-tested
against unindexed find.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE

DESIGN.md

@@ -484,6 +484,73 @@ Decisions taken where the brief was silent. Each is the simplest option consiste
 - **Mutex mode** (driver without `setIfAbsent`): a per-stream in-process promise chain serialises
   appends. It is safe only with a single writer process.
 
+### M6 — Documents
+
+- **Projection value** is `{ state, version, commit }`, or `{ deleted: true, version, commit }` for a
+  tombstone. Tombstones keep the version, so re-creating a document continues its stream.
+- **`expectedCommit` append hint.** Writes pass the projection's `commit` to `append` as
+  `expectedCommit`, so a write is exactly projection `getMany` + `setIfAbsent` + projection `batch`
+  (3 driver calls), even with a cold head cache. A hint taken from a projection can only be stale
+  (which causes a collision, then catch-up and retry), never ahead of the stream, so it cannot
+  create a gap. Non-KV backends may ignore it.
+- **Per-document in-process write lock.** Writes to the same document within one storage instance
+  are serialised (`KeyedMutex`), so optimistic retries (default 5) are spent only on cross-process
+  contention.
+- **Return values**: `set` → `void`; `insert` → id; `update` → new state, or `undefined` if the document
+  doesn't exist (pass `{ upsert: true }` to treat missing as `{}`); `delete` → whether a document
+  was deleted. No-op updates (resulting state equal to current) append nothing.
+- **Arity for documents**: in forms with an optional leading key (`get`, `set`, `update`), a non-null
+  object first argument selects the self form, and any other first argument is a key (so `undefined`
+  throws). `delete`/`history` use argument count. `getAt(version)` vs `getAt(key, version)` also uses
+  argument count, because a version and a numeric key are both numbers.
+- **The root document node has no own document** (consistent with KV).
+- **`keys()` on a document node** lists children holding a live document. Unlike KV `keys()`, it skips
+  children that only have nested documents or tombstones.
+- **`find` results** are `{ key, data }`. `findOne` returns the same shape or `undefined`.
+- **Filter semantics vs MongoDB** (see `src/documents/filter.ts`):
+  - Comparisons are type-bracketed (`$gt: 5` matches only numbers), and cross-type order follows
+    the index encoding: `null/missing < false < true < numbers < strings < other`.
+  - On array fields, every positive operator in one condition (`$eq $gt $gte $lt $lte $in`) must be
+    satisfied by the **same element** (like `$elemMatch`). In MongoDB, `{ s: { $gt: 5, $lt: 10 } }`
+    matches `[1, 20]`; here it doesn't. This makes index range scans exact. To get MongoDB's
+    behaviour, split the condition with `$and`.
+  - `{ f: null }` matches missing fields. `$not` works both at top level (negating a filter) and at
+    field level. Unknown operators throw.
+  - `$in` is not index-accelerated in v1.
+- **Index entries** for each field value: scalars as-is, arrays as their scalar elements (multikey;
+  empty arrays → `null`), and missing or non-scalar values as `null`. Every live document therefore
+  has at least one entry per index, so a full index scan sees all documents. Indexing a document
+  where more than one field of a compound index has multiple values throws (as MongoDB does).
+- **Index names** default to the fields joined by `,`.
+- **Unique indexes**: claims are scoped **per parent node** (`<node>/~u/<name>/<values>`), like one
+  MongoDB collection per parent. Value tuples where every field is missing/null are not claimed
+  (sparse). Claims are taken before appending, then released if the append fails. A claim value is
+  `[ownerKey, claimedAtMs]`. A claim held by another document is taken over only if that
+  document, after catching up from its stream, no longer holds the value, **and** the claim is older
+  than `claimGraceMs` (default 30 s). The owner might otherwise be between claiming and appending.
+  The residual risk is a writer stalling for longer than the grace period between claiming and
+  appending.
+- **Planner**: per index, count leading equality fields (k), then whether field k has a range;
+  score = 2k + (range ? 1 : 0). Ties prefer an index whose order satisfies `sort` after dropping
+  equality-constrained sort fields. With no usable predicate, an index matching `sort` is still used
+  as an ordered full index scan, which streams and stops early with `limit`. Range bounds are clamped
+  to the bound's type class. Index-backed results are re-read from projections, re-checked against the
+  full filter, de-duplicated, and stale entries are read-repaired.
+- **Result order** when `sort` is not given is the plan's order (index order, or child-key order for
+  a full scan). Ties in sorted results are broken by child key in the direction of the first sort
+  field, both for index-ordered and in-memory sorts.
+- **In-memory sort** keeps a bounded top `skip + limit` buffer when a limit is given. Without a limit,
+  it buffers all matches; this is the one place results are materialised.
+- **Full-scan warning**: `console.warn` once per filter *shape* (values replaced by `?`) per storage
+  instance, only for non-empty filters. On by default unless `NODE_ENV === 'production'`; set it with
+  `warnOnFullScan`.
+- **Catch-up** re-asserts unique claims for the caught-up state with plain writes, since the log is
+  the source of truth. A slow catch-up can briefly overwrite a newer projection with an older one;
+  the next write's `expectedVersion` check detects this and heals it. `catchUp()` walks the subtree
+  via event-stream `keys()` and returns the number of documents repaired.
+- **Projections for non-KV event stores** must be passed explicitly (`projections: KVNode`); the
+  projection key of a document is then `projections.prefix + docKey`.
+
 <!-- decisions appended per milestone -->
 
 ## 13. Benchmark baselines