M1: core path escaping, order-preserving index value encoding, errors

Scaffold TypeScript/Vitest project, add DESIGN.md (brief) and CLAUDE.md.
Property tests cover escape round-trips, separator-freedom and order preservation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE

.gitignore

@@ -0,0 +1,4 @@
+node_modules
+dist
+coverage
+*.log

CLAUDE.md

@@ -0,0 +1,20 @@
+# PotentialStorage
+
+Layered TypeScript storage library: KV → Events → Documents. The full design, milestones and
+implementation decisions are in [DESIGN.md](./DESIGN.md) — read it before changing anything.
+Record new decisions in DESIGN.md section 12.
+
+## Commands
+
+- `npm test` — run all tests (Vitest; includes `*.test-d.ts` type tests)
+- `npm run typecheck` — `tsc --noEmit` over src, test and bench
+- `npm run bench` — `vitest bench`
+- `npm run build` — emit `dist/`
+
+## Rules
+
+- No runtime dependencies in `src/`. Dev dependencies only.
+- Drivers stay flat and string-keyed; all path/index/escaping logic lives in the node classes.
+- Self vs child dispatch is by `arguments.length`; a passed-but-`undefined`/`null` key throws.
+- Multi-result reads are `AsyncIterable`; don't build large arrays internally.
+- Keep tests green and commit per milestone. Ask the owner before pushing.

DESIGN.md

@@ -0,0 +1,387 @@
+# PotentialStorage — Design
+
+This document is the design brief agreed with the project owner (sections 0–11), followed by
+implementation decisions made where the brief was silent (section 12) and benchmark baselines
+(section 13).
+
+## 0. Setup and workflow
+
+- Repository: `git@git.harvmaster.com:Harvmaster/PotentialStorage.git` (SSH; self-hosted Git, not GitHub). Clone it. If you can't reach it, stop and tell the owner rather than starting a fresh repo elsewhere.
+- Inspect what's already in the repo first. If it has tooling (package.json, tsconfig, test runner), keep it. If it's empty, set up:
+  - TypeScript (`strict: true`, `noUncheckedIndexedAccess: true`), ESM, Node ≥ 20
+  - Vitest for tests and `vitest bench` for benchmarks; `expectTypeOf` for type tests
+  - Package name `potential-storage` unless one already exists
+  - **No runtime dependencies in the core.** Dev dependencies are fine (e.g. `fast-check` for property tests).
+- Copy this brief into the repo as `DESIGN.md` and create a short `CLAUDE.md` pointing at it plus the build/test commands.
+- Work milestone by milestone (section 9). Commit after each milestone with a clear message, and keep tests green. Ask the owner before pushing.
+
+## 1. Principles
+
+1. **Three layers**, each usable on its own: **KV** → **Event** (append-only, built on KV) → **Document** (Mongo-like, built on Events or native).
+2. **Drivers vs stores.** Backends implement a tiny, flat, string-keyed *driver*. All path, child, index, escaping and typing logic lives once in the user-facing *node* classes, so every backend gets identical behaviour.
+3. **One navigation primitive: `child(key)`.** There are no collection/document classes. Every node is a store over its direct children, and `child(key)` re-roots to that key. Navigation is synchronous and does no I/O.
+4. **No methods on data.** Reads return plain values.
+5. **Performance is a requirement.** Batch operations are the primitive; scans are ordered and streamed; nothing replays or scans more than it must (section 8).
+
+## 2. Paths and key encoding (`src/core/path.ts`)
+
+- A node's path is `readonly string[]` of **unescaped** segments. Number keys are converted with `String()`.
+- The separator is a single character, configurable, default `/`.
+- **Escaping** (per segment): percent-encode `%`, the separator, and a **leading** `~`. Escaped segments never contain a raw separator. Write `escapeSegment` / `unescapeSegment` with round-trip property tests.
+- **Reserved segments** start with `~` (user segments can't, because a leading `~` is escaped):
+  - `~i` index entries
+  - `~u` unique-index claims
+  - `~e` event commits
+  - `~p` document projections root
+- Each node precomputes in its constructor:
+  - `selfKey` = escaped segments joined by the separator (e.g. `users/123`)
+  - `prefix` = `selfKey + sep` (`''` for the root)
+  - No string work per call beyond `prefix + escapeSegment(key)`.
+- Keys are ordered **lexicographically** (so `10` sorts before `9`). Document this. An order-preserving numeric key encoding is a possible later option, not part of v1.
+
+Example layout:
+
+```
+users/123                          → KV value of users/123
+users/123/posts/p1                 → KV value of a nested child
+users/~i/byEmail/sada@x.io/123     → index entry (value: empty / true)
+orders/o1/~e/000000000007          → event commit #7 of stream orders/o1
+~p/users/123                       → document projection { state, version, commit }
+```
+
+## 3. Layer 1: KV
+
+### 3.1 Driver interface (`src/kv/driver.ts`)
+
+```ts
+export interface ScanOptions {
+  gte?: string
+  lt?: string
+  limit?: number
+  reverse?: boolean
+  values?: boolean          // when false/omitted, yield value as undefined
+}
+
+export interface KVDriver {
+  getMany(keys: string[]): Promise<(unknown | undefined)[]>       // same order as keys
+  setMany(entries: [key: string, value: unknown][]): Promise<void>
+  deleteMany(keys: string[]): Promise<void>
+  scan(opts: ScanOptions): AsyncIterable<[key: string, value: unknown]>  // MUST be key-ordered
+
+  // Optional capabilities. Stores feature-detect them and fall back where possible.
+  setIfAbsent?(entries: [string, unknown][]): Promise<boolean[]>  // true = written
+  batch?(ops: BatchOp[]): Promise<void>                            // atomic multi-op write
+  deleteRange?(gte: string, lt: string): Promise<void>
+}
+
+export type BatchOp =
+  | { type: 'set'; key: string; value: unknown }
+  | { type: 'delete'; key: string }
+```
+
+- Values are passed through as JSON-serialisable values. Drivers that need bytes (HTTP, Redis, files) serialise internally via a pluggable `Codec` (default JSON).
+- Provide an abstract `BaseKVDriver` with default implementations of `deleteRange` (scan + deleteMany) and helpers, so a minimal driver implements only the four required methods.
+- **`InMemoryKVDriver`**: a sorted structure (sorted array with binary search is fine for v1; keep it behind an interface so it can become a B-tree). It implements every optional capability: `setIfAbsent`, atomic `batch` and `deleteRange`.
+- Write a reusable **driver conformance suite** `runKVDriverConformance(name, makeDriver)` covering ordering, `gte`/`lt` bounds, reverse, limit, `values` on/off, `getMany` order with missing keys, and each optional capability when present.
+
+### 3.2 `KVStorage` / `KVNode` (`src/kv/node.ts`)
+
+```ts
+const db = new KVStorage(driver, { separator: '/', indexes: { ... } })   // db is the root KVNode
+const users = db.child('users')
+const user  = users.child('123')
+```
+
+API of every `KVNode` (types in section 7):
+
+| Method | Meaning |
+|---|---|
+| `child(key)` | Sync. Returns a node at `path + key`. |
+| `as<S>()` | Type assertion for untyped use; returns the same node retyped. |
+| `get()` / `get(key)` | Own value at `selfKey` / value at child key. |
+| `getMany(keys)` | Batch child read, same order, `undefined` for missing. |
+| `set(value)` / `set(key, value)` | Write own value / child value. |
+| `setMany(entries)` | Batch child write. |
+| `delete()` / `delete(key)` | Delete own value / child value only (not descendants). |
+| `deleteMany(keys)` | Batch child delete. |
+| `deleteTree()` / `deleteTree(key)` | Delete own value **and all descendants** (uses `deleteRange` if available). |
+| `keys(opts?)` | `AsyncIterable<string>` of **direct** child keys (unescaped). Options: `limit`, `after`, `reverse`. |
+| `entries(opts?)` | `AsyncIterable<[key, value]>` of direct children that have an own value. |
+| `tree({ depth })` | Assemble an object from descendant keys up to `depth` (for field-per-key style). Bounded and explicit; never the default. |
+| `findBy(index, value \| { gte?, lt? }, opts?)` | Index lookup over direct children (section 3.3). |
+
+**Arity dispatch (important):**
+- Self vs child is decided by `arguments.length`, never by checking for `undefined`.
+- If a key argument is passed but is `undefined` or `null`, throw a `TypeError` (this prevents `users.get(maybeId)` silently reading self).
+- The root node (`path = []`) has no own value: `get()`, `set(v)` and `delete()` on root throw.
+- Implement one private `#resolve(args)` that every method uses.
+
+**Direct-children listing (`keys`)** uses a skip-scan:
+1. Scan `gte: prefix`, `lt: prefix + succ(sep)`.
+2. For each key, take the first segment after `prefix`. Skip reserved (`~…`) segments. Yield the segment (unescaped) if it differs from the previous one.
+3. When the next key is a descendant of the segment just yielded, re-seek the driver to `gte: prefix + escSeg + succ(sep)` (where `succ(c)` is the next char code) instead of iterating through the subtree. Keep using the same iterator while keys are siblings.
+
+Listing must be O(direct children × seek cost), not O(subtree). A child that has no own value but has descendants **is** listed.
+
+### 3.3 Indexes (`src/kv/indexes.ts`)
+
+Indexes are declared on **path patterns** in the `KVStorage` constructor. A pattern names a parent node, and its index applies to that node's direct children's values:
+
+```ts
+new KVStorage(driver, {
+  indexes: {
+    'users':         { byEmail: (u: User) => u.email },
+    'users/*/posts': { byDate:  (p: Post) => p.createdAt, byTag: (p: Post) => p.tags },
+  },
+})
+```
+
+- An index function returns `IndexValue | IndexValue[] | undefined`, where `IndexValue = string | number | boolean | null`. An array means multi-entry; `undefined` means not indexed.
+- Compile patterns into a **segment trie** (`*` matches one segment). A node resolves its index set **once, at construction**, in O(depth).
+- Entry key: `<nodePrefix>~i/<indexName>/<encodedValue>/<escapedChildKey>` with a `true` value.
+- **`encodeIndexValue` must be order-preserving and separator-safe:**
+  - Type tag first, so values order by type: `null` < `false` < `true` < numbers < strings.
+  - Numbers use an order-preserving float64 encoding (flip the sign bit for positives and all bits for negatives, then fixed-width hex).
+  - Strings are escaped like segments.
+  - This enables `findBy(index, { gte, lt })` range scans.
+- **Writes to an indexed node:**
+  1. Read the old value(s) (`getMany`) and compute the diff of index entries.
+  2. If the driver has `batch`, write the value and the index adds/removes atomically.
+  3. Otherwise, write index **adds first**, then the value, then index **removes**. Treat index entries as hints: `findBy` re-reads each value, re-checks the index function, and lazily deletes stale entries (read-repair).
+- Non-indexed nodes must **not** pay the read-before-write cost.
+
+## 4. Layer 2: Events (`src/events/`)
+
+An append-only log per stream. A stream is identified by a node path. Same `child()` navigation as KV.
+
+```ts
+const events = new EventStorage(kv)            // takes a KVStorage (or a KVNode as root)
+const orders = events.child('orders')
+
+await orders.append('o1', [{ type: 'created', total: 40 }], { expectedVersion: 0 })
+await orders.child('o1').append([{ type: 'paid' }])          // self form
+for await (const e of orders.read('o1', { from: 1 })) { /* { version, commit, at, data } */ }
+await orders.version('o1')
+for await (const id of orders.keys()) { }                    // child streams
+orders.child('o1').child('shipments')
+```
+
+- `EventStorage` is the abstract interface. `KVEventStorage` is the KV-backed implementation. Leave room for native backends (EventStoreDB, Kafka) implementing the same interface.
+- **Arity:** `append(events, opts?)` vs `append(key, events, opts?)`. Dispatch on `Array.isArray(args[0])`, plus the undefined-key rule from 3.2. `read`, `version` and `keys` follow the 3.2 rules.
+- **Versions:** an empty stream has version 0, and the first event has version 1. `expectedVersion: number | 'any'` (default `'any'`).
+- **Storage: one key per commit, densely numbered.**
+  - Key: `<streamSelfKey>/~e/<commitSeq zero-padded to 12>`
+  - Value: `{ firstVersion, at, events: E[] }`
+  - The commit sequence (not the event version) is the key so that **any** stale writer collides with the existing key. Keying by version would let a stale writer create overlapping version ranges.
+- **Append:**
+  1. Get the head `{ commitSeq, version }` (from the cache, or a reverse scan with `limit: 1`).
+  2. If `expectedVersion` is a number and doesn't match the head, throw `ConcurrencyError`.
+  3. Call `setIfAbsent(nextCommitKey)`.
+  4. On `false` (collision): refresh the head. If `expectedVersion` was given, throw `ConcurrencyError`; otherwise retry up to 5 times.
+  5. Without `setIfAbsent`, fall back to a per-stream in-process mutex. Document that this mode is single-writer only.
+- **Head cache:** a bounded LRU keyed by stream path. Update it on successful append; invalidate it on collision.
+- **`read({ from, to, reverse, limit })`** streams events with metadata. To start at version `from`, locate the containing commit by binary search over commit numbers using `getMany` probes (O(log commits) round trips), then forward-scan.
+- There is no update or delete. Keys inside `~e` are never rewritten.
+
+## 5. Layer 3: Documents (`src/documents/`)
+
+`DocumentStorage` is the interface. There are two implementations:
+- `EventSourcedDocumentStorage(events, { projections?: KVNode })` is the v1 target. Projections default to the `~p` root on the same KV.
+- `MongoDocumentStorage(client)` is a later milestone (section 9).
+
+```ts
+const docs  = new EventSourcedDocumentStorage(events, { indexes: { 'users': [{ fields: ['email'], unique: true }, { fields: ['age'] }] } })
+const users = docs.child('users')
+
+const id = await users.insert({ name: 'Ada', age: 36 })      // generates a time-sortable id
+await users.set('123', { name: 'Ada', age: 36 })
+await users.update('123', { $set: { name: 'Ada L.' }, $inc: { logins: 1 } })
+await users.child('123').update({ $unset: { nickname: true } })   // self form
+await users.get('123')
+await users.delete('123')
+for await (const { key, data } of users.find({ age: { $gte: 18 } }, { sort: { age: 1 }, limit: 20 })) { }
+await users.findOne({ email: 'ada@x.io' })
+await users.history('123')        // event-sourced only
+await users.getAt('123', 17)      // state as of version 17
+users.child('123').child('posts') // nested docs, same as KV
+```
+
+### 5.1 Events and projections
+
+- The doc at path `P` has its event stream at `P` in `EventStorage`. Its projection is at `~p/P` in the projections KV: `{ state, version }` (and `deleted: true` for tombstones).
+- Event types: `{ op: 'set', doc }`, `{ op: 'patch', set?: Record<path, value>, unset?: path[] }`, `{ op: 'delete' }`.
+- **Store resolved deltas, not operators.** For example, `$inc` resolves to `{ set: { logins: 6 } }`, so replay is deterministic.
+- Supported update operators in v1: `$set`, `$unset`, `$inc`, `$push`, `$pull`, `$addToSet`, with dot-path support.
+
+**Write path:**
+1. Read the projection.
+2. Compute the new state and patch.
+3. `append` with `expectedVersion = projection.version`.
+4. Write the projection and index entries (atomic `batch` if available).
+5. On `ConcurrencyError`: catch up the projection from the stream, recompute, and retry (bounded).
+
+**Reads** are a single projection `get`. Never replay on read.
+
+**Crash between append and projection write:**
+- The projection lags the stream. The next write's `expectedVersion` fails, which triggers catch-up, so the system self-heals.
+- Provide `get(key, { consistent: true })` that checks the stream head and catches up first.
+- Provide a `catchUp(node?)` maintenance method to repair a subtree.
+- `insert` ids are time-sortable (ULID-like). Implement a small generator, with no dependency.
+
+### 5.2 Queries and document indexes
+
+- Filter subset:
+  - `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`, `$exists`
+  - `$and`, `$or`, `$not`
+  - implicit equality and dot paths
+- Options: `sort`, `limit`, `skip`, `projection` (fields).
+- `find` on a node queries that node's **direct children**.
+- Indexes are declared per path pattern (same trie mechanism as 3.3).
+  - Each index has `fields` (compound allowed); entries go under `~p/<node>/~i/<name>/<enc(field1)>/<enc(field2)>/…/<childKey>`, using `encodeIndexValue`.
+  - `unique: true` additionally claims `~u/<name>/<enc(values)>` via `setIfAbsent`. Throw at construction if the driver lacks `setIfAbsent`.
+- **Planner:**
+  - Pick the index that covers the most leading equality fields, then at most one range field.
+  - Use the index order for `sort` when it matches.
+  - Otherwise do a full scan of children projections, with a one-time dev-mode `console.warn` naming the filter.
+
+## 6. Shared conventions
+
+- Every layer uses the same navigation (`child`, `as`, `keys`) and the same self/child arity rules.
+- All multi-result reads are `AsyncIterable` (streamed, cancellable). Never build large arrays internally.
+- Errors are typed classes exported from `src/errors.ts`: `ConcurrencyError`, `UniqueConstraintError`, `CapabilityError`, `InvalidKeyError`.
+- Public exports go through `src/index.ts`, with subpath exports per layer if convenient.
+
+## 7. Types (`src/types/schema.ts`)
+
+Each node is typed by a single **Spec** that describes its own value and its children by key:
+
+```ts
+export type Spec = { value?: unknown; children?: Record<string, Spec> }
+export type AnySpec = { value: unknown; children: Record<string, AnySpec> }   // untyped default
+
+export type ValueOf<S>    = S extends { value: infer V } ? V : never
+export type ChildrenOf<S> = S extends { children: infer C } ? C : {}
+export type KeyOf<S>      = Extract<keyof ChildrenOf<S>, string | number>
+export type ChildOf<S, K extends KeyOf<S>> = ChildrenOf<S>[K]
+
+// Helpers
+export type Many<V, C extends Record<string, Spec> = {}> =
+  { children: { [id: string]: { value: V; children: C } } }
+export type Fields<T> = { [K in keyof T & string]: { value: T[K] } }
+```
+
+Node signatures:
+
+```ts
+class KVNode<S extends Spec = AnySpec> {
+  child<K extends KeyOf<S>>(key: K): KVNode<ChildOf<S, K>>
+  get(): Promise<ValueOf<S> | undefined>
+  get<K extends KeyOf<S>>(key: K): Promise<ValueOf<ChildOf<S, K>> | undefined>
+  // set/delete/getMany/etc. follow the same pattern
+}
+```
+
+Example to cover with `expectTypeOf` tests:
+
+```ts
+type User = { username: string; age: number }
+type Post = { title: string; createdAt: string }
+type Schema = { children: {
+  users:   Many<User, { posts: Many<Post> }>
+  profile: { children: { [id: string]: { children: Fields<User> } } }   // field-per-key style
+} }
+
+const db = new KVStorage<Schema>(driver)
+db.child('users').get('123')                   // User | undefined
+db.child('users').child('123').get()           // User | undefined (same key)
+db.child('users').child('123').child('posts')  // KVNode<Many<Post>>
+db.child('users').child('123').child('postz')  // ✗ type error
+db.child('profile').child('9').get('age')      // number | undefined
+db.child('profile').child('9').get('agee')     // ✗ type error
+```
+
+Notes:
+- **Named levels** check keys at compile time; **dynamic `[id: string]` levels** accept any string.
+- One node's children can't mix an index signature with named keys (a TS restriction), so schemas alternate between dynamic and named levels. That alternation exists only in types; the runtime API is just `child()`.
+- `EventStorage` and `DocumentStorage` nodes use the same Spec machinery (the value type is the event type or the document type respectively).
+
+## 8. Performance requirements
+
+- Single-key ops: exactly one driver call (no hidden reads), except indexed writes (one `getMany` before the write) and document writes (projection read plus append plus projection write).
+- `*Many` ops: one driver call each (plus index work), never N calls.
+- `keys()`: skip-scan, O(direct children) seeks.
+- Event head lookup: cached; cold lookup is a single reverse scan with `limit: 1`.
+- Document `get`: one driver `get`.
+- Add `vitest bench` benchmarks for:
+  - KV `get`/`set`/`getMany` at 10k/100k keys
+  - `keys()` on a node with 1k children each having 100 descendants
+  - event append and read
+  - document `get`/`update`/indexed `find` vs full-scan `find`
+- Record baseline numbers in `DESIGN.md`.
+
+## 9. Milestones
+
+1. **Core:** `path.ts` (escape, unescape, encode, `succ`), `encodeIndexValue`, errors. Property tests for round-trips and order preservation.
+2. **KV driver:** `KVDriver`, `BaseKVDriver`, `InMemoryKVDriver`, conformance suite.
+3. **KV node:** full API from 3.2, including arity rules, skip-scan `keys`, `deleteTree`, `tree`. Add a model-based property test: random ops vs a reference `Map`.
+4. **KV indexes:** pattern trie, atomic and non-atomic write paths, read-repair, range `findBy`.
+5. **Events:** `KVEventStorage` with commit keys, `setIfAbsent` and mutex modes, head cache, binary-search `read({ from })`, concurrency tests (simulated interleaving with two storage instances on one driver).
+6. **Documents:** `EventSourcedDocumentStorage`, update operators, projections, catch-up, queries, planner, unique indexes, `history` and `getAt`.
+7. **Types:** Spec machinery wired through all three layers, plus `expectTypeOf` tests from section 7.
+8. **HTTP driver:**
+   - A `createKVHttpHandler(driver)` (framework-agnostic `Request → Response`) plus an `HttpKVDriver` client.
+   - Endpoints: `POST /getMany`, `POST /setMany`, `POST /deleteMany`, `POST /batch`, `POST /setIfAbsent`, `GET /scan?gte&lt&limit&reverse&values` (NDJSON stream).
+   - It must pass the conformance suite end-to-end.
+9. **Benchmarks** and a `DESIGN.md` update with results.
+10. **(Later, ask first)** `MongoDocumentStorage`:
+    - Path pattern maps to a collection (e.g. `users/*/posts` → `users.posts`), with a `_parent` field and a compound index `(_parent, _id)`.
+    - Native indexes map from the same index declarations.
+    - It should pass the document test suite where semantics allow (no `history`/`getAt`).
+
+## 10. Decisions already made (don't revisit without asking)
+
+- `child()`-only navigation; no collection/doc classes.
+- `get()` with no argument reads the node's **own key**. It never assembles the subtree; `tree()` does that explicitly.
+- A node's own value and its children are independent keys.
+- Self/child dispatch by arity; a passed-but-undefined key throws.
+- Indexes are declared on path patterns at construction.
+- Events use one key per commit, densely numbered.
+- Documents store resolved deltas, and reads hit projections only.
+- Keys sort lexicographically.
+- Values are JSON-serialisable; byte drivers use a codec.
+
+## 11. Open questions to raise with the owner (don't block on them)
+
+- Order-preserving numeric **keys** (not just index values)?
+- Should multi-writer safety be **required** (a hard error without `setIfAbsent`) rather than a documented single-writer fallback?
+- The hard-delete story for immutable event logs (compaction or crypto-shredding) for privacy deletion requests.
+
+## 12. Implementation decisions
+
+Decisions taken where the brief was silent. Each is the simplest option consistent with sections 1–10.
+
+### M1 — Core
+
+- **Separator** must be a single printable ASCII character other than `%` and `~`.
+- **Empty keys are rejected** (`InvalidKeyError`), as are `NaN`/`±Infinity` number keys. An empty segment
+  would produce `a//b`, which is ambiguous with the subtree layout.
+- **Separator escaping** uses `%` + two uppercase hex digits (`/` → `%2F`, `%` → `%25`, leading `~` → `%7E`).
+- **`InvalidKeyError` extends `TypeError`**, so "passed-but-undefined key throws a `TypeError`" (3.2) and
+  the typed error class (6) are the same throw.
+- **Index string encoding deviates from "escaped like segments".** Percent-escaping is not
+  order-preserving (`"a/"` → `"a%2F"` sorts below `"a&"`, and `"a"` + sep sorts above `"a!"`), which
+  would break `findBy` range scans. Instead every character the index encoder emits sorts strictly
+  above the separator: code units `<= succ(sep)` are written as `succ(sep)` followed by
+  `chr(succ(sep) + 1 + c)`. This is monotone and prefix-safe, so `a < b ⇔ enc(a)+sep+… < enc(b)+sep+…`
+  exactly (property-tested). Encodings are still separator-free.
+- **Type tags and number digits**: tags are the first five digits of the digit alphabet; numbers use
+  16 digits of `0-9a-f` when the separator sorts below `'0'` (e.g. `/`), otherwise the 16 characters
+  just above the separator. `-0` is stored as `0`; `NaN` cannot be indexed.
+- **Ordering** of strings is by UTF-16 code unit (JS `<`). Drivers that compare UTF-8 bytes agree
+  except for astral-plane characters vs. U+E000–U+FFFF; noted as a driver-author caveat.
+
+<!-- decisions appended per milestone -->
+
+## 13. Benchmark baselines
+
+_Filled in at milestone 9._

package.json

@@ -0,0 +1,49 @@
+{
+  "name": "potential-storage",
+  "version": "0.1.0",
+  "description": "Layered TypeScript storage: KV -> Events -> Documents",
+  "type": "module",
+  "engines": {
+    "node": ">=20"
+  },
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    },
+    "./kv": {
+      "types": "./dist/kv/index.d.ts",
+      "import": "./dist/kv/index.js"
+    },
+    "./events": {
+      "types": "./dist/events/index.d.ts",
+      "import": "./dist/events/index.js"
+    },
+    "./documents": {
+      "types": "./dist/documents/index.d.ts",
+      "import": "./dist/documents/index.js"
+    },
+    "./http": {
+      "types": "./dist/http/index.d.ts",
+      "import": "./dist/http/index.js"
+    }
+  },
+  "files": [
+    "dist"
+  ],
+  "scripts": {
+    "build": "tsc -p tsconfig.build.json",
+    "typecheck": "tsc --noEmit",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "bench": "vitest bench --run"
+  },
+  "devDependencies": {
+    "@types/node": "^22.20.5",
+    "fast-check": "^4.10.2",
+    "typescript": "^5.9.3",
+    "vitest": "^3.2.7"
+  }
+}

src/core/encoding.ts

@@ -0,0 +1,177 @@
+import { DEFAULT_SEPARATOR, succ } from './path.js'
+
+export type IndexValue = string | number | boolean | null
+
+/**
+ * Order-preserving, separator-safe encoding of index values.
+ *
+ * Every character the encoder emits sorts strictly above the separator. That makes encodings
+ * prefix-safe inside keys: `enc(a) + sep + …` sorts below `enc(b)` whenever `enc(a)` is a proper
+ * prefix of `enc(b)`, so `a < b ⇔ key(a, …) < key(b, …)` and range scans over index entries are exact.
+ *
+ * Layout: one type-tag character, then a body.
+ * - Type tags order `null < false < true < number < string`.
+ * - Numbers: IEEE-754 float64, big-endian; positive values have the sign bit flipped, negative
+ *   values have all bits flipped; written as 16 fixed-width "hex" digits. `-0` is normalised to `0`;
+ *   `NaN` is rejected.
+ * - Strings: each code unit `c` with `c <= E` (where `E = succ(sep)`) is written as `E` followed by
+ *   `chr(E + 1 + c)`; every other code unit is written as-is. This is monotone, so order is kept.
+ *
+ * The 16 digits are `0-9a-f` when the separator sorts below `'0'` (e.g. `/`), otherwise the 16
+ * characters immediately after the separator.
+ */
+export interface IndexValueCodec {
+  readonly sep: string
+  encode(value: IndexValue): string
+  decode(encoded: string): IndexValue
+}
+
+const codecs = new Map<string, IndexValueCodec>()
+
+export function indexValueCodec(sep: string = DEFAULT_SEPARATOR): IndexValueCodec {
+  let codec = codecs.get(sep)
+  if (!codec) {
+    codec = makeCodec(sep)
+    codecs.set(sep, codec)
+  }
+  return codec
+}
+
+export function encodeIndexValue(value: IndexValue, sep: string = DEFAULT_SEPARATOR): string {
+  return indexValueCodec(sep).encode(value)
+}
+
+export function decodeIndexValue(encoded: string, sep: string = DEFAULT_SEPARATOR): IndexValue {
+  return indexValueCodec(sep).decode(encoded)
+}
+
+/** Encodes a compound value as `enc(v1) + sep + enc(v2) + …`. */
+export function encodeIndexValues(values: readonly IndexValue[], sep: string = DEFAULT_SEPARATOR): string {
+  const codec = indexValueCodec(sep)
+  let out = ''
+  for (let i = 0; i < values.length; i++) {
+    if (i > 0) out += sep
+    out += codec.encode(values[i]!)
+  }
+  return out
+}
+
+export function isIndexValue(v: unknown): v is IndexValue {
+  return v === null || typeof v === 'string' || typeof v === 'boolean' || (typeof v === 'number' && !Number.isNaN(v))
+}
+
+function makeCodec(sep: string): IndexValueCodec {
+  const sepCode = sep.charCodeAt(0)
+  const digits =
+    sepCode < 0x30 ? '0123456789abcdef' : Array.from({ length: 16 }, (_, i) => String.fromCharCode(sepCode + 1 + i)).join('')
+  const digitValue = new Map<string, number>([...digits].map((d, i) => [d, i]))
+  const TAG_NULL = digits[0]!
+  const TAG_FALSE = digits[1]!
+  const TAG_TRUE = digits[2]!
+  const TAG_NUMBER = digits[3]!
+  const TAG_STRING = digits[4]!
+  const esc = succ(sep)
+  const escCode = esc.charCodeAt(0)
+
+  const view = new DataView(new ArrayBuffer(8))
+
+  function hex32(n: number): string {
+    let out = ''
+    for (let shift = 28; shift >= 0; shift -= 4) out += digits[(n >>> shift) & 0xf]
+    return out
+  }
+
+  function encodeNumber(n: number): string {
+    if (Number.isNaN(n)) throw new TypeError('NaN cannot be indexed')
+    view.setFloat64(0, n === 0 ? 0 : n) // normalise -0
+    let hi = view.getUint32(0)
+    let lo = view.getUint32(4)
+    if (hi & 0x80000000) {
+      hi = ~hi >>> 0
+      lo = ~lo >>> 0
+    } else {
+      hi = (hi ^ 0x80000000) >>> 0
+    }
+    return hex32(hi) + hex32(lo)
+  }
+
+  function decodeNumber(body: string): number {
+    if (body.length !== 16) throw new TypeError(`Bad encoded number ${JSON.stringify(body)}`)
+    let hi = 0
+    let lo = 0
+    for (let i = 0; i < 8; i++) hi = (hi << 4) | digitValue.get(body[i]!)!
+    for (let i = 8; i < 16; i++) lo = (lo << 4) | digitValue.get(body[i]!)!
+    hi >>>= 0
+    lo >>>= 0
+    if (hi & 0x80000000) {
+      hi = (hi ^ 0x80000000) >>> 0
+    } else {
+      hi = ~hi >>> 0
+      lo = ~lo >>> 0
+    }
+    view.setUint32(0, hi)
+    view.setUint32(4, lo)
+    return view.getFloat64(0)
+  }
+
+  function encodeString(s: string): string {
+    let out = ''
+    let start = 0
+    for (let i = 0; i < s.length; i++) {
+      const c = s.charCodeAt(i)
+      if (c <= escCode) {
+        out += s.slice(start, i) + esc + String.fromCharCode(escCode + 1 + c)
+        start = i + 1
+      }
+    }
+    return start === 0 ? s : out + s.slice(start)
+  }
+
+  function decodeString(body: string): string {
+    if (body.indexOf(esc) === -1) return body
+    let out = ''
+    for (let i = 0; i < body.length; i++) {
+      const c = body[i]!
+      if (c === esc) {
+        const next = body.charCodeAt(++i)
+        out += String.fromCharCode(next - escCode - 1)
+      } else out += c
+    }
+    return out
+  }
+
+  return {
+    sep,
+    encode(value) {
+      if (value === null) return TAG_NULL
+      switch (typeof value) {
+        case 'boolean':
+          return value ? TAG_TRUE : TAG_FALSE
+        case 'number':
+          return TAG_NUMBER + encodeNumber(value)
+        case 'string':
+          return TAG_STRING + encodeString(value)
+        default:
+          throw new TypeError(`Cannot index value of type ${typeof value}`)
+      }
+    },
+    decode(encoded) {
+      const tag = encoded[0]
+      const body = encoded.slice(1)
+      switch (tag) {
+        case TAG_NULL:
+          return null
+        case TAG_FALSE:
+          return false
+        case TAG_TRUE:
+          return true
+        case TAG_NUMBER:
+          return decodeNumber(body)
+        case TAG_STRING:
+          return decodeString(body)
+        default:
+          throw new TypeError(`Bad encoded index value ${JSON.stringify(encoded)}`)
+      }
+    },
+  }
+}

src/core/path.ts

@@ -0,0 +1,109 @@
+import { InvalidKeyError } from '../errors.js'
+
+export const DEFAULT_SEPARATOR = '/'
+
+/** Reserved segments. User segments can never start with `~` (a leading `~` is escaped). */
+export const RESERVED = {
+  index: '~i',
+  unique: '~u',
+  events: '~e',
+  projections: '~p',
+} as const
+
+export type Key = string | number
+
+/** The character immediately after `c` (by UTF-16 code unit). */
+export function succ(c: string): string {
+  return String.fromCharCode(c.charCodeAt(0) + 1)
+}
+
+export function validateSeparator(sep: string): void {
+  if (sep.length !== 1) throw new TypeError(`Separator must be a single character, got ${JSON.stringify(sep)}`)
+  const code = sep.charCodeAt(0)
+  if (code <= 0x20 || code >= 0x7f || sep === '%' || sep === '~') {
+    throw new TypeError(`Separator must be printable ASCII other than "%" and "~", got ${JSON.stringify(sep)}`)
+  }
+}
+
+function hex2(code: number): string {
+  return '%' + code.toString(16).toUpperCase().padStart(2, '0')
+}
+
+/** Converts a user key to a segment string, rejecting `undefined`/`null`/empty. */
+export function toSegment(key: unknown): string {
+  if (key === undefined || key === null) throw new InvalidKeyError(`Key must not be ${key}`)
+  if (typeof key === 'number') {
+    if (!Number.isFinite(key)) throw new InvalidKeyError(`Numeric key must be finite, got ${key}`)
+    return String(key)
+  }
+  if (typeof key !== 'string') throw new InvalidKeyError(`Key must be a string or number, got ${typeof key}`)
+  if (key === '') throw new InvalidKeyError('Key must not be empty')
+  return key
+}
+
+/**
+ * Escapes one path segment: percent-encodes `%`, the separator, and a leading `~`.
+ * The result never contains a raw separator and never starts with `~`.
+ */
+export function escapeSegment(segment: string, sep: string = DEFAULT_SEPARATOR): string {
+  // Fast path: nothing to escape (the overwhelmingly common case).
+  if (segment.indexOf('%') === -1 && segment.indexOf(sep) === -1 && segment.charCodeAt(0) !== 0x7e) return segment
+  let out = ''
+  for (let i = 0; i < segment.length; i++) {
+    const c = segment[i]!
+    if (c === '%' || c === sep || (i === 0 && c === '~')) out += hex2(c.charCodeAt(0))
+    else out += c
+  }
+  return out
+}
+
+export function unescapeSegment(escaped: string): string {
+  if (escaped.indexOf('%') === -1) return escaped
+  return escaped.replace(/%([0-9A-Fa-f]{2})/g, (_, h: string) => String.fromCharCode(parseInt(h, 16)))
+}
+
+export function isReservedSegment(escaped: string): boolean {
+  return escaped.charCodeAt(0) === 0x7e // '~'
+}
+
+/** Joins unescaped segments into an escaped key. */
+export function joinPath(path: readonly string[], sep: string = DEFAULT_SEPARATOR): string {
+  let out = ''
+  for (let i = 0; i < path.length; i++) {
+    if (i > 0) out += sep
+    out += escapeSegment(path[i]!, sep)
+  }
+  return out
+}
+
+/** Splits an escaped key into unescaped segments. */
+export function splitPath(key: string, sep: string = DEFAULT_SEPARATOR): string[] {
+  if (key === '') return []
+  return key.split(sep).map(unescapeSegment)
+}
+
+/**
+ * Precomputed key strings for a node path.
+ * - `selfKey`: escaped segments joined by the separator (`''` for the root)
+ * - `prefix`: `selfKey + sep` (`''` for the root) — every descendant key starts with it
+ * - `end`: exclusive upper bound of the subtree range (`selfKey + succ(sep)`; `undefined` for the root)
+ */
+export interface PathKeys {
+  readonly path: readonly string[]
+  readonly sep: string
+  readonly selfKey: string
+  readonly prefix: string
+  readonly end: string | undefined
+}
+
+export function pathKeys(path: readonly string[], sep: string): PathKeys {
+  const selfKey = joinPath(path, sep)
+  const root = path.length === 0
+  return {
+    path,
+    sep,
+    selfKey,
+    prefix: root ? '' : selfKey + sep,
+    end: root ? undefined : selfKey + succ(sep),
+  }
+}

src/errors.ts

@@ -0,0 +1,47 @@
+/** Base class for all PotentialStorage errors. */
+export class PotentialStorageError extends Error {
+  constructor(message: string) {
+    super(message)
+    this.name = new.target.name
+  }
+}
+
+/** An optimistic-concurrency check failed (stale `expectedVersion` or a lost commit race). */
+export class ConcurrencyError extends PotentialStorageError {
+  constructor(
+    readonly stream: string,
+    readonly expectedVersion: number | 'any',
+    readonly actualVersion: number,
+  ) {
+    super(`Concurrency conflict on "${stream}": expected version ${expectedVersion}, actual ${actualVersion}`)
+  }
+}
+
+/** A unique index already holds the given value for another key. */
+export class UniqueConstraintError extends PotentialStorageError {
+  constructor(
+    readonly index: string,
+    readonly values: readonly unknown[],
+    readonly existingKey?: string,
+  ) {
+    super(`Unique index "${index}" already contains ${JSON.stringify(values)}${existingKey === undefined ? '' : ` (key "${existingKey}")`}`)
+  }
+}
+
+/** The driver lacks an optional capability required by the requested feature. */
+export class CapabilityError extends PotentialStorageError {
+  constructor(readonly capability: string, detail?: string) {
+    super(`Driver lacks capability "${capability}"${detail ? `: ${detail}` : ''}`)
+  }
+}
+
+/**
+ * A key argument is invalid: passed but `undefined`/`null`, empty, or otherwise unusable.
+ * Extends `TypeError` so callers can catch either.
+ */
+export class InvalidKeyError extends TypeError {
+  constructor(message: string) {
+    super(message)
+    this.name = 'InvalidKeyError'
+  }
+}

src/index.ts

@@ -0,0 +1,3 @@
+export * from './errors.js'
+export * from './core/path.js'
+export * from './core/encoding.js'

test/core/encoding.test.ts

@@ -0,0 +1,72 @@
+import { describe, expect, it } from 'vitest'
+import fc from 'fast-check'
+import { decodeIndexValue, encodeIndexValue, type IndexValue } from '../../src/core/encoding.js'
+
+const separators = fc.constantFrom('/', ':', '.', '|', '#', '!')
+
+const indexValue: fc.Arbitrary<IndexValue> = fc.oneof(
+  fc.constant(null),
+  fc.boolean(),
+  fc.double({ noNaN: true }),
+  fc.integer(),
+  fc.string({ unit: 'binary' }),
+  fc.string({ unit: fc.constantFrom('/', ':', '0', '1', 'a', '%', '~', '\0', '!') }),
+)
+
+const rank = (v: IndexValue) => (v === null ? 0 : v === false ? 1 : v === true ? 2 : typeof v === 'number' ? 3 : 4)
+
+function compare(a: IndexValue, b: IndexValue): number {
+  const ra = rank(a)
+  const rb = rank(b)
+  if (ra !== rb) return ra - rb
+  if (typeof a === 'number' && typeof b === 'number') return a === b ? 0 : a < b ? -1 : 1
+  if (typeof a === 'string' && typeof b === 'string') return a === b ? 0 : a < b ? -1 : 1
+  return 0
+}
+
+const sign = (n: number) => (n < 0 ? -1 : n > 0 ? 1 : 0)
+const cmpStr = (a: string, b: string) => (a < b ? -1 : a > b ? 1 : 0)
+
+describe('encodeIndexValue', () => {
+  it('round-trips (property)', () => {
+    fc.assert(
+      fc.property(indexValue, separators, (v, sep) => {
+        const decoded = decodeIndexValue(encodeIndexValue(v, sep), sep)
+        if (typeof v === 'number' && v === 0) expect(decoded).toBe(0) // -0 normalised
+        else expect(decoded).toEqual(v)
+      }),
+    )
+  })
+
+  it('emits only characters above the separator (property)', () => {
+    fc.assert(
+      fc.property(indexValue, separators, (v, sep) => {
+        for (const c of encodeIndexValue(v, sep)) expect(c > sep).toBe(true)
+      }),
+    )
+  })
+
+  it('preserves order, including when followed by a separator and suffix (property)', () => {
+    fc.assert(
+      fc.property(indexValue, indexValue, fc.string(), fc.string(), separators, (a, b, sa, sb, sep) => {
+        const ea = encodeIndexValue(a, sep)
+        const eb = encodeIndexValue(b, sep)
+        const expected = sign(compare(a, b))
+        expect(cmpStr(ea, eb)).toBe(expected)
+        if (expected !== 0) expect(cmpStr(ea + sep + sa, eb + sep + sb)).toBe(expected)
+      }),
+      { numRuns: 2000 },
+    )
+  })
+
+  it('orders types null < false < true < numbers < strings', () => {
+    const values: IndexValue[] = ['', 'a', -Infinity, -1, 0, 1e300, Infinity, true, false, null]
+    const sorted = [...values].sort((a, b) => cmpStr(encodeIndexValue(a), encodeIndexValue(b)))
+    expect(sorted).toEqual([null, false, true, -Infinity, -1, 0, 1e300, Infinity, '', 'a'])
+  })
+
+  it('rejects NaN and non-index values', () => {
+    expect(() => encodeIndexValue(NaN)).toThrow()
+    expect(() => encodeIndexValue({} as never)).toThrow()
+  })
+})

test/core/path.test.ts

@@ -0,0 +1,82 @@
+import { describe, expect, it } from 'vitest'
+import fc from 'fast-check'
+import {
+  escapeSegment,
+  joinPath,
+  pathKeys,
+  splitPath,
+  succ,
+  toSegment,
+  unescapeSegment,
+  validateSeparator,
+} from '../../src/core/path.js'
+import { InvalidKeyError } from '../../src/errors.js'
+
+const separators = fc.constantFrom('/', ':', '.', '|', '#', '\\', '!')
+
+describe('escapeSegment', () => {
+  it('escapes %, the separator and a leading ~', () => {
+    expect(escapeSegment('a/b')).toBe('a%2Fb')
+    expect(escapeSegment('50%')).toBe('50%25')
+    expect(escapeSegment('~x')).toBe('%7Ex')
+    expect(escapeSegment('x~')).toBe('x~')
+    expect(escapeSegment('a:b', ':')).toBe('a%3Ab')
+    expect(escapeSegment('a/b', ':')).toBe('a/b')
+    expect(escapeSegment('plain')).toBe('plain')
+  })
+
+  it('round-trips (property)', () => {
+    fc.assert(
+      fc.property(fc.string({ unit: 'binary' }), separators, (s, sep) => {
+        expect(unescapeSegment(escapeSegment(s, sep))).toBe(s)
+      }),
+    )
+  })
+
+  it('never emits a raw separator or a leading ~ (property)', () => {
+    fc.assert(
+      fc.property(fc.string({ unit: 'binary' }), separators, (s, sep) => {
+        const e = escapeSegment(s, sep)
+        expect(e.includes(sep)).toBe(false)
+        expect(e.startsWith('~')).toBe(false)
+      }),
+    )
+  })
+
+  it('joins and splits paths (property)', () => {
+    fc.assert(
+      fc.property(fc.array(fc.string({ minLength: 1 }), { maxLength: 6 }), separators, (path, sep) => {
+        expect(splitPath(joinPath(path, sep), sep)).toEqual(path)
+      }),
+    )
+  })
+})
+
+describe('pathKeys', () => {
+  it('precomputes selfKey, prefix and subtree end', () => {
+    expect(pathKeys(['users', '123'], '/')).toMatchObject({ selfKey: 'users/123', prefix: 'users/123/', end: 'users/1230' })
+    expect(pathKeys([], '/')).toMatchObject({ selfKey: '', prefix: '', end: undefined })
+    expect(pathKeys(['a/b'], '/').selfKey).toBe('a%2Fb')
+  })
+})
+
+describe('misc', () => {
+  it('succ returns the next char', () => {
+    expect(succ('/')).toBe('0')
+  })
+
+  it('toSegment rejects undefined, null, empty and non-finite', () => {
+    expect(() => toSegment(undefined)).toThrow(InvalidKeyError)
+    expect(() => toSegment(null)).toThrow(TypeError)
+    expect(() => toSegment('')).toThrow(InvalidKeyError)
+    expect(() => toSegment(NaN)).toThrow(InvalidKeyError)
+    expect(toSegment(12)).toBe('12')
+  })
+
+  it('validates separators', () => {
+    expect(() => validateSeparator('//')).toThrow()
+    expect(() => validateSeparator('%')).toThrow()
+    expect(() => validateSeparator('~')).toThrow()
+    expect(() => validateSeparator(':')).not.toThrow()
+  })
+})

tsconfig.build.json

@@ -0,0 +1,5 @@
+{
+  "extends": "./tsconfig.json",
+  "compilerOptions": { "rootDir": "src", "outDir": "dist" },
+  "include": ["src"]
+}

tsconfig.json

@@ -0,0 +1,21 @@
+{
+  "compilerOptions": {
+    "target": "ES2022",
+    "module": "NodeNext",
+    "moduleResolution": "NodeNext",
+    "lib": ["ES2022"],
+    "types": ["node"],
+    "strict": true,
+    "noUncheckedIndexedAccess": true,
+    "noImplicitOverride": true,
+    "exactOptionalPropertyTypes": false,
+    "declaration": true,
+    "sourceMap": true,
+    "outDir": "dist",
+    "rootDir": ".",
+    "skipLibCheck": true,
+    "isolatedModules": true,
+    "verbatimModuleSyntax": true
+  },
+  "include": ["src", "test", "bench"]
+}

vitest.config.ts

@@ -0,0 +1,9 @@
+import { defineConfig } from 'vitest/config'
+
+export default defineConfig({
+  test: {
+    include: ['test/**/*.test.ts'],
+    typecheck: { enabled: true, include: ['test/**/*.test-d.ts'] },
+  },
+  bench: { include: ['bench/**/*.bench.ts'] },
+})