Commit Graph
18 Commits
Author SHA1 Message Date
Claude 3525dbc622 Name the package @harvmaster/storage
CI / Node 24 (pull_request) Successful in 12s
CI / Node 26 (pull_request) Successful in 10s
CI / Publish to staging registry (pull_request) Skipped
CI / Node 22 (push) Successful in 11s
CI / Publish to staging registry (push) Successful in 7s
CI / Node 22 (pull_request) Successful in 16s
CI / Node 24 (push) Successful in 13s
CI / Node 26 (push) Successful in 15s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 13:49:42 +00:00
Claude ad517965ee CI: publish every push to the staging registry
main → latest (the package.json version, skipped if already published), next → next,
other branches → dev-<branch> as <version>-<tag>.<timestamp> prereleases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 13:43:51 +00:00
Claude 631c75428b Plain operations: last write wins, hash-keyed events, simpler composite
CI / Node 22 (pull_request) Successful in 12s
CI / Node 26 (pull_request) Successful in 13s
CI / Node 24 (pull_request) Successful in 14s
- KV: one write primitive, write(changes) with { key, value? }; writeIfAbsent removed.
  The server executes exactly what clients send and never reads values, so they can be
  encrypted end to end. Adds setFields/getFields on the base class.
- HTTP: no server history or event ids; SSE relays every change (own writes included) and
  the client emits resync when a dropped stream reconnects.
- Composite: writes apply locally, then go to the source (undone locally if refused);
  pushed changes are applied and announced only where they differ from local values;
  offline writes go to a queue store (any KV storage); sync() = replay, diff, replay.
  Removes KVReplicaSet, KVOutbox, KeyCounter, SerialQueue and RetryTimer.
- Events: id = <time>_<sha256(time + canonical data)>, so concurrent appends never
  collide and retries can't duplicate; no versions or ConcurrencyError.
- Event-sourced documents: read, resolve, append, apply; catchUp folds in foreign events;
  getAt takes an event id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 13:33:07 +00:00
Claude 1f82ec9110 Break KVStorageComposite into focused classes
CI / Node 22 (pull_request) Successful in 11s
CI / Node 24 (pull_request) Successful in 12s
CI / Node 26 (pull_request) Successful in 12s
- KVReplicaSet: seed, apply, local scan, diff against the source, sync meta
- KVOutbox: persisted offline queue with in-order replay (load() resumes it)
- utils: KeyRange (also used for server authorization), KeyCounter,
  SerialQueue (also used by KVStorageHttp and the event-sourced store),
  RetryTimer (separate timers for sync and replay; previously one shared
  timer could cancel the other's retry)

The composite now only orchestrates. Behaviour unchanged; new unit tests for
the building blocks and the outbox.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 12:43:49 +00:00
Claude a8c9539ad9 Authenticated, scoped sync: signed requests and per-path subscriptions
CI / Node 24 (pull_request) Successful in 12s
CI / Node 22 (pull_request) Successful in 13s
CI / Node 26 (pull_request) Successful in 13s
- RequestAuth: clients sign `${timestamp}:${path}:${body}` (as in
  sync-server-example); headers carry public key, timestamp, signature and
  algorithm. Pluggable signers/verifiers; Ed25519 via WebCrypto built in.
  Timestamp window and replay protection.
- KVStorageHttpServer requires auth ('none' must be explicit). With auth, every
  key, scan range and subscription path must be under the caller's public key.
- POST /events with { paths }: subscribers only receive (and on resume only
  replay) changes under their authorized paths.
- KVStorageHttp: signer option; subscribe(path)/unsubscribe(path)/close()
  replace watch(); 401/403 reject instead of retrying.
- KVStorageComposite: prefix option scopes sync, cache and subscription.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 10:01:06 +00:00
Claude df7de27e20 Composite: don't undo writes made while a full sync is scanning
CI / Node 24 (pull_request) Successful in 11s
CI / Node 26 (pull_request) Successful in 12s
CI / Node 22 (pull_request) Successful in 13s
A write made during a full sync could be scanned in the replica but not in
the (read-ahead) source scan, so the diff removed it, and echo tracking then
consumed the server's echo. Keys written during a sync are now left out of
the diff. Deterministic regression test with a read-ahead source.

Also exclude -0 from the Extended JSON property test (JSON has no -0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 08:47:18 +00:00
Claude 4ffbc7e78d Add KVStorageComposite, IndexedDB storage and SSE change streams
CI / Node 24 (pull_request) Failing after 12s
CI / Node 22 (pull_request) Successful in 14s
CI / Node 26 (pull_request) Failing after 21s
- Every KV storage emits change events (base class owns writeMany/removeMany/
  writeIfAbsent; implementations provide protected apply* methods)
- KVStorageHttpServer streams changes over SSE (GET /events) with a resumable
  history; KVStorageHttp.watch() subscribes and resumes via Last-Event-ID
- KVStorageIndexedDB for local persistence
- KVStorageComposite keeps replicas of a source in sync: local reads, resume
  instead of full sync, offline outbox replayed in order, echo tracking
- Values may contain bigint/Uint8Array (Extended JSON over the network)
- Vendor sse-session, event-emitter and extended-json (no runtime deps)
- Freeze child nodes so shadowed state on children fails loudly
- CI on Node 22, 24 and 26

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 08:41:42 +00:00
Claude 08fb78721f Simplify: one abstract base class per layer
CI / Node 20 (pull_request) Successful in 10s
CI / Node 22 (pull_request) Successful in 11s
- BaseKVStorage / BaseEventStorage / BaseDocumentStorage define the API once;
  implementations provide only raw primitives (client and server share an API)
- Drop KV indexes, query planner, unique indexes; document stores take index fields
- Escape every char <= '/' so child keys stay contiguous; skip-scan is now simple
- Event-sourced documents = event storage + any document store for projections
- New folder layout (kv-storage, event-storage, document-storage, utils); tests only in test/

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 08:05:03 +00:00
Claude e27541c46a Add Prettier formatting and Gitea Actions CI (format, typecheck, test, build)
CI / Node 20 (push) Successful in 1m49s
CI / Node 22 (push) Successful in 1m52s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:29:38 +00:00
Claude 5a370f8c78 M9: benchmarks, chunked sorted map and JSON clone for the in-memory driver
Profiling showed document updates dominated by O(n) sorted-array splices; the
chunked map makes inserts O(log n + chunk). Baselines recorded in DESIGN.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:25:04 +00:00
Claude 76f791f54b M8: HTTP driver (framework-agnostic handler + HttpKVDriver client) passing conformance end-to-end
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:21:09 +00:00
Claude f6541507ac M7: Spec typing across KV, events and documents with expectTypeOf tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:19:11 +00:00
Claude 0c4e35b9b6 M6: EventSourcedDocumentStorage with projections, update operators, queries, planner, unique indexes, catch-up
Writes append resolved deltas with optimistic concurrency and self-heal lagging
projections. Reads hit projections only. Index-backed find is property-tested
against unindexed find.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:18:03 +00:00
Claude 67f0ba11f7 M5: KVEventStorage with dense commit keys, setIfAbsent and mutex modes, head cache, probe-based read
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:07:54 +00:00
Claude 8bc1c27c40 M4: KV indexes with pattern trie, atomic and non-atomic write paths, read-repair, range findBy
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:04:19 +00:00
Claude 8f8c880efd M3: KVNode/KVStorage with arity dispatch, skip-scan keys, deleteTree, tree
Children are listed in subtree order (escaped segment + separator) so the
skip-scan stays O(children) even when sibling keys extend each other with
characters below the separator. Model-based property test vs a reference Map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:02:58 +00:00
Claude f1ef4a71e9 M2: KV driver interface, BaseKVDriver, InMemoryKVDriver, conformance suite
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 06:53:54 +00:00
Claude 69108d1557 M1: core path escaping, order-preserving index value encoding, errors
Scaffold TypeScript/Vitest project, add DESIGN.md (brief) and CLAUDE.md.
Property tests cover escape round-trips, separator-freedom and order preservation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 06:52:06 +00:00