Commit Graph
20 Commits
Author SHA1 Message Date
Claude 7d755cecae Signed requests: strictly increasing timestamps per key
CI / Node 22 (push) Successful in 13s
CI / Node 24 (push) Successful in 13s
CI / Node 26 (push) Successful in 14s
CI / Node 24 (pull_request) Successful in 12s
CI / Chromium (pull_request) Successful in 48s
CI / Publish to staging registry (pull_request) Skipped
CI / Node 22 (pull_request) Successful in 13s
CI / Node 26 (pull_request) Successful in 12s
CI / Chromium (push) Successful in 47s
CI / Publish to staging registry (push) Successful in 10s
Ed25519 signatures are deterministic, so two identical requests from one key in the same
millisecond (e.g. two devices subscribing to the same path, or parallel identical reads)
had the same signature, and the server rejected the second as a replay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 14:36:25 +00:00
Claude 68d1a1c8e9 Add encryption, Chromium tests and a README
CI / Node 24 (push) Successful in 15s
CI / Node 26 (push) Successful in 16s
CI / Node 22 (pull_request) Successful in 13s
CI / Node 24 (pull_request) Failing after 3s
CI / Node 26 (pull_request) Failing after 4s
CI / Chromium (push) Failing after 1m8s
CI / Publish to staging registry (push) Skipped
CI / Chromium (pull_request) Failing after 1m3s
CI / Publish to staging registry (pull_request) Skipped
CI / Node 22 (push) Failing after 4s
- KVStorageEncrypted: AES-GCM per value (fresh IV, storage key as AAD), keys stay
  readable; decrypts and re-emits pushed changes and passes subscriptions through, so it
  can be a composite source. generateKey / exportKey / importKey / deriveKey (PBKDF2).
- KVStorageHttpServer: allowOrigins (CORS), including cache-control for SSE preflights.
- Chromium tests (Vitest browser mode + Playwright) against a real cross-origin server:
  conformance for IndexedDB, encrypted, HTTP and composite stores; signed requests and
  SSE; two devices syncing, offline writes surviving a reload. New CI job.
- Composite fixes found by those tests: sync() during a running sync schedules one more
  (changes between the first scan and the stream connecting were missed); a queued write
  the source refuses is dropped, restored and reported (onError) instead of blocking the
  queue forever.
- README with examples, mirrored in test/readme.test.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 14:25:22 +00:00
Claude 3525dbc622 Name the package @harvmaster/storage
CI / Node 24 (pull_request) Successful in 12s
CI / Node 26 (pull_request) Successful in 10s
CI / Publish to staging registry (pull_request) Skipped
CI / Node 22 (push) Successful in 11s
CI / Publish to staging registry (push) Successful in 7s
CI / Node 22 (pull_request) Successful in 16s
CI / Node 24 (push) Successful in 13s
CI / Node 26 (push) Successful in 15s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 13:49:42 +00:00
Claude ad517965ee CI: publish every push to the staging registry
main → latest (the package.json version, skipped if already published), next → next,
other branches → dev-<branch> as <version>-<tag>.<timestamp> prereleases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 13:43:51 +00:00
Claude 631c75428b Plain operations: last write wins, hash-keyed events, simpler composite
CI / Node 22 (pull_request) Successful in 12s
CI / Node 26 (pull_request) Successful in 13s
CI / Node 24 (pull_request) Successful in 14s
- KV: one write primitive, write(changes) with { key, value? }; writeIfAbsent removed.
  The server executes exactly what clients send and never reads values, so they can be
  encrypted end to end. Adds setFields/getFields on the base class.
- HTTP: no server history or event ids; SSE relays every change (own writes included) and
  the client emits resync when a dropped stream reconnects.
- Composite: writes apply locally, then go to the source (undone locally if refused);
  pushed changes are applied and announced only where they differ from local values;
  offline writes go to a queue store (any KV storage); sync() = replay, diff, replay.
  Removes KVReplicaSet, KVOutbox, KeyCounter, SerialQueue and RetryTimer.
- Events: id = <time>_<sha256(time + canonical data)>, so concurrent appends never
  collide and retries can't duplicate; no versions or ConcurrencyError.
- Event-sourced documents: read, resolve, append, apply; catchUp folds in foreign events;
  getAt takes an event id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 13:33:07 +00:00
Claude 1f82ec9110 Break KVStorageComposite into focused classes
CI / Node 22 (pull_request) Successful in 11s
CI / Node 24 (pull_request) Successful in 12s
CI / Node 26 (pull_request) Successful in 12s
- KVReplicaSet: seed, apply, local scan, diff against the source, sync meta
- KVOutbox: persisted offline queue with in-order replay (load() resumes it)
- utils: KeyRange (also used for server authorization), KeyCounter,
  SerialQueue (also used by KVStorageHttp and the event-sourced store),
  RetryTimer (separate timers for sync and replay; previously one shared
  timer could cancel the other's retry)

The composite now only orchestrates. Behaviour unchanged; new unit tests for
the building blocks and the outbox.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 12:43:49 +00:00
Claude a8c9539ad9 Authenticated, scoped sync: signed requests and per-path subscriptions
CI / Node 24 (pull_request) Successful in 12s
CI / Node 22 (pull_request) Successful in 13s
CI / Node 26 (pull_request) Successful in 13s
- RequestAuth: clients sign `${timestamp}:${path}:${body}` (as in
  sync-server-example); headers carry public key, timestamp, signature and
  algorithm. Pluggable signers/verifiers; Ed25519 via WebCrypto built in.
  Timestamp window and replay protection.
- KVStorageHttpServer requires auth ('none' must be explicit). With auth, every
  key, scan range and subscription path must be under the caller's public key.
- POST /events with { paths }: subscribers only receive (and on resume only
  replay) changes under their authorized paths.
- KVStorageHttp: signer option; subscribe(path)/unsubscribe(path)/close()
  replace watch(); 401/403 reject instead of retrying.
- KVStorageComposite: prefix option scopes sync, cache and subscription.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 10:01:06 +00:00
Claude df7de27e20 Composite: don't undo writes made while a full sync is scanning
CI / Node 24 (pull_request) Successful in 11s
CI / Node 26 (pull_request) Successful in 12s
CI / Node 22 (pull_request) Successful in 13s
A write made during a full sync could be scanned in the replica but not in
the (read-ahead) source scan, so the diff removed it, and echo tracking then
consumed the server's echo. Keys written during a sync are now left out of
the diff. Deterministic regression test with a read-ahead source.

Also exclude -0 from the Extended JSON property test (JSON has no -0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 08:47:18 +00:00
Claude 4ffbc7e78d Add KVStorageComposite, IndexedDB storage and SSE change streams
CI / Node 24 (pull_request) Failing after 12s
CI / Node 22 (pull_request) Successful in 14s
CI / Node 26 (pull_request) Failing after 21s
- Every KV storage emits change events (base class owns writeMany/removeMany/
  writeIfAbsent; implementations provide protected apply* methods)
- KVStorageHttpServer streams changes over SSE (GET /events) with a resumable
  history; KVStorageHttp.watch() subscribes and resumes via Last-Event-ID
- KVStorageIndexedDB for local persistence
- KVStorageComposite keeps replicas of a source in sync: local reads, resume
  instead of full sync, offline outbox replayed in order, echo tracking
- Values may contain bigint/Uint8Array (Extended JSON over the network)
- Vendor sse-session, event-emitter and extended-json (no runtime deps)
- Freeze child nodes so shadowed state on children fails loudly
- CI on Node 22, 24 and 26

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 08:41:42 +00:00
Claude 08fb78721f Simplify: one abstract base class per layer
CI / Node 20 (pull_request) Successful in 10s
CI / Node 22 (pull_request) Successful in 11s
- BaseKVStorage / BaseEventStorage / BaseDocumentStorage define the API once;
  implementations provide only raw primitives (client and server share an API)
- Drop KV indexes, query planner, unique indexes; document stores take index fields
- Escape every char <= '/' so child keys stay contiguous; skip-scan is now simple
- Event-sourced documents = event storage + any document store for projections
- New folder layout (kv-storage, event-storage, document-storage, utils); tests only in test/

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 08:05:03 +00:00
Claude e27541c46a Add Prettier formatting and Gitea Actions CI (format, typecheck, test, build)
CI / Node 20 (push) Successful in 1m49s
CI / Node 22 (push) Successful in 1m52s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:29:38 +00:00
Claude 5a370f8c78 M9: benchmarks, chunked sorted map and JSON clone for the in-memory driver
Profiling showed document updates dominated by O(n) sorted-array splices; the
chunked map makes inserts O(log n + chunk). Baselines recorded in DESIGN.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:25:04 +00:00
Claude 76f791f54b M8: HTTP driver (framework-agnostic handler + HttpKVDriver client) passing conformance end-to-end
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:21:09 +00:00
Claude f6541507ac M7: Spec typing across KV, events and documents with expectTypeOf tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:19:11 +00:00
Claude 0c4e35b9b6 M6: EventSourcedDocumentStorage with projections, update operators, queries, planner, unique indexes, catch-up
Writes append resolved deltas with optimistic concurrency and self-heal lagging
projections. Reads hit projections only. Index-backed find is property-tested
against unindexed find.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:18:03 +00:00
Claude 67f0ba11f7 M5: KVEventStorage with dense commit keys, setIfAbsent and mutex modes, head cache, probe-based read
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:07:54 +00:00
Claude 8bc1c27c40 M4: KV indexes with pattern trie, atomic and non-atomic write paths, read-repair, range findBy
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:04:19 +00:00
Claude 8f8c880efd M3: KVNode/KVStorage with arity dispatch, skip-scan keys, deleteTree, tree
Children are listed in subtree order (escaped segment + separator) so the
skip-scan stays O(children) even when sibling keys extend each other with
characters below the separator. Model-based property test vs a reference Map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 07:02:58 +00:00
Claude f1ef4a71e9 M2: KV driver interface, BaseKVDriver, InMemoryKVDriver, conformance suite
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 06:53:54 +00:00
Claude 69108d1557 M1: core path escaping, order-preserving index value encoding, errors
Scaffold TypeScript/Vitest project, add DESIGN.md (brief) and CLAUDE.md.
Property tests cover escape round-trips, separator-freedom and order preservation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Qxbkp4tA7G5SCWjzWuxmE
2026-10-23 06:52:06 +00:00