xo-sync-server-hono/docs/api.md

XO Sync Server API

This server stores encrypted client data as append-only events. The server does not interpret event payloads; data is accepted and returned as a base64 string.

Data Model

Stored events have this shape:

type StorageEvent = {
  storageIdentifier: string;
  data: string;      // base64 encoded encrypted payload
  hash: string;      // base64 sha256 hash of decoded data bytes
  created_at: number; // unix timestamp in milliseconds
};

storageIdentifier identifies the logical append-only storage feed. It is also the realtime subscription topic.

Errors

Validation errors return HTTP 400 with:

{
  "statusCode": 400,
  "error": "Validation Error",
  "details": [
    {
      "path": "storageIdentifier",
      "message": "Required"
    }
  ]
}

Unexpected errors return HTTP 500:

{
  "error": "Internal Server Error"
}

Stats / Bloom Negotiation

Use these endpoints before filtered reads to learn the server's event count, the inverse-bloom-filter size the client should use, and the server's current filter fingerprint.

GET /stats

Alias: GET /bloom

Query parameters:

storageIdentifier: string // required; can be comma-separated for batching

Example:

GET /stats?storageIdentifier=/wallet/a,/wallet/b

Response:

{
  "items": [
    {
      "storageIdentifier": "/wallet/a",
      "count": 42,
      "negotiatedBloomFilterSize": 256,
      "bloomFilter": "BASE64_SERVER_FILTER",
      "bloomFilterHash": "BASE64_SHA256_OF_SERVER_FILTER",
      "cached": false
    },
    {
      "storageIdentifier": "/wallet/b",
      "count": 1000,
      "negotiatedBloomFilterSize": 1000,
      "bloomFilter": "BASE64_SERVER_FILTER",
      "bloomFilterHash": "BASE64_SHA256_OF_SERVER_FILTER",
      "cached": true
    }
  ]
}

negotiatedBloomFilterSize is currently max(count, 256). Clients should build their inverse bloom filter using this size before calling /data with a filter. count is not a sync check; two peers can have the same count but different events. Use bloomFilter or bloomFilterHash as the server-side set fingerprint.

POST /stats

Alias: POST /bloom

Request body:

{
  "storageIdentifiers": ["/wallet/a", "/wallet/b"]
}

Response is the same as GET /stats.

Read Data

GET /data

Reads events for one storage identifier.

Query parameters:

storageIdentifier: string; // required
bloomFilter?: string;      // optional base64 serialized inverse bloom filter
count?: number;            // required when bloomFilter is supplied

Without a filter, the server returns all events:

GET /data?storageIdentifier=/wallet/a

Response:

[
  {
    "storageIdentifier": "/wallet/a",
    "data": "BASE64_PAYLOAD",
    "hash": "BASE64_SHA256_HASH",
    "created_at": 1760000000000
  }
]

With a compatible filter, the server returns only events that appear missing from the client:

GET /data?storageIdentifier=/wallet/a&bloomFilter=BASE64_FILTER&count=256

If count does not match the current negotiated size, the server falls back to returning the full event list for correctness.

If the supplied filter matches the server's current filter, the response is an empty list because the client already has all current server events. If the filters differ, the server builds or reuses its current filter, computes the difference, then iterates storage rows and returns only rows whose hashes appear missing from the client filter.

Append Data

POST /data

Appends one event to a storage feed.

Request body:

{
  "storageIdentifier": "/wallet/a",
  "data": "BASE64_PAYLOAD"
}

Response:

{
  "storageIdentifier": "/wallet/a",
  "data": "BASE64_PAYLOAD",
  "hash": "BASE64_SHA256_HASH",
  "created_at": 1760000000000
}

Side effects:

• Invalidates the cached bloom filter for storageIdentifier.
• Broadcasts a realtime storage-event to subscribers of that storage feed.

Stream Data

GET /data/stream

Opens a Server-Sent Events stream for one storage identifier.

Query parameters are the same as GET /data:

storageIdentifier: string;
bloomFilter?: string;
count?: number;

Connection behavior:

1. The server opens an SSE connection.
2. The server subscribes the connection to storage:${storageIdentifier}.
3. The server calls the same handler as GET /data.
4. If the handler returns data, the server sends it as the initial sync event.
5. The connection remains open for live storage-event broadcasts.

Initial sync event:

event: sync
data: [{"storageIdentifier":"/wallet/a","data":"...","hash":"...","created_at":1760000000000}]

Live append event:

event: storage-event
data: {"storageIdentifier":"/wallet/a","data":"...","hash":"...","created_at":1760000001000}

SSE also sends:

retry: 3000

Reconnect behavior uses the same sync mechanism as the first connection. Clients should reconnect with a fresh local inverse bloom filter:

GET /data/stream?storageIdentifier=/wallet/a&bloomFilter=BASE64_FILTER&count=256

The server sends any missing persistent events as the initial sync event, then continues with live storage-event broadcasts. The server does not use Last-Event-ID for SSE replay.

Recommended Sync Flow

1. Call POST /stats with all storage identifiers the client wants to sync.
2. For each storage identifier, compare the server bloomFilterHash with the client's local filter hash if available.
3. If filters differ, build a local inverse bloom filter using negotiatedBloomFilterSize.
4. Call GET /data or GET /data/stream with the client's bloomFilter and count.
5. Apply returned server events locally.
6. Compare the server bloomFilter from /stats against local events to find local events the server is missing, then upload those with POST /data.
7. Keep /data/stream open if realtime updates are needed.
8. On reconnect, repeat the stats and bloom-filtered stream request.

Notes

• Storage is append-only. There is no update or delete endpoint.
• Payloads are opaque to the server.
• hash is computed by the server from decoded data bytes using SHA-256.
• /bloom currently exists as an alias for /stats; reconciliation happens in /data, not /bloom.
• The bloom-filter cache stores the current server filter for each storageIdentifier; it does not store event payloads or a copy of all event hashes. The cache is invalidated when new data is appended.