xo-sync-server-hono/docs/api.md

# XO Sync Server API

This server stores encrypted client data as append-only events. The server does
not interpret event payloads; `data` is accepted and returned as a base64 string.

## Data Model

Stored events have this shape:

```ts
type StorageEvent = {
  storageIdentifier: string;
  data: string;      // base64 encoded encrypted payload
  hash: string;      // base64 sha256 hash of decoded data bytes
  created_at: number; // unix timestamp in milliseconds
};
```

`storageIdentifier` identifies the logical append-only storage feed. It is also
the realtime subscription topic.

## Errors

Validation errors return HTTP `400` with:

```json
{
  "statusCode": 400,
  "error": "Validation Error",
  "details": [
    {
      "path": "storageIdentifier",
      "message": "Required"
    }
  ]
}
```

Unexpected errors return HTTP `500`:

```json
{
  "error": "Internal Server Error"
}
```

## Stats / Bloom Negotiation

Use these endpoints before filtered reads to learn the server's event count,
the inverse-bloom-filter size the client should use, and the server's current
filter fingerprint.

### `GET /stats`

Alias: `GET /bloom`

Query parameters:

```ts
storageIdentifier: string // required; can be comma-separated for batching
```

Example:

```http
GET /stats?storageIdentifier=/wallet/a,/wallet/b
```

Response:

```json
{
  "items": [
    {
      "storageIdentifier": "/wallet/a",
      "count": 42,
      "negotiatedBloomFilterSize": 256,
      "bloomFilter": "BASE64_SERVER_FILTER",
      "bloomFilterHash": "BASE64_SHA256_OF_SERVER_FILTER",
      "cached": false
    },
    {
      "storageIdentifier": "/wallet/b",
      "count": 1000,
      "negotiatedBloomFilterSize": 1000,
      "bloomFilter": "BASE64_SERVER_FILTER",
      "bloomFilterHash": "BASE64_SHA256_OF_SERVER_FILTER",
      "cached": true
    }
  ]
}
```

`negotiatedBloomFilterSize` is currently `max(count, 256)`. Clients should build
their inverse bloom filter using this size before calling `/data` with a filter.
`count` is not a sync check; two peers can have the same count but different
events. Use `bloomFilter` or `bloomFilterHash` as the server-side set
fingerprint.

### `POST /stats`

Alias: `POST /bloom`

Request body:

```json
{
  "storageIdentifiers": ["/wallet/a", "/wallet/b"]
}
```

Response is the same as `GET /stats`.

## Read Data

### `GET /data`

Reads events for one storage identifier.

Query parameters:

```ts
storageIdentifier: string; // required
bloomFilter?: string;      // optional base64 serialized inverse bloom filter
count?: number;            // required when bloomFilter is supplied
```

Without a filter, the server returns all events:

```http
GET /data?storageIdentifier=/wallet/a
```

Response:

```json
[
  {
    "storageIdentifier": "/wallet/a",
    "data": "BASE64_PAYLOAD",
    "hash": "BASE64_SHA256_HASH",
    "created_at": 1760000000000
  }
]
```

With a compatible filter, the server returns only events that appear missing from
the client:

```http
GET /data?storageIdentifier=/wallet/a&bloomFilter=BASE64_FILTER&count=256
```

If `count` does not match the current negotiated size, the server falls back to
returning the full event list for correctness.

If the supplied filter matches the server's current filter, the response is an
empty list because the client already has all current server events. If the
filters differ, the server builds or reuses its current filter, computes the
difference, then iterates storage rows and returns only rows whose hashes appear
missing from the client filter.

## Append Data

### `POST /data`

Appends one event to a storage feed.

Request body:

```json
{
  "storageIdentifier": "/wallet/a",
  "data": "BASE64_PAYLOAD"
}
```

Response:

```json
{
  "storageIdentifier": "/wallet/a",
  "data": "BASE64_PAYLOAD",
  "hash": "BASE64_SHA256_HASH",
  "created_at": 1760000000000
}
```

Side effects:

- Invalidates the cached bloom filter for `storageIdentifier`.
- Broadcasts a realtime `storage-event` to subscribers of that storage feed.

## Stream Data

### `GET /data/stream`

Opens a Server-Sent Events stream for one storage identifier.

Query parameters are the same as `GET /data`:

```ts
storageIdentifier: string;
bloomFilter?: string;
count?: number;
```

Connection behavior:

1. The server opens an SSE connection.
2. The server subscribes the connection to `storage:${storageIdentifier}`.
3. The server calls the same handler as `GET /data`.
4. If the handler returns data, the server sends it as the initial `sync` event.
5. The connection remains open for live `storage-event` broadcasts.

Initial sync event:

```txt
event: sync
data: [{"storageIdentifier":"/wallet/a","data":"...","hash":"...","created_at":1760000000000}]
```

Live append event:

```txt
event: storage-event
data: {"storageIdentifier":"/wallet/a","data":"...","hash":"...","created_at":1760000001000}
```

SSE also sends:

```txt
retry: 3000
```

Reconnect behavior uses the same sync mechanism as the first connection. Clients
should reconnect with a fresh local inverse bloom filter:

```http
GET /data/stream?storageIdentifier=/wallet/a&bloomFilter=BASE64_FILTER&count=256
```

The server sends any missing persistent events as the initial `sync` event, then
continues with live `storage-event` broadcasts. The server does not use
`Last-Event-ID` for SSE replay.

## Recommended Sync Flow

1. Call `POST /stats` with all storage identifiers the client wants to sync.
2. For each storage identifier, compare the server `bloomFilterHash` with the
   client's local filter hash if available.
3. If filters differ, build a local inverse bloom filter using
   `negotiatedBloomFilterSize`.
4. Call `GET /data` or `GET /data/stream` with the client's `bloomFilter` and
   `count`.
5. Apply returned server events locally.
6. Compare the server `bloomFilter` from `/stats` against local events to find
   local events the server is missing, then upload those with `POST /data`.
7. Keep `/data/stream` open if realtime updates are needed.
8. On reconnect, repeat the stats and bloom-filtered stream request.

## Notes

- Storage is append-only. There is no update or delete endpoint.
- Payloads are opaque to the server.
- `hash` is computed by the server from decoded `data` bytes using SHA-256.
- `/bloom` currently exists as an alias for `/stats`; reconciliation happens in
  `/data`, not `/bloom`.
- The bloom-filter cache stores the current server filter for each
  `storageIdentifier`; it does not store event payloads or a copy of all event
  hashes. The cache is invalidated when new data is appended.