LibAuth Comparison

This commit is contained in:
Jim Hamill
2025-02-03 11:40:38 +00:00
parent 14506d4ca6
commit f2b182c882
6 changed files with 1152 additions and 72 deletions
-2
View File
@@ -41,8 +41,6 @@ function toPascalCase(str: string) {
}
function convertNavToPascalCase(nav: Array<any>) {
console.log(nav);
return nav.map((item) => {
const result = { ...item };
@@ -77,6 +77,20 @@ ul.VPDocOutlineItem.root {
font-size: 12px !important;
}
details[open] {
/*
border-top: 1px dashed var(--vp-c-text-1);
border-bottom: 1px dashed var(--vp-c-text-1);
*/
padding-bottom: 3em;
border-bottom: 1px dashed var(--vp-c-text-1);
margin-bottom: 6em;
}
details summary {
color: var(--primary) !important;
}
/* Customization of VitePress CSS */
:root {
+4 -61
View File
@@ -67,9 +67,9 @@ const txResult = await walletHD.send({
General structure:
- All blockchain adapters will implement `IBlockhain`
- All storage adapters will implement `IStorage`
- All wallets will implement `IWallet`
- All blockchain adapters will implement `BaseBlockchain`
- All storage adapters will implement `BaseStorage`
- All wallets will implement `BaseWallet`
- Likewise for any other "service" we need in XO Stack (e.g. BCMR?, Prices?).
Most (almost all?) service methods in our interfaces will be `async`. This is so that we can handle situations where things might be remote (or require OS approval of some sort). For example:
@@ -93,7 +93,7 @@ Wallets can be **composed** of other Wallets. For example:
- WalletHD **has many** WalletP2PKH
- SomeCustomWallet **may have many** WalletHD (or other kinds of wallet).
The only requirement is that a Wallet **MUST** implement the IWallet interface.
The only requirement is that a Wallet **MUST** implement the BaseWallet interface.
### Blockchain
@@ -156,63 +156,6 @@ Do we have to store a whole graph?
Do we just broadcast to one of our nodes and then broadcast to ALL of our nodes ~5s later to see if there's contention?
### Electrum 4.0.1 Notes
1. Maybe default constructor generic to `ElectrumClientEvents`:
```ts
class ElectrumClient<ElectrumEvents extends ElectrumClientEvents = ElectrumClientEvents>
```
2. `RequestResponse` might be typed incorrectly: It now takes `RPCParameter | RPCParameter[]` which are defined as follows: `string | number | boolean | null;` (note that it only accept parameters, not responses which are often/maybe-always JSON).
Previously, it would accept:
```
export type RequestResponse =
| object
| string
| number
| boolean
| null
| RequestResponse[];
```
Perhaps we could type it: `export type RequestResponse = RPCParameter & object | Array<RPCParameter & object>;` (replace `object` with something like `RPCResponse`, but I think that might be taken already?)
3. `RPCNotification` might be typed incorrectly (or maybe this is for future proofing?). The `params` property is marked as optional (but I think all notifications will have at least one). Might be more elegant to have it required and just return an empty array.
4. The current approach still requires polyfills because `@electrum/sockets` itself is imported by `@electrum/network` and `@electrum/sockets` exports `ElectrumTcpSocket`.
A workable approach might be:
1. Move socket interface back to `@electum/network`.
2. Move the `ElectrumWebSocket` back to `@electrum/network`.
3. Rename `@electrum/sockets` to `@electrum/tcp-socket` and only include the TCP Socket in there.
4. This will still allow third-parties to define their own sockets (e.g. `@custom/electrum-libp2p-socket`).
5. This is what's required for polyfilling in Vite currently:
```sh
npm install events
npm install vite-plugin-node-polyfills
```
```js
[
nodePolyfills,
{
include: ['events', 'net', 'tls'],
// NOTE: Not disabling these causes an error on Vite Polyfills V0.22.
globals: {
Buffer: false,
global: false,
process: false,
},
},
],
```
## Markdown Extensions (TODO: Remove me later)
Check out the documentation for the [full list of markdown extensions](https://vitepress.dev/guide/markdown).
+828
View File
@@ -0,0 +1,828 @@
# XO/Libauth Comparison
**NOTE: The below is still a WIP. On 2025-01-31, we floated the idea of using OP_RETURN as a backup mechanic of important data (P2SH data params, etc) and the below does not yet reflect this correctly (but a very rough approach is described near the bottom).**
## High Level Overview
<div class="columns">
<div class="column">
**Libauth**
- Functional approach
- Highly tree-shakable
</div>
<div class="column">
**XO**
- Object-Oriented approach
- Built on-top of LibAuth.
- Declarative-style API (for ease of use)
- Adapter-based (can swap in/out blockchain adapters, storage adapters, etc).
- Layered packages:
1. @xocash/primitives
Network-naive layer offering common BCH primitives (Address, HDNodes, PrivateKey, PublicKey, etc).
2. @xocash/stack
Network-aware layer, offering blockchain, storage, wallet and P2P interfaces/adapters.
3. @xocash/wallet | @xocash/dapp
Higher-level conveniences over @xocash/stack, providing sensible default adapters for given use-cases.
- Moderately tree-shakable
- Most bundlers cannot tree-shake unused class methods (but may be able to in future).
</div>
</div>
### Details
At the high level, we're using a more OO and declarative approach for XO. There are trade-offs here (particularly with regards to tree-shaking), but we are expecting most apps that leverage XO to use all (or most) of the WebASM Crypto Utilities provided in LibAuth (which will likely constitute a large portion of the bundle size).
This is still in flux, but we are anticipating (at least) three packages/layers:
1. **@xocash/primitives**
This is just an OO wrapper over LibAuth to make BCH primitives easier for less technical devs to use.
It includes primitives like Address, HDPrivateKey, HDPublicKey, Mnemonic, PrivateKey, PublicKey, etc.
Each primitive is immutable. The interfaces here are generally `from`, `to` and `methods` that can be performed upon the Primitive. E.g.
```ts
// NOTE: Each `from` method is static and throws on invalid input.
const cashAddress = Address.fromLegacy('someLegacyAddress').toCashAddr();
```
These are all network-naive - the aim is that the only package dependency here is LibAuth.
2. **@xocash/stack**
This is a level above primitives that features various network-aware components to build apps with XO.
It includes the modular "building blocks": Blockchain Adapters, Storage Adapters, P2P Adapters (TBD), Wallets and Utils.
These will lean on Dependency Injection patterns so that adapters can be swapped in/out.
For example, when instantiating a wallet, the dev can use choose which Blockchain Adapter (e.g. Electrum-Cash, Chaingraph, Local/Memory) they wish to use and which storage adapter (Memory, IndexedDB, SQLite, Mongo, etc) they wish to use - or create their own Adapters provided they satisfy the XO interfaces.
(NOTE: Some adapters might end up in their own package if they drag in heavy dependencies).
3. **@xocash/conveniences** (almost definitely won't be named this)
This is a higher-level package that just instantiates the above for common scenarios: E.g. Building a Wallet, building a Dapp, etc.
Essentially, the intent is to just provide sensible defaults (adapters) for those common use-cases.
(To better manage package dependecies, it's possible that this might end up as an explicit package per use-case: e.g. @xocash/dapp, @xocash/wallet, etc).
<details>
<summary>Why the OO approach?</summary>
The main reasons are for cleaner documentation/IDE-autocomplete and easier DX: We want to try to handle a lot of the state for developers and an OO approach allows us to encapsulate that nicely and provide declarative interfaces to access it.
For example, instantiation of a HD Wallet might look something like this:
```ts
// Create an instance of a HDWallet.
const walletHD = await WalletHD.from(
// Adapters
{
blockchain: blockchainElectrum,
cache: cachesStore,
// NOTE: This will likely change to "Activities"
storage: walletsStore,
},
// Settings.
// NOTE: Currently, this supports other forms of instantation (e.g. Seed or XPriv).
// See WalletHDGenesisData: https://gitlab.com/GeneralProtocols/xo/stack/-/blob/development/packages/stack/src/wallet/wallet-hd.ts?ref_type=heads#L43
{
derivationPath: "m/44'/145'/0'",
mnemonic:
'page pencil stock planet limb cluster assault speak off joke private pioneer',
},
);
// Send a transaction by providing a tx template.
// NOTE: There's currently a TxBuilder that populates UTXOs to meet the output amounts and append change.
await walletHD.send({
outputs: [
{
lockingBytecode: Address.from('someAddress').toLockscriptBytes(),
valueSatoshis: 20000n
}
]
});
```
Tree-shaking is a big down-side of this in the sense that currently (with the exception of Google's `clojure`) no bundlers can shake unused class methods. However, apparently this is possible so **may** change in future: https://github.com/rollup/rollup/issues/349#issuecomment-2560719286
</details>
## Wallets and Wallet-Set/Root-Seed/Account
<div class="columns">
<div class="column">
**Libauth**
- Multiple keys supported per WalletSet.
- WalletSet ID is randomly generated (?).
</div>
<div class="column">
**Tentative XO Approach**
- Single root-seed that is used to derive other wallets.
- Keys are not stored in activities (only info on how to derive keys from the root seed).
- Root seed also deterministically derives: WalletID, Encryption Key and Signing Key (maybe others?)
</div>
</div>
We're considering deviating a bit from the LibAuth approach documented [here](https://github.com/bitjson/libauth/blob/d3d610de088cd98f5c391fe62dce81874eb5d95a/docs/engine/wallet-sets.md#create-a-wallet-set) as there's a few additional things we'd like to try to aim for:
1. Only supporting a single root-seed per WalletSet.
Part of the intent here is to make it so that a user only requires a single detail for Wallet Recovery/Sync on another device (but this also ties in closely with the below items).
2. Allowing existing HD Wallet mnemonics to used to create a "WalletSet".
This is more of a convenience and that it **may** be achievable is actually probably more of a consequence using a single-root seed.
3. Storing other metadata (e.g. Tx Received Time/Fiat-Amount, Tx Memos, Tx Categories, Frozen Coins, etc) using Activities.
Ideally, we want to use "Activities" to store other information about a user's wallet(s), so that other pieces of state can be synced across devices seamlessly.
4. The ability to backup "Action" (P2SH) data to an external store (or stores) prior to broadcasting to Blockchain.
This is to prevent, for example, loss of P2SH Wallet/AddressData in case user loses access to their device: We want to make sure that all data can be recoverable on another device via a root-seed only.
5. Storing only the instructions (e.g. path) as to how keys are derived (as opposed to any keys themselves) in Activities.
### Key Derivation
To minimize the amount of data necessary for recovery and maintain backwards compatibility with existing HD Wallets, we use the root-seed to derive all other keys in a (near) deterministic sense.
The below is very rough (I'm not suggesting this as a practical implementation, but just to illustrate the approach vaguely).
| Key | Derivation (Rough Example) | Comments |
|------------------------|---------------------------------------------------------------|---------------------------------------|
| Default HD Wallet | Root Seed -> m/44'/145'/0' | Backwards compat with current Wallets |
| WalletId | sha256(sha256(rootSeed) + sha256('walletId')) | Identifier for Wallet |
| EncryptionKey | sha256(sha256(rootSeed) + sha256('encryptionKey')) | Encrpytion Key for Activities |
| SigningKey | sha256(sha256(rootSeed) + sha256('signingKey')) | Signing key for Identity/Activity |
| Intra-Wallet Templates | sha256(sha256(rootSeed) + sha256('template:${templateHash}')) | Derive seed per template |
| Web-based Dapps | sha256(sha256(rootSeed) + sha256(`domain:${domain}')) | Derive seed per domain |
NOTE: WalletID, EncryptionKey and SigningKey could **probably** use a BIP32 Derivation Path instead of hashing a new seed from the root seed (and this might be more ideal/sensible?).
If we use derivation indexes for Templates and/or Web-based Dapps, In a multi-device setup, the devices might become "unsynced" for a period, we could end up with overlapping derivation indexes used across devices. This presents a security concern:
> **If the same private key is used to sign both an ECDSA AND Schnorr, this Private Key becomes recoverable (it is compromised).**
However, if we derive a deterministic root "key" for use with each template/dapp, (I think) this helps mitigate some of the very dangerous consequences of re-using derivation indexes (and therefore keys) in the event that wallets become desynced.
For example, provided a given template is audited to not contain both ECDSA and Schnorr signing, we are guarded against the above (and, in the worst case, the key is only compromised for the given template - or domain).
(Are there other attack vectors we need to consider?)
### Wallet Structure
The current structure I have for Wallets is very OO - all Wallets inherit from an abstract `BaseWallet` class that contains a common interface that derived Wallets must implement.
(Very early draft of the interface: https://stack-generalprotocols-xo-ad8a04d164920d0360d523e66b59226eb1795.gitlab.io/api/@xocash/stack/wallet/base-wallet.html )
The idea is that a Wallet **can be** composed of other wallets. A rough example of this being done for WalletHD (which is composed of WalletP2PKH) is available [here](https://gitlab.com/GeneralProtocols/xo/stack/-/blob/development/packages/stack/src/wallet/wallet-hd.ts?ref_type=heads).
For WalletSets, I think this should allow something similar to the the Wallet "Shares" concept described [here](https://todo).
At a very rough high-level (this has not yet been implemented), instantiating a WalletSet might then look something like:
```ts
// Some root wallet seed.
const seed = new Uint8Array(someSeed);
// Create an instance of our Activities store for this wallet.
// NOTE: The way I'm Instantiating this here probably looks weird.
// The reason for it is that I think the Activities scheme might actually have utility outside of Wallets.
// For example, we might want to sync non-wallet data across services and it might be a decent way of doing this.
// In terms of complexity, this will likely be one of those things we abstract over in our @xocash/convenience layer.
// (Making this generically usable might just be a pipe-dream of mine, so might not be done like this in future).
const activitiesStore = await Activities.from({
id: WalletSet.deriveId(seed)
encryption: new Encrypter({ key: WalletSet.deriveEncryptionKey(seed) }),
signingKey: new Signer({ key: WalletSet.deriveSigningKey(seed) }),
requiredStores: [localStore, gpWalletSyncServer],
optionalStores: [someOtherSyncServer], // Maybe even an OP_RETURN storage mechanic??? TODO: Decide whether you want to go into this below.
// NOTE: These define how snapshots are compiled.
// E.g. for a given ActivityType, it may have a policy where "Latest Wins" or "Earlist-Wins" - or might even have a custom, more-complex policy (defined using callbacks).
// See "Activities" section below for more details.
policies: XO_DEFAULT_WALLET_POLICIES
})
// Create a WalletSet
const walletSet = WalletSet.from(
// Adapters
{
blockchain: blockchainElectrum,
cache: cachesStore,
activities: activitiesStore,
},
// Genesis Data
{
seed // or mnemonic: 'some mnemonic words' for importing legacy
}
)
```
... where `WalletSet` might be something like (NOTE: Very very rough - don't take literally):
```ts
export class WalletSet extends BaseWallet {
// The root seed.
protected readonly genesisData: Seed | Mnemonic;
// Wallets indexed by walletId.
// Near equivalent of "shares" in LibAuth - but a bit more abstract in the sense it uses the BaseWallet interface.
// Templates will likely also be wrapped under that interface (called WalletTemplated or something), but allow additional methods.
protected wallets: { [walletId: string]: BaseWallet };
// Metadata for this wallet.
// See the below section for how this might work.
protected activities: Activities;
static async from(adapters: WalletAdapters, genesisData: WalletSetGenesisData) {
// Compile a snapshot of activities.
const walletState = adapters.activities.compileSnapshot();
// Instantiate other wallets from our compiled state (which is derived from our "Activities").
const walletPromises = this.walletState['shares'].map((walletShare) => {
// Create an instance of the wallet share depending on what type it is, etc.
if (walletShare.type === 'TemplatedWallet') {
return await WalletTemplated.from(adapters, {
seed: genesisData.seed,
template: walletShare.template,
walletData: walletShare.walletData,
derivationPath: walletShare.derivationPath,
etc,
})
}
if (walletShare.type === 'WalletHD') {
return await WalletHD.from(adapters, rootHDNode.derive
{
seed: genesisData.seed,
derivationPath: walletShare.derivationPath,
},
}
// etc
});
// Wait for all of the Wallet Shares to instantiate.
const wallets = Promise.all(walletSharePromises)
// Return the new WalletSet instance.
return new WalletSet(adapters, { rootHDNode, walletId, encryptionKey, signingKey, wallets, etc });
}
async send(txTemplate, filterCallback = someDefaultFilterThatOnlyUsesWalletsWhereWalletTypeIsWalletHD) {
// Filter for wallets who's UTXOs we can populate the tx with.
const walletsFiltered = filterCallback(this.wallets);
// Get unspents we can populate the tx with.
const unspents = Promise.all(walletsFiltered.map(wallet => await wallet.getUnspents()));
// ... some PoC code here under signTransaction/send:
// https://gitlab.com/GeneralProtocols/xo/stack/-/blob/2def0383ec0d1d78f2579861136faddf7aacd4a7/packages/stack/src/wallet/wallet-hd.ts#L340
}
// ... other methods.
}
```
## Activities (and "eventually consistent syncing")
**NOTE: This section is outdated and WIP. Still worth reading, but we're trending towards the OP_RETURN approach (below) if viable.**
<div class="columns">
<div class="column">
**Libauth**
- Multiple keys per WalletSet.
- Wallet ID is arbitrarily generated.
</div>
<div class="column">
**Tentative XO Approach**
- Each activity is replicated immediately with other devices (configurable).
- Activities are "append-only" (though some optimizations done locally for tomb-stoning irrelevant data)
- Activity payloads are "atomic".
- Final state is a built "snapshot" of Activity data.
- UUIDV7 as ActivityID (UUIDV7 includes timestamps).
- Syncing occurs by comparing Activity IDs (optimized using Inverse Bloom Filters)
</div>
</div>
For XO, we want to try to make it so that:
1. Activities can be immediately synced to backup server(s) - or other wallets/devices.
2. We can also use Activities to sync both Address/Wallet Data and other Wallet Metadata (Memo's, Frozen Coins, Time Received, etc).
3. Data stored on backup servers is encrypted (and, in the worst case that this is compromised, it does not reveal keys).
<details>
<summary>On the note of encryption</summary>
TODO: Do we encrypt Activity IDs (UUIDV7)? Might be overkill. Also, incurrs additional compute + bandwidth costs (+28 bytes per UUIDV7 due to AES-GCM header).
There's also the problem that, when syncing from an encrypted remote, we would have to ensure we encrypt using the same AES-GCM IV, otherwise the values we generate locally for set-comparison would not be identical to the encrypted remote's values.
</details>
In the structure being considered currently, we index each activity using a UUIDV7 (which contains a timestamp).
<details>
<summary>Why UUIDV7?</summary>
The original LibAuth proposal suggests using a millisecond timestamp as index (appended by entity), but we probably need more granularity as several Activities might occur on the same millisecond timestamp. UUIDV7 itself contains a timestamp (and is lexicographically sortable by timestamp), but then attaches additional randomness to enforce uniqueness.
</details>
```ts
export type Activities = { [uuidv7: string]: Activity };
```
... where the payload Activity itself looks something like this:
```ts
export type Activity<Type extends string, ValueType> = {
// NOTE: Timestamp will be automatically generated if not provided.
timestamp?: bigint;
type: Type;
// NOTE: May want to make this a Uint8Array in future.
id: string;
value: ValueType;
};
```
... and `Type` defines the type of Activity that has taken place. E.g.
```ts
// Standardized Activities.
// NOTE: This would be a common standard expected to be supported by most Wallets.
export type WalletName = Activity<'WalletName', string>;
export type WalletDescription = Activity<'WalletDescription', string>;
export type TransactionFirstSeen = Activity<'TransactionFirstSeen', string>;
export type TransactionMemo = Activity<'TransactionMemo', string>;
// XO-specific Activities.
// NOTE: In practice, the below might be standardized, but the concept here is that some Wallets/Services might have "custom" Acitvity/Event types that they wish to support.
export type XOWalletColor = Activity<'XOWalletColor', string>;
export type XOTransactionCategory = Activity<'XOTransactionCategory', string>;
export type XOFrozenCoin = Activity<'XOFrozenCoin', boolean>;
export type XOTemplateAction = Activity<'XOTemplateAction', TemplateActionPayload>;
export type XOTemplateActionResult = Activity<'XOTemplateActionResult', TemplateActionResultPayload>;
```
<details>
<summary>"Atomic" Activities</summary>
For XO, we want to also be able to store additional metadata. A lot of this metadata will be mutable (e.g. `WalletName`, `TrasnactionMemo`, etc). To ensure we ONLY mutate what we intend to mutate (and maintain the "eventual consistency" approach), we try to keep these structures relevant to the precise payload that they're mutating.
For example, if we had a structure like so:
```ts
export type WalletDetails = Activity<'WalletDetails', {
name: string;
description: string
}>;
```
... it can open us up to a scenario like the following:
1. *Wallet A* and *Wallet B* are not connected to each other.
2. *Wallet A* creates a `WalletDetails` activity and sets the name to "New Wallet Name", while not modifying the `description`.
3. *Wallet B* creates a `WalletDetails` activity and sets the description to "New Wallet Description", while not modifying the `name`.
4. When we sync these devices under a "latest wins" policy for `WalletDetails`, there will be data loss: We will prioritize the latest WalletDetails activity (*Wallet B*), which means that the name set by Wallet A would be lost as the `WalletDetails` structure of *Wallet B* would take precedence over that of *Wallet A*.
The unfortunate trade-off here is that: we end up with more activities and syncing becomes more expensive. However, by grouping by what might mutate, we end up with much better consistency/reliability.
Immutable data (e.g. something like a `XOTemplateAction`) can obviously have more complex Activity payloads as we would never modify "just a part" of that.
</details>
Like the LibAuth proposal, the aim is that state will become "eventually consistent". We do this by syncing these individual activities between devices and then each device compiles a "snapshot" of the final state.
Snapshots are created by checking the "policy" assigned to each type. For example, if we have multiple WalletName activities like so:
```ts
// NOTE: Timestamp, if not provided, is automatically generated.
await activities.addActivity<WalletName>({
type: 'WalletName',
// In practice, ID would be either a hash or UUID.
id: 'wallet1',
value: 'Old Wallet Name',
});
await activities.addActivity<WalletName>({
type: 'WalletName',
id: 'wallet1',
value: 'New Wallet Name',
});
```
... and we have a policy for the `WalletName` type such that the "latest timestamp takes precedence" (this is the default policy), the resulting snapshot (WalletState) will be compiled as following:
```ts
// Snapshot takes the form Type:EntityId:Value
// NOTE: We might want to modify this to have the "value" just contain the raw Activity type so that we have access to the timestamp too.
{
"WalletName": {
"wallet1": "New Wallet Name"
}
}
```
This approach should also allow decent extensibility: If a Wallet (or Dapp) does not support a given typeof activity, it will still be compiled in the snapshot, but just left unused.
### Optimizing the Activity Sync process
#### Optimizing UUID Set Comparison - Inverse Bloom Filters vs Naive Set Comparison
Because we are using UUIDV7, each activity will carry a universally unique ID. The naive approach to syncing wallets/servers with each other is just to perform a set comparison of the UUIDs:
1. Wallet A and GP WalletSync server are syncing with each other.
2. Wallet A sends WalletSync Server the set of Activity UUIDs it currently has.
3. WalletSync server responds to Wallet A with the activities it has that Wallet A **DOES NOT** have.
4. WalletSync server sends Wallet A the set of Activity UUIDs it currently has.
5. Wallet A responds to WalletSync Server with the activities it has that WalletSync Server **DOES NOT** have.
With many Activities (e.g. 100,000), this naive set comparison would become very bandwidth heavy. We can optimize this by attempting other strategies first and then falling back to a naive set comparison in the event that these other strategies fail. For example:
1. First, attempt to use an Inverse Bloom Filter to compare values in the set.
2. (Maybe) fallback to a Merkle Tree.
3. Otherwise, if the above are uneconomical (e.g. this is a new device syncing from scratch), just send across the full UUID set (the "naive" approach).
#### Using "msgpackr" to encode JSON-structures during Transport
[Msgpackr](https://www.npmjs.com/package/msgpackr) is a simple protocol for encoding JSON structures in a binary form. It is very efficient (comparable to native `JSON.stringify` and `JSON.parse`) and has the perk that it can encode Uint8Arrays as actual binary data (as opposed to a JSON string type representation (e.g. `someData: <Uint8Array: 0xbabecafe>`).
As we'll likely be storing a lot of binary data in our acitvities (e.g. data associated with P2SH contracts), for that kind of data, we reduce the bandwidth required by about 50%.
#### (Maybe) "Pruning" Activities
For Wallets, where the payload of an activity is readable, we can very likely prune no-longer-relevant data (i.e. data that is not included in the Snapshot/Final-State).
To give an example, imagine a `FreezeCoin` Activity like so where we may freeze/unfreeze the same coin many times.
```ts
await activities.addActivity<FreezeCoin>({
type: 'FreezeCoin',
// In practice, this would refer to an actual outpoint ID.
id: '0xOutpoint1',
value: true,
});
await activities.addActivity<FreezeCoin>({
type: 'FreezeCoin',
id: '0xOutpoint1',
value: false,
});
await activities.addActivity<FreezeCoin>({
type: 'FreezeCoin',
id: '0xOutpoint1',
value: true,
});
await activities.addActivity<FreezeCoin>({
type: 'FreezeCoin',
id: '0xOutpoint1',
value: false,
});
```
In practice, the only relevant value here is the last one (the one that will be used in our snapshot). The wallet does not need to **keep** the Activity Payload of the previous FreezeCoin activities. To save space (and bandwidth), by tombstoning the Activity Payload by setting it to `null` (indicating that a newer activity takes precedence over this activity).
However, because we intend to encrypt Activity Data on WalletSync servers, these servers themselves cannot "inspect" the Activity Data and apply policies (they must store ALL activities). So, we probably still want to keep the ActivityID (UUIDV7) locally to prevent a WalletSync server from continously re-sending the Activity payload itself. E.g. When the Set Comparison happens between a Wallet and WalletSync server, the WalletSync Server will be able to see that the Wallet already has that particular ActivityID and will not attempt to resend the payload.
This might also yield other performance improvements when compiling snapshots in the sense that we can filter out all `null` payloads from consideration:
```ts
// Pseudo-code for clarity.
const relevantActivitiesToBuildSnapshotWith = activities.filter(activity => activity !== null);
```
#### (Maybe) Using enums to represent "type"
The Type field is currently a string. This is to enhance readability (particularly during Development). But we might want to consider using an enum-like scheme and allocating windows to specific wallets/services. E.g.
```
STANDARD: 0 - 1000
XO: 10_000 - 10_999
SELENE: 11_000 - 11_999
etc
```
Where:
```
WALLET_NAME = 1;
WALLET_DESCRIPTION = 2;
etc
XO_WALLET_COLOR = 10_000;
etc
```
This would save bandwidth and storage requirements, but might be overkill.
**TODO: This will almost certainly need to be done for OP_RETURN storage. But, we might want the OP_RETURN adapter to use an on-the-fly map (String <-> Number) so that we can keep plaintext "types" in our backup files. This means that the OP_RETURN adapter will only support a subset of Activity Types - which might actually be desirable to prevent abuse of OP_RETURN storage of decorative Wallet Metadata?**
#### (Maybe) Other structures like CRDTs
TODO: Looked into this, but it's probably more complex than we need - and might not be more optimal in the long-run.
### Alternative Storage Mechanisms
#### (Maybe) OP_RETURN data carriers
As opposed to WalletSync Servers, there is the possibility that we can store Wallet Metadata on-chain itself in an encrypted form using OP_RETURN (or NFT Commitments).
In this case, we probably only want to store essential data (e.g. the data parameters of P2SH contracts) as opposed to decorative data (Tx Memo, Tx Category, Wallet Name, Wallet Description, Received Timestamp, etc).
Approach:
1. Can store about ~195 bytes (223 - 28 bytes AES-GCM overhead) using AES-GCM with recommended security params (but could split data across several txs if need be)
2. Use a HDWallet for derivation of addresses that store this data.
Perks:
1. Does not require WalletSync Servers (Wallet Data/State is synced from chain)
2. Removes the "peer discovery" problem (wallet only has to have blockchain connection to find its data)
3. Natural DoS protection: The tx's to store on-chain costs sats.
- TODO: Compare with concern of using WalletSync servers: How do we defend these against spam?
- JN suggestion: WalletSync server requires an on-chain tx to "authenticate" a wallet before our WalletSync server will allow it to store data.
Negatives:
1. We turn the chain into a data-carrier.
2. This might become expensive (particularly with other metadata like tx memos, categories, etc).
3. Might be slower than WalletSync
4. Data would be public (albeit encrypted)
Other considerations:
1. Ideally we want to avoid using sequence numbers for data spread across multiple txs.
- We can probably follow a tx chain instead which means we eliminate the risk caused by accidentally using the same derivation path twice (we wouldn't know which sequence number corresponds to which activity if this happened).
2. NFT commitments are another possibility.
- We're capped to 40 bytes for commitments, but we could spread the payload across multiple commitment outputs (first 28 bytes would be AES-GCM header).
- This might be more expensive in practice (if we ever need to unlock these... Tx V5 detached signatures might mitigate, or even a custom P2SH unlock that references a sig at a hard-coded input?)
3. To save space, action data params would have to be ordered (we want to be able to omit the field name).
4. Action data params might not have fixed length (e.g. a payout address might be P2PKH, P2SH20 or P2SH32), so we'd probably need to length-prefix each data variable.
5. Outpoints would probably be 33 bytes each (TxHash:IndexID) - might be a lot of data for more complex actions?
6. **Technically, the storage approach I have above probably won't work as-is: Though Activities are "append only", they fundamentally use mutable Object Stores indexed by a key (UUIDV7). OP_RETURN stores would be immutable.**
7. Need to be cautious about circular dependencies: An `OpReturnStore` would have to leverage `Blockchain` - and `Blockchain` leverages `BaseStore`. That design could be a problem.
### Peer-finding for Device-to-Device Sync
TODO: LibP2P overview
- Peers are found via DHT lookup (but can first priotize a HTTP-lookup first for speed)
- Can create a network that preferences BCH nodes.
- Can (probably) use BCH as a DoS-prevention mechanic.
## Templates
<div class="columns">
<div class="column">
**Libauth (V2)**
- Lock/Unlock Centric
- Intra-Wallet support
</div>
<div class="column">
**Tentative XO Approach**
- Action-centric (Transaction Shapes)
- CashASM placeholders allowed in Transaction Shape fields
- Each Transaction Shape can use Locking/Unlocking Scripts defined in the template
- Aiming for Intra-Wallet support + WebDapp support
</div>
</div>
This is a big topic and very much in-flux. To try to give a brief gist, the idea is that templates have actions that can be executed - and these Actions contain placeholders and support CashASM evaluations.
Example of interface for an Action (still WIP):
```ts
interface Action {
// Human-readble name
name: string,
// Human-readable description
description: string,
// A list of transaction shapes that this action will execute sequentially.
// NOTE: We need the capability to execute multiple transactions in an action so that:
// 1. We can create minting transactions required for token genesis.
// 2. We can create "consolidation transactions" so that subsequent shapes that impose constraints on input counts, etc, will not have additional UTXOs appended.
// 3. Some contracts will simply require multi-tx flows for a given action.
transactions: Array<TransactionShape>,
// A list of "data" that should be returned to the Dapp.
// This represents the "state" that the Dapp (or Wallet) MUST store.
// The format is name (or alias) to a CashASM evaluation. Using a CashASM over raw variable names gives us flexibility in that:
// 1. We can conceal values derived by the wallet from the consuming service.
// 2. In future, format values for direct consumption by a follow-up action (e.g. imagine a generic settlement service)
// 3. In future, "transactions" may be made optional so that that an action can be used to just collect data (e.g. collect pubKeys for multisig setup or collect pubkeys/payout address for AnyHedge P2P Setup). TODO: Go into more detail about this.
resolves: { [name: string]: CashASM }
}
```
Actions are then invoked something like:
```ts
const createOfferResult = await wallet.executeAction(
BCHGURU_TEMPLATE,
{
// The action from the template to invoke.
action: 'bch_createOffer',
// The data to pass into this action.
// NOTE: In practice, these will need to be converted to Uint8Array. An 'encodeData' util function will be provided to help.
data: {
oraclePublicKey: '0x02d09db08af1ff4e8453919cc866a4be427d7bfe18f2c05e5444c196fcf6fd2818',
player1Prediction: 100_000_000,
betAmount: 250_000,
// ...
}
}
);
```
The templates themselves are quite similar to LibAuth. It might be better to link to this document as many of the concepts/ideas will likely carry across to XO.
Example of a Template Action: https://gitlab.com/developers.cash/cashconnect-temp#actions (note there's a link to full examples near the bottom - if you get time, would love feedback on the other parts... there's still some unsolved problems I have, but I think the general direction might be feasible.)
A potential mechanism for chaining these actions (this is very rough, but might help inspire something better): https://gitlab.com/developers.cash/cashconnect-temp/-/blob/main/Action-Chaining.md
### Template Sandboxes
In the [CashConnect proposal](https://gitlab.com/developers.cash/cashconnect-temp#sandboxing-and-recoverability), a sandbox key is assigned per domain. This is derived from the user's root seed in the form:
```ts
const domainSandboxSeed = sha256(sha256(rootSeed) + sha256(domain));
```
For CashConnect, the intent of this was to:
1. Allow recovery from root-seed only (this is not perfect, but if a user re-visits a domain, they will be able to recover their key associated with that domain).
- *The assumption of CashConnect was that the Wallet did not have the capability to store any P2SH data (this responsibility is relegated to the Dapp/Service).*
2. Give the domain a permissable sandbox account ("key") it could use for construction of actions
- Create transactions that consolidate amounts for follow-up transactions
- Create Token Genesis transactions and
- Authenticate with services using a unique sandboxed id.
<details>
<summary>More details on the Sandbox Account</summary>
In CashConnect, this is just the "default" key that is provided to the Dapp.
```ts
const providedData: CompilationDataBCH = {
bytecode: {
// Data required by this action.
... dataProvidedToTemplateAction,
// This is a change address provided by the wallet (used by the tx builder to auto-append change outputs).
changeLockingBytecode,
},
keys: {
privateKeys: {
sandboxKey: theDerivedKeyForThisDappOrTemplate
},
},
};
```
It is used for signing requests (e.g. authenticating with the service) and also as a means to allow creating Token Genesis and Amount Consolidation Transactions (needed to conform to some tx shapes).
For example, a template might include the following scripts:
```ts
{
//... other template fields
scripts: {
// Sandbox-specific scripts
SandboxP2PKH: {
script: 'OP_DUP OP_HASH160 <$(<wallet.public_key> OP_HASH160)> OP_EQUALVERIFY OP_CHECKSIG',
lockingType: 'p2pkh',
},
SandboxP2PKH_unlock: {
script: '<wallet.schnorr_signature.all_outputs_all_utxos> <wallet.public_key>',
unlocks: 'SandboxP2PKH'
}
}
}
```
Using BCHouse as an example (which requires a Token Genesis), an action to create a campaign might then look as follows:
```ts
{
actions: {
createCampaign: {
name: '...',
description: '...',
transactions: [
// First transaction is to create a Token Genesis Output so that we can mint an NFT.
{
// Have Tx Builder automatically append inputs from default wallet to meet required sats/token amounts.
inputs: ['*'],
outputs: [
{
lockingBytecode: {
script: '#SandboxP2PKH',
},
valueSatoshis: 1000n,
},
// Have TX Builder automatically append any change outputs
'*'
],
},
// Second transaction actually creates the campaign (using the outpoint from above as input).
{
// ...
inputs: [
{
unlockingBytecode: {
script: '#SandboxP2PKH_unlock',
},
sequenceNumber: 0,
outpointTransactionHash: '$(<tx0_hash>)',
outpointIndex: 0,
},
// Have Tx Builder automatically append additional inputs from default wallet to meet required sats/token amounts.
'*'
],
outputs: [
{
lockingBytecode: {
script: '#CampaignMain',
},
valueSatoshis: 1000n,
token: {
category: '$(<tx0_hash>)',
amount: 0n,
nft: {
capability: 'minting',
commitment: '$(<0x0000000000000000>)',
},
},
},
// Have TX Builder automatically append any change outputs
'*'
],
}
]
}
}
}
```
The other main use-case is Amount Consolidation where a contract may require that only a single input is provided. In that case, we can first shuffle the required funds into a P2PKH Sandbox Account and then use that as the outpoint in the follow-up transaction.
</details>
For intra-wallet (non WebDapp cases), we might want to do similar for an XO Connect, but instead of using the domain, we use the template hash (and probably template-instantiation data to ensure each instance gets its own sandbox?).
>we could end up with overlapping derivation indexes used across devices.
The above problem would still exist, but we would:
1. at worst, confine it to the given template
2. avoid the Schnorr/ECDSA re-use problem (assuming template was audited to not mix Schnorr/ECDSA):
For Web-based Dapps, domain-based derivation is something I keep tossing up on. If we guarantee the storage/recoverability of state via OP_RETURN as a data-carrier, we can perhaps use something arbitrary/random - but I'm not really sure I see the benefit of this as we would likely need to map this ID to a domain anyway.
#
<style>
.columns {
display: flex;
}
.column {
flex: 1;
}
</style>
-6
View File
@@ -109,12 +109,6 @@ export abstract class BaseWallet<
/**
* Get the address that this wallet should receive to.
*
* @remarks
*
* Not all wallets will have an address (e.g. a HD Wallet), but should still be able to return an address that:
* a) Change will be sent to.
* b) Funds should be received to.
*/
abstract getReceivingAddress(): Promise<Address>;
+306 -3
View File
@@ -1,3 +1,33 @@
import { BaseWallet } from './base-wallet.js';
import { type TransactionTemplate } from './types.js';
import { TransactionBuilder } from './transaction-builder.js';
import { type WalletServices } from './types.js';
import {
WalletP2PKH,
WalletP2PKHFactory,
useWalletP2PKH,
} from './wallet-p2pkh.js';
import {
type BaseBlockchain,
type BlockchainTransactions,
type BlockchainUTXOs,
} from '../blockchain';
import {
type BaseStore,
type StoreValueProxy,
StoreInMemory,
} from '../storage';
import {
type TokenBalances,
HDPrivateNode,
ExtMap,
calculateBalanceSats,
calculateBalanceTokens,
encodeExtendedJson,
} from '@xocash/primitives';
/**
* Placeholder.
*
@@ -7,8 +37,281 @@
* See remarks on Storage - it's semi-related and I'm still playing with approaches.
*/
// https://github.com/bitjson/libauth/blob/d3d610de088cd98f5c391fe62dce81874eb5d95a/src/lib/engine/types/wallet-set-types.ts
export class WalletSet {
constructor() {
console.log('TODO');
/*
export type WalletSetGenesisData = {
type: 'WalletSet';
} & ({ seed: string } | { mnemonic: string } | { xpriv: string });
export type WalletSetEvents = {
shouldMonitorUpdated: boolean;
stateUpdated: string | null;
};
export class WalletSet extends BaseWallet<WalletSetEvents> {
// Mutable State.
public shouldMonitor = false;
// Store values that are proxied for convenience/clarity.
// NOTE: We define these for clarity so that we can see the state stored with this wallet.
public genesisData: StoreValueProxy<WalletSetGenesisData>;
public wallets: { [id: string]: BaseWallet<any> } = {};
constructor(
protected adapters: WalletAdapters,
protected genesisData: WalletSetGenesisData,
) {
super();
// Initialize proxies to our stored data.
this.genesisData = this.store.proxy('genesis');
}
static async from<T extends typeof WalletSet>(
this: T,
services: WalletServices,
genesisData?: WalletSetGenesisData,
) {
// Assign in memory stores as defaults in case otherwise not provided.
const servicesFinal = {
...{
store: new StoreInMemory(),
cache: new StoreInMemory(),
},
...services,
};
// If we are attempting to create a new wallet...
if (genesisData) {
// Ensure that the provided store does not already contain data.
if (Object.keys(await servicesFinal.store.all()).length) {
throw new Error(`Provided store already contains data`);
}
// Store the wallet genesis data.
await servicesFinal.store.set<WalletSetGenesisData>(
'genesis',
genesisData,
);
}
// Retrieve our genesis data and extract the private key.
const retrievedGenesisData =
await servicesFinal.store.getOrFail<WalletSetGenesisData>('genesis');
let hdPrivateNode: HDPrivateNode;
// Instantiate the HDPrivateNode from the type specified in metadata.
if ('seed' in retrievedGenesisData) {
hdPrivateNode = HDPrivateNode.fromSeed(
hexToBin(retrievedGenesisData.seed),
);
} else if ('mnemonic' in retrievedGenesisData) {
hdPrivateNode = HDPrivateNode.fromMnemonic(retrievedGenesisData.mnemonic);
} else {
hdPrivateNode = HDPrivateNode.fromXPriv(retrievedGenesisData.xpriv);
}
// Return a new instance of WalletHD.
return new this(
servicesFinal.blockchain,
servicesFinal.store,
servicesFinal.cache,
hdPrivateNode,
walletFactory,
) as InstanceType<T>;
}
async start() {
// Get the genesis data.
const genesisData = await this.genesisData.getOrFail();
// Declare storage for our start promises.
const startPromises: Promise<void>[] = [];
// Wait for all of our start promises to complete.
await Promise.all(startPromises);
}
async stop() {
// Declare storage for our stop promises.
const stopPromises: Promise<void>[] = [];
// Wait for all of our stop promises to complete.
await Promise.all(stopPromises);
}
async destroy() {
this.removeAllListeners();
}
async getTransactions() {
// Declare storage for our fetch promises.
const fetchPromises = [];
// Wait for all of our fetch promises to complete.
const txs = await Promise.all(fetchPromises);
// Flatten (and deduplicate) the transactions.
const txsFlattened = ExtMap.flatten(txs);
// Emit an event to notify that transactions have been updated.
this.emit('transactionsUpdated', txsFlattened);
// Return the deduplicated transactions.
return txsFlattened;
}
async getUnspents() {
// Declare storage for our fetch promises.
const fetchPromises = [];
// Iterate over each chain path and fetch unspents for each wallet.
for (const chainWallets of Object.values(this.wallets)) {
for (const wallet of chainWallets) {
fetchPromises.push(wallet.getUnspents());
}
}
// Wait for all of our fetch promises to complete.
const unspents = await Promise.all(fetchPromises);
// Flatten them.
const unspentsFlattened = ExtMap.flatten(unspents);
// Return the unspents.
return unspentsFlattened;
}
async getUnspentDirectives() {
// Declare storage for our fetch promises.
const fetchPromises = [];
// Iterate over each chain path and fetch unspent directives for each wallet.
for (const chainWallets of Object.values(this.wallets)) {
for (const wallet of chainWallets) {
fetchPromises.push(wallet.getUnspentDirectives());
}
}
// Wait for all of our fetch promises to complete.
const unspentDirectives = await Promise.all(fetchPromises);
// Flatten them.
const unspentsFlattened = unspentDirectives.flat();
// Return the unspents.
return unspentsFlattened;
}
async getBalanceSats() {
// Fetch our unspents.
const unspents = await this.getUnspents();
// Get the individual UTXOs.
const utxos = unspents
.map((blockchainUTXO) => blockchainUTXO.utxo)
.toArray();
// Calculate the balance.
return calculateBalanceSats(utxos);
}
async getBalanceTokens(): Promise<TokenBalances> {
// Fetch our unspents.
const unspents = await this.getUnspents();
// Get the individual UTXOs.
const utxos = unspents
.map((blockchainUTXO) => blockchainUTXO.utxo)
.toArray();
// Calculate the token balances.
return calculateBalanceTokens(utxos);
}
async getAddresses() {
const chainPaths = [CHAIN_EXTERNAL, CHAIN_INTERNAL];
let addressPromises = [];
for (const chainPath of chainPaths) {
addressPromises.push(
...this.wallets[chainPath].map((wallet) =>
wallet.getReceivingAddress(),
),
);
}
const addresses = await Promise.all(addressPromises);
return ExtMap.fromArray(addresses, (address) => address.toLockscriptHex());
}
async getReceivingAddress(chainPath = CHAIN_EXTERNAL) {
// TODO: Derive a new wallet, not the first one you lazy fuckhead.
// You'll probably need to refactor your scan() approach.
let receivingWallet = this.wallets[chainPath][0];
// If there is no receiving wallet, create one.
// TODO: This is fucked. You need to rethink deriveWallets.
if (!receivingWallet) {
this.deriveWallets(1, chainPath);
receivingWallet = this.wallets[chainPath][0];
}
// Return the receiving address of the derived wallet.
return receivingWallet.privateKey.derivePublicKey().deriveAddress();
}
async signTransaction(
txTemplate: Partial<TransactionTemplate>,
): Promise<Uint8Array> {
// Populate the final transaction template by merging with defaults.
// NOTE: This is just a convenience to make "send" easier to use.
const txTemplateFinal: TransactionTemplate = {
...{
version: 2,
locktime: 0,
inputs: [],
outputs: [],
},
...txTemplate,
};
// Get the receiving address of this wallet.
// NOTE: As per BIP44 spec, we set the change address to use the internal chain.
const changeAddress = await this.getReceivingAddress(CHAIN_EXTERNAL);
// Get the change locking bytecode (change address) for this wallet.
const changeLockingBytecode = changeAddress.toLockscriptBytes();
// Fetch unspents that are usable by this wallet.
const unspents = await this.getUnspentDirectives();
// Create an instance of our transaction builder and populate with our Wallet's UTXOs.
// NOTE: We must deal with tokens first and satoshis last.
// The reason is that the change output will be determined based on transaction fee.
const txBuilder = new TransactionBuilder(txTemplateFinal, unspents)
.addTokensFromUnspents()
.addTokenChangeOutputs(changeLockingBytecode)
.addSatoshisFromUnspents()
.addSatoshiChangeOutput(changeLockingBytecode);
// Generate the transaction.
const generatedTxBytes = txBuilder.toBytes();
return generatedTxBytes;
}
async sendTransaction(txTemplate: Partial<TransactionTemplate>) {
// Sign the transaction template.
const generatedTxBytes = await this.signTransaction(txTemplate);
// Broadcast the transaction to the blockchain.
await this.blockchain.broadcastTransaction(generatedTxBytes);
}
}
*/