base material
This commit is contained in:
emergent-reasons
@@ -0,0 +1,320 @@
|
||||
---
|
||||
title: 'Vision to Reality 4: Revving the Engine'
|
||||
date: 2026-08-05
|
||||
description: This article covers recent improvements to the XO stack and draws up a short roadmap.
|
||||
author: Jonathan Silverblood
|
||||
tags: [templates, engine, technical audience]
|
||||
---
|
||||
|
||||
# Vision to Reality 4: Revving the Engine
|
||||
|
||||
This article draws up a short roadmap to set expectations for the upcoming BCH Blaze hackathon in November 2026 and covers recent improvements to the XO stack.
|
||||
|
||||
----
|
||||
|
||||
```mermaid
|
||||
mindmap
|
||||
root )Responsibilities(
|
||||
|
||||
[INVITATIONS]
|
||||
(Connects)
|
||||
📱 Apps
|
||||
👛 Wallets
|
||||
👥 Users
|
||||
(Coordinates)
|
||||
⚡ Actions
|
||||
🧩 Variables
|
||||
📥 Inputs
|
||||
📤 Outputs
|
||||
|
||||
[TEMPLATES]
|
||||
(Describes)
|
||||
🧩 Variables
|
||||
💸 Transactions
|
||||
📥 Inputs
|
||||
📤 Outputs
|
||||
🏷️ Lockscripts
|
||||
(Defines)
|
||||
⚡ Actions
|
||||
💸 Transactions
|
||||
💾 Data
|
||||
|
||||
[ENGINE]
|
||||
(Generates)
|
||||
🔑 Keys
|
||||
🪪 Identities
|
||||
✉️ Invitations
|
||||
💸 Transactions
|
||||
(Manages)
|
||||
🤫 Secrets
|
||||
🧩 Variables
|
||||
📤 Outputs
|
||||
🏷️ Lockscripts
|
||||
```
|
||||
|
||||
----
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Tools and tutorials
|
||||
|
||||
In preparation for the Blaze hackathon we are planning to release additional articles and tooling to help participants build on XO.
|
||||
|
||||
- Publish article on high-level XO concepts
|
||||
- Publish article on best practices when making templates
|
||||
- Publish article on current tools and limitations
|
||||
- Release alpha version of a command-line or text interface wallet
|
||||
- Release alpha version of graphical XO interactive debug tool
|
||||
|
||||
### Token Support
|
||||
|
||||
Currently, the XO engine is capable of working with tokens only in a very limited capacity. For example, while it is possible to define token balance estimations in templates, they are currently not being used.
|
||||
|
||||
Full token support is a target goal and is one of the key area for improvement leading up to the hackathon, with explicit support for token genesis, token minting, coin selection, balance estimation and much more.
|
||||
|
||||
|
||||
### Improved naming
|
||||
|
||||
In the templates today, the words `locking script` and `locking bytecode` are mixed and the result is vagueness and inconsistency.
|
||||
|
||||
All uses that directly refer to actual underlying bytecode will be renamed to either `lockingBytecode` or `unlockingBytecode`. Similarly all references to `lockingScript` or `unlockingScript` will refer to named and documented entities that uses `lockingBytecode` or `unlockingBytecode` respectively in their definitions.
|
||||
|
||||
As a result, we will also introduce an `unlockingScripts` section similar to the `lockingScripts`. Here unlocking scripts can be described properly with names, descriptions and other metadata.
|
||||
|
||||
|
||||
### Default outputs instead of inheritance
|
||||
|
||||
One of the many goals of the XO stack is to ensure that no funds are lost. To this end, we previously added data to locking scripts that allowed the engine to classify and handle unknown transactions, when those transactions have outputs to monitored lockingscripts in the engine.
|
||||
|
||||
Previously, any output to a known locking script that lacked classification by the engine, would inherit properties of the locking script and become an implicit default output.
|
||||
|
||||
Going forward, we are making this explicit by adding a reference to an explicit default output in lockingscripts and moving all related functionality to this definition.
|
||||
|
||||
This has two big impacts. First, it makes it more explicit that a template is responsible for handling default outputs on locking scripts. Second, it encourages template authors to provide explicit names and descriptions for the default locking script and for any uncoordinated outputs that spends to it.
|
||||
|
||||
### Events
|
||||
|
||||
Currently users of the XO stack have to poll for data when they need it. Going forward, we will implement proper event emission so that your application can know when new data is available. This will include all blockchain triggered changes, but also information on when the engine starts and completes state synchronization, when wallet balance estimations are updated and much more.
|
||||
|
||||
|
||||
### Improved documentation
|
||||
|
||||
While we are still doing early alpha releases and breaking changes are still to be expected, we will make a concentrated effort to improve the technical package documentation ahead of the BCH Blaze hackathon.
|
||||
|
||||
This includes tutorials on how to get started, better structure on the available documentation and more practical examples.
|
||||
|
||||
|
||||
## Recent changes
|
||||
|
||||
A lot has happened since our last article and we've been hard at work making template, engine and tooling updates, as well as presenting at BCH Bliss.
|
||||
|
||||
Here is a short summary of what has improved:
|
||||
|
||||
- new BIP43 compliant derivation scheme
|
||||
- improved reservation handling
|
||||
- added expiration date on invitations and resources
|
||||
- introduce intents as a concept
|
||||
- cleared up and isolated state better
|
||||
- dynamic template roles instances
|
||||
- added simple automated coin selection
|
||||
- improved change handling with policies
|
||||
- implemented a large number of new primitive data types
|
||||
- various quality-of-life and developer experience improvements
|
||||
|
||||
|
||||
### Derivation
|
||||
|
||||
Under the hood, a lot has changed with regards to how XO derives cryptographic keys. Previously, we used a scheme that was a variation of BIP44 but that was **not** BIP44 compliant.
|
||||
|
||||
The new derivation paths are shorter and simpler and are instead based on BIP43:
|
||||
|
||||
> m/XO'/ACCOUNT_OR_TEMPLATE'/USE_CASE'/INDEX
|
||||
|
||||
Where `XO = 2415` and each template has their own hardened keyspace. Keys are further separated within each keyspace by a hardened `USE_CASE`.
|
||||
|
||||
This setup isolates keys that a template author might be able to influence from keys created by other templates, or used by the engine in various ways.
|
||||
|
||||
For account-wide usage, the `ACCOUNT_OR_TEMPLATE` is set to 0, allowing for a non-templated keyspace, which also has hardened `USE_CASE` isolation.
|
||||
|
||||
|
||||
### Reservations
|
||||
|
||||
When preparing to take action, the engine reserves resources that will be used by the action. This part of the code has been improved with better error reporting, more clear naming and it is now possible for applications to reserve resources manually.
|
||||
|
||||
Technically, the reservation has changed from being a boolean, to being `reservedBy` with a reference. Requesting a reservation for an already reserved resource returns an error.
|
||||
|
||||
|
||||
### Expirations
|
||||
|
||||
Participation in invitations now come with expiration dates, so that reserved resources can be restored when actions are not completed within expected timeframes.
|
||||
|
||||
Note that the expiration dates are indicative rather than enforced, and serve only to prevent permanent reservation of resources. If an action is attempted after an expiration has happened and the resources are still available, the action will still succeed.
|
||||
|
||||
|
||||
### Intents
|
||||
|
||||
We have added a concept of reuseable intents which defines the preconditions to creating resources in the engine.
|
||||
|
||||
This allowed some work to be specified upfront with the new `generate:` keyword, rather than being inferred by the engine after a requirement has been encountered.
|
||||
|
||||
For example, here is an intent to create a P2PKH change output:
|
||||
```ts
|
||||
{
|
||||
output: 'changeOutput',
|
||||
role: 'receiver',
|
||||
generate: [ 'ownerKey' ],
|
||||
}
|
||||
```
|
||||
|
||||
Further, this reconciles all differences between starting actions and follow-up actions, and serves as the framework for creating various structures such as locking scripts, outputs and other data.
|
||||
|
||||
|
||||
### State
|
||||
|
||||
We have moved all state representations to only exist in outputs or locking scripts, rather than being defined in requirements in some places.
|
||||
|
||||
Note that we are further planning to move the inherited output handling for uncoordinated outputs from the locking scripts into a referenced `defaultOutput`, which will fully move **all** templated state representation to exist only in outputs (as originally intended).
|
||||
|
||||
|
||||
### Slots
|
||||
|
||||
Templates can now specify a minimum and maximum number of instances each role requires via the `slots:` keyword.
|
||||
|
||||
This removes the need for fixed roles and allows for arbitrary and fungible role takers, such as pledgers in fundraising campaigns or signers in multisig wallets.
|
||||
|
||||
|
||||
### Coin Selection
|
||||
|
||||
Before automated coin selection, applications had to provide their inputs manually and the engine was only able to unlock inputs from the same template.
|
||||
|
||||
With the initial implementation of naive coin selection, the engine can now pick coins that are selectable from any template to satisfy requirements for satoshis when performing an action.
|
||||
|
||||
|
||||
### Change
|
||||
|
||||
In previous versions of the engine, change was not handled automatically and instead implementations had to provide the change output by hand.
|
||||
|
||||
Note that change is a difficult concept to get right, as there are usecases that have conflicting requirements. It is easy to break user expectation when doing naive coin selection:
|
||||
|
||||
- Spending from a multisig template and generating a singlesig change.
|
||||
- Spending from a privacy focused template and generating a non-private change.
|
||||
|
||||
The long-term solution for this is introduction of a change policy which controls how the engine automates creation of change outputs.
|
||||
|
||||
As a result, templates can now define a change policy if they have specific needs:
|
||||
|
||||
```ts
|
||||
// Sets optional default values to be used with this template.
|
||||
defaults:
|
||||
{
|
||||
change:
|
||||
{
|
||||
output: 'changeOutput',
|
||||
role: 'receiver',
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
The engine follows a strict order when choosing what change policy to apply, allowing for fine-grained control by the application:
|
||||
|
||||
- `override` policy if manually provided by the application.
|
||||
- `default` policy in the template linked to the spent input.
|
||||
- `fallback` policy previously configured by the application.
|
||||
|
||||
*If no change policy is available, the engine will refuse to take action when change is required.*
|
||||
|
||||
With this policy framework in place, a privacy focused wallet can guarantee that change will never go to a non-private output, either by setting an `override` policy, or by only allowing templates with acceptable privacy.
|
||||
|
||||
At the same time, if a wallet does not have strict change requirements they can set a `fallback` policy instead, and will then use `default` policies from the templates they are spending from when available.
|
||||
|
||||
Currently, the engine only supports the template `default` and engine `fallback` policies, the manual `override` policy will be supported soon.
|
||||
|
||||
|
||||
|
||||
### Primitives
|
||||
|
||||
A number of new primitives have been added, and all primitives are validated on construction, including new run-time validation.
|
||||
|
||||
|
||||
**New primitive: Satoshis**
|
||||
|
||||
In various places in our code we used to have **bigints** for satoshi values, but this has been changed in favour of a dedicated `satoshis` primitive, which additionally also validates that the amount falls in reasonable ranges representable by the **CashVM**.
|
||||
|
||||
|
||||
**New primitive: Transaction Hash**
|
||||
|
||||
We used to represent transaction hashes as strings, but this has been changed in favour of a dedicated `transaction hash` primitive.
|
||||
|
||||
This change makes it less likely that another string is used in a place where we should be using a transaction hash and the primitive further validates the length and format of transaction hashes.
|
||||
|
||||
|
||||
**New primitive: Extended Public Keys**
|
||||
|
||||
Previously we used to have a single public key for each participant in an invitation, which resulted in key re-use and potential for adversaries to control what data to sign with a key, risking cryptographic information leak.
|
||||
|
||||
By adding an `extended public key` primitive and using that in our invitations, each update to an invitation now signs with a unique key. This reduces possible attack surface by removing key re-use in invitations.
|
||||
|
||||
|
||||
**New primitive: Timestamps**
|
||||
|
||||
Representing time in script has long been a solved problem and well defined by the **CashVM**, but representing time to end-users in templates required better tooling on the engine side.
|
||||
|
||||
By adding an `timestamp` primitive we can extend support for end-user presentation of time, starting with the `iso8601` standard formatting.
|
||||
|
||||
```ts
|
||||
// Example CashASM demonstrating timestamp usage.
|
||||
description: 'Last seen: <myTimestamp.iso8601>',
|
||||
```
|
||||
|
||||
|
||||
## Other improvements
|
||||
|
||||
### Passing Identifiers
|
||||
|
||||
Previously it was required that developers pass full objects for things like `invitations` every time they needed to interact with them. This has been changed so that only identifiers are needed for items that are cached by default.
|
||||
|
||||
```ts
|
||||
const { invitationIdentifier } = await engine.createInvitation();
|
||||
|
||||
// Later, when signing the invitation
|
||||
await engine.signInvitiation(invitationIdentifier);
|
||||
```
|
||||
|
||||
### Isolated database per-seed.
|
||||
|
||||
To avoid conflicts when using the same application loaded with different seeds, we have added database isolation on a per-seed basis.
|
||||
|
||||
This also makes it possible to run multiple instances of the engine with different seeds, allowing multiple accounts within the same application.
|
||||
|
||||
|
||||
### Improved invitation handling
|
||||
|
||||
Invitations can now be easily shared with import and export functionality, creating a flexible starting point for building transport protocols and standards for sending invitations between different wallets.
|
||||
|
||||
To prevent possible leakage of cryptographic entropy, each step of an invitation is now signed by a unique key, rather than a shared key per-invitation.
|
||||
|
||||
After executing an action from an invitation, the resulting `data` or `transaction` data structures are now returned to the caller.
|
||||
|
||||
|
||||
### Template validation
|
||||
|
||||
We have improved validation prior to importing templates with both syntax and range checks, and ensured that all errors are properly reported with actionable error messages.
|
||||
|
||||
|
||||
### Applicaton state
|
||||
|
||||
In additional to templated state, we have added support for engine and application state.
|
||||
|
||||
This is used by the engine to remember things like how to create default receiving addresses, and can be used by applications to store things like wallet settings.
|
||||
|
||||
|
||||
### Rigorous testing
|
||||
|
||||
Many parts of the XO stack have significantly improved their automated testing for additional quality assurance.
|
||||
|
||||
|
||||
### P2PKH
|
||||
|
||||
P2PKH is the current industry standard for wallets. We have continued to improve the P2PKH template with better documentation, more human language and more explicit handling.
|
||||
|
||||
Reference in New Issue
Block a user