To make off-chain data integration secure and more efficient, Substrate provides off-chain workers. The off-chain worker subsystem allows execution of long-running and possibly non-deterministic tasks (e.g. web requests, encryption/decryption and signing of data, random number generation, CPU-intensive computations, enumeration/aggregation of on-chain data, etc.) which could otherwise require longer than the block execution time.
The code for the off-chain workers is stored on-chain, and has access to the on-chain environment, but is never executed as part of block-processing. Off-chain workers make it easy to run the correct code and allow longer running tasks to be performed without holding up the blockchain. But, the on-chain code allows verification mechanisms, such as voting, averaging, and challenging to be implemented in the state-transition function of the blockchain it is running on. It is up to the blockchain/oracle network designer and those running nodes on the network to agree on exactly who should be doing what, when, and the rewards and punishments for participants.
Want to see a walkthrough of the code? Check out this callback example.
If you want traditional oracles to be run by multiple nodes and authorities on your network, this is very difficult to achieve, especially if you want to update them to new versions. Off-chain workers solve this issue by publishing this piece of infrastructure across the network, allowing nodes to be aware when they are out of date. This makes running oracle networks much easier to deploy and maintain over time as these networks grow in size, complexity and sophistication.
Because they are a part of the runtime code, instead of running a separate process, they are run by Substrate itself in the same way as the state transition function is run on every block import - they are either executed in the sandboxed WebAssembly (Wasm) environment, or if the on-chain code version matches the node implementation, they benefit from performance improvements running natively.
Off-Chain Worker APIs
Since off-chain workers are running in a sandboxed environment, they can only access a limited set of APIs for security reasons. They do have full access to any on-chain state and have additional capabilities to communicate with the external world via extended set of APIs.
Local Storage. An additional, local key-value database shared between all off-chain workers.
HTTP requests API. A fully-featured HTTP client allowing to access and fetch data from external services.
A secure, local entropy source for random number generation.
Access to local node precise time and ability to sleep/resume work.
Access to the local keystore to sign and verify statements or transactions.
Ability to submit transactions (either signed or unsigned) back on-chain to publish computation results.
Note: Some of these APIs are still under development. More information and usage documentation will be published as and when the APIs are available for use.
Get started on playing with Substrate off-chain workers by going to the GitHub repo, learn by example with this walkthrough of a callback, or talking to the developers in the Substrate technical channel.