Patract Hub (https://patract.io) develops local open source toolkits and one-stop cloud smart IDE, committed to provide free development toolkits and infrastructure services for the entire smart contract ecosystem. 8 weeks ago, we applied a treasury proposal #15 for Europa v0.1 , and now we have finished the development (https://github.com/patractlabs/europa) and recorded a [YouTube demo] (https://www.youtube.com/watch?v=z1SWq0vfgYY). We are delayed behind the required schedule for 2 weeks and we are very sorry about that.
Summary of Europa's future plan:
v0.1: Have an independent runtime environment to facilitate more subsequent expansion directions.
The independent runtime environment of excluded nodes can be expanded more without the constraints of the node environment and WASM compilation, and can be easily integrated with other components. In this version, it is more like simulating the Ganache project in Ethereum ecosystem, enabling contract developers to develop without having to build a contract blockchain. Developers can quickly fire up a personal Substrate chain, which can be used to run tests, execute commands, and inspect state while controlling how the chain operates.
- Build and run in an independent executable file
- Can start from different workspaces, and a new workspace is an empty chain
- Can receive extrinsics to set_code, deploy and call contracts directly
- Can interact with Redspot
- Can use RPC to jump over some blocks and revert to a specified height to revert states
Europa is kind of another implementation for Substrate client in our design. We know that the runtime of a blockchain is the business logic that defines its behavior, and the Substrate runtime need to run by an executor and environment. So that we design the executor and environment more like a "sandbox" to run a Substrate runtime.
In v0.1, the primary target is to establish this "sandbox" environment to run a runtime. Then only in this way, we can start to fork the wasmi and other components to extends the features or the contracts execution environment.
For this purpose, we implement following features for a sandbox framework:
In the process of implementation, we move parts of the requirements in v0.3 to v0.1. We regard Europa as a "framework" at start, so that all substrate runtime could integrate Europa client directly.
Same as Substrate, Europa use bin/europa as an example for blockchain projects which based on Substrate to show how to use Europa, just like what bin/node and bin/node-template do in Substrate. Developers just need to create a new runtime directory, remove the pallets about consensus ("pallet-babe", "pallet-grandpa" and "pallet-session”), because Europa doesn't have the consensus process. Developers need also remove build.rs for runtime crate, and implement bin which contains service.rs and other parts. Then the sandbox can run for this blockchain runtime.
Thus, we can find that the structure of bin/europa is as same as bin/node in Substrate. And on the other hand, bin/europa is also our particular implementation for pallet-contracts smart contracts platform sandbox. It can be used directly to debug smart-contracts for ink!.
Sandbox is just used to execute the runtime, so that we remove all components about consensus. Thus, we use sc-consensus-manual-seal crate to produce block. This crate have a good abstract, so we do not need to fork it. We just modify the part of commands stream for manual-seal. The async stream could receive information from different place (e.g. transaction pool, RPC and others) to drive seal-engine to produce blocks.
As sandbox need to be easily debugged, we remove all WASM components. As we currently know, WASM causes many problems when debugging in details. At first, We need to extend many features for low level libraries, so if we base on the runtime to do something which need to be compiled into WASM, we will meet many unexpected problems.
Europa runtime should remove build.rs in runtime crate and remove [build-dependencies] in runtime crate (cargo.toml file). On the other hand, a sandbox doesn’t need WASM features.
In Substrate, there is a way to provide all existing state under a block, but there is no way to export the modified state-kvs for a block. However, the developers are concerned mostly about the state changes between blocks. So that, they can check whether the changes match their expectations for debugging.
Thus, in europa sandbox, we store the mapping of blockhash and state kvs in the state-kv database, so that developers could export the state changes to look up the details.
You can run the export command while the europa node is running:
# in a shell window
$ ./target/debug/europa
Nov 12 17:10:14.524 INFO Europa Dev Node
Nov 12 17:10:14.524 INFO ✌️ version 0.1.0-7b4463c-x86_64-linux-gnu
Nov 12 17:10:14.524 INFO ❤️ by patract labs <https://github.com/patractlabs>, 2020-2020
Nov 12 17:10:14.524 INFO 📋 Chain specification: Development
# ...
Nov 12 17:21:23.544 INFO Accepted a new tcp connection from 127.0.0.1:44210.
Nov 12 17:21:32.238 INFO 🙌 Starting consensus session on top of parent 0xc7e1ce585807b34b7fecabe1242cafb2628c958b984ec0aee7727cdd34117529
Nov 12 17:21:32.252 INFO 🎁 Prepared block for proposing at 1 [hash: 0x0109608217316a298c88135cf39a87cc31c37729fbe567b4a1a9f8dcdb81ebeb; parent_hash: 0xc7e1…7529; extrinsics (2): [0x2194…baf8, 0x0931…58bb]]
Nov 12 17:21:32.267 INFO Instant Seal success: CreatedBlock { hash: 0x0109608217316a298c88135cf39a87cc31c37729fbe567b4a1a9f8dcdb81ebeb, aux: ImportedAux { header_only: false, clear_justification_requests: false, needs_justification: false, bad_justification: false, needs_finality_proof: false, is_new_best: true } }
# open another shell window, the following command would print the changes for block height 1
$ ./target/debug/europa state-kv 1
Nov 12 15:53:27.699 INFO modified state for block:0x6c119a8f7de42e330aca8b9d3587937aacbbc203cc21650b60644c2f2d33e7fb
Nov 12 15:53:27.699 INFO key:26aa394eea5630e07c48ae0c9558cef702a5c1b19ab7a04f536c519aca4983ac|value:[DELETED]
Nov 12 15:53:27.699 INFO key:26aa394eea5630e07c48ae0c9558cef70a98fdbe9ce6c55837576c60c7af3850|value:05000000
# ...
And Europa also has a RPC method to export the state-kvs while running:
You can call Europa’s RPC like this:
{
"id":100,
"jsonrpc":"2.0",
"method":"europa_modifiedStateKvs",
"params":[ 1 ]
}
will return:
{
"jsonrpc": "2.0",
"result": {
"0x26aa394eea5630e07c48ae0c9558cef702a5c1b19ab7a04f536c519aca4983ac": null,
"0x26aa394eea5630e07c48ae0c9558cef70a98fdbe9ce6c55837576c60c7af3850": "0x05000000",
// ...
}
}
One feature of sandbox is that the developers can operate blockchain state easily. Assuming a situation like this:
A developer is developing a module which has a call of foo, while the call foo needs other basic states (e.g. An account could be a validator only if he had registered his information on chain). Then, when this developer does a test and the result doesn’t match his expectation (had modified related states), something must be wrong for foo. He found the reason and fixed it, and wanted to try it again to test, however the current state is not the old state.
So the sandbox provides a way to "go back" to the old state, allowing developers to do a rollback operation. Thus we provide a RPC method, named: europa_backwardToHeight. It allows developers to recover to any old state as the current best state, then the following operations are based on that old state.
By contrast, with this "backward” RPC method, we also provide a "forward" method, named europa_forwardToHeight. It sends a command to let Europa to produce a batch of blocks to the height which is assigned by this RPC call. This is very useful for the developments which are related to the block number. Many DeFi DApps need to allow users to claim something after a specific block. So, using this RPC to do integration test is very suitable.
And for the feature #4, we also provide a RPC, named europa_modifiedStateKvs to export the modified state kvs.
As above, in current version, Europa provides 3 RPC methods:
europa_forwardToHeight: allow Europa to produce empty blocks to reach to the assigned height.europa_backwardToHeight: allow Europa to revert to the assigned height and state.europa_modifiedStateKvs: allow Europa to export modified state kvs and child state kvs for a block.Substrate could use the command -d/--base-path to assign a directory to store data, thus different environment could use different directory to distinguish. But -d directory is not recorded, so that developers should manage those different paths by themselves. Europa provide a concept of "workspace" to do this work based on the "base path". Developers could use command -w/--worksace to assign different workspace directory to run different Europa node instance. Europa also provides commands to operate "workspaces”, like setting default workspace, listing all existed workspace, or deleting a workspace.
In our workspace management, developers can switch different environments by setting different workspace. By using this feature, developers can switch between different environments to do test easily. On the other hand, this feature provides a friendly way for integration testing.
The building process for this project is as same as Substrate.
When building finished, current executable file in target directory is named europa.
Run Europa
The following is builded in debug mode. If you want to build in release mode, using release to replace debug.
$ ./target/debug/europa
# if you what to specify a directory, add `-d` or `--base-path`
$ ./target/debug/europa -d database
Access Europa
Now, you can use the Official Portal (https://polkadot.js.org/apps/) to access Europa:
click the left tab and switch to DEVELOPMENT - Local Node.
click Settings - Developer, and paste:
{
"Address": "AccountId",
"LookupSource": "AccountId"
}
Other third parties can access Europa like accessing any Substrate node.
Export modified state kvs
$ ./target/debug/europa state-kv 1
# if you have specified a directory, add `-d` or `--base-path`
$ ./target/debug/europa state-kv -d database 1
RPC call is same as other RPCs in Substrate. You can do POST requests by rpc/ws like this:
{
"id":1,
"jsonrpc":"2.0",
"method":"europa_forwardToHeight",
"params":[ 5 ]
}
Just replace "method" and "params" to corresponding parameters.
Specify another workspace
$ ./target/debug/europa -w another-workspace
# if you have specify a directory, add `-d` or `--base-path`
$ ./target/debug/europa -d database -w another-workspace
Stop the Europa, then execute:
# another-workspace is the workspace name that we want to set as default.
$ ./target/debug/europa workspace default another-workspace
Nov 12 17:28:41.980 INFO Current default workspace:
Nov 12 17:28:41.981 INFO default
Nov 12 17:28:41.981 INFO
Nov 12 17:28:41.981 INFO Set [another-workspace] as default workspace.
Then, start Europa. Europa will use "another-workspace" as default workspace.
$ ./target/debug/europa
# ...
Nov 12 17:29:33.862 INFO 💾 Database: RocksDb at .sub/another-workspace/chains/dev/db
Nov 12 17:29:33.862 INFO 📖 Workspace: another-workspace | Current workspace list: ["default", "another-workspace"]
Nov 12 17:29:33.862 INFO ⛓ Native runtime: europa-1 (europa-1.tx1.au1)
Delete workspace
$ ./target/debug/europa workspace delete another-workspace
Nov 12 17:30:49.549 INFO Current default workspace:
Nov 12 17:30:49.549 INFO another-workspace
Nov 12 17:30:49.549 INFO
Nov 12 17:30:49.550 INFO Delete workspace [another-workspace].
Nov 12 17:30:49.550 INFO delete default record: [another-workspace]
Nov 12 17:30:49.550 INFO delete workspace:[another-workspace] from workspace list
As I describe above, we have implemented all features required in proposal.
Europa can be compiled and run directly.
Europa can start from different workspaces and do management.
Europa can run any Substrate runtime, and currently Europa has already integrated
pallet-contracts
Europa can interact with any third parties, including “Official Portal”. Thus Redspot can also access Europa as a normal node.
Europa has provided some RPCs to support this.
That's all, we think Europa can become a very useful tool for Substrate developers. We need more time to do the research for Europa v0.2, then we will post the detailed proposal in about a week.