Code generation algorithm

Intro

To generate our codebase dynamically, we implement a mini custom framework where we define each pallets config restricted by specific rules on how to write it.

Our algorithm parses the configs and inserts the code into a pre-coded base template located in /generator/template

Writing Pallet Configs

Every pallet should have it's own config file defined inside the /src/pallets/configs folder

The config file should export a const object which implement the IPalletConfig interface

When implementing the interface we also define all possible Traits and GenesisConfig fields of the implementing pallet.

The interface consist of name, metadata, runtime and dependencies fields

• name Defines the name of the pallet and is an enum value of ESupportedPallets, this is because when later we define dependencies of another pallet we can link it to existing config through the unique name using the enum

• metada It simply contains some relevant metadata of the pallet, nothing special here

• runtime This object contains most of the required data for generating the code for the pallet needed to insert into the node. It contains runtime modules needed for it to be implemented inside construct_runtime, values for Trait implementation and data for the genesis config. If some code is unique and couldn't easily been abstracted it's put inside the additionalChainSpecCode and additionalRuntimeLibCode fields

• dependencies This field defines the pallets dependency data to put inside the Cargo.toml file, additional pallets it needs (This is where the ESupportedPallets enum comes in handy, because we can now recursively find dependencies and insert them into the code if not already inserted), and also other standard deps needed for the pallet to function

Diving deeper into IPalletRuntimeConfig

interface IPalletRuntimeConfig<Traits extends string, GenesisFields extends string> {
  // The construct runtime config for the pallet
  constructRuntime: IPalletConstructRuntimeConfig;

  // Dynamic object of { Pallet Trait : string | Trait config}
  // if string, simply put the Trait name = string inside trait implementation for runtime
  // if config, put the data from config inside parameter_types macro and then insert name into implementation
  palletTraits: DynamicObject<IPalletTraitsConfig | string, Traits, AllKeysRequired>

  // In here we configure our genesis config struct
  genesisConfig?: IPalletGenesisConfig<GenesisFields>;

  // Rust syntax for code that can't be automatically inserted by the generator
  additionalChainSpecCode?: {
    additionalCode?: string[];
    additionalVariables?: string[];
  };
  additionalRuntimeLibCode?: string[];
}

Substrate Project Generation

The main files our Algorithm touches are:

1. The runtime lib.rs - This is where we import and implement new Modules/Pallets. We can think of it as the main file of our Substrate Node

   const runtimeLibPath = `${projectPath}/runtime/src/lib.rs`;

2. The manifest Cargo.toml - This is where we define any external dependencies needed for our pallets to run

   const runtimeManifestPath = `${projectPath}/runtime/Cargo.toml`;

3. The chain_spec.rs - This is the file where we define the initital values for our blockchain. We can think of it as a genesis file of our chain

   const chainspecPath = `${projectPath}/node/src/chain_spec.rs`;

What happens exactly?

1. The business logic of recursively resolving dependencies and order of writing the pallets into the codebase and file writing is inside /src/services/codeGenerator.service.ts file.

2. But before the file is written, it's first read and converted into a string, where specific utility functions mutate it to include the new code. - /src/utils/substrateManifest.util.ts - Manages the Manifest file - /src/utils/substrateRuntime.util.ts - Manages the Runtime and Chainspec files

3. All the code is written into /generator/temporary/${Date.now()}-${userId} folder.

To prevent collision due to async nature of a web server, we must use userId to uniquelly identify one session's code and not collide with a concurrent request.