System Channel and Application Channel in Hyperledger Fabric
By: Kc Tam
Tags: Hyperledger Fabric, Enterprise Blockchain, Channel, Tutorial, System Channel

Overview

It begins with a question I have been asked several times in the past few months. Some have completed the tutorial Add an Org to a Channel, successfully adding a new organization to a channel. When they try to add this organization to a new channel, they encounter problems. The cause drives me to some system design of Hyperledger Fabric, in particular, on system channel.

The term channel we always refer to in fact is Application Channel, in which channel members are sharing the same ledger and chaincode for a specific business purpose. We can have as many Application Channels as possible to meet our business needs. Besides, there is one System Channel in which consortium configuration is stored. Whenever an update on the consortium, such as a new organization joining the consortium, we need to update this System Channel. And only those organizations defined in the consortium can be added as members of Application Channels. In the tutorial mentioned above we only act on Application Channel, not the System Channel. As a result we cannot add this new organization to a new channel.

In this article, we first replicate this situation, that is, after completing the tutorial, we cannot add this new organization into a new channel. By inspecting the configuration and blocks, we will locate the root cause. In the final part we will add this new organization to System Channel as a consortium member. With this, we are able to create a new channel that this new organization can be a member of.

We use Test Network as our base, with proper modification to meet our demonstration. A repository is created such that you can try in your environment.

Note: there is another article in which system channel is involved: adding an orderer in an ordering cluster. You can do a quick reference to that article, and see similar process when update configuration in the system channel.

Test Network: Modified for Demonstration

In this demonstration, we are using the Test Network (v2.0+). It is a two-peer-org setup, one peer on each org, plus a Solo-based ordering service. It comes with good scripts ( network.sh ) to bring up containers, create channels and deploy sample chaincode (fabcar). Besides it also provides scripts to implement the tutorial Adding an Org to a Channel.

For easy demonstration some files are modified such that we do not need to switch between files. Here is the repository for the demonstration.

Here is the highlight of modification.

cryptogen

We use the cryptogen (not Fabric CA Server) for simplicity. In the original directory structure organizations/cryptogen/ , we have

crypto-config-orderer.yaml

crypto-config-org1.yaml

crypto-config-org2.yaml

We add one more crypto-config-org3.yaml , with a similar setup as other peer organizations (one peer, one admin and one user). The crypto material for Org3 will be generated upfront as well.

channel artifacts

The configuration file is configtx/configtx.yaml .

To prepare Org3 material, we add back the Org3 definition in the organizations section. Again, they are similar to Org1 and Org2.

We keep the two profiles: TwoOrgsOrdererGenesis and TwoOrgsChannel, as we need to remain the first part of demonstration. Meanwhile, we add one more profile: Channel1–3, and make Org1 and Org3 as the members of this new channel.

docker compose file

The only docker compose file we are using is docker/docker-compose-test-net.yaml . Inside we add peer0.org3.example.com. The container setup is largely similar to other peers, except for the correct crypto material and port number to be used.

repository

With these configuration files modified, we have generated certain material for our demonstration. Detail is skipped here. The material we have generated in the repository are

crypto material for all organizations, including Org3, using cryptogen

genesis block, configuration update transactions for mychannel, and configuration transaction for channel1–3.

the content to be inserted into configuration in JSON, named org3.json , is generated and kept in peersOrganization/org3.example.com/

Finally, we have set up several files for terminal setup for different organizations. They are terminalorg1 , terminalorg2 , terminalorg3 and terminalorderer , with proper environment variables. We will source the file to set our terminal for the specific organization.

We first replicate the finding.

Tutorial Replay

Here we follow the tutorial Adding an Org to a Channel. Instead of repeating every command, here we just showed the screenshots of the results on main steps.

Bring up containers

We simply bring up all the containers, including peer0.org3.example.com.

Join mychannel and check status

After joining peer0.org1.example.com and peer0.org2.example.com to mychannel, here is the status of the three organizations.

peer0.org1.example.com: join mychannel

peer0.org2.example.com: join mychannel

peer0.org3.example.com: not joining mychannel yet

Prepare transaction update and sign by both organizations

Following the tutorial, here is the signing from both Org1 and Org2.

Signing from Org1 (peer0.org1.example.com)

Signing from Org2 (peer0.org2.example.com) and submit to ordering service

After submitting, we see new blocks arriving to both peers (now the blockchain height is 4).

peer0.org1.example.com (new block committed)

peer0.org2.example.com (new block committed)

Join peer0.org3 to mychannel

With this, we can fetch block zero of mychannel and join peer0.org3.example.com.

Fetch channel genesis block from peer0.org1.example.com

Join peer0.org3.example.com with genesis block file

Peer will get back the whole blockchain (see the blockchain height, and block hash, same as peer0.org1.example.com and peer0.org2.example.com).

peer0.org3.example.com gets the blockchain from other peers

Here is what we have completed so far.

After completing the tutorial, peer0.org3 is now a member of mychannel.

We have repeated the whole process of tutorial. Now we can simulate the problem.

Simulating the Problem

Create the genesis block file for channel1–3. Remember that in channel1–3 we have defined Org3 as a channel member.

peer channel create -o localhost:7050 -c channel1-3 --ordererTLSHostnameOverride orderer.example.com -f ./channel-artifacts/channel1-3.tx --outputBlock ./channel-artifacts/channel1-3.block --tls --cafile $ORDERER_CA

Here is what we expect: even if we have added an organization to an application channel, this organization is NOT YET in the consortium, that is, this organization is not yet in configuration of the system channel. We will take a look at both system channel and application channel, and their relation with our channel configuration file configtx.yaml .

Observation: System Channel and Application Channel

As we mentioned before, there is one system channel when the consortium network is up and running. Inside the system channel we find two main configuration settings: orderers and consortium members. This system channel needs to be updated when any change of ordering service (e.g. one more orderer added to a cluster) or new consortium member (organization) is added.

We will take a look at the configuration file and the blocks being fetched from both channels.

Configuration file: configtx.yaml

We first take a look at the configuration file configtx.yaml . The part we are interested in is the profiles.

The original configtx.yaml only has these two profiles defined.

The first profile TwoOrgsOrdererGenesis is used when we create the genesis block. In this profile, Org1 and Org2 are included. The command we are using (not shown in this document).

configtxgen -profile TwoOrgsOrdererGenesis -channelID system-channel -outputBlock ./system-genesis-block/genesis.block

As we can see the command is acted on system-channel.

Meanwhile, the second profile TwoOrgsChannel is used when we generate transaction updates for the application channel. And for this setup, again, Org1 and Org2 are included in this application channel. This is valid as Org1 and Org2 is already in the consortium (system channel).

The command is acting on mychannel.

configtxgen -profile TwoOrgsChannel -outputCreateChannelTx ./channel-artifacts/mychannel.tx -channelID mychannel

Besides the configuration file, we can also fetch blocks from both system channel and application channel and make some observations.

Configuration Block captured in System Channel

We use Terminal for Orderer in order to fetch the system channel.

source terminalorderer peer channel fetch config config_block_sys.pb -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com -c system-channel --tls --cafile $ORDERER_CA

We can see that there is a blockchain of length 2, and the configuration block is block #0.

We can see that inside the SampleConsortium, only Org1MSP and Org2MSP defined.

Extract from the configuration block from system-channel

Configuration Block captured in Application Channel

We use Terminal for Org1 to capture the latest configuration update (should be block #3).

After decoding, we can see that inside the Application, Org1MSP, Org2MSP and Org3MSP are defined. This is the result after we have completed the tutorial, adding a new organization to a channel (mychannel).

Extract from the configuration block from mychannel (application channel)

Observation

Even though Org3 is being added in an application channel (mychannel), it is not yet in consortium and therefore we cannot create a new channel in which Org3 is a member. What we need is to add it back to the system channel.

block file for channel1–3 cannot be created as Org3 is not yet in consortium

Add Org3 to System Channel and then Create Channel with Org3

Now we understand that, in order to allow new channels to be created for a new organization, this organization must be added in consortium first. Here we demonstrate how to add organization to the consortium (system channel) such that we can add a new channel.

Fetch configuration block from system channel

Note: all commands here is done in terminal for Orderer.

source terminalorderer peer channel fetch config config_block_sys.pb -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com -c system-channel --tls --cafile $ORDERER_CA

Construct configuration update

With this, we go through the similar process of adding Org3 to Application Channel as before. But now this process is acting upon the configuration block from the system channel.

Here I just listed all the commands.

configtxlator proto_decode --input config_block_sys.pb --type common.Block | jq .data.data[0].payload.data.config > config.json jq -s '.[0] * {"channel_group":{"groups":{"Consortiums":{"groups":{"SampleConsortium":{"groups": {"Org3MSP":.[1]}}}}}}}' config.json ./organizations/peerOrganizations/org3.example.com/org3.json > modified_config.json configtxlator proto_encode --input config.json --type common.Config --output config.pb configtxlator proto_encode --input modified_config.json --type common.Config --output modified_config.pb configtxlator compute_update --channel_id system-channel --original config.pb --updated modified_config.pb --output org3_update.pb configtxlator proto_decode --input org3_update.pb --type common.ConfigUpdate | jq . > org3_update.json echo '{"payload":{"header":{"channel_header":{"channel_id":"'system-channel'", "type":2}},"data":{"config_update":'$(cat org3_update.json)'}}}' | jq . > org3_update_in_envelope.json configtxlator proto_encode --input org3_update_in_envelope.json --type common.Envelope --output org3_update_in_envelope.pb

Now we have the configuration update transaction.

Sign the update with orderer admin

Keep working in the Terminal for Orderer as orderer admin is needed to sign and submit the transaction.

peer channel update -f org3_update_in_envelope.pb -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com -c system-channel --tls --cafile $ORDERER_CA

And we fetch the latest configuration block and see again if org3 is already there.

peer channel fetch config config_block_sys_update.pb -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com -c system-channel --tls --cafile $ORDERER_CA

We see now a new block (block #2) is received. After decoding we can see three organizations inside.

We see Org3MSP is already included in the Consortium.

Create Channel with Org3

Now create new channel channel1–3, which includes Org3 as a channel member.

Go to Terminal for Org1

peer channel create -o localhost:7050 -c channel1-3 --ordererTLSHostnameOverride orderer.example.com -f ./channel-artifacts/channel1-3.tx --outputBlock ./channel-artifacts/channel1-3.block --tls --cafile $ORDERER_CA

Block file for channel1–3 is now created successfully

Now we receive block 0 for channel1–3 ( channel1–3.block ). We can join peer0.org1.example.com and peer0.org3.example.com to this channel.

Terminal for Org1

Terminal for Org3

This screenshot shows the final summary of channel members for both mychannel and channel1–3.

mychannel : Org1, Org2 and Org3

: Org1, Org2 and Org3 channel1–3: Org1 and Org3

Top: Org1, Middle: Org2, Bottom: Org3

And this is what we have now.

block file for channel1–3 is created after Org3 is added in consortium

Summary

We begin with a problem we meet: according to the tutorial, we can add an organization to a channel, but we cannot add new channels with this newly added organization. The reason behind is that an organization needs to be in consortium before we can create new channels with this newly added organization. We find that we are in fact dealing with two different types of channels: one system channel, keeping the consortium configuration, and one or more application channels, which is the “channel” we always refer to, where shared ledger and business logic is enforced among selected consortium members. In this article we have deep-dived into configuration files, blocks captured on the system channel and application channel. We also demonstrated how to add a new organization into a consortium, such that this organization can later be added into another application channel. Hope this provides a good reference when you study Hyperledger Fabric.