Hyperledger Fabric Multi Host 対応(3)

# Hyperledger Fabric Multi Host 対応(3)

Hyperledger Fabric のマルチホスト構成について、実装方法の検討とFabcar を使用した実装例を考えてみる。

今回は、fabcar の環境をマルチホスト化する。  

---
## 構成

Fabcar の構成イメージは、以下の図の通り。

![single.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/327451/a511c914-4976-65f3-03db-9fc6904ba9fa.png)


今回は、fabcar をベースとして、org1とorg2を別々のホスト上に分離して構築する。  
orderer は、org1 側に構築

構成のイメージは、以下の図の通り。

![multi.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/327451/3bb70116-0d7d-4c34-9c5f-5369178d4b78.png)


---

## 構築方法

### hosts の修正

```
$ sudo vi /etc/hosts
```

以下のように、ノードを配置したホストのIPを設定する。※ordererと、org1は、192.168.1.100に、org2を192.168.1.101に配置した例は、以下の通り

```
192.168.2.101   orderer.example.com
192.168.2.101   ca.org1.example.com
192.168.2.101   peer0.org1.example.com
192.168.2.101   peer1.org1.example.com
192.168.2.102   ca.org2.example.com
192.168.2.102   peer0.org2.example.com
192.168.2.102   peer1.org2.example.com
```

念のために疎通も確認  
ホスト１側(peer0.org2.example.com にping)

```
$ ping peer0.org2.example.com 
PING peer0.org2.example.com (192.168.2.102) 56(84) bytes of data.
64 bytes from ca.org2.example.com (192.168.2.102): icmp_seq=1 ttl=64 time=0.463 ms
64 bytes from ca.org2.example.com (192.168.2.102): icmp_seq=2 ttl=64 time=0.648 ms
^C
--- peer0.org2.example.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1029ms
rtt min/avg/max/mdev = 0.463/0.555/0.648/0.095 ms
```

ホスト2側(peer0.org1.example.com にping)

```
$ ping peer0.org2.example.com 
PING peer0.org2.example.com (192.168.2.102) 56(84) bytes of data.
64 bytes from ca.org2.example.com (192.168.2.102): icmp_seq=1 ttl=64 time=0.087 ms
64 bytes from ca.org2.example.com (192.168.2.102): icmp_seq=2 ttl=64 time=0.063 ms
64 bytes from ca.org2.example.com (192.168.2.102): icmp_seq=3 ttl=64 time=0.061 ms
^C
--- peer0.org2.example.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2032ms
rtt min/avg/max/mdev = 0.061/0.070/0.087/0.013 ms
```

---

### Docker-compose の修正

#### ホスト１側の修正  

修正する docker-composeは以下の３つ
1. docker-compose-ca.yaml  
1. docker-compose-cli.yaml
1. docker-compose-couch.yaml

##### docker-compose のバックアップ

```
$ cd ~/fabric-samples/first-network/

$ mkdir org

$ cp docker-compose-ca.yaml org/

$ cp docker-compose-cli.yaml org/

$ cp docker-compose-couch.yaml org/

$ ls org
docker-compose-ca.yaml  docker-compose-cli.yaml  docker-compose-couch.yaml

```

##### docker-compose-ca.yaml の修正

ca0 と、ca1 が定義されているので、ca0 のみにする。

```
$ vi docker-compose-ca.yaml 

# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

version: '2'

networks:
  byfn:

services:
  ca0:
    image: hyperledger/fabric-ca:$IMAGE_TAG
    environment:
      - FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
      - FABRIC_CA_SERVER_CA_NAME=ca-org1
      - FABRIC_CA_SERVER_TLS_ENABLED=true
      - FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org1.example.com-cert.pem
      - FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/${BYFN_CA1_PRIVATE_KEY}
      - FABRIC_CA_SERVER_PORT=7054
    ports:
      - "7054:7054"
    command: sh -c 'fabric-ca-server start --ca.certfile /etc/hyperledger/fabric-ca-server-config/ca.org1.example.com-cert.pem --ca.keyfile /etc/hyperledger/fabric-ca-server-config/${BYFN_CA1_PRIVATE_KEY} -b admin:adminpw -d'
    volumes:
      - ./crypto-config/peerOrganizations/org1.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
    container_name: ca_peerOrg1
    networks:
      - byfn


```

##### docker-compose-cli.yaml の修正

org2 定義の削除と、extra_hosts の追加

```
$ vi docker-compose-cli.yaml 


# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

version: '2'

volumes:
  orderer.example.com:
  peer0.org1.example.com:
  peer1.org1.example.com:

networks:
  byfn:

services:

  orderer.example.com:
    extends:
      file:   base/docker-compose-base.yaml
      service: orderer.example.com
    container_name: orderer.example.com
    networks:
      - byfn
    extra_hosts:
      - "ca.org1.example.com:192.168.2.101"
      - "peer0.org1.example.com:192.168.2.101"
      - "peer1.org1.example.com:192.168.2.101"
      - "orderer.example.com:192.168.2.101"
      - "ca.org2.example.com:192.168.2.102"
      - "peer0.org2.example.com:192.168.2.102"
      - "peer1.org2.example.com:192.168.2.102"


  peer0.org1.example.com:
    container_name: peer0.org1.example.com
    extends:
      file:  base/docker-compose-base.yaml
      service: peer0.org1.example.com
    networks:
      - byfn
    extra_hosts:
      - "ca.org1.example.com:192.168.2.101"
      - "peer0.org1.example.com:192.168.2.101"
      - "peer1.org1.example.com:192.168.2.101"
      - "orderer.example.com:192.168.2.101"
      - "ca.org2.example.com:192.168.2.102"
      - "peer0.org2.example.com:192.168.2.102"
      - "peer1.org2.example.com:192.168.2.102"


  peer1.org1.example.com:
    container_name: peer1.org1.example.com
    extends:
      file:  base/docker-compose-base.yaml
      service: peer1.org1.example.com
    networks:
      - byfn
    extra_hosts:
      - "ca.org1.example.com:192.168.2.101"
      - "peer0.org1.example.com:192.168.2.101"
      - "peer1.org1.example.com:192.168.2.101"
      - "orderer.example.com:192.168.2.101"
      - "ca.org2.example.com:192.168.2.102"
      - "peer0.org2.example.com:192.168.2.102"
      - "peer1.org2.example.com:192.168.2.102"


  cli:
    container_name: cli
    image: hyperledger/fabric-tools:$IMAGE_TAG
    tty: true
    stdin_open: true
    environment:
      - SYS_CHANNEL=$SYS_CHANNEL
      - GOPATH=/opt/gopath
      - CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
      #- FABRIC_LOGGING_SPEC=DEBUG
      - FABRIC_LOGGING_SPEC=INFO
      - CORE_PEER_ID=cli
      - CORE_PEER_ADDRESS=peer0.org1.example.com:7051
      - CORE_PEER_LOCALMSPID=Org1MSP
      - CORE_PEER_TLS_ENABLED=true
      - CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.crt
      - CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.key
      - CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
      - CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
    command: /bin/bash
    volumes:
        - /var/run/:/host/var/run/
        - ./../chaincode/:/opt/gopath/src/github.com/chaincode
        - ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
        - ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
        - ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
    depends_on:
      - orderer.example.com
      - peer0.org1.example.com
      - peer1.org1.example.com
    networks:
      - byfn
    extra_hosts:
      - "ca.org1.example.com:192.168.2.101"
      - "peer0.org1.example.com:192.168.2.101"
      - "peer1.org1.example.com:192.168.2.101"
      - "orderer.example.com:192.168.2.101"
      - "ca.org2.example.com:192.168.2.102"
      - "peer0.org2.example.com:192.168.2.102"
      - "peer1.org2.example.com:192.168.2.102"
```

##### docker-compose-couch.yaml の修正

org2 定義の削除

```
$ vi docker-compose-couch.yaml 


# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

version: '2'

networks:
  byfn:

services:
  couchdb0:
    container_name: couchdb0
    image: hyperledger/fabric-couchdb
    # Populate the COUCHDB_USER and COUCHDB_PASSWORD to set an admin user and password
    # for CouchDB.  This will prevent CouchDB from operating in an "Admin Party" mode.
    environment:
      - COUCHDB_USER=
      - COUCHDB_PASSWORD=
    # Comment/Uncomment the port mapping if you want to hide/expose the CouchDB service,
    # for example map it to utilize Fauxton User Interface in dev environments.
    ports:
      - "5984:5984"
    networks:
      - byfn

  peer0.org1.example.com:
    environment:
      - CORE_LEDGER_STATE_STATEDATABASE=CouchDB
      - CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb0:5984
      # The CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME and CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD
      # provide the credentials for ledger to connect to CouchDB.  The username and password must
      # match the username and password set for the associated CouchDB.
      - CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=
      - CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=
    depends_on:
      - couchdb0

  couchdb1:
    container_name: couchdb1
    image: hyperledger/fabric-couchdb
    # Populate the COUCHDB_USER and COUCHDB_PASSWORD to set an admin user and password
    # for CouchDB.  This will prevent CouchDB from operating in an "Admin Party" mode.
    environment:
      - COUCHDB_USER=
      - COUCHDB_PASSWORD=
    # Comment/Uncomment the port mapping if you want to hide/expose the CouchDB service,
    # for example map it to utilize Fauxton User Interface in dev environments.
    ports:
      - "6984:5984"
    networks:
      - byfn

  peer1.org1.example.com:
    environment:
      - CORE_LEDGER_STATE_STATEDATABASE=CouchDB
      - CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb1:5984
      # The CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME and CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD
      # provide the credentials for ledger to connect to CouchDB.  The username and password must
      # match the username and password set for the associated CouchDB.
      - CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=
      - CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=
    depends_on:
      - couchdb1

```

#### ホスト2側の修正  

修正する docker-composeは以下の３つ
1. docker-compose-ca.yaml  
1. docker-compose-cli.yaml
1. docker-compose-couch.yaml

##### docker-compose のバックアップ

```
$ cd ~/fabric-samples/first-network/

$ mkdir org

$ cp docker-compose-ca.yaml org/

$ cp docker-compose-cli.yaml org/

$ cp docker-compose-couch.yaml org/

$ ls org
docker-compose-ca.yaml  docker-compose-cli.yaml  docker-compose-couch.yaml

```

##### docker-compose-ca.yaml の修正

ca0 と、ca1 が定義されているので、ca1 のみにする。

```
$ vi docker-compose-ca.yaml 

# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

version: '2'

networks:
  byfn:

services:
  ca1:
    image: hyperledger/fabric-ca:$IMAGE_TAG
    environment:
      - FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
      - FABRIC_CA_SERVER_CA_NAME=ca-org2
      - FABRIC_CA_SERVER_TLS_ENABLED=true
      - FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem
      - FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/${BYFN_CA2_PRIVATE_KEY}
      - FABRIC_CA_SERVER_PORT=8054
    ports:
      - "8054:8054"
    command: sh -c 'fabric-ca-server start --ca.certfile /etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem --ca.keyfile /etc/hyperledger/fabric-ca-server-config/${BYFN_CA2_PRIVATE_KEY} -b admin:adminpw -d'
    volumes:
      - ./crypto-config/peerOrganizations/org2.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
    container_name: ca_peerOrg2
    networks:
      - byfn
```

##### docker-compose-cli.yaml の修正

orderer、org1 定義の削除と、extra_hosts の追加

```
$ vi docker-compose-cli.yaml 

# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

version: '2'

volumes:
  orderer.example.com:
  peer0.org2.example.com:
  peer1.org2.example.com:

networks:
  byfn:

services:

  peer0.org2.example.com:
    container_name: peer0.org2.example.com
    extends:
      file:  base/docker-compose-base.yaml
      service: peer0.org2.example.com
    networks:
      - byfn
    extra_hosts:
      - "ca.org1.example.com:192.168.2.101"
      - "peer0.org1.example.com:192.168.2.101"
      - "peer1.org1.example.com:192.168.2.101"
      - "orderer.example.com:192.168.2.101"
      - "ca.org2.example.com:192.168.2.102"
      - "peer0.org2.example.com:192.168.2.102"
      - "peer1.org2.example.com:192.168.2.102"

  peer1.org2.example.com:
    container_name: peer1.org2.example.com
    extends:
      file:  base/docker-compose-base.yaml
      service: peer1.org2.example.com
    networks:
      - byfn
    extra_hosts:
      - "ca.org1.example.com:192.168.2.101"
      - "peer0.org1.example.com:192.168.2.101"
      - "peer1.org1.example.com:192.168.2.101"
      - "orderer.example.com:192.168.2.101"
      - "ca.org2.example.com:192.168.2.102"
      - "peer0.org2.example.com:192.168.2.102"
      - "peer1.org2.example.com:192.168.2.102"

  cli:
    container_name: cli
    image: hyperledger/fabric-tools:$IMAGE_TAG
    tty: true
    stdin_open: true
    environment:
      - SYS_CHANNEL=$SYS_CHANNEL
      - GOPATH=/opt/gopath
      - CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
      #- FABRIC_LOGGING_SPEC=DEBUG
      - FABRIC_LOGGING_SPEC=INFO
      - CORE_PEER_ID=cli
      - CORE_PEER_ADDRESS=peer0.org1.example.com:7051
      - CORE_PEER_LOCALMSPID=Org1MSP
      - CORE_PEER_TLS_ENABLED=true
      - CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.crt
      - CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.key
      - CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
      - CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
    command: /bin/bash
    volumes:
        - /var/run/:/host/var/run/
        - ./../chaincode/:/opt/gopath/src/github.com/chaincode
        - ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
        - ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
        - ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
    depends_on:
      - peer0.org2.example.com
      - peer1.org2.example.com
    networks:
      - byfn
    extra_hosts:
      - "ca.org1.example.com:192.168.2.101"
      - "peer0.org1.example.com:192.168.2.101"
      - "peer1.org1.example.com:192.168.2.101"
      - "orderer.example.com:192.168.2.101"
      - "ca.org2.example.com:192.168.2.102"
      - "peer0.org2.example.com:192.168.2.102"
      - "peer1.org2.example.com:192.168.2.102"
```




##### docker-compose-couch.yaml の修正

org1 定義の削除

```
$ vi docker-compose-couch.yaml 

# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

version: '2'

networks:
  byfn:

services:
  couchdb2:
    container_name: couchdb2
    image: hyperledger/fabric-couchdb
    # Populate the COUCHDB_USER and COUCHDB_PASSWORD to set an admin user and password
    # for CouchDB.  This will prevent CouchDB from operating in an "Admin Party" mode.
    environment:
      - COUCHDB_USER=
      - COUCHDB_PASSWORD=
    # Comment/Uncomment the port mapping if you want to hide/expose the CouchDB service,
    # for example map it to utilize Fauxton User Interface in dev environments.
    ports:
      - "7984:5984"
    networks:
      - byfn

  peer0.org2.example.com:
    environment:
      - CORE_LEDGER_STATE_STATEDATABASE=CouchDB
      - CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb2:5984
      # The CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME and CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD
      # provide the credentials for ledger to connect to CouchDB.  The username and password must
      # match the username and password set for the associated CouchDB.
      - CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=
      - CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=
    depends_on:
      - couchdb2

  couchdb3:
    container_name: couchdb3
    image: hyperledger/fabric-couchdb
    # Populate the COUCHDB_USER and COUCHDB_PASSWORD to set an admin user and password
    # for CouchDB.  This will prevent CouchDB from operating in an "Admin Party" mode.
    environment:
      - COUCHDB_USER=
      - COUCHDB_PASSWORD=
    # Comment/Uncomment the port mapping if you want to hide/expose the CouchDB service,
    # for example map it to utilize Fauxton User Interface in dev environments.
    ports:
      - "8984:5984"
    networks:
      - byfn

  peer1.org2.example.com:
    environment:
      - CORE_LEDGER_STATE_STATEDATABASE=CouchDB
      - CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb3:5984
      # The CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME and CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD
      # provide the credentials for ledger to connect to CouchDB.  The username and password must
      # match the username and password set for the associated CouchDB.
      - CORE_LEDGER_STATE_COUCHDBCONFIG_USERNAME=
      - CORE_LEDGER_STATE_COUCHDBCONFIG_PASSWORD=
    depends_on:
      - couchdb3
```

---

### 起動shellの修正

以下の4点について追加を実施

* 証明書のコピー
* channel 情報のコピー
* ホスト2側コンテナの起動処理呼び出し
* ホスト2側コンテナの終了処理呼び出し

#### 証明書のコピー

generateCerts() 関数の中で、証明書を生成している。  
証明書生成後に、証明書コピーのshellを呼び出すように追加

```
function generateCerts() {
  which cryptogen
  if [ "$?" -ne 0 ]; then
    echo "cryptogen tool not found. exiting"
    exit 1
  fi
  echo
  echo "##########################################################"
  echo "##### Generate certificates using cryptogen tool #########"
  echo "##########################################################"

  if [ -d "crypto-config" ]; then
    rm -Rf crypto-config
  fi
  set -x
  cryptogen generate --config=./crypto-config.yaml
  res=$?
  set +x
  if [ $res -ne 0 ]; then
    echo "Failed to generate certificates..."
    exit 1
  fi
  echo
  echo "Generate CCP files for Org1 and Org2"
  ./ccp-generate.sh

  ################
  # copy crypto file to host2
  #
  echo #############################
  echo # copy crypto file to host2 #
  echo #############################
  ./copy-crypto.sh
}
```

#### channel 情報のコピー

channel 情報は、generateChannelArtifacts() で生成している。  
channel 情報生成後に、channel 情報コピーのshellを呼び出すように追加

```
function generateChannelArtifacts() {
  which configtxgen
  if [ "$?" -ne 0 ]; then
    echo "configtxgen tool not found. exiting"
    exit 1
  fi

  echo "##########################################################"
  echo "#########  Generating Orderer Genesis block ##############"
  echo "##########################################################"
  # Note: For some unknown reason (at least for now) the block file can't be
  # named orderer.genesis.block or the orderer will fail to launch!
  echo "CONSENSUS_TYPE="$CONSENSUS_TYPE
  set -x
  if [ "$CONSENSUS_TYPE" == "solo" ]; then
    configtxgen -profile TwoOrgsOrdererGenesis -channelID $SYS_CHANNEL -outputBlock ./channel-artifacts/genesis.block
  elif [ "$CONSENSUS_TYPE" == "kafka" ]; then
    configtxgen -profile SampleDevModeKafka -channelID $SYS_CHANNEL -outputBlock ./channel-artifacts/genesis.block
  elif [ "$CONSENSUS_TYPE" == "etcdraft" ]; then
    configtxgen -profile SampleMultiNodeEtcdRaft -channelID $SYS_CHANNEL -outputBlock ./channel-artifacts/genesis.block
  else
    set +x
    echo "unrecognized CONSESUS_TYPE='$CONSENSUS_TYPE'. exiting"
    exit 1
  fi
  res=$?
  set +x
  if [ $res -ne 0 ]; then
    echo "Failed to generate orderer genesis block..."
    exit 1
  fi
  echo
  echo "#################################################################"
  echo "### Generating channel configuration transaction 'channel.tx' ###"
  echo "#################################################################"
  set -x
  configtxgen -profile TwoOrgsChannel -outputCreateChannelTx ./channel-artifacts/channel.tx -channelID $CHANNEL_NAME
  res=$?
  set +x
  if [ $res -ne 0 ]; then
    echo "Failed to generate channel configuration transaction..."
    exit 1
  fi

  echo
  echo "#################################################################"
  echo "#######    Generating anchor peer update for Org1MSP   ##########"
  echo "#################################################################"
  set -x
  configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate ./channel-artifacts/Org1MSPanchors.tx -channelID $CHANNEL_NAME -asOrg Org1MSP
  res=$?
  set +x
  if [ $res -ne 0 ]; then
    echo "Failed to generate anchor peer update for Org1MSP..."
    exit 1
  fi

  echo
  echo "#################################################################"
  echo "#######    Generating anchor peer update for Org2MSP   ##########"
  echo "#################################################################"
  set -x
  configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate \
    ./channel-artifacts/Org2MSPanchors.tx -channelID $CHANNEL_NAME -asOrg Org2MSP
  res=$?
  set +x
  if [ $res -ne 0 ]; then
    echo "Failed to generate anchor peer update for Org2MSP..."
    exit 1
  fi
  echo

  ################
  # copy channel-artifacts file to host2
  #
  echo ########################################
  echo # copy channel-artifacts file to host2 #
  echo ########################################
  ./copy-channel.sh
}
```

#### ホスト2側コンテナの起動処理呼び出し

ホスト1のコンテナは、networkUp() で起動している。  
ホスト1のコンテナ起動前に、ホスト2コンテナの起動処理を追加する。  

```
# Generate the needed certificates, the genesis block and start the network.
function networkUp() {
  checkPrereqs
  # generate artifacts if they don't exist
  if [ ! -d "crypto-config" ]; then
    generateCerts
    replacePrivateKey
    generateChannelArtifacts
  fi
  COMPOSE_FILES="-f ${COMPOSE_FILE}"
  if [ "${CERTIFICATE_AUTHORITIES}" == "true" ]; then
    COMPOSE_FILES="${COMPOSE_FILES} -f ${COMPOSE_FILE_CA}"
    export BYFN_CA1_PRIVATE_KEY=$(cd crypto-config/peerOrganizations/org1.example.com/ca && ls *_sk)
    export BYFN_CA2_PRIVATE_KEY=$(cd crypto-config/peerOrganizations/org2.example.com/ca && ls *_sk)
  fi
  if [ "${CONSENSUS_TYPE}" == "kafka" ]; then
    COMPOSE_FILES="${COMPOSE_FILES} -f ${COMPOSE_FILE_KAFKA}"
  elif [ "${CONSENSUS_TYPE}" == "etcdraft" ]; then
    COMPOSE_FILES="${COMPOSE_FILES} -f ${COMPOSE_FILE_RAFT2}"
  fi
  if [ "${IF_COUCHDB}" == "couchdb" ]; then
    COMPOSE_FILES="${COMPOSE_FILES} -f ${COMPOSE_FILE_COUCH}"
  fi

  ################
  # upnode to host2
  #
  echo ####################
  echo # upnode  to host2 #
  echo ####################
  ./up-node.s

  IMAGE_TAG=$IMAGETAG docker-compose ${COMPOSE_FILES} up -d 2>&1
  docker ps -a
  if [ $? -ne 0 ]; then
    echo "ERROR !!!! Unable to start network"
    exit 1
  fi

  if [ "$CONSENSUS_TYPE" == "kafka" ]; then
    sleep 1
    echo "Sleeping 10s to allow $CONSENSUS_TYPE cluster to complete booting"
    sleep 9
  fi

  if [ "$CONSENSUS_TYPE" == "etcdraft" ]; then
    sleep 1
    echo "Sleeping 15s to allow $CONSENSUS_TYPE cluster to complete booting"
    sleep 14
  fi

  # now run the end to end script
  docker exec cli scripts/script.sh $CHANNEL_NAME $CLI_DELAY $LANGUAGE $CLI_TIMEOUT $VERBOSE $NO_CHAINCODE
  if [ $? -ne 0 ]; then
    echo "ERROR !!!! Test failed"
    exit 1
  fi
}
```


#### ホスト2側コンテナの終了処理呼び出し

ホスト1のコンテナは、networkDown() で終了している。  
ホスト1のコンテナ終了後に、ホスト2コンテナの終了処理を追加する。  

```
# Tear down running network
function networkDown() {
  # stop org3 containers also in addition to org1 and org2, in case we were running sample to add org3
  # stop kafka and zookeeper containers in case we're running with kafka consensus-type
  docker-compose -f $COMPOSE_FILE -f $COMPOSE_FILE_COUCH -f $COMPOSE_FILE_KAFKA -f $COMPOSE_FILE_RAFT2 -f $COMPOSE_FILE_CA -f $COMPOSE_FILE_ORG3 down --volumes --remove-orphans
  ################
  # downnode to host2
  #
  echo ######################
  echo # downnode  to host2 #
  echo ######################
  ./down-node.sh


  # Don't remove the generated artifacts -- note, the ledgers are always removed
  if [ "$MODE" != "restart" ]; then
    # Bring down the network, deleting the volumes
    #Delete any ledger backups
    docker run -v $PWD:/tmp/first-network --rm hyperledger/fabric-tools:$IMAGETAG rm -Rf /tmp/first-network/ledgers-backup
    #Cleanup the chaincode containers
    clearContainers
    #Cleanup images
    removeUnwantedImages
    # remove orderer block and other channel configuration transactions and certs
    rm -rf channel-artifacts/*.block channel-artifacts/*.tx crypto-config ./org3-artifacts/crypto-config/ channel-artifacts/org3.json
    # remove the docker-compose yaml file that was customized to the example
    rm -f docker-compose-e2e.yaml
  fi
}
```

---

### サポートツールの作成

　作成するのは、以下のツール

* 証明書コピーツール
* channel 情報コピーツール
* ホスト2側コンテナ起動処理呼び出しツール
* ホスト2側コンテナ終了処理呼び出しツール


#### 証明書コピーツール
fabric-samples/first-network/crypto-config の下に各種証明書が作成されているので、ホスト2側にscpを使用してコピーする。  
※ 本来であれば、ホスト1の公開鍵とホスト2側の証明書をコピーすべきであるが、本手順では、すべての証明書をコピーしている。

```
$ vi copy-crypto.sh

scp -r ~/fabric-samples/first-network/crypto-config <ユーザ名>@<ホスト2_IPアドレス>:~/fabric-samples/first-network/.
scp ~/fabric-samples/first-network/connection-org2.json <ユーザ名>@<ホスト2_IPアドレス>:~/fabric-samples/first-network/.
scp ~/fabric-samples/first-network/connection-org2.yaml <ユーザ名>@<ホスト2_IPアドレス>:~/fabric-samples/first-network/.
```

実行権限を付与する。

```
$ chmod +x copy-crypto.sh
```

#### channel 情報コピーツール

fabric-samples/first-network/channel-artifacts  の下にchannel 情報が作成されているので、ホスト2側にscpを使用してコピーする。  

```
$ vi copy-channel.sh

scp -r ~/fabric-samples/first-network/channel-artifacts <ユーザ名>@<ホスト2_IPアドレス>:~/fabric-samples/first-network/.
```

実行権限を付与する。

```
$ chmod +x copy-channel.sh
```

#### ホスト2側コンテナ起動処理呼び出しツール


##### ホスト1側

ssh を使用してホスト２側のコンテナ起動shellを呼び出す

```
$ vi up-node.sh

ssh <ユーザ名>@<ホスト2_IPアドレス> "cd ~/fabric-samples/first-network/;./up-node.sh"
```

実行権限を付与する。

```
$ chmod +x up-node.sh
```

##### ホスト2側

ホスト1からの起動指示を受けて、コンテナを起動させるshell を作成

```
$ vi up-node.sh

export BYFN_CA2_PRIVATE_KEY=$(cd crypto-config/peerOrganizations/org2.example.com/ca && ls *_sk)
docker-compose -f docker-compose-cli.yaml -f docker-compose-ca.yaml -f docker-compose-couch.yaml up  -d
```

実行権限を付与する。

```
$ chmod +x up-node.sh
```

#### ホスト2側コンテナ終了処理呼び出しツール


##### ホスト1側

ssh を使用してホスト２側のコンテナ終了shellを呼び出す

```
$ vi down-node.sh

ssh <ユーザ名>@<ホスト2_IPアドレス> "cd ~/fabric-samples/first-network/;./down-node.sh"
```

実行権限を付与する。

```
$ chmod +x up-node.sh
```

##### ホスト2側

ホスト1からの終了指示を受けて、コンテナを終了させるshell を作成
docker コンテナをダウンさせた後に、chaincode コンテナなどもきれいに消去する。

```
$ vi down-node.sh

#!/bin/bash

# Obtain CONTAINER_IDS and remove them
# TODO Might want to make this optional - could clear other containers
function clearContainers() {
  CONTAINER_IDS=$(docker ps -a | awk '($2 ~ /dev-peer.*/) {print $1}')
  if [ -z "$CONTAINER_IDS" -o "$CONTAINER_IDS" == " " ]; then
    echo "---- No containers available for deletion ----"
  else
    docker rm -f $CONTAINER_IDS
  fi
}

# Delete any images that were generated as a part of this setup
# specifically the following images are often left behind:
# TODO list generated image naming patterns
function removeUnwantedImages() {
  DOCKER_IMAGE_IDS=$(docker images | awk '($1 ~ /dev-peer.*/) {print $3}')
  if [ -z "$DOCKER_IMAGE_IDS" -o "$DOCKER_IMAGE_IDS" == " " ]; then
    echo "---- No images available for deletion ----"
  else
    docker rmi -f $DOCKER_IMAGE_IDS
  fi
}


export BYFN_CA2_PRIVATE_KEY=$(cd crypto-config/peerOrganizations/org2.example.com/ca && ls *_sk)
docker-compose -f docker-compose-cli.yaml -f docker-compose-ca.yaml -f docker-compose-couch.yaml down

# Bring down the network, deleting the volumes
#Delete any ledger backups
docker run -v $PWD:/tmp/first-network --rm hyperledger/fabric-tools rm -Rf /tmp/first-network/ledgers-backup
#Cleanup the chaincode containers
clearContainers
#Cleanup images
removeUnwantedImages
# remove orderer block and other channel configuration transactions and certs
rm -rf channel-artifacts/*.block channel-artifacts/*.tx crypto-config 

docker system prune --volumes -f



```

実行権限を付与する。

```
$ chmod +x up-node.sh
```

---

### ccp-template の修正

connection プロファイルは、org1,2 両方での Endorsement に対応できるようにorg1, org2 の両方を含んだものとして作成する。

#### connection プロファイル作成ツールの修正

以下のように修正する。

```
$ vi ccp-generate.sh

#!/bin/bash

function one_line_pem {
    echo "`awk 'NF {sub(/\\n/, ""); printf "%s\\\\\\\n",$0;}' $1`"
}

function json_ccp {
    local PP=$(one_line_pem $5)
    local CP=$(one_line_pem $6)
    local PP2=$(one_line_pem ${11})
    local CP2=$(one_line_pem ${12})
    sed -e "s/\${ORG}/$1/" \
        -e "s/\${P0PORT}/$2/" \
        -e "s/\${P1PORT}/$3/" \
        -e "s/\${CAPORT}/$4/" \
        -e "s#\${PEERPEM}#$PP#" \
        -e "s#\${CAPEM}#$CP#" \
        -e "s/\${ORG2}/$7/" \
        -e "s/\${P0PORT2}/$8/" \
        -e "s/\${P1PORT2}/$9/" \
        -e "s/\${CAPORT2}/${10}/" \
        -e "s#\${PEERPEM2}#$PP2#" \
        -e "s#\${CAPEM2}#$CP2#" \
        -e "s/\${ORG0}/${13}/" \
        ccp-template.json 
}

function yaml_ccp {
    local PP=$(one_line_pem $5)
    local CP=$(one_line_pem $6)
    local PP2=$(one_line_pem ${11})
    local CP2=$(one_line_pem ${12})
    sed -e "s/\${ORG}/$1/" \
        -e "s/\${P0PORT}/$2/" \
        -e "s/\${P1PORT}/$3/" \
        -e "s/\${CAPORT}/$4/" \
        -e "s#\${PEERPEM}#$PP#" \
        -e "s#\${CAPEM}#$CP#" \
        -e "s/\${ORG2}/$7/" \
        -e "s/\${P0PORT2}/$8/" \
        -e "s/\${P1PORT2}/$9/" \
        -e "s/\${CAPORT2}/${10}/" \
        -e "s#\${PEERPEM2}#$PP2#" \
        -e "s#\${CAPEM2}#$CP2#" \
        -e "s/\${ORG0}/${13}/" \
        ccp-template.yaml | sed -e $'s/\\\\n/\\\n        /g'
}

ORG=1
P0PORT=7051
P1PORT=8051
CAPORT=7054
PEERPEM=crypto-config/peerOrganizations/org1.example.com/tlsca/tlsca.org1.example.com-cert.pem
CAPEM=crypto-config/peerOrganizations/org1.example.com/ca/ca.org1.example.com-cert.pem
ORG2=2
P0PORT2=9051
P1PORT2=10051
CAPORT2=8054
PEERPEM2=crypto-config/peerOrganizations/org2.example.com/tlsca/tlsca.org2.example.com-cert.pem
CAPEM2=crypto-config/peerOrganizations/org2.example.com/ca/ca.org2.example.com-cert.pem
ORG0=1

echo "$(json_ccp $ORG $P0PORT $P1PORT $CAPORT $PEERPEM $CAPEM $ORG2 $P0PORT2 $P1PORT2 $CAPORT2 $PEERPEM2 $CAPEM2 $ORG0)" > connection-org1.json
echo "$(yaml_ccp $ORG $P0PORT $P1PORT $CAPORT $PEERPEM $CAPEM $ORG2 $P0PORT2 $P1PORT2 $CAPORT2 $PEERPEM2 $CAPEM2 $ORG0)" > connection-org1.yaml

ORG=1
P0PORT=7051
P1PORT=8051
CAPORT=7054
PEERPEM=crypto-config/peerOrganizations/org1.example.com/tlsca/tlsca.org1.example.com-cert.pem
CAPEM=crypto-config/peerOrganizations/org1.example.com/ca/ca.org1.example.com-cert.pem
ORG2=2
P0PORT2=9051
P1PORT2=10051
CAPORT2=8054
PEERPEM2=crypto-config/peerOrganizations/org2.example.com/tlsca/tlsca.org2.example.com-cert.pem
CAPEM2=crypto-config/peerOrganizations/org2.example.com/ca/ca.org2.example.com-cert.pem
ORG0=2

echo "$(json_ccp $ORG $P0PORT $P1PORT $CAPORT $PEERPEM $CAPEM $ORG2 $P0PORT2 $P1PORT2 $CAPORT2 $PEERPEM2 $CAPEM2 $ORG0)" > connection-org2.json
echo "$(yaml_ccp $ORG $P0PORT $P1PORT $CAPORT $PEERPEM $CAPEM $ORG2 $P0PORT2 $P1PORT2 $CAPORT2 $PEERPEM2 $CAPEM2 $ORG0)" > connection-org2.yaml
```


#### JSON テンプレートの修正

以下のように修正する。

```
$ vi ccp-template.json

{
    "name": "first-network-org${ORG0}",
    "version": "1.0.0",
    "client": {
        "organization": "Org${ORG0}",
        "connection": {
            "timeout": {
                "peer": {
                    "endorser": "300"
                }
            }
        }
    },
    "organizations": {
        "Org${ORG}": {
            "mspid": "Org${ORG}MSP",
            "peers": [
                "peer0.org${ORG}.example.com",
                "peer1.org${ORG}.example.com"
            ],
            "certificateAuthorities": [
                "ca.org${ORG}.example.com"
            ]
        },
        "Org${ORG2}": {
            "mspid": "Org${ORG2}MSP",
            "peers": [
                "peer0.org${ORG2}.example.com",
                "peer1.org${ORG2}.example.com"
            ],
            "certificateAuthorities": [
                "ca.org${ORG2}.example.com"
            ]
        }
    },
    "peers": {
        "peer0.org${ORG}.example.com": {
            "url": "grpcs://peer0.org${ORG}.example.com:${P0PORT}",
            "tlsCACerts": {
                "pem": "${PEERPEM}"
            },
            "grpcOptions": {
                "ssl-target-name-override": "peer0.org${ORG}.example.com",
                "hostnameOverride": "peer0.org${ORG}.example.com"
            }
        },
        "peer1.org${ORG}.example.com": {
            "url": "grpcs://peer1.org${ORG}.example.com:${P1PORT}",
            "tlsCACerts": {
                "pem": "${PEERPEM}"
            },
            "grpcOptions": {
                "ssl-target-name-override": "peer1.org${ORG}.example.com",
                "hostnameOverride": "peer1.org${ORG}.example.com"
            }
        },
        "peer0.org${ORG2}.example.com": {
            "url": "grpcs://peer0.org${ORG2}.example.com:${P0PORT2}",
            "tlsCACerts": {
                "pem": "${PEERPEM2}"
            },
            "grpcOptions": {
                "ssl-target-name-override": "peer0.org${ORG2}.example.com",
                "hostnameOverride": "peer0.org${ORG2}.example.com"
            }
        },
        "peer1.org${ORG2}.example.com": {
            "url": "grpcs://peer1.org${ORG2}.example.com:${P1PORT2}",
            "tlsCACerts": {
                "pem": "${PEERPEM2}"
            },
            "grpcOptions": {
                "ssl-target-name-override": "peer1.org${ORG2}.example.com",
                "hostnameOverride": "peer1.org${ORG2}.example.com"
            }
        }
    },
    "certificateAuthorities": {
        "ca.org${ORG}.example.com": {
            "url": "https://ca.org${ORG}.example.com:${CAPORT}",
            "caName": "ca-org${ORG}",
            "tlsCACerts": {
                "pem": "${CAPEM}"
            },
            "httpOptions": {
                "verify": false
            }
        },
        "ca.org${ORG2}.example.com": {
            "url": "https://ca.org${ORG2}.example.com:${CAPORT2}",
            "caName": "ca-org${ORG2}",
            "tlsCACerts": {
                "pem": "${CAPEM2}"
            },
            "httpOptions": {
                "verify": false
            }
        }
    }
}
```

#### YAML テンプレートの修正

以下のように修正する。

```
$ vi ccp-template.yaml

---
name: first-network-org${ORG0}
version: 1.0.0
client:
  organization: Org${ORG0}
  connection:
    timeout:
      peer:
        endorser: '300'
organizations:
  Org${ORG}:
    mspid: Org${ORG}MSP
    peers:
    - peer0.org${ORG}.example.com
    - peer1.org${ORG}.example.com
    certificateAuthorities:
    - ca.org${ORG}.example.com
  Org${ORG2}:
    mspid: Org${ORG2}MSP
    peers:
    - peer0.org${ORG2}.example.com
    - peer1.org${ORG2}.example.com
    certificateAuthorities:
    - ca.org${ORG2}.example.com
peers:
  peer0.org${ORG}.example.com:
    url: grpcs://peer0.org${ORG}.example.com:${P0PORT}
    tlsCACerts:
      pem: |
        ${PEERPEM}
    grpcOptions:
      ssl-target-name-override: peer0.org${ORG}.example.com
      hostnameOverride: peer0.org${ORG}.example.com
  peer1.org${ORG}.example.com:
    url: grpcs://peer1.org${ORG}.example.com:${P1PORT}
    tlsCACerts:
      pem: |
        ${PEERPEM}
    grpcOptions:
      ssl-target-name-override: peer1.org${ORG}.example.com
      hostnameOverride: peer1.org${ORG}.example.com
  peer0.org${ORG2}.example.com:
    url: grpcs://peer0.org${ORG2}.example.com:${P0PORT2}
    tlsCACerts:
      pem: |
        ${PEERPEM2}
    grpcOptions:
      ssl-target-name-override: peer0.org${ORG2}.example.com
      hostnameOverride: peer0.org${ORG2}.example.com
  peer1.org${ORG2}.example.com:
    url: grpcs://peer1.org${ORG2}.example.com:${P1PORT2}
    tlsCACerts:
      pem: |
        ${PEERPEM2}
    grpcOptions:
      ssl-target-name-override: peer1.org${ORG}.example.com
      hostnameOverride: peer1.org${ORG}.example.com
certificateAuthorities:
  ca.org${ORG}.example.com:
    url: https://ca.org${ORG}.example.com:${CAPORT}
    caName: ca-org${ORG}
    tlsCACerts:
      pem: |
        ${CAPEM}
    httpOptions:
      verify: false
  ca.org${ORG2}.example.com:
    url: https://ca.org${ORG2}.example.com:${CAPORT2}
    caName: ca-org${ORG2}
    tlsCACerts:
      pem: |
        ${CAPEM2}
    httpOptions:
      verify: false
```


### javascript の修正

#### ホスト1側

* asLocalhost: true  を　asLocalhost: false にする。

```
cd ~/fabric-samples/fabcar/javascript
```

##### enrollAdmin.js

修正なし

##### registerUser.js

```
$ vi registerUser.js 

以下の1か所を修正

        await gateway.connect(ccpPath, { wallet, identity: 'admin', discovery: { enabled: true, asLocalhost: false } });
```

##### query.js

```
$ vi query.js 

以下の1か所を修正

        await gateway.connect(ccpPath, { wallet, identity: 'user1', discovery: { enabled: true, asLocalhost: false } });
```

##### invoke.js

```
$ vi invoke.js 

以下の1か所を修正

        await gateway.connect(ccpPath, { wallet, identity: 'user1', discovery: { enabled: true, asLocalhost: false } });
```



#### ホスト2側

* org1 を org2に修正する。  
* asLocalhost: true  を　asLocalhost: false にする。

```
cd ~/fabric-samples/fabcar/javascript
```

##### enrollAdmin.js

```
$ vi enrollAdmin.js

以下の3か所を修正

const ccpPath = path.resolve(__dirname, '..', '..', 'first-network', 'connection-org2.json');

        const caInfo = ccp.certificateAuthorities['ca.org2.example.com'];

        const identity = X509WalletMixin.createIdentity('Org2MSP', enrollment.certificate, enrollment.key.toBytes());

```

##### registerUser.js

```
$ vi registerUser.js 

以下の4か所を修正

const ccpPath = path.resolve(__dirname, '..', '..', 'first-network', 'connection-org2.json');

        await gateway.connect(ccpPath, { wallet, identity: 'admin', discovery: { enabled: true, asLocalhost: false } });

        const secret = await ca.register({ affiliation: 'org2.department1', enrollmentID: 'user1', role: 'client' }, adminIdentity);

        const userIdentity = X509WalletMixin.createIdentity('Org2MSP', enrollment.certificate, enrollment.key.toBytes());
```

##### query.js

```
$ vi query.js 

以下の2か所を修正

const ccpPath = path.resolve(__dirname, '..', '..', 'first-network', 'connection-org2.json');

        await gateway.connect(ccpPath, { wallet, identity: 'user1', discovery: { enabled: true, asLocalhost: false } });

```

##### invoke.js

ホスト1と同じでは、どちらで登録したかわからなくなるので、CAR12 をCAR13 に変更する。

```
$ vi invoke.js 

以下の3か所を修正

const ccpPath = path.resolve(__dirname, '..', '..', 'first-network', 'connection-org2.json');

        await gateway.connect(ccpPath, { wallet, identity: 'user1', discovery: { enabled: true, asLocalhost: false } });

        await contract.submitTransaction('createCar', 'CAR13', 'Honda', 'Accord', 'Black', 'Tom');
```

---
## 動作確認

### ブロックチェーンの起動

ホスト1で以下を実行

```
$ ./startFabric.sh golang
```

正常終了したら、コンテナの起動状況を確認
* ホスト1側

```
$ docker ps
CONTAINER ID        IMAGE                                                                                                    COMMAND                  CREATED             STATUS              PORTS                                        NAMES
ead481ee3523        dev-peer0.org1.example.com-fabcar-1.0-5c906e402ed29f20260ae42283216aa75549c571e2e380f3615826365d8269ba   "chaincode -peer.add…"   20 minutes ago      Up 20 minutes                                                    dev-peer0.org1.example.com-fabcar-1.0
0aa90fed6280        hyperledger/fabric-tools:latest                                                                          "/bin/bash"              20 minutes ago      Up 20 minutes                                                    cli
2ba0ead340ad        hyperledger/fabric-peer:latest                                                                           "peer node start"        20 minutes ago      Up 20 minutes       0.0.0.0:8051->8051/tcp                       peer1.org1.example.com
d5f9df28c351        hyperledger/fabric-peer:latest                                                                           "peer node start"        21 minutes ago      Up 20 minutes       0.0.0.0:7051->7051/tcp                       peer0.org1.example.com
8912e8dd7cd0        hyperledger/fabric-orderer:latest                                                                        "orderer"                21 minutes ago      Up 21 minutes       0.0.0.0:7050->7050/tcp                       orderer.example.com
0713dc760aed        hyperledger/fabric-couchdb                                                                               "tini -- /docker-ent…"   21 minutes ago      Up 21 minutes       4369/tcp, 9100/tcp, 0.0.0.0:5984->5984/tcp   couchdb0
b8b092a8a65f        hyperledger/fabric-ca:latest                                                                             "sh -c 'fabric-ca-se…"   21 minutes ago      Up 20 minutes       0.0.0.0:7054->7054/tcp                       ca_peerOrg1
8025cf39ac2f        hyperledger/fabric-couchdb                                                                               "tini -- /docker-ent…"   21 minutes ago      Up 20 minutes       4369/tcp, 9100/tcp, 0.0.0.0:6984->5984/tcp   couchdb1
```

* ホスト2側

```
$ docker ps
CONTAINER ID        IMAGE                                                                                                    COMMAND                  CREATED             STATUS              PORTS                                        NAMES
d0b65d8623b7        dev-peer0.org2.example.com-fabcar-1.0-264b0a1cb5efbecaac5cf8990339c24474dc8435c6e10f10f2be565d555d0e94   "chaincode -peer.add…"   19 minutes ago      Up 19 minutes                                                    dev-peer0.org2.example.com-fabcar-1.0
5471941392d4        hyperledger/fabric-tools:latest                                                                          "/bin/bash"              21 minutes ago      Up 21 minutes                                                    cli
f5430b9460c4        hyperledger/fabric-peer:latest                                                                           "peer node start"        21 minutes ago      Up 21 minutes       0.0.0.0:10051->10051/tcp                     peer1.org2.example.com
e00ee67e009f        hyperledger/fabric-peer:latest                                                                           "peer node start"        21 minutes ago      Up 21 minutes       0.0.0.0:9051->9051/tcp                       peer0.org2.example.com
266d138e73e9        hyperledger/fabric-couchdb                                                                               "tini -- /docker-ent…"   21 minutes ago      Up 21 minutes       4369/tcp, 9100/tcp, 0.0.0.0:8984->5984/tcp   couchdb3
e4bb1ba70cf1        hyperledger/fabric-couchdb                                                                               "tini -- /docker-ent…"   21 minutes ago      Up 21 minutes       4369/tcp, 9100/tcp, 0.0.0.0:7984->5984/tcp   couchdb2
395b0d58e832        hyperledger/fabric-ca:latest                                                                             "sh -c 'fabric-ca-se…"   21 minutes ago      Up 21 minutes       7054/tcp, 0.0.0.0:8054->8054/tcp             ca_peerOrg2
```


### Admin 証明書/ユーザ証明書の取得

ホスト1, 2で実行する

* ホスト1側

javascriptフォルダに移動して、Wallet フォルダを削除
```
$ cd javascript
$ rm -rf wallet/
```

```
$ node enrollAdmin.js 

$ node registerUser.js 
```

* ホスト2側

javascriptフォルダに移動して、Wallet フォルダを削除
```
$ cd ~/fabric-samples/fabcar/javascript
$ rm -rf wallet/
```

```
$ node enrollAdmin.js 

$ node registerUser.js 
```


### query / invoke 実行

#### 初期値を確認

ホスト１, ２ ともに同じものとなっているか。

* ホスト1側

```
$ node query.js 
Wallet path: ~/fabcar/javascript/wallet
Transaction has been evaluated, result is: [{"Key":"CAR0", "Record":{"colour":"blue","make":"Toyota","model":"Prius","owner":"Tomoko"}},{"Key":"CAR1", "Record":{"colour":"red","make":"Ford","model":"Mustang","owner":"Brad"}},{"Key":"CAR2", "Record":{"colour":"green","make":"Hyundai","model":"Tucson","owner":"Jin Soo"}},{"Key":"CAR3", "Record":{"colour":"yellow","make":"Volkswagen","model":"Passat","owner":"Max"}},{"Key":"CAR4", "Record":{"colour":"black","make":"Tesla","model":"S","owner":"Adriana"}},{"Key":"CAR5", "Record":{"colour":"purple","make":"Peugeot","model":"205","owner":"Michel"}},{"Key":"CAR6", "Record":{"colour":"white","make":"Chery","model":"S22L","owner":"Aarav"}},{"Key":"CAR7", "Record":{"colour":"violet","make":"Fiat","model":"Punto","owner":"Pari"}},{"Key":"CAR8", "Record":{"colour":"indigo","make":"Tata","model":"Nano","owner":"Valeria"}},{"Key":"CAR9", "Record":{"colour":"brown","make":"Holden","model":"Barina","owner":"Shotaro"}}]
```

* ホスト2側

```
$ node query.js 
Wallet path: ~/fabcar/javascript/wallet
Transaction has been evaluated, result is: [{"Key":"CAR0", "Record":{"colour":"blue","make":"Toyota","model":"Prius","owner":"Tomoko"}},{"Key":"CAR1", "Record":{"colour":"red","make":"Ford","model":"Mustang","owner":"Brad"}},{"Key":"CAR2", "Record":{"colour":"green","make":"Hyundai","model":"Tucson","owner":"Jin Soo"}},{"Key":"CAR3", "Record":{"colour":"yellow","make":"Volkswagen","model":"Passat","owner":"Max"}},{"Key":"CAR4", "Record":{"colour":"black","make":"Tesla","model":"S","owner":"Adriana"}},{"Key":"CAR5", "Record":{"colour":"purple","make":"Peugeot","model":"205","owner":"Michel"}},{"Key":"CAR6", "Record":{"colour":"white","make":"Chery","model":"S22L","owner":"Aarav"}},{"Key":"CAR7", "Record":{"colour":"violet","make":"Fiat","model":"Punto","owner":"Pari"}},{"Key":"CAR8", "Record":{"colour":"indigo","make":"Tata","model":"Nano","owner":"Valeria"}},{"Key":"CAR9", "Record":{"colour":"brown","make":"Holden","model":"Barina","owner":"Shotaro"}}]
```

#### ホスト1で、invoke実行

```
$ node invoke.js 
Wallet path: ~/fabric-samples/fabcar/javascript/wallet
Transaction has been submitted
```

query で確認

ホスト1,2 ともに CAR12が追加されている

* ホスト1側

```
$ node query.js 
Wallet path: ~/fabric-samples/fabcar/javascript/wallet
Transaction has been evaluated, result is: [{"Key":"CAR0", "Record":{"colour":"blue","make":"Toyota","model":"Prius","owner":"Tomoko"}},{"Key":"CAR1", "Record":{"colour":"red","make":"Ford","model":"Mustang","owner":"Brad"}},{"Key":"CAR12", "Record":{"colour":"Black","make":"Honda","model":"Accord","owner":"Tom"}},{"Key":"CAR2", "Record":{"colour":"green","make":"Hyundai","model":"Tucson","owner":"Jin Soo"}},{"Key":"CAR3", "Record":{"colour":"yellow","make":"Volkswagen","model":"Passat","owner":"Max"}},{"Key":"CAR4", "Record":{"colour":"black","make":"Tesla","model":"S","owner":"Adriana"}},{"Key":"CAR5", "Record":{"colour":"purple","make":"Peugeot","model":"205","owner":"Michel"}},{"Key":"CAR6", "Record":{"colour":"white","make":"Chery","model":"S22L","owner":"Aarav"}},{"Key":"CAR7", "Record":{"colour":"violet","make":"Fiat","model":"Punto","owner":"Pari"}},{"Key":"CAR8", "Record":{"colour":"indigo","make":"Tata","model":"Nano","owner":"Valeria"}},{"Key":"CAR9", "Record":{"colour":"brown","make":"Holden","model":"Barina","owner":"Shotaro"}}]
```

* ホスト2側

```
$ node query.js 
Wallet path: ~/fabric-samples/fabcar/javascript/wallet
Transaction has been evaluated, result is: [{"Key":"CAR0", "Record":{"colour":"blue","make":"Toyota","model":"Prius","owner":"Tomoko"}},{"Key":"CAR1", "Record":{"colour":"red","make":"Ford","model":"Mustang","owner":"Brad"}},{"Key":"CAR12", "Record":{"colour":"Black","make":"Honda","model":"Accord","owner":"Tom"}},{"Key":"CAR2", "Record":{"colour":"green","make":"Hyundai","model":"Tucson","owner":"Jin Soo"}},{"Key":"CAR3", "Record":{"colour":"yellow","make":"Volkswagen","model":"Passat","owner":"Max"}},{"Key":"CAR4", "Record":{"colour":"black","make":"Tesla","model":"S","owner":"Adriana"}},{"Key":"CAR5", "Record":{"colour":"purple","make":"Peugeot","model":"205","owner":"Michel"}},{"Key":"CAR6", "Record":{"colour":"white","make":"Chery","model":"S22L","owner":"Aarav"}},{"Key":"CAR7", "Record":{"colour":"violet","make":"Fiat","model":"Punto","owner":"Pari"}},{"Key":"CAR8", "Record":{"colour":"indigo","make":"Tata","model":"Nano","owner":"Valeria"}},{"Key":"CAR9", "Record":{"colour":"brown","make":"Holden","model":"Barina","owner":"Shotaro"}}]
```

#### ホスト2で、invoke実行

```
$ node invoke.js 
Wallet path: ~/fabric-samples/fabcar/javascript/wallet
Transaction has been submitted
```

query で確認

ホスト1,2 ともに CAR13が追加されている

* ホスト1側

```
$ node query.js 
Wallet path: ~/fabric-samples/fabcar/javascript/wallet
Transaction has been evaluated, result is: [{"Key":"CAR0", "Record":{"colour":"blue","make":"Toyota","model":"Prius","owner":"Tomoko"}},{"Key":"CAR1", "Record":{"colour":"red","make":"Ford","model":"Mustang","owner":"Brad"}},{"Key":"CAR12", "Record":{"colour":"Black","make":"Honda","model":"Accord","owner":"Tom"}},{"Key":"CAR13", "Record":{"colour":"Black","make":"Honda","model":"Accord","owner":"Tom"}},{"Key":"CAR2", "Record":{"colour":"green","make":"Hyundai","model":"Tucson","owner":"Jin Soo"}},{"Key":"CAR3", "Record":{"colour":"yellow","make":"Volkswagen","model":"Passat","owner":"Max"}},{"Key":"CAR4", "Record":{"colour":"black","make":"Tesla","model":"S","owner":"Adriana"}},{"Key":"CAR5", "Record":{"colour":"purple","make":"Peugeot","model":"205","owner":"Michel"}},{"Key":"CAR6", "Record":{"colour":"white","make":"Chery","model":"S22L","owner":"Aarav"}},{"Key":"CAR7", "Record":{"colour":"violet","make":"Fiat","model":"Punto","owner":"Pari"}},{"Key":"CAR8", "Record":{"colour":"indigo","make":"Tata","model":"Nano","owner":"Valeria"}},{"Key":"CAR9", "Record":{"colour":"brown","make":"Holden","model":"Barina","owner":"Shotaro"}}]
```

* ホスト2側

```
$ node query.js 
Wallet path: ~/fabric-samples/fabcar/javascript/wallet
Transaction has been evaluated, result is: [{"Key":"CAR0", "Record":{"colour":"blue","make":"Toyota","model":"Prius","owner":"Tomoko"}},{"Key":"CAR1", "Record":{"colour":"red","make":"Ford","model":"Mustang","owner":"Brad"}},{"Key":"CAR12", "Record":{"colour":"Black","make":"Honda","model":"Accord","owner":"Tom"}},{"Key":"CAR13", "Record":{"colour":"Black","make":"Honda","model":"Accord","owner":"Tom"}},{"Key":"CAR2", "Record":{"colour":"green","make":"Hyundai","model":"Tucson","owner":"Jin Soo"}},{"Key":"CAR3", "Record":{"colour":"yellow","make":"Volkswagen","model":"Passat","owner":"Max"}},{"Key":"CAR4", "Record":{"colour":"black","make":"Tesla","model":"S","owner":"Adriana"}},{"Key":"CAR5", "Record":{"colour":"purple","make":"Peugeot","model":"205","owner":"Michel"}},{"Key":"CAR6", "Record":{"colour":"white","make":"Chery","model":"S22L","owner":"Aarav"}},{"Key":"CAR7", "Record":{"colour":"violet","make":"Fiat","model":"Punto","owner":"Pari"}},{"Key":"CAR8", "Record":{"colour":"indigo","make":"Tata","model":"Nano","owner":"Valeria"}},{"Key":"CAR9", "Record":{"colour":"brown","make":"Holden","model":"Barina","owner":"Shotaro"}}]
```

これで、動作確認はOKです。

---

## 今後やりたいこと
より実用に近いシステムとするため、以下の対応も今後継続していきたい。  
* orderer の raft化
orderer が、org1 のみとなっているため、org2 側にも分散させる。solo では、複数orderer に対応できないので、raft化も一緒に対応
* 証明書の配布の最適化
現状は、org1でpeerなどの証明書をまとめて生成している。org2 側の秘密鍵なども含めて一緒に生成しているため、各orgの証明書は、各orgで生成し、公開鍵を配布するようにする。
* 起動shell の最適化
現状は、org1側から、org2側の起動および、chaincode のinstall を行っている。  
各org毎に起動するように変更。  
ただし、chaincode は各orgで同期を取ってインストールおよびインスタンス化する必要があるので、どちらかのorgから投入するようにする。
* 名前解決のDNS化
hostsでの静的な解決の場合は、今後org追加時などの対応が困難になる。DNSで名前の解決ができるようにする。
* orgの追加
org3 を別のホストに追加する場合の手順追加。