Question: How do I resolve the error "Cache of region boundaries are out of date" when running an HBase read-replica Amazon EMR cluster with Phoenix?

When I try to connect to Apache HBase on an Amazon EMR read-replica cluster using Apache Phoenix, I get an error message similar to the following:Error: ERROR 1108 (XCL08): Cache of region boundaries are out of date. (state=XCL08,code=1108) org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108(XCL08): Cache of region boundaries are out of date.at org.apache.phoenix.exception.SQLExceptionCode$14.newException(SQLExceptionCode.java:365)at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)at org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:189)at org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:169)at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:140)

Short descriptionPhoenix tries to connect to the hbase:meta table by default. However, because the hbase:meta table belongs to the primary cluster, Phoenix can't to connect the read-replica cluster. To resolve this problem, modify hbase-site.xml to point to the hbase:meta_cluster-id table that belongs to the HBase read-replica cluster.

ResolutionBefore you begin, confirm that Phoenix is installed on the primary cluster as well as the read-replica cluster. Phoenix can't connect to HBase from a read-replica cluster if Phoenix isn't installed on the primary cluster.On a running cluster1.    Add the following configurations to the HBase configuration file ( /etc/phoenix/conf/hbase-site.xml) on the master node. Replace cluster-id with the ID of your read-replica cluster.<property>   <name>hbase.balancer.tablesOnMaster</name>   <value>hbase:meta</value></property><property>   <name>hbase.meta.table.suffix</name>   <value>cluster-id</value></property>2.    Restart the Phoenix service:For Amazon EMR release versions 5.29 and earlier:sudo stop phoenix-queryserversudo start phoenix-queryserverFor Amazon EMR release versions 5.30 and later:sudo systemctl stop phoenix-queryserver.servicesudo systemctl start phoenix-queryserver.serviceIn Amazon EMR release versions 5.21.0 and later, you can also make these configuration changes by overriding the cluster configuration for the master instance group:1.    Open the Amazon EMR console.2.    In the cluster list, choose the active read-replica cluster that you want to reconfigure.3.    Open the cluster details page for the cluster and go to Configurations tab.4.    In the Filter dropdown list, choose the master instance group.5.    In Reconfigure dropdown list, choose either Edit in table.6.    Choose Add configuration, and then add the following two configurations:Classification: phoenix-hbase-siteProperty: hbase.balancer.tablesOnMasterValue: hbase:metaClassification: phoenix-hbase-siteProperty: hbase.meta.table.suffixValue: ${emr.clusterId}7.    Choose Save changes.For more information about this process, see Reconfigure an instance group in the console.On a new clusterAdd a configuration object similar to the following when you launch a cluster using Amazon EMR release version 4.6.0 or later:[  {    "Classification": "phoenix-hbase-site",     "Configurations": [      ],     "Properties": {      "hbase.balancer.tablesOnMaster" : "hbase:meta",      "hbase.meta.table.suffix" : "${emr.clusterId}"      }  },  {    "Classification": "hbase-site",     "Configurations": [      ],     "Properties": {      "hbase.meta.table.suffix" : "${emr.clusterId}"      }  }]Related informationHBase on Amazon S3 (Amazon S3 storage mode)Apache Phoenix