av9ash/GitBugs huggingface dataset:
Bug: Optimize table.jsp code
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/Jun/20 09:15
Affects Version/s: 3.0.0-alpha-1

# Rename buildRegionServerLink to buildRegionLink, for it returns the link of region, not regionserver.
 # Add buildRegionDeployedServerTag and moreRegionsToRender, this two function can be used everywhere in table.jsp, and with them we can reduce a lot of same code.
 # Fix bug which locality stats tab can not sort. This is because we set wrong id for locality stats tab.
 # We use too much "if (withReplica)..." statements, remove it and let the page always show the replica id of regions. This can help to keep the code simple and the UI style union.
 # When there is no primary region on regionserver, show 0 instead of null
Bug: MetricsTable#updateFlushTime is wrong
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 13/Apr/20 10:01
Affects Version/s: 2.2.1

MetricsRegionServer does an update on the MetricsRegionServerSource, MetricsTable etc.
While doing updateFlushTime, the time taken for flush is rightly updated in the RegionServerSource but at the MetricsTable level we update the memstoresize instead of the time.
This applies from 1.1 version onwards.
Bug: Add support to run LoadIncrementalHFiles in a distributed manner
Status: Patch Available | Priority: Minor | Resolution: 
Created: 11/Jun/20 14:04
Affects Version/s: 1.4.0

LoadIncrementalHFiles takes a very long time to complete when running HBase on top of S3 and attempting to bulkload 500K-700K files.

The root cause of this is a combination of the higher latency of S3 (as compared to HDFS) as well as the calls made by LoadIncrementalHFiles to the underlying filesystem(each file is opened, seeked to the trailer offset at the end, and then the trailer is read).

Increasing the parallelism does not yield any significant improvement. This seems to stem from the fact that once the trailer is read the stream is not consumed to the end. This causes the underlying HTTP connection to be aborted and it cannot be re-used.

 

The proposed solution would be to also add support to run LoadIncrementalHFiles on multiple machines as a map reduce job.
Bug: Moving the meta region causes MetricsException when using above 2.6.0 hadoop version
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 20/May/20 02:48
Affects Version/s: 1.4.13

When the meta region moved out from a region server then moved in to the one again, we got MetricsException below and the metric source for coprocessor couldn't be re-registered. it happened with hadoop 2.8.5
{noformat}
2020-04-21 15:13:02,956 WARN  [HBase-Metrics2-1] util.MBeans: Error creating MBean object name: Hadoop:service=HBase,name=RegionServer,sub=Coprocessor.Region.C
P_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint
org.apache.hadoop.metrics2.MetricsException: org.apache.hadoop.metrics2.MetricsException: Hadoop:service=HBase,name=RegionServer,sub=Coprocessor.Region.CP_org.
apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint already exists!
    at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:127)
    at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:102)
    at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:121)
    at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:64)
    at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:222)
    at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:100)
    at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:268)
    at org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:239)
    at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:320)
    at com.sun.proxy.$Proxy10.postStart(Unknown Source)
    at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:193)
    at org.apache.hadoop.metrics2.impl.JmxCacheBuster$JmxCacheBusterRunnable.run(JmxCacheBuster.java:109)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834) 
{noformat}
Bug: JsonBean representation of metrics at /jmx endpoint now quotes all numbers
Status: Closed | Priority: Minor | Resolution: Fixed
Created: 29/Jun/20 19:53
Affects Version/s: 1.3.6

HBASE-20571 had a fix to look for NaN or Infinity in numbers, and to quote those as strings. The order of the `if-else` block is different in branch-1 (https://github.com/apache/hbase/commit/2d493556f3c8ae87fb92422b525bf7c9345e6ccd) and branch-2 (https://github.com/apache/hbase/commit/39ea1efa885e2f27f41af59228e0a12c4ded08f8)

HBASE-23015 changed the JsonBean.java code in a meaningful way, and the order of the changes were consistent between branch-1 ([https://github.com/apache/hbase/commit/f77c14d18150f55ee892f8d24a5ee231c1ae7e20#diff-87e9e2722b9210eebfd8c820c5d72a46L319-L324]) and branch-2 ([https://github.com/apache/hbase/commit/761aef6d9d0b8a455842de4d5eac7d9486f00633#diff-2c8f5dd222141c69112c5c5b5f70cf55R319-R324]) Unfortunately, they need to be reversed since the order is different between branch-1 and branch-2.
Bug: Remove hbase-native-client code from the master branch
Status: Closed | Priority: Minor | Resolution: Fixed
Created: 24/May/20 02:12

Per recent discussions in the slack channel, there is not much value
in having this code in the master branch. This code has it's dedicated
git repo at https://github.com/apache/hbase-native-client.

It was already removed from branch-1 (HBASE-19419) and branch-2
(HBASE-19176)
Bug: Bring back hedged reads metrics to branch-1
Status: Closed | Priority: Minor | Resolution: Fixed
Created: 26/May/20 09:38
Affects Version/s: 1.4.13

Seems at some point, the hedged metrics were added to MetricsRegionServerWrapperImpl (they were added without mentioning Jira ticket, but may be it was https://issues.apache.org/jira/browse/HBASE-12220 ?)

 

However, because of compilation issue, it was reverted: https://issues.apache.org/jira/browse/HBASE-12246

 

And never re-applied for branch-1: https://issues.apache.org/jira/browse/HBASE-12246?focusedCommentId=16425102&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16425102
Bug: The store file open and close thread pool should be shared at the region level
Status: Resolved | Priority: Minor | Resolution: Won't Fix
Created: 26/May/20 15:06

For now, we provide threads per column family evenly in general, but  there are some cases that some column families have much more store files than others( maybe that's the life, right? ). So in that case, some Stores have beed done quickly while others are struggling.We should share the thread pool at the region level in case of data skew.
Bug: Delete old hbase-thirdparty releases from dist.a.o
Status: Resolved | Priority: Minor | Resolution: Done
Created: 30/Apr/20 08:12

On dist.apache.org only the latest releases should be available.

Currently these are available there:

hbase-thirdparty-3.0.0/
hbase-thirdparty-3.1.1/
hbase-thirdparty-3.2.0/
hbase-thirdparty-3.3.0/
Bug: Break long region names in the web UI
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 21/Jun/20 12:08
Affects Version/s: 3.0.0-alpha-1

Before this patch, when it comes to the long region name, the UI content will be out of the screen, making it unreadable. Like this:

!image-2020-06-21-20-18-37-041.png|width=542,height=50!

!image-2020-06-21-20-19-25-183.png|width=531,height=23!

!image-2020-06-21-20-20-02-782.png|width=542,height=146!

 

After this patch, the long region name wil be break to the new line, like this:

!image-2020-06-21-20-27-23-474.png|width=529,height=35!

!image-2020-06-21-20-28-36-464.png|width=533,height=33!

!image-2020-06-21-20-29-07-819.png|width=531,height=117!
Bug: connection#close throws NPE
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 16/Jun/20 12:19
Affects Version/s: 2.2.3

In the ConnectionImplementation construction method, if getRegistry throws exception， registry will be null，and then close will throw NPE
{code:java}
try {
  this.registry = AsyncRegistryFactory.getRegistry(conf);
  ...
} catch (Throwable e) {
  // avoid leaks: registry, rpcClient, ...
  LOG.debug("connection construction failed", e);
  close();
  throw e;
}
{code}
Bug: master-status UI make date type fields sortable
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 28/Jun/20 16:00
Affects Version/s: 2.1.5

Revisit of HBASE-21207, HBASE-22543

date type values such as regionserver list 'Start time' field on master-status page, are not sorted by time.

HBASE-21207, HBASE-22543 missed it. so before this fix, date sorted as String.

The first field of it is 'day'. therefore always Friday goes first Wednesday goes last, no matter what date it is.

   * SCREEN_SHOT1.png

 

this fix make date type values sorted by time and date.
Bug: Clone snapshot with optional table level property overrides
Status: Open | Priority: Minor | Resolution: 
Created: 19/Jun/20 01:47
Affects Version/s: 1.6.0

We have this use case where we try to clone from snapshots of a table with replication enabled. Cloning process just reads the TableDescriptor from the manifest and uses it for the new target table.

In our case, the moment we clone it, replication kicks in and attempts to replicate table to peers but since the target table doesn't exist, it keeps throwing TNFE in a loop. I propose to add additional *optional* parameters to clone_snapshot that takes the target table property overrides (table leve/cf level).

For example:

{noformat}
clone_snapshot 'snapshotName', 'tableName'    # default
clone_snapshot 'snapshotName', 'tableName', {NAME => 'f1', REPLICATION_SCOPE => 0 }
clone_snapshot 'snapshotName, 'tableName', {MERGE_ENABLED => false}
.....
{noformat}
 

Implementation wise, this just adds a {{ModifyTableProcedure}} as a child procedure (if args are provided) for {{CloneSnapshotProcedure}} after {{CLONE_SNAPSHOT_ADD_TO_META}}, so we reuse most of the "modifyTable" code. 

Also, since clone puts the table in a disabled state, using "ModifyTableProcedure" shouldn't cause any issues.

 Thoughts? Is there any easier way to do it?
Bug: [branch-1] Support ZooKeeper 3.6.0+
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 09/Jun/20 17:10

Fix compilation issues against ZooKeeper 3.6.0. Backwards compatible changes with 3.4 and 3.5. Tested with:

{{  mvn clean install -Dtest=org.apache.hadoop.hbase.zookeeper.**}}

{{  mvn clean install -Dzookeeper.version=3.5.8 -Dtest=org.apache.hadoop.hbase.zookeeper.**}}

{{  mvn clean install -Dzookeeper.version=3.6.0 -Dtest=org.apache.hadoop.hbase.zookeeper.**}}
Bug: Add more details when print CompactionConfiguration info
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 14/May/20 07:27

Now the CompactionConfiguration info logs look like
{code}
2020-05-14 15:32:45,468 INFO  [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16020] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory
2020-05-14 15:32:45,468 INFO  [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16020] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory
2020-05-14 15:32:45,469 INFO  [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16020] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory
2020-05-14 15:32:45,469 INFO  [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16020] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory
{code}

We don't have any details on which region and which column family the CompactionConfiguration for. It would be better to include the region name and column family detail also in this log.
Bug: Region move to different rsgroup server should fail fast.
Status: Open | Priority: Minor | Resolution: 
Created: 23/Apr/20 05:28

When I tried to move a region (which was the part of the Table T1 present in RSG1 rsgroup) to RegionServer of another RSGroup RSG2, the move was not successful and yet the output was "Took 3.4 seconds" instead of an error. Also, during the move, the master logs did not show the destination address as the RS address, it showed localhost which I think is misleading.

 

STEPS TO RETRACE THE SCENARIO:
 # Create a RSGroup 'RSG1' and add region servers into it.
 # Create another RSGroup 'RSG2' and add region servers into it also.
 # Create a table 't1' with multiple regions and move it to rsgroup 'RSG1'.
 # Now try to move a region of table t1 from 'RSG1' to region server of 'RSG2'.
Bug: REST and Thrift server do not handle the "doAs" parameter case insensitively
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 27/Apr/20 15:46

Hadoop does a case-insensitve comparison on the doAs parameter name when handling principal impersonation.

The HBase Rest and Thrift servers do not do that, they only accept the "doAs" form.

According to HADOOP-11083, the the correct Hadoop behaviour is accepting doAs in any case.
Bug: Unsupported rollback should not be logged as CODE-BUG 
Status: Resolved | Priority: Minor | Resolution: Cannot Reproduce
Created: 17/Apr/20 15:02

When executing rollback from a state that does not support rollback functionality UnsupportedOperationException is thrown. The Procedure framework does not handle this case specially and logs an ERROR message because it is caught as a Throwable.

This scenario should have a cleaner log message.
{noformat}
2020-04-08 20:57:10,137 ERROR org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught runtime exception for pid=14791, state=FAILED:SERVER_CRASH_START, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via AssignProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: 6f48ff190e182c204056eb8b227ff6e8 owned by pid=9419, CANNOT run 'this' (pid=14797).; ServerCrashProcedure server=example.com,16020,1584086541944, splitWal=true, meta=false
java.lang.UnsupportedOperationException: unhandled state=SERVER_CRASH_GET_REGIONS
        at org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.rollbackState(ServerCrashProcedure.java:265)
        at org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.rollbackState(ServerCrashProcedure.java:59)
        at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
        at org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:982)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1663)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1595)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1438)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1200(ProcedureExecutor.java:78)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:2058)
 {noformat}
Bug: Contribute a Test class that shows some examples for using the Async Client API
Status: Resolved | Priority: Minor | Resolution: Incomplete
Created: 27/May/20 20:04

类似于  [https://github.com/apache/hbase/blob/master/hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/AsyncClientExample.java]

但最初是以测试的形式进行，以使验证和环境变得容易。

这基本上是一些示例，说明如何将CompletableFuture API与Async Client一起使用-鉴于CompletableFuture API的表达力和大小，从头开始做一个新手可能会有些痛苦，但是使用其他示例代码要容易得多建立或修补。
Bug: HBase Rest: Request for region detail of a table which doesn't exits is success(200 success code) instead of 404
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 01/Jun/20 04:51
Affects Version/s: 2.2.3

Request for region detail of a table which doesn't exits is success(200 success code) however response doesn't have any region data.

However response for a request of schema detail for a table which doesn't exits, is 404(Not found)
Bug: [Hadoop3.3] Update license for org.ow2.asm:*
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 24/Apr/20 18:03

Hadoop 3.3 brings a few Jetty dependencies which transitively brings in org.ow2.asm:asm-analysis, org.ow2.asm:asm-commons, org.ow2.asm:asm-tree.

When testing with the latest Jetty (9.4.26.v20200117) I found its org.ow2.asm:* updated from 7.1 to 7.2, which changed the declared license from "BSD" to "BSD-3-Clause License" (The actual license text did not change). The HBase's license checker doesn't accept it.

File the jira to update it to "BSD 3-Clause License" so that HBase can build.
{noformat}
[INFO] |  |  |  +- org.eclipse.jetty.websocket:javax-websocket-server-impl:jar:9.4.26.v20200117:test
[INFO] |  |  |  |  +- org.eclipse.jetty:jetty-annotations:jar:9.4.26.v20200117:test
[INFO] |  |  |  |  |  +- org.eclipse.jetty:jetty-plus:jar:9.4.26.v20200117:test
[INFO] |  |  |  |  |  |  \- org.eclipse.jetty:jetty-jndi:jar:9.4.26.v20200117:test
[INFO] |  |  |  |  |  \- org.ow2.asm:asm-commons:jar:7.2:test
[INFO] |  |  |  |  |     +- org.ow2.asm:asm-tree:jar:7.2:test
[INFO] |  |  |  |  |     \- org.ow2.asm:asm-analysis:jar:7.2:test
{noformat}
{noformat}
This product includes asm-analysis licensed under the BSD-3-Clause.

ERROR: Please check ^^^^^^^^^^^^ this License for acceptability here:

https://www.apache.org/legal/resolved

If it is okay, then update the list named 'non_aggregate_fine' in the LICENSE.vm file.
If it isn't okay, then revert the change that added the dependency.

More info on the dependency:

<groupId>org.ow2.asm</groupId>
<artifactId>asm-analysis</artifactId>
<version>7.2</version>
{noformat}
Bug: Update PolicyBasedChaosMonkey to handle uncaught exceptions
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 29/Jun/20 23:32

Running {{ServerKillingChaosMonkey}} via {{RESTApiClusterManager}} for any duration of time slowly leaks region servers. I see failures on the RESTApi side go unreported on the ChaosMonkey side. It seems like {{RuntimeExceptions}} are being thrown and lost.

{{PolicyBasedChaosMonkey}} uses a primitive means of thread management anyway. Update to use a thread pool, thread groups, and an uncaughtExceptionHandler.
Bug: Use a client property for SaslConnection MAX_ATTEMPTS (currently hardcoded to 5)
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 03/Jun/20 05:47
Affects Version/s: 1.3.2

The no.of retries for Sasl handshake is hardcoded to 5 (MAX_ATTEMPTS). Use cases which have strict SLAs on their operation timeouts need this to be configurable to meet their SLAs.

Sample Scenario:
 * RS is slow in accepting connections but is slow in responding to requests.
 * Client makes a Sasl Connection attempt but times out on read (can be configurable via hbase.ipc.client.socket.timeout.read) and retries 5 times at this time (this is not configurable) and eventually bails out.

Make this configurable by a new client property (say *hbase.security.relogin.maxretries)* so that clients/usecases can override and meet their timeout SLAs.
Bug: [Chaos Monkey] abstract logging through the class hierarchy
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 30/Apr/20 21:51
Affects Version/s: 2.3.0

Running chaos monkey and watching the logs, it's very difficult to tell what actions are actually running. There's lots of shared methods through the class hierarchy that extends from {{abstract class Action}}, and each class comes with its own {{Logger}}. As a result, the logs have useless stuff like

{noformat}
INFO actions.Action: Started regionserver...
{noformat}

Add {{protected abstract Logger getLogger()}} to the class's internal interface, and have the concrete implementations provide their logger.
Bug: CacheConfig details logged at Store open is not really useful
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 26/May/20 18:19
Affects Version/s: 3.0.0-alpha-1

CacheConfig constructor is logging 'this' object at INFO level. This log comes during Store open(As CacheConfig instance for that store is created). As the log is at CacheConfig only, we don't get to know this is for which region:store. So not really useful log.
{code}
blockCache=org.apache.hadoop.hbase.io.hfile.CombinedBlockCache@7bc02941, cacheDataOnRead=true, cacheDataOnWrite=true, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
{code}
Also during every compaction also this logs keeps coming. This is because during compaction we create new CacheConfig based on the HStore level CacheConfig object.  We can avoid this log with every compaction happening.
Bug: SyncTable logging improvements
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 09/Jun/20 10:29
Affects Version/s: 2.2.5

While troubleshooting mismatches in replication deployment, SyncTable logging can provide some insights on what is diverging between two clusters. One caveat, though, is that it logs diverging row key as hexdecimal values, which is not so useful for operators trying to figure out which rows are mismatching, ideally, this info should be human readable, so that operators could have the exact value they could use for querying the tables with other tools, such as hbase shell.

Another issue is that while rows mismatches are logged as info, cell values mismatches are only logged as debug. In general, any of the mismatches would already be quite verbose, so ideally both should be logged in debug level.