Bug: When getting rowkey, the program enters the org.apache.hadoop.hbase.regionserver.KeyValueHeap#generalizedSeek and an endless loop occurs
Status: Open | Priority: Major | Resolution: 
Created: 28/Dec/21 14:17

1. Phenomenon description:

Get rowkey: 350ef10764bc4af960b6b468479aff1e_ The data of 10032279726411 is the master of the program card and enters the dead cycle

2. Data information:

hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/250853d176294585ae0339d11f9714c1
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:directory/1637459509892/Put/vlen=0/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_name/1637459509892/Put/vlen=0/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wareid/1637459509892/Put/vlen=14/seqid=0
hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/85ab3a19f409476aada64ab5bf906653
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:directory/1637373810660/Put/vlen=0/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_name/1637373810660/Put/vlen=0/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wareid/1637373810660/Put/vlen=14/seqid=0
hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/93cc194d62fc419792991613332b990d
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:directory/1637112759915/Put/vlen=0/seqid=1555467268
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:product_ext_nick/1637057318457/Put/vlen=27/seqid=1554882684
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:store_info_ex_pc/1637125253154/Put/vlen=10329/seqid=1555632639
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_name/1637112759915/Put/vlen=0/seqid=1555467268
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:warehouse/1637125253154/Put/vlen=17843/seqid=1555632639
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wareid/1637112759915/Put/vlen=14/seqid=1555467268
hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/a001e8a2dd454029ae38554b88018109
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:directory/1637027051359/Put/vlen=0/seqid=1554430204
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:dredisprice/1636877442107/Put/vlen=5/seqid=1552727948
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:lastupdatetime/1636829185081/Put/vlen=10/seqid=1552173184
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:store_info_ex_pc/1636979778239/Put/vlen=10297/seqid=1553901027
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_name/1637027051359/Put/vlen=0/seqid=1554430204
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:warehouse/1636979778239/Put/vlen=17299/seqid=1553901027
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wareid/1637027051359/Put/vlen=14/seqid=1554430204
hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/a62976c426574d43af4b5d3fef8af921
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:directory/1637545456304/Put/vlen=0/seqid=1556484345
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:product_ext_nick/1637629087047/Put/vlen=18/seqid=1556602639
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:union_spec_ids/1637582728941/Put/vlen=213/seqid=1556506804
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_name/1637545456304/Put/vlen=0/seqid=1556484345
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wareid/1637545456304/Put/vlen=14/seqid=1556484345
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wstate/1637549762970/Put/vlen=2/seqid=1556484528
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wyn/1637549762970/Put/vlen=1/seqid=1556484528
hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/ff8f38c495ba46398ae9c02207950f7d
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:directory/1636594256197/Put/vlen=0/seqid=1549353741
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:dredisprice/1636646590929/Put/vlen=5/seqid=1550021079
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:lastupdatetime/1635940338088/Put/vlen=10/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:product_ext_nick/1634995561817/Put/vlen=41/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:product_id/1623293068319/Put/vlen=14/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:sku_union_spec_ids/1623293068319/Put/vlen=213/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:store_info_ex_pc/1636191109189/Put/vlen=10297/seqid=1544129517
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:union_expand_ids/1623293068319/Put/vlen=0/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:union_spec_ids/1635935424287/Put/vlen=213/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:unlimit_catid/1623293068319/Put/vlen=4/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_id/1623293068319/Put/vlen=6/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_name/1636594256197/Put/vlen=0/seqid=1549353741
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:ware_type/1623293068319/Put/vlen=1/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:ware_weight/1623293068319/Put/vlen=3/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:warehouse/1636405561106/Put/vlen=17321/seqid=1546667844
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wareid/1636594256197/Put/vlen=14/seqid=1549353741
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:warename/1623293068319/Put/vlen=99/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:warepid/1623293068319/Put/vlen=14/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:waretitle/1627360205219/Put/vlen=54/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:winsdate/1623293068319/Put/vlen=10/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wpid2/1623293068319/Put/vlen=1/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wredisprice/1623293074488/Put/vlen=5/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wstate/1635418719233/Put/vlen=1/seqid=0
K: 350ef10764bc4af960b6b468479aff1e_10032279726411/d:wyn/1623293070700/Put/vlen=1/seqid=0

3. Abnormal process:

When executed to: d:vender_ name column ScanQueryMatcher.MatchCode qcode = matcher.match(cell); The return value is: seek_ NEXT_ COL，The method calling process is：

seekOrSkipToNextColumn-》seekAsDirection-》reseek-》heap.resee-》generalizedSeek

However, when the program executes the generalizedseek method, it cannot exit，At this time, the data information of heap is：

2021-12-28 19:02:26,801 INFO  [main] regionserver.StoreScanner(844): current:StoreFileScanner[HFileScanner for reader reader=[hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/93cc194d62fc419792991613332b990d], compression=lz4, cacheConf=cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, firstKey=Optional[34f5c0cb717779d9ede9a8b80e46d4d4_10039682866610/d:product_ext_nick/1637109574351/Put/seqid=0], lastKey=Optional[351eb49327ecd6099a11efe0d1347724_10020558744609/d:commentcount/1637134716442/Put/seqid=0], avgKeyLen=70, avgValueLen=76, entries=3831303, length=251907468, cur=350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_name/1637112759915/Put/vlen=0/seqid=1555467268]
2021-12-28 19:02:26,801 INFO  [main] regionserver.StoreScanner(847): keyValueScanner:StoreFileScanner[HFileScanner for reader reader=[hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/a001e8a2dd454029ae38554b88018109], compression=lz4, cacheConf=cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, firstKey=Optional[34f5c0c9f049e2fa0b0b18b50a4f8b82_10039161263881/d:product_ext_nick/1637024214199/Put/seqid=0], lastKey=Optional[351eb49327ecd6099a11efe0d1347724_10020558744609/d:commentcount/1637044327190/Put/seqid=0], avgKeyLen=71, avgValueLen=48, entries=9028492, length=456744125, cur=350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_name/1637027051359/Put/vlen=0/seqid=1554430204]
2021-12-28 19:02:26,801 INFO  [main] regionserver.StoreScanner(847): keyValueScanner:StoreFileScanner[HFileScanner for reader reader=[hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/ff8f38c495ba46398ae9c02207950f7d], compression=lz4, cacheConf=cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, firstKey=Optional[34f5c0c91665cb0a1a3360076a6294f2_10037000082814/d:averagescore/1631432545837/Put/seqid=0], lastKey=Optional[351eb49327ecd6099a11efe0d1347724_10020558744609/d:wyn/1626850561189/Put/seqid=0], avgKeyLen=69, avgValueLen=26, entries=539934002, length=17765279323, cur=350ef10764bc4af960b6b468479aff1e_10032279726411/d:vender_name/1636594256197/Put/vlen=0/seqid=1549353741]
2021-12-28 19:02:26,801 INFO  [main] regionserver.StoreScanner(847): keyValueScanner:StoreFileScanner[HFileScanner for reader reader=[hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/250853d176294585ae0339d11f9714c1], compression=lz4, cacheConf=cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, firstKey=Optional[34f5c0dcb467c26bd5355982cc01a095_43099352773/d:directory/1637459509892/Put/seqid=0], lastKey=Optional[351eb48a36179802a0f8b38452e26b7a_10039387550202/d:wareid/1637459509892/Put/seqid=0], avgKeyLen=68, avgValueLen=12, entries=336963, length=9094030, cur=350ef10764bc4af960b6b468479aff1e_10032279726411/d:wareid/1637459509892/Put/vlen=14/seqid=-1]
2021-12-28 19:02:26,801 INFO  [main] regionserver.StoreScanner(847): keyValueScanner:StoreFileScanner[HFileScanner for reader reader=[hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/a62976c426574d43af4b5d3fef8af921], compression=lz4, cacheConf=cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, firstKey=Optional[34f5c0cca42ad68aa73b72953336d214_10038013927618/d:catname/1637623282535/Put/seqid=0], lastKey=Optional[351eb49327ecd6099a11efe0d1347724_10020558744609/d:union_spec_ids/1637582728941/Put/seqid=0], avgKeyLen=71, avgValueLen=42, entries=18764748, length=899013217, cur=350ef10764bc4af960b6b468479aff1e_10032279726411/d:wareid/1637545456304/Put/vlen=14/seqid=1556484345]
2021-12-28 19:02:26,801 INFO  [main] regionserver.StoreScanner(847): keyValueScanner:StoreFileScanner[HFileScanner for reader reader=[hdfs://snake/hbase/data/realtime_search/WD_MERGED_DATA/2fc7bd24c7d6d8d500f04b2480660de9/d/85ab3a19f409476aada64ab5bf906653], compression=lz4, cacheConf=cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, firstKey=Optional[34f5c0dcb467c26bd5355982cc01a095_43099352773/d:directory/1637373810660/Put/seqid=0], lastKey=Optional[351eb48a36179802a0f8b38452e26b7a_10039387550202/d:wareid/1637373810660/Put/seqid=0], avgKeyLen=68, avgValueLen=12, entries=334692, length=9047191, cur=350ef10764bc4af960b6b468479aff1e_10032279726411/d:wareid/1637373810660/Put/vlen=14/seqid=-1]

Because cur of multiple storefilescanners in heap are the  d:vender_name  columns,So in the while loop of the generalizedSeek method，After entering the while loop, the comparator will be compared infinitely getComparator(). Compare (seekkey, topkey), the next cycle will not exit,and The value of the scan variable is not changing

!image-2021-12-28-22-17-36-088.png!