Bug: HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.4.0
Status: Open | Priority: Major | Resolution: 
Created: 08/Oct/24 10:05

Test env:

Hbase 2.5.10-hadoop3

Hadoop 3.4.0

 

*You can use the attached files to reproduce the issue.*

I have noticed two error message:

1) The first is that *_NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.beginFileLease(long, org.apache.hadoop.hdfs.DFSOutputStream)._* I think this is due to the incompatible change in Hadoop 3.4.0 change [https://github.com/apache/hadoop/pull/5536/files#diff-405edd18f9ff05d7ae8c25efef90266db403ecbe0e75ae8ce8fb68bb02f03041|https://github.com/apache/hadoop/pull/4813/files#diff-405edd18f9ff05d7ae8c25efef90266db403ecbe0e75ae8ce8fb68bb02f03041]

 
{code:java}
2024-10-08 06:08:27,401 ERROR [master/zbt-ubuntu:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper (FanOutOneBlockAsyncDFSOutputHelper.java:<clinit>(316)) - Couldn't properly initialize access to HDFS internals. Please update your WAL Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more information.
java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.beginFileLease(long, org.apache.hadoop.hdfs.DFSOutputStream)
    at java.lang.Class.getDeclaredMethod(Class.java:2130)
    at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createLeaseManager(FanOutOneBlockAsyncDFSOutputHelper.java:205)
    at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.<clinit>(FanOutOneBlockAsyncDFSOutputHelper.java:308)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140)
    at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:139)
    at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:179)
    at org.apache.hadoop.hbase.wal.WALFactory.<init>(WALFactory.java:207)
    at org.apache.hadoop.hbase.wal.WALFactory.<init>(WALFactory.java:186)
    at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:388)
    at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
    at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:949)
    at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2428)
    at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:566)
    at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
    at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
    at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:563)
    at java.lang.Thread.run(Thread.java:748){code}
 

 

2) The second error is *_java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper._* But i have added the hbase-asyncfs dependency in {*}pom.xml{*}, i don't figure out why there is this error?

 
{code:java}
2024-10-08 06:08:32,615 WARN  [RS_OPEN_META-regionserver/zbt-ubuntu:0-0] handler.AssignRegionHandler (AssignRegionHandler.java:handleException(173)) - Fatal error occurred while opening region hbase:meta,,1.1588230740, aborting...
java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper
    at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:54)
    at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:193)
    at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:180)
    at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:117)
    at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:696)
    at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:127)
    at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriterInternal(AbstractFSWAL.java:916)
    at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.lambda$rollWriter$8(AbstractFSWAL.java:948)
    at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
    at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:948)
    at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:582)
    at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:523)
    at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:160)
    at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
    at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:298)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:2393)
    at org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:149)
    at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748){code}