Bug: [WASB] Retry not getting implemented when using wasb scheme in hadoop-azure 2.7.4 
Status: Open | Priority: Minor | Resolution: 
Created: 16/Mar/22 12:02
Affects Version/s: 2.7.4

I am using prestodb to read data from blob.
Presto is using  hadoop-azure-2.7.4 jar.

I'm using *wasb* scheme to query the data on blob. I'm afraid for some reason the hadoop-azure library is not retrying when getting IO exception.

Attaching the stack trace below,
{code:java}
com.facebook.presto.spi.PrestoException: Error reading from wasb://oemdpv3prd-v1@oemdpv3prd.blob.core.windows.net/data/pipelines/hudi/kafka/telemetrics_v2/dp.hmi.quectel.bms.data.packet.v2/dt=2022-01-15/e576abc3-942a-434d-be02-6899798258eb-0_5-13327-290407_20220115211203.parquet at position 65924529
    at com.facebook.presto.hive.parquet.HdfsParquetDataSource.readInternal(HdfsParquetDataSource.java:66)
    at com.facebook.presto.parquet.AbstractParquetDataSource.readFully(AbstractParquetDataSource.java:60)
    at com.facebook.presto.parquet.AbstractParquetDataSource.readFully(AbstractParquetDataSource.java:51)
    at com.facebook.presto.parquet.reader.ParquetReader.readPrimitive(ParquetReader.java:247)
    at com.facebook.presto.parquet.reader.ParquetReader.readColumnChunk(ParquetReader.java:330)
    at com.facebook.presto.parquet.reader.ParquetReader.readBlock(ParquetReader.java:313)
    at com.facebook.presto.hive.parquet.ParquetPageSource$ParquetBlockLoader.load(ParquetPageSource.java:182)
    at com.facebook.presto.hive.parquet.ParquetPageSource$ParquetBlockLoader.load(ParquetPageSource.java:160)
    at com.facebook.presto.common.block.LazyBlock.assureLoaded(LazyBlock.java:291)
    at com.facebook.presto.common.block.LazyBlock.getLoadedBlock(LazyBlock.java:282)
    at com.facebook.presto.operator.ScanFilterAndProjectOperator$RecordingLazyBlockLoader.load(ScanFilterAndProjectOperator.java:314)
    at com.facebook.presto.operator.ScanFilterAndProjectOperator$RecordingLazyBlockLoader.load(ScanFilterAndProjectOperator.java:300)
    at com.facebook.presto.common.block.LazyBlock.assureLoaded(LazyBlock.java:291)
    at com.facebook.presto.common.block.LazyBlock.getLoadedBlock(LazyBlock.java:282)
    at com.facebook.presto.operator.project.InputPageProjection.project(InputPageProjection.java:69)
    at com.facebook.presto.operator.project.PageProjectionWithOutputs.project(PageProjectionWithOutputs.java:56)
    at com.facebook.presto.operator.project.PageProcessor$ProjectSelectedPositions.processBatch(PageProcessor.java:323)
    at com.facebook.presto.operator.project.PageProcessor$ProjectSelectedPositions.process(PageProcessor.java:197)
    at com.facebook.presto.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:315)
    at com.facebook.presto.operator.WorkProcessorUtils$YieldingIterator.computeNext(WorkProcessorUtils.java:79)
    at com.facebook.presto.operator.WorkProcessorUtils$YieldingIterator.computeNext(WorkProcessorUtils.java:65)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
    at com.facebook.presto.operator.project.MergingPageOutput.getOutput(MergingPageOutput.java:113)
    at com.facebook.presto.operator.ScanFilterAndProjectOperator.processPageSource(ScanFilterAndProjectOperator.java:295)
    at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:242)
    at com.facebook.presto.operator.Driver.processInternal(Driver.java:418)
    at com.facebook.presto.operator.Driver.lambda$processFor$9(Driver.java:301)
    at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:722)
    at com.facebook.presto.operator.Driver.processFor(Driver.java:294)
    at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077)
    at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
    at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:599)
    at com.facebook.presto.$gen.Presto_0_256_2ed7f73____20220112_133056_1.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: undefined
    at com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.core.Utility.initIOException(Utility.java:598)
    at com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.blob.BlobInputStream.dispatchRead(BlobInputStream.java:264)
    at com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.blob.BlobInputStream.readInternal(BlobInputStream.java:448)
    at com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.blob.BlobInputStream.read(BlobInputStream.java:420)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
    at java.io.DataInputStream.read(DataInputStream.java:149)
    at org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsInputStream.read(NativeAzureFileSystem.java:735)
    at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:65)
    at org.apache.hadoop.fs.FSInputStream.readFully(FSInputStream.java:78)
    at org.apache.hadoop.fs.BufferedFSInputStream.readFully(BufferedFSInputStream.java:113)
    at org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:107)
    at com.facebook.presto.hive.parquet.HdfsParquetDataSource.readInternal(HdfsParquetDataSource.java:58)
    ... 36 more
Caused by: com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.StorageException: Blob hash mismatch (integrity check failed), Expected value is gRfj04hQvdOqOaAdqo+lGw==, retrieved 4V4j3H2KabAsssDyGnJN9w==.
    at com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.blob.CloudBlob$10.postProcessResponse(CloudBlob.java:1714)
    at com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.blob.CloudBlob$10.postProcessResponse(CloudBlob.java:1613)
    at com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:148)
    at com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.blob.CloudBlob.downloadRangeInternal(CloudBlob.java:1468)
    at com.facebook.presto.hadoop.$internal.com.microsoft.azure.storage.blob.BlobInputStream.dispatchRead(BlobInputStream.java:255)
    ... 47 more{code}
Please suggest a way to get the retry implemented.


I want the retry to be implemented if there is an IO exception while calling *read* method below, 
{code:java}
org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsInputStream.read(NativeAzureFileSystem.java:735){code}