Repository: influxdata/influxdb-java
Pull request #773 (2021-09-13 05:40:44)

fix: continuous integration
1. Fixed sys env expansion in GitHub workflow
2. Fixed flaky tests:
     - `BatchOptionsTest.testJitterDuration`
     - `MessagePackBatchOptionsTest.testWriteWithRetryOnRecoverableError`
3. Fixed `MessagePackInfluxDBTest.testChunkingQueryPost` - MessagePack response doesn't contains last empty response

codecov-commenter: # [Codecov](https://codecov.io/gh/influxdata/influxdb-java/pull/773?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata) Report
> Merging [#773](https://codecov.io/gh/influxdata/influxdb-java/pull/773?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata) (8f8c660) into [master](https://codecov.io/gh/influxdata/influxdb-java/commit/920aa065129763e2aa706a782a19b0f80eaf8ffc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata) (920aa06) will **decrease** coverage by `0.03%`.
> The diff coverage is `n/a`.

> :exclamation: Current head 8f8c660 differs from pull request most recent head 5989497. Consider uploading reports for the commit 5989497 to get more accurate results
[![Impacted file tree graph](https://codecov.io/gh/influxdata/influxdb-java/pull/773/graphs/tree.svg?width=650&height=150&src=pr&token=QKip8tza8W&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata)](https://codecov.io/gh/influxdata/influxdb-java/pull/773?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata)

```diff
@@             Coverage Diff              @@
##             master     #773      +/-   ##
============================================
- Coverage     54.99%   54.95%   -0.04%     
  Complexity      390      390              
============================================
  Files            70       70              
  Lines          2575     2575              
  Branches        270      268       -2     
============================================
- Hits           1416     1415       -1     
  Misses         1093     1093              
- Partials         66       67       +1     
```


| [Impacted Files](https://codecov.io/gh/influxdata/influxdb-java/pull/773?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata) | Coverage Δ | |
|---|---|---|
| [src/main/java/org/influxdb/dto/Query.java](https://codecov.io/gh/influxdata/influxdb-java/pull/773/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata#diff-c3JjL21haW4vamF2YS9vcmcvaW5mbHV4ZGIvZHRvL1F1ZXJ5LmphdmE=) | `65.00% <0.00%> (-2.50%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/influxdata/influxdb-java/pull/773?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/influxdata/influxdb-java/pull/773?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata). Last update [920aa06...5989497](https://codecov.io/gh/influxdata/influxdb-java/pull/773?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=influxdata).

majst01: I have seen you pushed directly to master, please never do this !
bednar: @majst01 The GitHub Checks pass five times, so I think that the flaky tests are fixed.

Patch:
diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml
index d427485..3d7f399 100644
--- a/.github/workflows/pr.yml
+++ b/.github/workflows/pr.yml
@@ -12,7 +12,7 @@ jobs:
     strategy:
       matrix:
         jdk: [3-openjdk-16-slim, 3-jdk-14, 3-jdk-8-slim]
-        influxdb: [1.1, 1.6, 1.8, 2.0]
+        influxdb: ['1.1', '1.6', '1.8', '2.0']
 
     steps:
       - name: Checkout
@@ -23,6 +23,9 @@ jobs:
         run: '["${{ secrets.DOCKER_REGISTRY_TOKEN }}" == ""] && echo "::set-output name=is_fork_pr::true" || echo "::set-output name=is_fork_pr::false"'
 
       - name: Build project
+        env:
+          MAVEN_JAVA_VERSION: "${{ matrix.jdk }}"
+          INFLUXDB_VERSION: "${{ matrix.influxdb }}"
         run: ./compile-and-test.sh
 
       - name: codecov
diff --git a/src/main/java/org/influxdb/impl/BatchProcessor.java b/src/main/java/org/influxdb/impl/BatchProcessor.java
index 4430072..28c45b6 100644
--- a/src/main/java/org/influxdb/impl/BatchProcessor.java
+++ b/src/main/java/org/influxdb/impl/BatchProcessor.java
@@ -20,6 +20,7 @@ import java.util.concurrent.ThreadFactory;
 import java.util.concurrent.TimeUnit;
 import java.util.function.BiConsumer;
 import java.util.function.Consumer;
+import java.util.function.Supplier;
 import java.util.logging.Level;
 import java.util.logging.Logger;
 
@@ -46,6 +47,7 @@ public final class BatchProcessor {
   private final BatchWriter batchWriter;
   private boolean dropActionsOnQueueExhaustion;
   Consumer<Point> droppedActionHandler;
+  Supplier<Double> randomSupplier;
 
   /**
    * The Builder to create a BatchProcessor instance.
@@ -318,20 +320,21 @@ public final class BatchProcessor {
     } else {
         this.queue = new LinkedBlockingQueue<>();
     }
+    this.randomSupplier = Math::random;
 
     Runnable flushRunnable = new Runnable() {
       @Override
       public void run() {
         // write doesn't throw any exceptions
         write();
-        int jitterInterval = (int) (Math.random() * BatchProcessor.this.jitterInterval);
+        int jitterInterval = (int) (randomSupplier.get() * BatchProcessor.this.jitterInterval);
         BatchProcessor.this.scheduler.schedule(this,
                 BatchProcessor.this.flushInterval + jitterInterval, BatchProcessor.this.flushIntervalUnit);
       }
     };
     // Flush at specified Rate
     this.scheduler.schedule(flushRunnable,
-            this.flushInterval + (int) (Math.random() * BatchProcessor.this.jitterInterval),
+            this.flushInterval + (int) (randomSupplier.get() * BatchProcessor.this.jitterInterval),
             this.flushIntervalUnit);
   }
 


Test patch:
diff --git a/src/test/java/org/influxdb/BatchOptionsTest.java b/src/test/java/org/influxdb/BatchOptionsTest.java
index 351321b..7378746 100644
--- a/src/test/java/org/influxdb/BatchOptionsTest.java
+++ b/src/test/java/org/influxdb/BatchOptionsTest.java
@@ -7,13 +7,13 @@ import org.influxdb.dto.BatchPoints;
 import org.influxdb.dto.Point;
 import org.influxdb.dto.Query;
 import org.influxdb.dto.QueryResult;
-import org.jetbrains.annotations.NotNull;
+import org.influxdb.impl.BatchProcessor;
+import org.influxdb.impl.BatchProcessorTest;
 import org.junit.jupiter.api.Assertions;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.platform.runner.JUnitPlatform;
 import org.junit.runner.RunWith;
-import org.mockito.Mockito;
 import org.mockito.invocation.InvocationOnMock;
 import org.mockito.stubbing.Answer;
 
@@ -21,12 +21,15 @@ import static org.mockito.Mockito.*;
 
 import java.io.IOException;
 import java.text.MessageFormat;
+import java.util.ArrayList;
+import java.util.List;
 import java.util.concurrent.CountDownLatch;
 import java.util.concurrent.Executors;
 import java.util.concurrent.ThreadFactory;
 import java.util.concurrent.TimeUnit;
 import java.util.function.BiConsumer;
 import java.util.function.Consumer;
+import java.util.function.Supplier;
 
 
 @RunWith(JUnitPlatform.class)
@@ -234,15 +237,20 @@ public class BatchOptionsTest {
    * @throws InterruptedException
    */
   @Test
-  public void testJitterDuration() throws InterruptedException {
+  public void testJitterDuration() throws Exception {
 
     String dbName = "write_unittest_" + System.currentTimeMillis();
     try {
-      BatchOptions options = BatchOptions.DEFAULTS.flushDuration(100).jitterDuration(500);
+      // prepare points before start BatchProcessor
+      List<Point> points = prepareSomePoints(0, 19);
+      BatchOptions options = BatchOptions.DEFAULTS.flushDuration(100).jitterDuration(1000);
       influxDB.query(new Query("CREATE DATABASE " + dbName));
       influxDB.setDatabase(dbName);
       influxDB.enableBatch(options);
-      write20Points(influxDB);
+      BatchProcessor batchProcessor = BatchProcessorTest.getPrivateField(influxDB, "batchProcessor");
+      // random always return 1.0 to be sure that first query is null
+      BatchProcessorTest.setPrivateField(batchProcessor, "randomSupplier", (Supplier<Double>) () -> 1.0);
+      points.forEach(influxDB::write);
 
       Thread.sleep(100);
 
@@ -251,7 +259,7 @@ public class BatchOptionsTest {
       Assertions.assertNull(result.getResults().get(0).getError());
 
       //wait for at least one flush
-      Thread.sleep(1000);
+      Thread.sleep(1500);
       result = influxDB.query(new Query("select * from weather", dbName));
       Assertions.assertEquals(20, result.getResults().get(0).getSeries().get(0).getValues().size());
     }
@@ -596,6 +604,7 @@ public class BatchOptionsTest {
 
       verify(spy, times(2)).write(any(BatchPoints.class));
 
+      Thread.sleep(1_500);
       QueryResult result = influxDB.query(new Query("select * from m0", dbName));
       Assertions.assertNotNull(result.getResults().get(0).getSeries());
       Assertions.assertEquals(200, result.getResults().get(0).getSeries().get(0).getValues().size());
@@ -668,14 +677,7 @@ public class BatchOptionsTest {
   }
 
   void writeSomePoints(InfluxDB influxDB, int firstIndex, int lastIndex) {
-    for (int i = firstIndex; i <= lastIndex; i++) {
-      Point point = Point.measurement("weather")
-              .time(i,TimeUnit.HOURS)
-              .addField("temperature", (double) i)
-              .addField("humidity", (double) (i) * 1.1)
-              .addField("uv_index", "moderate").build();
-      influxDB.write(point);
-    }
+    prepareSomePoints(firstIndex, lastIndex).forEach(influxDB::write);
   }
 
   void write20Points(InfluxDB influxDB) {
@@ -686,6 +688,19 @@ public class BatchOptionsTest {
     writeSomePoints(influxDB, 0, n - 1);
   }
 
+  List<Point> prepareSomePoints(int firstIndex, int lastIndex) {
+    List<Point> points = new ArrayList<>();
+    for (int i = firstIndex; i <= lastIndex; i++) {
+      Point point = Point.measurement("weather")
+              .time(i, TimeUnit.HOURS)
+              .addField("temperature", (double) i)
+              .addField("humidity", (double) (i) * 1.1)
+              .addField("uv_index", "moderate").build();
+      points.add(point);
+    }
+    return points;
+  }
+
   private BatchPoints createBatchPoints(String dbName, String measurement, int n) {
     BatchPoints batchPoints = BatchPoints.database(dbName).build();
     for (int i = 1; i <= n; i++) {
diff --git a/src/test/java/org/influxdb/InfluxDBTest.java b/src/test/java/org/influxdb/InfluxDBTest.java
index 60dd666..38b5fcf 100644
--- a/src/test/java/org/influxdb/InfluxDBTest.java
+++ b/src/test/java/org/influxdb/InfluxDBTest.java
@@ -1330,7 +1330,7 @@ public class InfluxDBTest {
         batchPoints.point(point3);
         this.influxDB.write(batchPoints);
 
-        CountDownLatch countDownLatch = new CountDownLatch(3);
+        CountDownLatch countDownLatch = new CountDownLatch(2);
 
         Thread.sleep(2000);
         Query query = new Query("SELECT * FROM disk", dbName, true);
diff --git a/src/test/java/org/influxdb/impl/BatchProcessorTest.java b/src/test/java/org/influxdb/impl/BatchProcessorTest.java
index 213f6d3..df28da3 100644
--- a/src/test/java/org/influxdb/impl/BatchProcessorTest.java
+++ b/src/test/java/org/influxdb/impl/BatchProcessorTest.java
@@ -226,14 +226,27 @@ public class BatchProcessorTest {
       }
     }
 
+    @Test
+    @SuppressWarnings("unchecked")
+    public void randomSupplier() {
+      InfluxDB mockInfluxDB = mock(InfluxDBImpl.class);
+      BatchProcessor batchProcessor = BatchProcessor.builder(mockInfluxDB).actions(Integer.MAX_VALUE)
+                .interval(1, TimeUnit.NANOSECONDS).build();
+
+      Double random = batchProcessor.randomSupplier.get();
+      assertTrue(random >= 0);
+      assertTrue(random < 1);
+      Assertions.assertNotEquals(random, batchProcessor.randomSupplier.get());
+    }
+
     @SuppressWarnings("unchecked")
-    static <T> T getPrivateField(final Object obj, final String name) throws Exception {
+    public static <T> T getPrivateField(final Object obj, final String name) throws Exception {
       Field field = obj.getClass().getDeclaredField(name);
       field.setAccessible(true);
       return (T) field.get(obj);
     }
 
-    static void setPrivateField(final Object obj, final String name, final Object value) throws Exception {
+    public static void setPrivateField(final Object obj, final String name, final Object value) throws Exception {
       Field field = obj.getClass().getDeclaredField(name);
       field.setAccessible(true);
       field.set(obj, value);