Repository: influxdata/influxdb-java (java)

Issue:
Client-side skipping of NaN values
A bit related to #537, but instead of validation I'd like to propose skipping NaNs to silence Telegraf/InfluxDB logs - they can't be reliably filtered, and some libraries (i.e. Kafka) use NaNs in their metric values.

Currently InfluxDB treats NaNs as invalid input, performs partial write and logs a warning. `InfluxDB#write` finishes with `InfluxDBException.UnableToParseException`.

I'd like to discuss having an option to silently drop invalid points during writes, or a clarification that clients shouldn't pass invalid values to points, so that logging in Apache Flink (apache/flink#8513) can be fixed in some way.

Pull request description:
Fix #614: drop NaN and infinity values from fields
Fox for #614 - drop NaN and +/- infinity values from fields while serializing Point to line protocol. They are unsupported by InfluxDB and all issues relating to them are old and got no progress.

I went with the simplest solution, assuming that if these values will be supported in the future then probably a change in driver will be required, unless InfluxDB aligns exactly with current Java serialization (NaN to U+FFFD - invalid character, and infinity to `∞`).

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index dab7799..85aef0a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,8 @@
 ### Fixes
 
 - Add new annotation called TimeColumn for timestamp field in POJO bean, this can set Point time and precision field correctly, also avoid UnableToParseException when flush Point to influx.
+- Skip fields with NaN and infinity values when writing to InfluxDB
+  [Issue #614](https://github.com/influxdata/influxdb-java/issues/614)
 
 ## 2.15 [2019-02-22]
 
diff --git a/src/main/java/org/influxdb/dto/Point.java b/src/main/java/org/influxdb/dto/Point.java
index b1385ac..2bdb418 100644
--- a/src/main/java/org/influxdb/dto/Point.java
+++ b/src/main/java/org/influxdb/dto/Point.java
@@ -405,13 +405,15 @@ public class Point {
   }
 
   /**
-   * calculate the lineprotocol entry for a single Point.
+   * Calculate the lineprotocol entry for a single Point.
+   * <p>
+   * NaN and infinity values are silently dropped as they are unsupported:
+   * https://github.com/influxdata/influxdb/issues/4089
    *
-   * Documentation is WIP : https://github.com/influxdb/influxdb/pull/2997
+   * @see <a href="https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_reference/">
+   *     InfluxDB line protocol reference</a>
    *
-   * https://github.com/influxdb/influxdb/blob/master/tsdb/README.md
-   *
-   * @return the String without newLine.
+   * @return the String without newLine, empty when there are no fields to write
    */
   public String lineProtocol() {
     return lineProtocol(null);
@@ -419,8 +421,15 @@ public class Point {
 
   /**
    * Calculate the lineprotocol entry for a single point, using a specific {@link TimeUnit} for the timestamp.
+   * <p>
+   * NaN and infinity values are silently dropped as they are unsupported:
+   * https://github.com/influxdata/influxdb/issues/4089
+   *
+   * @see <a href="https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_reference/">
+   *     InfluxDB line protocol reference</a>
+   *
    * @param precision the time precision unit for this point
-   * @return the String without newLine
+   * @return the String without newLine, empty when there are no fields to write
    */
   public String lineProtocol(final TimeUnit precision) {
 
@@ -431,7 +440,10 @@ public class Point {
 
     escapeKey(sb, measurement);
     concatenatedTags(sb);
-    concatenatedFields(sb);
+    int writtenFields = concatenatedFields(sb);
+    if (writtenFields == 0) {
+      return "";
+    }
     formatedTime(sb, precision);
 
     return sb.toString();
@@ -447,10 +459,11 @@ public class Point {
     sb.append(' ');
   }
 
-  private void concatenatedFields(final StringBuilder sb) {
+  private int concatenatedFields(final StringBuilder sb) {
+    int fieldCount = 0;
     for (Entry<String, Object> field : this.fields.entrySet()) {
       Object value = field.getValue();
-      if (value == null) {
+      if (value == null || isNotFinite(value)) {
         continue;
       }
       escapeKey(sb, field.getKey());
@@ -471,6 +484,8 @@ public class Point {
       }
 
       sb.append(',');
+
+      fieldCount++;
     }
 
     // efficiently chop off the trailing comma
@@ -478,6 +493,8 @@ public class Point {
     if (sb.charAt(lengthMinusOne) == ',') {
       sb.setLength(lengthMinusOne);
     }
+
+    return fieldCount;
   }
 
   static void escapeKey(final StringBuilder sb, final String key) {
@@ -505,6 +522,11 @@ public class Point {
     }
   }
 
+  private static boolean isNotFinite(final Object value) {
+    return value instanceof Double && !Double.isFinite((Double) value)
+            || value instanceof Float && !Float.isFinite((Float) value);
+  }
+
   private void formatedTime(final StringBuilder sb, final TimeUnit precision) {
     if (this.time == null) {
       return;


Test patch:
diff --git a/src/test/java/org/influxdb/dto/PointTest.java b/src/test/java/org/influxdb/dto/PointTest.java
index d1a9621..e8aec9c 100644
--- a/src/test/java/org/influxdb/dto/PointTest.java
+++ b/src/test/java/org/influxdb/dto/PointTest.java
@@ -483,6 +483,44 @@ public class PointTest {
       assertThat(tags).isEqualTo(correctOrder);
     }
 
+    @Test
+    public void lineProtocolSkippingOfNanFields() {
+		String lineProtocol;
+
+		lineProtocol = Point
+				.measurement("test")
+				.time(1, TimeUnit.MILLISECONDS)
+				.addField("float-valid", 1f)
+				.addField("float-nan", Float.NaN)
+				.addField("float-inf1", Float.NEGATIVE_INFINITY)
+				.addField("float-inf2", Float.POSITIVE_INFINITY)
+				.tag("host", "serverA")
+				.build()
+				.lineProtocol(TimeUnit.MILLISECONDS);
+		assertThat(lineProtocol).isEqualTo("test,host=serverA float-valid=1.0 1");
+
+		lineProtocol = Point
+				.measurement("test")
+				.time(1, TimeUnit.MILLISECONDS)
+				.addField("double-valid", 1d)
+				.addField("double-nan", Double.NaN)
+				.addField("double-inf1", Double.NEGATIVE_INFINITY)
+				.addField("double-inf2", Double.POSITIVE_INFINITY)
+				.tag("host", "serverA")
+				.build()
+				.lineProtocol(TimeUnit.MILLISECONDS);
+		assertThat(lineProtocol).isEqualTo("test,host=serverA double-valid=1.0 1");
+
+		lineProtocol = Point
+				.measurement("test")
+				.time(1, TimeUnit.MILLISECONDS)
+				.addField("double-nan", Double.NaN)
+				.tag("host", "serverA")
+				.build()
+				.lineProtocol(TimeUnit.MILLISECONDS);
+		assertThat(lineProtocol).isEqualTo("");
+    }
+
 
   @Test
   public void testAddFieldsFromPOJONullCheck() {