Repository: google/gson (Java)
Date: 2021-10-11 13:33:25+00:00
Task type: Bug Fix

Issue:
newJsonWriter not return a new JSON writer configured for the settings on this Gson instance!
Hi,

while working on my project with Retrofit, I found an issue with `newJsonWriter` method in Gson class.

I used `GsonConverterFactory` which is a wrapper class of Gson in Retrofit.

Gson enables to escape HTML characters as a default.

However, Retrofit doesn't encode the request body as the same as Gson does.

Here is GsonConverterFactory code from Retrofit. I can see there is no any other options it gives to gson.

```
public final class GsonConverterFactory extends Converter.Factory {
  /**
   * Create an instance using a default {@link Gson} instance for conversion. Encoding to JSON and
   * decoding from JSON (when no charset is specified by a header) will use UTF-8.
   */
  public static GsonConverterFactory create() {
    return create(new Gson());
  }

  /**
   * Create an instance using {@code gson} for conversion. Encoding to JSON and
   * decoding from JSON (when no charset is specified by a header) will use UTF-8.
   */
  @SuppressWarnings("ConstantConditions") // Guarding public API nullability.
  public static GsonConverterFactory create(Gson gson) {
    if (gson == null) throw new NullPointerException("gson == null");
    return new GsonConverterFactory(gson);
  }

  private final Gson gson;

  private GsonConverterFactory(Gson gson) {
    this.gson = gson;
  }

  @Override
  public Converter<ResponseBody, ?> responseBodyConverter(Type type, Annotation[] annotations,
      Retrofit retrofit) {
    TypeAdapter<?> adapter = gson.getAdapter(TypeToken.get(type));
    return new GsonResponseBodyConverter<>(gson, adapter);
  }

  @Override
  public Converter<?, RequestBody> requestBodyConverter(Type type,
      Annotation[] parameterAnnotations, Annotation[] methodAnnotations, Retrofit retrofit) {
    TypeAdapter<?> adapter = gson.getAdapter(TypeToken.get(type));
    return new GsonRequestBodyConverter<>(gson, adapter);
  }
}
```

```
final class GsonRequestBodyConverter<T> implements Converter<T, RequestBody> {
  private static final MediaType MEDIA_TYPE = MediaType.get("application/json; charset=UTF-8");
  private static final Charset UTF_8 = Charset.forName("UTF-8");

  private final Gson gson;
  private final TypeAdapter<T> adapter;

  GsonRequestBodyConverter(Gson gson, TypeAdapter<T> adapter) {
    this.gson = gson;
    this.adapter = adapter;
  }

  @Override public RequestBody convert(T value) throws IOException {
    Buffer buffer = new Buffer();
    Writer writer = new OutputStreamWriter(buffer.outputStream(), UTF_8);
    JsonWriter jsonWriter = gson.newJsonWriter(writer);
    adapter.write(jsonWriter, value);
    jsonWriter.close();
    return RequestBody.create(MEDIA_TYPE, buffer.readByteString());
  }
}

```
As above, it uses `gson.newJsonWriter(writer)`.

```
  /**
   * Returns a new JSON writer configured for the settings on this Gson instance.
   */
  public JsonWriter newJsonWriter(Writer writer) throws IOException {
    if (generateNonExecutableJson) {
      writer.write(JSON_NON_EXECUTABLE_PREFIX);
    }
    JsonWriter jsonWriter = new JsonWriter(writer);
    if (prettyPrinting) {
      jsonWriter.setIndent("  ");
    }
    jsonWriter.setSerializeNulls(serializeNulls);
    return jsonWriter;
  }
```

I look through `newJsonWriter`, but it does not apparently configure whole settings from Gson.

For example, it is `HtmlSafe` option in my case.

I guess HtmlSafe is garanteed as true in Gson, though, it can be false in JsonWriter when not using `setHtmlSafe(true)'.

Could you check this issue?


Issue discussion:
This problem takes me a whole afternoon to find out where make it authorization error.
This should really be fixed, but I have to admit that it worries me how many repercussions it might have for all projects currently using retrofit with GsonConverterFactory.create() to have an update at some point suddenly start escaping html characters when they are not used to having that happen because of this bug...

Of course, any sane json parser on the other end won't care either way, but you never know what brain-dead or broken parsers might exist on the other end.

But yeah, just a long way of saying, I really think this should be fixed, but I'm almost afraid to ask.
Hi, can you fix it？

Solution patch:
diff --git a/gson/src/main/java/com/google/gson/Gson.java b/gson/src/main/java/com/google/gson/Gson.java
--- a/gson/src/main/java/com/google/gson/Gson.java
+++ b/gson/src/main/java/com/google/gson/Gson.java
@@ -756,6 +756,15 @@ public void toJson(JsonElement jsonElement, Appendable writer) throws JsonIOExce
 
   /**
    * Returns a new JSON writer configured for the settings on this Gson instance.
+   *
+   * <p>The following settings are considered:
+   * <ul>
+   *   <li>{@link GsonBuilder#disableHtmlEscaping()}</li>
+   *   <li>{@link GsonBuilder#generateNonExecutableJson()}</li>
+   *   <li>{@link GsonBuilder#serializeNulls()}</li>
+   *   <li>{@link GsonBuilder#setLenient()}</li>
+   *   <li>{@link GsonBuilder#setPrettyPrinting()}</li>
+   * </ul>
    */
   public JsonWriter newJsonWriter(Writer writer) throws IOException {
     if (generateNonExecutableJson) {
@@ -765,12 +774,19 @@ public JsonWriter newJsonWriter(Writer writer) throws IOException {
     if (prettyPrinting) {
       jsonWriter.setIndent("  ");
     }
+    jsonWriter.setHtmlSafe(htmlSafe);
+    jsonWriter.setLenient(lenient);
     jsonWriter.setSerializeNulls(serializeNulls);
     return jsonWriter;
   }
 
   /**
    * Returns a new JSON reader configured for the settings on this Gson instance.
+   *
+   * <p>The following settings are considered:
+   * <ul>
+   *   <li>{@link GsonBuilder#setLenient()}</li>
+   * </ul>
    */
   public JsonReader newJsonReader(Reader reader) {
     JsonReader jsonReader = new JsonReader(reader);


Test patch:
diff --git a/gson/src/test/java/com/google/gson/GsonTest.java b/gson/src/test/java/com/google/gson/GsonTest.java
--- a/gson/src/test/java/com/google/gson/GsonTest.java
+++ b/gson/src/test/java/com/google/gson/GsonTest.java
@@ -19,7 +19,10 @@
 import com.google.gson.internal.Excluder;
 import com.google.gson.stream.JsonReader;
 import com.google.gson.stream.JsonWriter;
+import com.google.gson.stream.MalformedJsonException;
 import java.io.IOException;
+import java.io.StringReader;
+import java.io.StringWriter;
 import java.lang.reflect.Field;
 import java.lang.reflect.Type;
 import java.text.DateFormat;
@@ -82,4 +85,71 @@ private static final class TestTypeAdapter extends TypeAdapter<Object> {
     }
     @Override public Object read(JsonReader in) throws IOException { return null; }
   }
+
+  public void testNewJsonWriter_Default() throws IOException {
+    StringWriter writer = new StringWriter();
+    JsonWriter jsonWriter = new Gson().newJsonWriter(writer);
+    jsonWriter.beginObject();
+    jsonWriter.name("test");
+    jsonWriter.nullValue();
+    jsonWriter.name("<test2");
+    jsonWriter.value(true);
+    jsonWriter.endObject();
+
+    try {
+      // Additional top-level value
+      jsonWriter.value(1);
+      fail();
+    } catch (IllegalStateException expected) {
+      assertEquals("JSON must have only one top-level value.", expected.getMessage());
+    }
+
+    jsonWriter.close();
+    assertEquals("{\"\\u003ctest2\":true}", writer.toString());
+  }
+
+  public void testNewJsonWriter_Custom() throws IOException {
+    StringWriter writer = new StringWriter();
+    JsonWriter jsonWriter = new GsonBuilder()
+      .disableHtmlEscaping()
+      .generateNonExecutableJson()
+      .setPrettyPrinting()
+      .serializeNulls()
+      .setLenient()
+      .create()
+      .newJsonWriter(writer);
+    jsonWriter.beginObject();
+    jsonWriter.name("test");
+    jsonWriter.nullValue();
+    jsonWriter.name("<test2");
+    jsonWriter.value(true);
+    jsonWriter.endObject();
+
+    // Additional top-level value
+    jsonWriter.value(1);
+
+    jsonWriter.close();
+    assertEquals(")]}'\n{\n  \"test\": null,\n  \"<test2\": true\n}1", writer.toString());
+  }
+
+  public void testNewJsonReader_Default() throws IOException {
+    String json = "test"; // String without quotes
+    JsonReader jsonReader = new Gson().newJsonReader(new StringReader(json));
+    try {
+      jsonReader.nextString();
+      fail();
+    } catch (MalformedJsonException expected) {
+    }
+    jsonReader.close();
+  }
+
+  public void testNewJsonReader_Custom() throws IOException {
+    String json = "test"; // String without quotes
+    JsonReader jsonReader = new GsonBuilder()
+      .setLenient()
+      .create()
+      .newJsonReader(new StringReader(json));
+    assertEquals("test", jsonReader.nextString());
+    jsonReader.close();
+  }
 }