Repository: google/gson (Java)
Date: 2021-10-29 15:49:11+00:00
Task type: Bug Fix

Issue:
`FieldNamingPolicy.upperCaseFirstLetter` uppercases trailing non-letters
The internal method `FieldNamingPolicy.upperCaseFirstLetter(String)` which is used by some of the `FieldNamingPolicy` constants erroneously uppercases trailing non-letters. The reason for this is that for the last character it is not checked whether it is a letter or not.
There are some Unicode characters which are not letters, but for which `toUpperCase` returns a different character. An example for this is `U+0345` for which the uppercase character is `U+0399`.

Example:
```java
// Remains unchanged when it is not trailing
Integer.toHexString(upperCaseFirstLetter("\u0345_").charAt(0))
// Erroneously uppercased when trailing
Integer.toHexString(upperCaseFirstLetter("\u0345").charAt(0))
```

However, despite this character being allowed in a field name (`Character.isJavaIdentifierPart('\u0345')` is `true`), this only affects non-ASCII characters and it is therefore rather unlikely that this issue ever occurs in reality.


Issue discussion:
As you say, this is unlikely to matter in practice. But if you want to send a PR to fix it, we'd be happy to merge it.
Hi, we will volunteer to work on this issue in October/November. For the project maintainer, please let us know if this is okay and if you want it fixed. 
@komminen, I already have an alternative implementation which should fix this bug in a [separate project](https://github.com/Marcono1234/gson-record-type-adapter-factory/blob/9ae1f0a37d8492df06baf27365e9f894b861351d/src/main/java/marcono1234/gson/recordadapter/RecordComponentNamingStrategy.java#L128-L148); I just have not created a pull request proposing this implementation yet.
However, if you think that this implementation is flawed or can be improved, feel free to propose an alternative fix for this. I would also recommend using `\u2170` (or a similar char) in the tests then because (unlike the example char mentioned in the description) it can be used as Java identifier start and part (`Character.isJavaIdentifierStart` and `Character.isJavaIdentifierPart`) and might therefore be more realistic test data.

Otherwise I would create a pull request some time in the future, proposing my implementation to fix this issue.

Solution patch:
diff --git a/gson/src/main/java/com/google/gson/FieldNamingPolicy.java b/gson/src/main/java/com/google/gson/FieldNamingPolicy.java
--- a/gson/src/main/java/com/google/gson/FieldNamingPolicy.java
+++ b/gson/src/main/java/com/google/gson/FieldNamingPolicy.java
@@ -71,7 +71,7 @@ public enum FieldNamingPolicy implements FieldNamingStrategy {
    */
   UPPER_CAMEL_CASE_WITH_SPACES() {
     @Override public String translateName(Field f) {
-      return upperCaseFirstLetter(separateCamelCase(f.getName(), " "));
+      return upperCaseFirstLetter(separateCamelCase(f.getName(), ' '));
     }
   },
 
@@ -89,7 +89,7 @@ public enum FieldNamingPolicy implements FieldNamingStrategy {
    */
   LOWER_CASE_WITH_UNDERSCORES() {
     @Override public String translateName(Field f) {
-      return separateCamelCase(f.getName(), "_").toLowerCase(Locale.ENGLISH);
+      return separateCamelCase(f.getName(), '_').toLowerCase(Locale.ENGLISH);
     }
   },
 
@@ -112,7 +112,7 @@ public enum FieldNamingPolicy implements FieldNamingStrategy {
    */
   LOWER_CASE_WITH_DASHES() {
     @Override public String translateName(Field f) {
-      return separateCamelCase(f.getName(), "-").toLowerCase(Locale.ENGLISH);
+      return separateCamelCase(f.getName(), '-').toLowerCase(Locale.ENGLISH);
     }
   },
 
@@ -135,15 +135,15 @@ public enum FieldNamingPolicy implements FieldNamingStrategy {
    */
   LOWER_CASE_WITH_DOTS() {
     @Override public String translateName(Field f) {
-      return separateCamelCase(f.getName(), ".").toLowerCase(Locale.ENGLISH);
+      return separateCamelCase(f.getName(), '.').toLowerCase(Locale.ENGLISH);
     }
   };
 
   /**
    * Converts the field name that uses camel-case define word separation into
-   * separate words that are separated by the provided {@code separatorString}.
+   * separate words that are separated by the provided {@code separator}.
    */
-  static String separateCamelCase(String name, String separator) {
+  static String separateCamelCase(String name, char separator) {
     StringBuilder translation = new StringBuilder();
     for (int i = 0, length = name.length(); i < length; i++) {
       char character = name.charAt(i);
@@ -158,21 +158,25 @@ static String separateCamelCase(String name, String separator) {
   /**
    * Ensures the JSON field names begins with an upper case letter.
    */
-  static String upperCaseFirstLetter(String name) {
-    int firstLetterIndex = 0;
-    int limit = name.length() - 1;
-    for(; !Character.isLetter(name.charAt(firstLetterIndex)) && firstLetterIndex < limit; ++firstLetterIndex);
+  static String upperCaseFirstLetter(String s) {
+    int length = s.length();
+    for (int i = 0; i < length; i++) {
+      char c = s.charAt(i);
+      if (Character.isLetter(c)) {
+        if (Character.isUpperCase(c)) {
+          return s;
+        }
 
-    char firstLetter = name.charAt(firstLetterIndex);
-    if(Character.isUpperCase(firstLetter)) { //The letter is already uppercased, return the original
-      return name;
-    }
-
-    char uppercased = Character.toUpperCase(firstLetter);
-    if(firstLetterIndex == 0) { //First character in the string is the first letter, saves 1 substring
-      return uppercased + name.substring(1);
+        char uppercased = Character.toUpperCase(c);
+        // For leading letter only need one substring
+        if (i == 0) {
+          return uppercased + s.substring(1);
+        } else {
+          return s.substring(0, i) + uppercased + s.substring(i + 1);
+        }
+      }
     }
 
-    return name.substring(0, firstLetterIndex) + uppercased + name.substring(firstLetterIndex + 1);
+    return s;
   }
 }


Test patch:
diff --git a/gson/src/test/java/com/google/gson/FieldNamingPolicyTest.java b/gson/src/test/java/com/google/gson/FieldNamingPolicyTest.java
new file mode 100644
--- /dev/null
+++ b/gson/src/test/java/com/google/gson/FieldNamingPolicyTest.java
@@ -0,0 +1,130 @@
+package com.google.gson;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotEquals;
+import java.lang.reflect.Field;
+import java.util.Locale;
+import org.junit.Test;
+import com.google.gson.functional.FieldNamingTest;
+
+/**
+ * Performs tests directly against {@link FieldNamingPolicy}; for integration tests
+ * see {@link FieldNamingTest}.
+ */
+public class FieldNamingPolicyTest {
+  @Test
+  public void testSeparateCamelCase() {
+    // Map from original -> expected
+    String[][] argumentPairs =  {
+      { "a", "a" },
+      { "ab", "ab" },
+      { "Ab", "Ab" },
+      { "aB", "a_B" },
+      { "AB", "A_B" },
+      { "A_B", "A__B" },
+      { "firstSecondThird", "first_Second_Third" },
+      { "__", "__" },
+      { "_123", "_123" }
+    };
+
+    for (String[] pair : argumentPairs) {
+      assertEquals(pair[1], FieldNamingPolicy.separateCamelCase(pair[0], '_'));
+    }
+  }
+
+  @Test
+  public void testUpperCaseFirstLetter() {
+    // Map from original -> expected
+    String[][] argumentPairs =  {
+      { "a", "A" },
+      { "ab", "Ab" },
+      { "AB", "AB" },
+      { "_a", "_A" },
+      { "_ab", "_Ab" },
+      { "__", "__" },
+      { "_1", "_1" },
+      // Not a letter, but has uppercase variant (should not be uppercased)
+      // See https://github.com/google/gson/issues/1965
+      { "\u2170", "\u2170" },
+      { "_\u2170", "_\u2170" },
+      { "\u2170a", "\u2170A" },
+    };
+
+    for (String[] pair : argumentPairs) {
+      assertEquals(pair[1], FieldNamingPolicy.upperCaseFirstLetter(pair[0]));
+    }
+  }
+
+  /**
+   * Upper casing policies should be unaffected by default Locale.
+   */
+  @Test
+  public void testUpperCasingLocaleIndependent() throws Exception {
+    class Dummy {
+      @SuppressWarnings("unused")
+      int i;
+    }
+
+    FieldNamingPolicy[] policies = {
+      FieldNamingPolicy.UPPER_CAMEL_CASE,
+      FieldNamingPolicy.UPPER_CAMEL_CASE_WITH_SPACES
+    };
+
+    Field field = Dummy.class.getDeclaredField("i");
+    String name = field.getName();
+    String expected = name.toUpperCase(Locale.ROOT);
+
+    Locale oldLocale = Locale.getDefault();
+    // Set Turkish as Locale which has special case conversion rules
+    Locale.setDefault(new Locale("tr"));
+
+    try {
+      // Verify that default Locale has different case conversion rules
+      assertNotEquals("Test setup is broken", expected, name.toUpperCase());
+
+      for (FieldNamingPolicy policy : policies) {
+        // Should ignore default Locale
+        assertEquals("Unexpected conversion for " + policy, expected, policy.translateName(field));
+      }
+    } finally {
+        Locale.setDefault(oldLocale);
+    }
+  }
+
+  /**
+   * Lower casing policies should be unaffected by default Locale.
+   */
+  @Test
+  public void testLowerCasingLocaleIndependent() throws Exception {
+    class Dummy {
+      @SuppressWarnings("unused")
+      int I;
+    }
+
+    FieldNamingPolicy[] policies = {
+      FieldNamingPolicy.LOWER_CASE_WITH_DASHES,
+      FieldNamingPolicy.LOWER_CASE_WITH_DOTS,
+      FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES,
+    };
+
+    Field field = Dummy.class.getDeclaredField("I");
+    String name = field.getName();
+    String expected = name.toLowerCase(Locale.ROOT);
+
+    Locale oldLocale = Locale.getDefault();
+    // Set Turkish as Locale which has special case conversion rules
+    Locale.setDefault(new Locale("tr"));
+
+    try {
+      // Verify that default Locale has different case conversion rules
+      assertNotEquals("Test setup is broken", expected, name.toLowerCase());
+
+      for (FieldNamingPolicy policy : policies) {
+        // Should ignore default Locale
+        assertEquals("Unexpected conversion for " + policy, expected, policy.translateName(field));
+      }
+    } finally {
+        Locale.setDefault(oldLocale);
+    }
+  }
+}