Repository: ahawker/ulid

Issue:
Non-Crockford's Base32 letters converted differently in Java or Python implementations
Hi Andrew, 

first of all, thanks for the amazing library, we've been using a lot!

I have a doubt regarding how we fix the conversion of ULIDs which are not following Crockford's Base32 standard.

We are using Lua to generate some guids (https://github.com/Tieske/ulid.lua) and for some reason, we get from time to time letters outside the Crockford's Base32. 
While trying to fix this on our side (we're not sure how this is happening to be honest), we realised that Java and Python implementations silently corrects this issue in different ways:

### Java
```java
ULID.Value ulidValueFromString = ULID.parseULID("01BX73KC0TNH409RTFD1JXKmO0")
--> "01BX73KC0TNH409RTFD1JXKM00"
```
`mO` is silently converted into `M0`

### Python
```python
In [1]: import ulid

In [2]: u = ulid.from_str('01BX73KC0TNH409RTFD1JXKmO0')

In [3]: u
Out[3]: <ULID('01BX73KC0TNH409RTFD1JXKQZ0')>

In [4]: u.str
Out[4]: '01BX73KC0TNH409RTFD1JXKQZ0'
```
`mO` is silently converted into `QZ`

Shouldn't the python library behave as the Java one as per the [Crockford's Base32](http://crockford.com/wrmg/base32.html) spec, converting `L` and `I` to `1` and `O` to `0` and only upper casing lower case letters instead of changing them?

Thanks a lot in advance!

Eddie

Comments:

Patch:
diff --git a/ulid/base32.py b/ulid/base32.py
index 83f8a8a..f7377b6 100644
--- a/ulid/base32.py
+++ b/ulid/base32.py
@@ -31,11 +31,11 @@ DECODING = array.array(
      0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x00, 0x01,
      0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0xFF, 0xFF,
      0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E,
-     0x0F, 0x10, 0x11, 0xFF, 0x12, 0x13, 0xFF, 0x14, 0x15, 0xFF,
+     0x0F, 0x10, 0x11, 0x01, 0x12, 0x13, 0x01, 0x14, 0x15, 0x00,
      0x16, 0x17, 0x18, 0x19, 0x1A, 0xFF, 0x1B, 0x1C, 0x1D, 0x1E,
      0x1F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x0A, 0x0B, 0x0C,
-     0x0D, 0x0E, 0x0F, 0x10, 0x11, 0xFF, 0x12, 0x13, 0xFF, 0x14,
-     0x15, 0xFF, 0x16, 0x17, 0x18, 0x19, 0x1A, 0xFF, 0x1B, 0x1C,
+     0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x01, 0x12, 0x13, 0x01, 0x14,
+     0x15, 0x00, 0x16, 0x17, 0x18, 0x19, 0x1A, 0xFF, 0x1B, 0x1C,
      0x1D, 0x1E, 0x1F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
      0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
      0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,


Test patch:
diff --git a/tests/test_base32.py b/tests/test_base32.py
index ab2df67..cac8214 100644
--- a/tests/test_base32.py
+++ b/tests/test_base32.py
@@ -9,6 +9,14 @@ import pytest
 from ulid import base32
 
 
+@pytest.fixture(scope='session')
+def decoding_alphabet():
+    """
+    Fixture that yields the entire alphabet that is valid for base32 decoding.
+    """
+    return base32.ENCODING + 'lLiIoO'
+
+
 def test_encode_handles_ulid_and_returns_26_char_string(valid_bytes_128):
     """
     Assert that :func:`~ulid.base32.encode` encodes a valid 128 bit bytes object into a :class:`~str`
@@ -235,3 +243,12 @@ def test_decode_randomness_raises_on_non_ascii_str(invalid_str_encoding):
     """
     with pytest.raises(ValueError):
         base32.decode_randomness(invalid_str_encoding)
+
+
+def test_decode_table_has_value_for_entire_decoding_alphabet(decoding_alphabet):
+    """
+    Assert that :attr:`~ulid.base32.DECODING` stores a valid value mapping for all characters that
+    can be base32 decoded.
+    """
+    for char in decoding_alphabet:
+        assert base32.DECODING[ord(char)] != 0xFF, 'Character "{}" decoded improperly'.format(char)
diff --git a/tests/test_bugs.py b/tests/test_bugs.py
new file mode 100644
index 0000000..6ab8fcb
--- /dev/null
+++ b/tests/test_bugs.py
@@ -0,0 +1,21 @@
+"""
+    test_bugs
+    ~~~~~~~~~
+
+    Tests for validating reported bugs have been fixed.
+"""
+from ulid import api
+
+
+def test_github_issue_58():
+    """
+    Assert that :func:`~ulid.api.from_str` can properly decode strings that
+    contain Base32 "translate" characters.
+
+    Base32 "translate" characters are: "iI, lL, oO".
+
+    Issue: https://github.com/ahawker/ulid/issues/58
+    """
+    value = '01BX73KC0TNH409RTFD1JXKmO0'
+    instance = api.from_str(value)
+    assert instance.str == '01BX73KC0TNH409RTFD1JXKM00'