Fix date listener bug

@JakeWharton 

The same fix as https://github.com/square/android-times-square/pull/81 but with a test.