mojombo/chronic — pull request #202 by davispuh (2013-08-05 21:05:01+00:00)
Title: Parser improvements

Current implementation have reached it's limits and it's very hard to add some specific new date/time format parsing, thus there's need for some refactoring. While general design/idea with *Tokens* and *Tags* are very good there's missing some steps in implementation and due that over years there have been made several hacks (especially in `pre_normalize`). It's not good to use so many regexp pre-processing if it all could be done with proper parser.
Also currently codebase is quite complicated in some areas and thus it's not easy to contribute. It requires lot of time to properly investigate how it does work.

So my idea is to implement proper parsing and refactor some code parts. It would make it much easier to add new formats for parsing. Some formats which are parsed with current implementation - they does work because of pure luck I would say and overall this library seems quite hackish...

For example I wondered why this date `2013-08-05T22:55:03+03:00` even worked, then I noticed that `05T22:55:03+03:00` is parsed as TimeZone and whole `:text` is passed to generic `Time.parse`. I think that's not good at all...

Ok, now more about my idea. It's quite simple, first we divide whole input string by smallest possible tokens (like numbers, separators (dash, colon, etc.)) then we group them together so they make Date, Time and TimeZone (I haven't decided yet what would be best way for this grouping) and only finally we handle those groups based on which combination is most likely correct.
Also I think there are way too many handlers and probably most are unneeded and kinda duplicate code as in end it doesn't matter if it's *ScalarMonth* or *RepeaterMonthName* because that's same thing. Basically handlers should just create date/time from already parsed numbers.

Now about this PR. These are first changes in refactoring about which I was talking. I'll send some few more PRs, but it will take some time and changes probably will be quite big, even this PR isn't small, but here I barely changed anything.
About changes in this PR.
* Improved `Handler.match` so pattern array can contain arrays of tags. It means it will match any of them (simple or), eg. `Chronic::Handler.new([[:separator_on, :separator_at], :scalar], :handler)` will match `on 2` or `at 2`
* Replaced `SeparatorSlashOrDash` to separate `SeparatorSlash` and `SeparatorDash`
* Added more types for separators: dot, colon, space, quote, T, W
* Refactored Scalar and Ordinal classes so they're cleaner and no need to match `\d` all the time, can do it only once
* Created Date and Time classes for Date/Time related functionality
* Added Sign class for parsing `-` and `+`
* Pass `options` as 2nd arg to any `Repeater.new` otherwise it wasn't possible for them to receive any options
* Added `:hours24` option to be able to force 24-hour clock (resolves #201). When `:hours24` is `true` then there's no ambiguous time and it will be always correct. If it's `false` then `03:00:00` will be treated as 12-hour clock, but when `nil` it's same behavior as currently was.
* Added tests for `Handler.match` and `:hours24`

Some changes in this PR aren't really used yet such as so many Separators, but they'll be needed in future.

All tests does pass and everything is backwards compatible. But still should review carefully as I consider any date/time format for which there's no tests as unsupported (because I don't know about them). Also I think it's good time to release new version now as current state is somewhat okay.