Ditch reliance on ActiveSupport / Numeric monkey patching

Fix #489, Fix #490