matplotlib/matplotlib — Issue #617: Round-off error in ScalarFormatter
Author: tibnor
Created: 2011-12-11 08:43:23+00:00
State: open

My problem is that the ScalarFormatter gives round-off error when using offsets, here is my example:

from pylab import *
plot((0.500000001,0.500000002),(0.500000001,0.500000002))
savefig('ex.pdf')

It gives this significant and offset: 1e-9+5e-1
And the following ticklabels:
2.19999996
2.00000005
1.80000004
etc.

Expected is:
2.2
2.0
1.8
etc.

The bug is due to this function in ScalarFormatter (/usr/share/pyshared/matplotlib/ticker.py)

def _set_format(self):
# set the format string to format all the ticklabels
# The floating point black magic (adding 1e-15 and formatting
# to 8 digits) may warrant review and cleanup.
locs = (np.asarray(self.locs)-self.offset) / 10**self.orderOfMagnitude+1e-15
sigfigs = [len(str('%1.8f'% loc).split('.')[1].rstrip('0')) \
for loc in locs]
sigfigs.sort()
self.format = '%1.' + str(sigfigs[-1]) + 'f'
if self._usetex:
self.format = '$%s$' % self.format
elif self._useMathText:
self.format = '$\mathdefault{%s}$' % self.format

Simply setting setting the formater to '%1.6f'% solves the problem, but I am not sure how it effects other cases.