Repository: scipy/scipy

Issue:
mmwrite errors with large (but still 64-bit) integers
There are several related problems with mmwrite that start appearing for integers not much larger than 2^53.

(1) First, for example, running the two lines:

from scipy import io, sparse, identity
io.mmwrite('out.mtx', sparse.coo_matrix(9123456789012345*identity(1, dtype='int64')))

results in an output file out.mtx containing the wrong integer 9123456789012344.

(2) Second, even worse, once the integer gets sufficiently close to (or larger than) 10^16, it is forced into a floating point number, try for example:

from scipy import io, sparse, identity
io.mmwrite('out.mtx', sparse.coo_matrix(9999999999999999*identity(1, dtype='int64')))

which results in out.mtx containing the expression "1e+16". As a consequence, out.mtx is no longer a valid MatrixMarket file, since the header still specifies "MatrixMarket matrix coordinate integer" but the file now contains a floating point number.

(3) I have found that it is possible to prevent the conversion to floating point by passing a sufficiently large precision argument (even though the documentation for mmwrite leads one to believe that this argument has no effect for integers). For example, try:

from scipy import io, sparse, identity
io.mmwrite('out.mtx', sparse.coo_matrix(3498573948572958754*identity(1, dtype='int64')), precision=50)

Good news: out.mtx is now a valid MatrixMarket file, since it contains an integer. Bad news: the integer it contains, 3498573948572958720, is wrong!

All three integers in the above examples are safely less than 2^63, and in all three cases there is no problem representing the numbers internally --- "print"ing the matrix shows the correct integer --- the problem is entirely the fault of mmwrite.

I am using version:
Python 2.7.10 (default, Jun 29 2016, 06:24:32)
[GCC 4.9.3] on linux2
scipy version 0.17.0



Discussion:
The issue is fairly simple, so should be a simple project to fix it for someone who has time for this.

For the first example, apparently the error is related to

```
In[]: '%.16g' % 9123456789012345
Out[]: '9123456789012344'
```

cause by the following lines in mmio.py - inside  `_write()` in class `MMFile`:

```
754   # formats for row indices, col indices and data columns
755   fmt = ('%i', '%i') + ('%%.%dg' % precision,) * (IJV.shape[1]-2)
```

Being `field` an integer, should `fmt` had its own integer format string `%d`? Something like:

```
    # formats for row indices, col indices and data columns
    if field == 'integer':
        fmt = ('%i', '%i') + ('%d',) * (IJV.shape[1]-2)
    else:
        fmt = ('%i', '%i') + ('%%.%dg' % precision,) * (IJV.shape[1]-2)
```

Constructing the IJV array also looks wrong, you end up casting your 
row/col indices to float which can again lose precision (in principle, 
it requires crazy big sparse matrix dimensions). Best to not use 
numpy.savetxt, and just do a for loop instead, similar to what mmfile 
does for dense matrices.

Let me attempt to solve this bug, please.

@PetrKosvanec go for it! Issues don't typically get assigned, so just send a PR when you're ready.

@PetrKosvanec Hi. Are you still working on this or can I give it a shot?
@jotasi 3 months since the last comment is a long time, just go for it I'd say!
@rgommers Alright, I will.
Go for it, please. I am looking for a job. And finished MITx's edx course
6.00.2x in between. I am sorry that I did not attempt this.

On Thu, Jan 5, 2017 at 8:49 AM, jotasi <notifications@github.com> wrote:

> @rgommers <https://github.com/rgommers> Alright, I will.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/scipy/scipy/issues/6397#issuecomment-270583714>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/ATPXNEq8dGJT_2b5SWvCQ4IuZkFHLlj_ks5rPKCOgaJpZM4JRPyk>
> .
>



-- 
Petr

''in the meantime", not in between


On Thu, Jan 5, 2017 at 8:07 PM, Petr Kosvanec <kosvanec@gmail.com> wrote:

> Go for it, please. I am looking for a job. And finished MITx's edx course
> 6.00.2x in between. I am sorry that I did not attempt this.
>
> On Thu, Jan 5, 2017 at 8:49 AM, jotasi <notifications@github.com> wrote:
>
>> @rgommers <https://github.com/rgommers> Alright, I will.
>>
>> —
>> You are receiving this because you were mentioned.
>> Reply to this email directly, view it on GitHub
>> <https://github.com/scipy/scipy/issues/6397#issuecomment-270583714>, or mute
>> the thread
>> <https://github.com/notifications/unsubscribe-auth/ATPXNEq8dGJT_2b5SWvCQ4IuZkFHLlj_ks5rPKCOgaJpZM4JRPyk>
>> .
>>
>
>
>
> --
> Petr
>



-- 
Petr

@PetrKosvanec Thanks for letting me know. Good luck with the job search!
Thank you. Good luck with the bug(s)!

On Thu, Jan 5, 2017 at 8:14 PM, jotasi <notifications@github.com> wrote:

> @PetrKosvanec <https://github.com/PetrKosvanec> Thanks for letting me
> know. Good luck with the job search!
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/scipy/scipy/issues/6397#issuecomment-270731161>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/ATPXNB6HXxJVasWl6q248PMp3wuHoPUvks5rPUEugaJpZM4JRPyk>
> .
>



-- 
Petr

While trying to write the tests for this, I realized that there is actually also a problem with reading matrices with large integers. When reading such a matrix, there is no check for the bit size of the integers. It will always default to signed 32 bit integers which results in an integer overflow as soon as one hits `2**31`. This holds true for both sparse and dense matrices. For example:

```
In [1]: import numpy as np

In [2]: from scipy import io, sparse

In [3]: io.mmwrite('outDense.mtx', np.array([[2**31]], dtype=np.int64))

In [4]: io.mmread('outDense.mtx')
Out[4]: array([[-2147483648]], dtype=int32)

In [5]: io.mmwrite('outSparse.mtx', sparse.csr_matrix(np.array([[2**31]], dtype=np.int64)))

In [6]: io.mmread('outSparse.mtx').todense()
Out[6]: matrix([[-2147483648]], dtype=int32)
```
While `outDense.mtx` contains the correct:

> %%MatrixMarket matrix array integer symmetric
> %
> 1 1
> 2147483648

and `outSparse.mtx`:

> %%MatrixMarket matrix coordinate integer symmetric
> %
> 1 1 1
> 1 1 2147483648

Should I open a new issue for that or is it similar enough to treat it here as well? If so, what would be the best way to tackle this? As far as I understand the MatrixMarket format allows only for "integer" as type string in this case such that one cannot distinguish bit sizes by that. The only options that I came up with so far are:

 - Changing the default to a larger bit size but this changing to e.g. signed 64 bit integers does not account for unsigned 64 bit integers so that does not quite cut it and that approach will waste memory in most cases.

 - Better might be to read element wise, check each element's size, and to change the dtype accordingly.

 - Not support such large integers (at least for reading)

Am I overlooking some simple solution to this (or am I totally off about the usage of `mmwrite` or `mmread` and thus the bug)?
Hi,

I'm the original submitter of this bug.

Unfortunately, indeed, I have not been able to find a specification of
the MatrixMarket format that clearly defines exactly what "integer"
means.

So I guess that means the door is open for it to be implementation
dependent; in which case you might decide to resolve my bug report by
saying "will not fix".

Mathematica, for example, has built-in support for MatrixMarket
format, but it throws an error if you try to export data containing a
signed integer greater than 2**31 in absolute value.  If you try to
import a file containing a large integer of this type, it silently
truncates to the lowest 32 bits.  I filed a bug report about this and
the existence of a problem was denied.  (Previously I have submitted
several bug reports to Wolfram that were fixed quite rapidly.)

I ran into this problem doing some computations where I needed to pass
vectors with large integer entries back and forth between python and
Mathematica, and I was hoping (perhaps naively) that since Mathematica
and python both support arbitrary precision integers, they should both
be able to read/write arbitrary integers in .mtx format.

tl;dr I don't know what to recommend, but I greatly appreciate that
someone is thinking about it!

Mark


On Thu, Jan 5, 2017 at 3:11 PM, jotasi <notifications@github.com> wrote:
>
> While trying to write the tests for this, I realized that there is actually also a problem with reading matrices with large integers. When reading such a matrix, there is no check for the bit size of the integers. It will always default to signed 32 bit integers which results in an integer overflow as soon as one hits 2**31. This holds true for both sparse and dense matrices. For example:
>
> In [1]: import numpy as np
>
> In [2]: from scipy import io, sparse
>
> In [3]: io.mmwrite('outDense.mtx', np.array([[2**31]], dtype=np.int64))
>
> In [4]: io.mmread('outDense.mtx')
> Out[4]: array([[-2147483648]], dtype=int32)
>
> In [5]: io.mmwrite('outSparse.mtx', sparse.csr_matrix(np.array([[2**31]], dtype=np.int64)))
>
> In [6]: io.mmread('outSparse.mtx').todense()
> Out[6]: matrix([[-2147483648]], dtype=int32)
>
> While outDense.mtx contains the correct:
>
> %%MatrixMarket matrix array integer symmetric
> %
> 1 1
> 2147483648
>
> and outSparse.mtx:
>
> %%MatrixMarket matrix coordinate integer symmetric
> %
> 1 1 1
> 1 1 2147483648
>
> Should I open a new issue for that or is it similar enough to treat it here as well? If so, what would be the best way to tackle this? As far as I understand the MatrixMarket format allows only for "integer" as type string in this case such that one cannot distinguish bit sizes by that. The only options that I came up with so far are:
>
> Changing the default to a larger bit size but this changing to e.g. signed 64 bit integers does not account unsigned 64 bit integers so that does not quit cut it and that approach will waste memory in most cases.
>
> Better might be to read element wise, check each elements size, and to change the dtype accordingly.
>
> Not support such large integers (at least for reading)
>
> Am I overlooking some simple solution to this (or am I totally off about the usage of mmwrite or mmread and thus the bug)?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or mute the thread.

> Changing the default to a larger bit size but this changing to e.g. signed 64 bit integers does not account for unsigned 64 bit integers so that does not quite cut it and that approach will waste memory in most cases.

Silent overflow seems bad. The memory issue doesn't seem so bad, nor does the uint64 corner case. This solution seems preferable - either always ``int64`` or use the native ``int`` type by default (so raise an error on 32-bit systems for values that are too large).

> Better might be to read element wise, check each element's size, and to change the dtype accordingly.

This type of behavior is usually not preferred, dtype shouldn't depend on numerical values.

> Should I open a new issue for that or is it similar enough to treat it here as well?

no need for a new issue if you're tackling it together with this one I'd say.


Thank you for the clarification on whats the preferred behavior. I agree that silent overflow should definitively be fixed in some way. Using the native `int` type seems like a good solution, especially since I double checked and saw that `mmwrite` does not support unsigned integers either and raises an error if you try to use an array with that dtype.

Resolving patch:
diff --git a/scipy/io/mmio.py b/scipy/io/mmio.py
--- a/scipy/io/mmio.py
+++ b/scipy/io/mmio.py
@@ -17,7 +17,7 @@
 
 from numpy import (asarray, real, imag, conj, zeros, ndarray, concatenate,
                    ones, ascontiguousarray, vstack, savetxt, fromfile,
-                   fromstring)
+                   fromstring, can_cast)
 from numpy.compat import asbytes, asstr
 
 from scipy._lib.six import string_types
@@ -179,7 +179,7 @@ def _validate_symmetry(self, symmetry):
             raise ValueError('unknown symmetry type %s, must be one of %s' %
                              (symmetry, self.SYMMETRY_VALUES))
 
-    DTYPES_BY_FIELD = {FIELD_INTEGER: 'i',
+    DTYPES_BY_FIELD = {FIELD_INTEGER: 'int',
                        FIELD_REAL: 'd',
                        FIELD_COMPLEX: 'D',
                        FIELD_PATTERN: 'd'}
@@ -493,6 +493,7 @@ def _parse_body(self, stream):
         dtype = self.DTYPES_BY_FIELD.get(field, None)
 
         has_symmetry = self.has_symmetry
+        is_integer = field == self.FIELD_INTEGER
         is_complex = field == self.FIELD_COMPLEX
         is_skew = symm == self.SYMMETRY_SKEW_SYMMETRIC
         is_herm = symm == self.SYMMETRY_HERMITIAN
@@ -506,7 +507,9 @@ def _parse_body(self, stream):
                 line = stream.readline()
                 if not line or line.startswith(b'%'):
                     continue
-                if is_complex:
+                if is_integer:
+                    aij = int(line)
+                elif is_complex:
                     aij = complex(*map(float, line.split()))
                 else:
                     aij = float(line)
@@ -541,7 +544,9 @@ def _parse_body(self, stream):
                 l = line.split()
                 i, j = map(int, l[:2])
                 i, j = i-1, j-1
-                if is_complex:
+                if is_integer:
+                    aij = int(l[2])
+                elif is_complex:
                     aij = complex(*map(float, l[2:]))
                 else:
                     aij = float(l[2])
@@ -564,35 +569,39 @@ def _parse_body(self, stream):
                 # empty matrix
                 return coo_matrix((rows, cols), dtype=dtype)
 
-            try:
-                if not _is_fromfile_compatible(stream):
-                    flat_data = fromstring(stream.read(), sep=' ')
-                else:
-                    # fromfile works for normal files
-                    flat_data = fromfile(stream, sep=' ')
-            except Exception:
-                # fallback - fromfile fails for some file-like objects
-                flat_data = fromstring(stream.read(), sep=' ')
-
-                # TODO use iterator (e.g. xreadlines) to avoid reading
-                # the whole file into memory
-
+            I = zeros(entries, dtype='intc')
+            J = zeros(entries, dtype='intc')
             if is_pattern:
-                flat_data = flat_data.reshape(-1, 2)
-                I = ascontiguousarray(flat_data[:, 0], dtype='intc')
-                J = ascontiguousarray(flat_data[:, 1], dtype='intc')
-                V = ones(len(I), dtype='int8')  # filler
+                V = ones(entries, dtype='int8')
+            elif is_integer:
+                V = zeros(entries, dtype='int')
             elif is_complex:
-                flat_data = flat_data.reshape(-1, 4)
-                I = ascontiguousarray(flat_data[:, 0], dtype='intc')
-                J = ascontiguousarray(flat_data[:, 1], dtype='intc')
-                V = ascontiguousarray(flat_data[:, 2], dtype='complex')
-                V.imag = flat_data[:, 3]
+                V = zeros(entries, dtype='complex')
             else:
-                flat_data = flat_data.reshape(-1, 3)
-                I = ascontiguousarray(flat_data[:, 0], dtype='intc')
-                J = ascontiguousarray(flat_data[:, 1], dtype='intc')
-                V = ascontiguousarray(flat_data[:, 2], dtype='float')
+                V = zeros(entries, dtype='float')
+
+            entry_number = 0
+            for line in stream:
+                if not line or line.startswith(b'%'):
+                    continue
+
+                if entry_number+1 > entries:
+                    raise ValueError("'entries' in header is smaller than "
+                                     "number of entries")
+                l = line.split()
+                I[entry_number], J[entry_number] = map(int, l[:2])
+
+                if not is_pattern:
+                    if is_integer:
+                        V[entry_number] = int(l[2])
+                    elif is_complex:
+                        V[entry_number] = complex(*map(float, l[2:]))
+                    else:
+                        V[entry_number] = float(l[2])
+                entry_number += 1
+            if entry_number < entries:
+                raise ValueError("'entries' in header is larger than "
+                                 "number of entries")
 
             I -= 1  # adjust indices (base 1 -> base 0)
             J -= 1
@@ -634,7 +643,10 @@ def _write(self, stream, a, comment='', field=None, precision=None,
             if field is not None:
 
                 if field == self.FIELD_INTEGER:
-                    a = a.astype('i')
+                    if not can_cast(a.dtype, 'int'):
+                        raise OverflowError("mmwrite does not support integer "
+                                            "dtypes larger than native 'int'.")
+                    a = a.astype('int')
                 elif field == self.FIELD_REAL:
                     if a.dtype.char not in 'fd':
                         a = a.astype('d')
@@ -659,6 +671,9 @@ def _write(self, stream, a, comment='', field=None, precision=None,
         if field is None:
             kind = a.dtype.kind
             if kind == 'i':
+                if not can_cast(a.dtype, 'int'):
+                    raise OverflowError("mmwrite does not support integer "
+                                        "dtypes larger than native 'int'.")
                 field = 'integer'
             elif kind == 'f':
                 field = 'real'
@@ -739,22 +754,21 @@ def _write(self, stream, a, comment='', field=None, precision=None,
             # write shape spec
             stream.write(asbytes('%i %i %i\n' % (rows, cols, coo.nnz)))
 
-            # make indices and data array
+            template = self._field_template(field, precision-1)
+
             if field == self.FIELD_PATTERN:
-                IJV = vstack((coo.row, coo.col)).T
-            elif field in [self.FIELD_INTEGER, self.FIELD_REAL]:
-                IJV = vstack((coo.row, coo.col, coo.data)).T
+                for r, c in zip(coo.row+1, coo.col+1):
+                    stream.write(asbytes("%i %i\n" % (r, c)))
+            elif field in (self.FIELD_INTEGER, self.FIELD_REAL):
+                for r, c, d in zip(coo.row+1, coo.col+1, coo.data):
+                    stream.write(asbytes(("%i %i " % (r, c)) +
+                                         (template % d)))
             elif field == self.FIELD_COMPLEX:
-                IJV = vstack((coo.row, coo.col, coo.data.real,
-                              coo.data.imag)).T
+                for r, c, d in zip(coo.row+1, coo.col+1, coo.data):
+                    stream.write(asbytes(("%i %i " % (r, c)) +
+                                         (template % (d.real, d.imag))))
             else:
                 raise TypeError('Unknown field type %s' % field)
-            IJV[:, :2] += 1  # change base 0 -> base 1
-
-            # formats for row indices, col indices and data columns
-            fmt = ('%i', '%i') + ('%%.%dg' % precision,) * (IJV.shape[1]-2)
-            # save to file
-            savetxt(stream, IJV, fmt=fmt)
 
 
 def _is_fromfile_compatible(stream):


Test patch:
diff --git a/scipy/io/tests/test_mmio.py b/scipy/io/tests/test_mmio.py
--- a/scipy/io/tests/test_mmio.py
+++ b/scipy/io/tests/test_mmio.py
@@ -6,9 +6,10 @@
 import shutil
 
 import numpy as np
-from numpy import array,transpose, pi
+from numpy import array, transpose, pi
 from numpy.testing import (TestCase, run_module_suite, assert_equal,
-                           assert_array_equal, assert_array_almost_equal)
+                           assert_array_equal, assert_array_almost_equal,
+                           assert_raises)
 
 import scipy.sparse
 from scipy.io.mmio import mminfo, mmread, mmwrite
@@ -28,21 +29,38 @@ def check(self, a, info):
         b = mmread(self.fn)
         assert_array_almost_equal(a, b)
 
+    def check_exact(self, a, info):
+        mmwrite(self.fn, a)
+        assert_equal(mminfo(self.fn), info)
+        b = mmread(self.fn)
+        assert_equal(a, b)
+
     def test_simple_integer(self):
-        self.check([[1, 2], [3, 4]],
-                   (2, 2, 4, 'array', 'integer', 'general'))
+        self.check_exact([[1, 2], [3, 4]],
+                         (2, 2, 4, 'array', 'integer', 'general'))
+
+    def test_32bit_integer(self):
+        a = array([[2**31-1, 2**31-2], [2**31-3, 2**31-4]], dtype=np.int32)
+        self.check_exact(a, (2, 2, 4, 'array', 'integer', 'general'))
+
+    def test_64bit_integer(self):
+        a = array([[2**31, 2**32], [2**63-2, 2**63-1]], dtype=np.int64)
+        if (np.intp(0).itemsize < 8):
+            assert_raises(OverflowError, mmwrite, self.fn, a)
+        else:
+            self.check_exact(a, (2, 2, 4, 'array', 'integer', 'general'))
 
     def test_simple_upper_triangle_integer(self):
-        self.check([[0, 1], [0, 0]],
-                   (2, 2, 4, 'array', 'integer', 'general'))
+        self.check_exact([[0, 1], [0, 0]],
+                         (2, 2, 4, 'array', 'integer', 'general'))
 
     def test_simple_lower_triangle_integer(self):
-        self.check([[0, 0], [1, 0]],
-                   (2, 2, 4, 'array', 'integer', 'general'))
+        self.check_exact([[0, 0], [1, 0]],
+                         (2, 2, 4, 'array', 'integer', 'general'))
 
     def test_simple_rectangular_integer(self):
-        self.check([[1, 2, 3], [4, 5, 6]],
-                   (2, 3, 6, 'array', 'integer', 'general'))
+        self.check_exact([[1, 2, 3], [4, 5, 6]],
+                         (2, 3, 6, 'array', 'integer', 'general'))
 
     def test_simple_rectangular_float(self):
         self.check([[1, 2], [3.5, 4], [5, 6]],
@@ -57,12 +75,12 @@ def test_simple_complex(self):
                    (2, 2, 4, 'array', 'complex', 'general'))
 
     def test_simple_symmetric_integer(self):
-        self.check([[1, 2], [2, 4]],
-                   (2, 2, 4, 'array', 'integer', 'symmetric'))
+        self.check_exact([[1, 2], [2, 4]],
+                         (2, 2, 4, 'array', 'integer', 'symmetric'))
 
     def test_simple_skew_symmetric_integer(self):
-        self.check([[1, 2], [-2, 4]],
-                   (2, 2, 4, 'array', 'integer', 'skew-symmetric'))
+        self.check_exact([[1, 2], [-2, 4]],
+                         (2, 2, 4, 'array', 'integer', 'skew-symmetric'))
 
     def test_simple_skew_symmetric_float(self):
         self.check(array([[1, 2], [-2.0, 4]], 'f'),
@@ -85,7 +103,6 @@ def test_random_rectangular_float(self):
 
 
 class TestMMIOSparseCSR(TestMMIOArray):
-
     def setUp(self):
         self.tmpdir = mkdtemp()
         self.fn = os.path.join(self.tmpdir, 'testfile.mtx')
@@ -99,21 +116,42 @@ def check(self, a, info):
         b = mmread(self.fn)
         assert_array_almost_equal(a.todense(), b.todense())
 
+    def check_exact(self, a, info):
+        mmwrite(self.fn, a)
+        assert_equal(mminfo(self.fn), info)
+        b = mmread(self.fn)
+        assert_equal(a.todense(), b.todense())
+
     def test_simple_integer(self):
-        self.check(scipy.sparse.csr_matrix([[1, 2], [3, 4]]),
-                   (2, 2, 4, 'coordinate', 'integer', 'general'))
+        self.check_exact(scipy.sparse.csr_matrix([[1, 2], [3, 4]]),
+                         (2, 2, 4, 'coordinate', 'integer', 'general'))
+
+    def test_32bit_integer(self):
+        a = scipy.sparse.csr_matrix(array([[2**31-1, -2**31+2],
+                                           [2**31-3, 2**31-4]],
+                                          dtype=np.int32))
+        self.check_exact(a, (2, 2, 4, 'coordinate', 'integer', 'general'))
+
+    def test_64bit_integer(self):
+        a = scipy.sparse.csr_matrix(array([[2**32+1, 2**32+1],
+                                           [-2**63+2, 2**63-2]],
+                                          dtype=np.int64))
+        if (np.intp(0).itemsize < 8):
+            assert_raises(OverflowError, mmwrite, self.fn, a)
+        else:
+            self.check_exact(a, (2, 2, 4, 'coordinate', 'integer', 'general'))
 
     def test_simple_upper_triangle_integer(self):
-        self.check(scipy.sparse.csr_matrix([[0, 1], [0, 0]]),
-                   (2, 2, 1, 'coordinate', 'integer', 'general'))
+        self.check_exact(scipy.sparse.csr_matrix([[0, 1], [0, 0]]),
+                         (2, 2, 1, 'coordinate', 'integer', 'general'))
 
     def test_simple_lower_triangle_integer(self):
-        self.check(scipy.sparse.csr_matrix([[0, 0], [1, 0]]),
-                   (2, 2, 1, 'coordinate', 'integer', 'general'))
+        self.check_exact(scipy.sparse.csr_matrix([[0, 0], [1, 0]]),
+                         (2, 2, 1, 'coordinate', 'integer', 'general'))
 
     def test_simple_rectangular_integer(self):
-        self.check(scipy.sparse.csr_matrix([[1, 2, 3], [4, 5, 6]]),
-                   (2, 3, 6, 'coordinate', 'integer', 'general'))
+        self.check_exact(scipy.sparse.csr_matrix([[1, 2, 3], [4, 5, 6]]),
+                         (2, 3, 6, 'coordinate', 'integer', 'general'))
 
     def test_simple_rectangular_float(self):
         self.check(scipy.sparse.csr_matrix([[1, 2], [3.5, 4], [5, 6]]),
@@ -128,12 +166,12 @@ def test_simple_complex(self):
                    (2, 2, 4, 'coordinate', 'complex', 'general'))
 
     def test_simple_symmetric_integer(self):
-        self.check(scipy.sparse.csr_matrix([[1, 2], [2, 4]]),
-                   (2, 2, 3, 'coordinate', 'integer', 'symmetric'))
+        self.check_exact(scipy.sparse.csr_matrix([[1, 2], [2, 4]]),
+                         (2, 2, 3, 'coordinate', 'integer', 'symmetric'))
 
     def test_simple_skew_symmetric_integer(self):
-        self.check(scipy.sparse.csr_matrix([[1, 2], [-2, 4]]),
-                   (2, 2, 3, 'coordinate', 'integer', 'skew-symmetric'))
+        self.check_exact(scipy.sparse.csr_matrix([[1, 2], [-2, 4]]),
+                         (2, 2, 3, 'coordinate', 'integer', 'skew-symmetric'))
 
     def test_simple_skew_symmetric_float(self):
         self.check(scipy.sparse.csr_matrix(array([[1, 2], [-2.0, 4]], 'f')),
@@ -156,6 +194,178 @@ def test_random_rectangular_float(self):
         a = scipy.sparse.csr_matrix(a)
         self.check(a, (20, 15, 300, 'coordinate', 'real', 'general'))
 
+    def test_simple_pattern(self):
+        a = scipy.sparse.csr_matrix([[0, 1.5], [3.0, 2.5]])
+        p = np.zeros_like(a.todense())
+        p[a.todense() > 0] = 1
+        info = (2, 2, 3, 'coordinate', 'pattern', 'general')
+        mmwrite(self.fn, a, field='pattern')
+        assert_equal(mminfo(self.fn), info)
+        b = mmread(self.fn)
+        assert_array_almost_equal(p, b.todense())
+
+
+_32bit_integer_dense_example = '''\
+%%MatrixMarket matrix array integer general
+2  2
+2147483647
+2147483646
+2147483647
+2147483646
+'''
+
+_32bit_integer_sparse_example = '''\
+%%MatrixMarket matrix coordinate integer symmetric
+2  2  2
+1  1  2147483647
+2  2  2147483646
+'''
+
+_64bit_integer_dense_example = '''\
+%%MatrixMarket matrix array integer general
+2  2
+          2147483648
+-9223372036854775806
+         -2147483648
+ 9223372036854775807
+'''
+
+_64bit_integer_sparse_general_example = '''\
+%%MatrixMarket matrix coordinate integer general
+2  2  3
+1  1           2147483648
+1  2  9223372036854775807
+2  2  9223372036854775807
+'''
+
+_64bit_integer_sparse_symmetric_example = '''\
+%%MatrixMarket matrix coordinate integer symmetric
+2  2  3
+1  1            2147483648
+1  2  -9223372036854775807
+2  2   9223372036854775807
+'''
+
+_64bit_integer_sparse_skew_example = '''\
+%%MatrixMarket matrix coordinate integer skew-symmetric
+2  2  3
+1  1            2147483648
+1  2  -9223372036854775807
+2  2   9223372036854775807
+'''
+
+_over64bit_integer_dense_example = '''\
+%%MatrixMarket matrix array integer general
+2  2
+         2147483648
+9223372036854775807
+         2147483648
+9223372036854775808
+'''
+
+_over64bit_integer_sparse_example = '''\
+%%MatrixMarket matrix coordinate integer symmetric
+2  2  2
+1  1            2147483648
+2  2  19223372036854775808
+'''
+
+class TestMMIOReadLargeIntegers(TestCase):
+    def setUp(self):
+        self.tmpdir = mkdtemp()
+        self.fn = os.path.join(self.tmpdir, 'testfile.mtx')
+
+    def tearDown(self):
+        shutil.rmtree(self.tmpdir)
+
+    def check_read(self, example, a, info, dense, over32, over64):
+        with open(self.fn, 'w') as f:
+            f.write(example)
+        assert_equal(mminfo(self.fn), info)
+        if (over32 and (np.intp(0).itemsize < 8)) or over64:
+            assert_raises(OverflowError, mmread, self.fn)
+        else:
+            b = mmread(self.fn)
+            if not dense:
+                b = b.todense()
+            assert_equal(a, b)
+
+    def test_read_32bit_integer_dense(self):
+        a = array([[2**31-1, 2**31-1],
+                   [2**31-2, 2**31-2]], dtype=np.int64)
+        self.check_read(_32bit_integer_dense_example,
+                        a,
+                        (2, 2, 4, 'array', 'integer', 'general'),
+                        dense=True,
+                        over32=False,
+                        over64=False)
+
+    def test_read_32bit_integer_sparse(self):
+        a = array([[2**31-1, 0],
+                   [0, 2**31-2]], dtype=np.int64)
+        self.check_read(_32bit_integer_sparse_example,
+                        a,
+                        (2, 2, 2, 'coordinate', 'integer', 'symmetric'),
+                        dense=False,
+                        over32=False,
+                        over64=False)
+
+    def test_read_64bit_integer_dense(self):
+        a = array([[2**31, -2**31],
+                   [-2**63+2, 2**63-1]], dtype=np.int64)
+        self.check_read(_64bit_integer_dense_example,
+                        a,
+                        (2, 2, 4, 'array', 'integer', 'general'),
+                        dense=True,
+                        over32=True,
+                        over64=False)
+
+    def test_read_64bit_integer_sparse_general(self):
+        a = array([[2**31, 2**63-1],
+                   [0, 2**63-1]], dtype=np.int64)
+        self.check_read(_64bit_integer_sparse_general_example,
+                        a,
+                        (2, 2, 3, 'coordinate', 'integer', 'general'),
+                        dense=False,
+                        over32=True,
+                        over64=False)
+
+    def test_read_64bit_integer_sparse_symmetric(self):
+        a = array([[2**31, -2**63+1],
+                   [-2**63+1, 2**63-1]], dtype=np.int64)
+        self.check_read(_64bit_integer_sparse_symmetric_example,
+                        a,
+                        (2, 2, 3, 'coordinate', 'integer', 'symmetric'),
+                        dense=False,
+                        over32=True,
+                        over64=False)
+
+    def test_read_64bit_integer_sparse_skew(self):
+        a = array([[2**31, -2**63+1],
+                   [2**63-1, 2**63-1]], dtype=np.int64)
+        self.check_read(_64bit_integer_sparse_skew_example,
+                        a,
+                        (2, 2, 3, 'coordinate', 'integer', 'skew-symmetric'),
+                        dense=False,
+                        over32=True,
+                        over64=False)
+
+    def test_read_over64bit_integer_dense(self):
+        self.check_read(_over64bit_integer_dense_example,
+                        None,
+                        (2, 2, 4, 'array', 'integer', 'general'),
+                        dense=True,
+                        over32=True,
+                        over64=True)
+
+    def test_read_over64bit_integer_sparse(self):
+        self.check_read(_over64bit_integer_sparse_example,
+                        None,
+                        (2, 2, 2, 'coordinate', 'integer', 'symmetric'),
+                        dense=False,
+                        over32=True,
+                        over64=True)
+
 
 _general_example = '''\
 %%MatrixMarket matrix coordinate real general