Repository: pandas-dev/pandas

Issue:
error in common na_values definition
Note for  0.14.1: need to add back `#N/A N/A` to na_values defaults

in pandas 0.12

i found a little bug:

this:
_NA_VALUES = set(['-1.#IND', '1.#QNAN', '1.#IND', '-1.#QNAN',
                 '#N/A N/A', 'NA', '#NA', 'NULL', 'NaN',
                 'nan', ''])

must be:
_NA_VALUES = set(['-1.#IND', '1.#QNAN', '1.#IND', '-1.#QNAN',
                 '#N/A','N/A', 'NA', '#NA', 'NULL', 'NaN',
                 'nan', ''])

thank you
Glauco



Discussion:
this is not a bug. you can always specify your own `na_values` if you would like. Or are you advocating `N/A` be added to the 'common list'?

No, he's saying that currently one of the values is: '#N/A N/A'
 but that should actually be two values '#N/A' and 'N/A' right? in other
words, there's a missing closing single-quote and comma.

Exactly . Take a look at the snippet. 

Inviato da Aifon

> Il giorno 27/nov/2013, alle ore 19:41, Jeff Tratner notifications@github.com ha scritto:
> 
> No, he's saying that currently one of the values is: '#N/A N/A'
> but that should actually be two values '#N/A' and 'N/A' right? in other
> words, there's a missing closing single-quote and comma.
> —
> Reply to this email directly or view it on GitHub.

Il contenuto di questa e-mail è rivolto unicamente alle persone alle quali è indirizzato e può contenere informazioni la cui riservatezza è tutelata legalmente. Sono vietati la riproduzione, la diffusione e l’uso di questa e-mail in mancanza di autorizzazione del destinatario. Se avete ricevuto questa e-mail per errore vogliate cortesemente contattarci  immediatamente.

This e-mail is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. Unauthorised reproduction, dissemination or use of this e-mail or of the information herein by anyone other than the intended recipient is prohibited. If you have received this e-mail by mistake, please notify us immediately.

yeah, it's just easy to miss the point because it's a typo and subtle to
boot :)


Resolving patch:
diff --git a/pandas/io/parsers.py b/pandas/io/parsers.py
--- a/pandas/io/parsers.py
+++ b/pandas/io/parsers.py
@@ -438,7 +438,7 @@ def read_fwf(filepath_or_buffer, colspecs='infer', widths=None, **kwds):
 # no longer excluding inf representations
 # '1.#INF','-1.#INF', '1.#INF000000',
 _NA_VALUES = set(['-1.#IND', '1.#QNAN', '1.#IND', '-1.#QNAN',
-                 '#N/A N/A', 'NA', '#NA', 'NULL', 'NaN',
+                 '#N/A','N/A', 'NA', '#NA', 'NULL', 'NaN',
                  'nan', ''])
 
 


Test patch:
diff --git a/pandas/io/tests/test_parsers.py b/pandas/io/tests/test_parsers.py
--- a/pandas/io/tests/test_parsers.py
+++ b/pandas/io/tests/test_parsers.py
@@ -683,6 +683,31 @@ def test_non_string_na_values(self):
             tm.assert_frame_equal(result6,good_compare)
             tm.assert_frame_equal(result7,good_compare)
 
+    def test_default_na_values(self):
+        _NA_VALUES = set(['-1.#IND', '1.#QNAN', '1.#IND', '-1.#QNAN',
+                          '#N/A','N/A', 'NA', '#NA', 'NULL', 'NaN',
+                          'nan', ''])
+
+        nv = len(_NA_VALUES)
+        def f(i, v):
+            if i == 0:
+                buf = ''
+            elif i > 0:
+                buf = ''.join([','] * i)
+
+            buf = "{0}{1}".format(buf,v)
+
+            if i < nv-1:
+                buf = "{0}{1}".format(buf,''.join([','] * (nv-i-1)))
+
+            return buf
+
+        data = StringIO('\n'.join([ f(i, v) for i, v in enumerate(_NA_VALUES) ]))
+
+        expected = DataFrame(np.nan,columns=range(nv),index=range(nv))
+        df = self.read_csv(data, header=None)
+        tm.assert_frame_equal(df, expected)
+
     def test_custom_na_values(self):
         data = """A,B,C
 ignore,this,row