Repository: pandas-dev/pandas

Issue:
Google Finance DataReader returns columns with object type instead of float64
Hello,

Google Finance DataReader returns columns with object type instead of float64

```
In [112]: import pandas.io.data as web

In [113]: import datetime

In [114]: start = datetime.datetime(2010, 1, 1)

In [115]: end = datetime.datetime(2013, 1, 27)

In [116]: f=web.DataReader("F", 'google', start, end)

In [117]: f.dtypes
Out[117]:
Open       object
High       object
Low        object
Close     float64
Volume      int64
dtype: object
```



Discussion:
so #8792 validated this for Close. Didn't realize that OHL were still having issues. (Its because of a minus sign that I think is messed up). So will mark as a bug. Appreciate a PR to convert all of this data automatically on import.

I understand that you could be interested by a PR but I have done a totally different approach about DataReader and pandas/io/data.py needs a lot of change. So I don't know if it can be accepted.

I wrote a DataReaderBase class.

DataReaderGoogleFinanceDaily inherits from DataReaderBase
(so others DataReader's like DataReaderGoogleFinanceIntraday, DataReaderYahooFinanceDaily, DataReaderFRED...)

I used a factory pattern using DataReaderFactory class where every DataReaders are "registered"

API changed slightly and I'm sure you will not like this

it looks like

```
data = MyDataReader("GoogleFinanceDaily").get(symbol, start, end)
```

I can probably work to unify to actual DataReader API.

With this kind of idea it's now very easy to use requests (and have cache for queries).
see https://github.com/pydata/pandas/issues/8713

About this particular issue I just defined this function

```
def to_float(x):
    try:
        return(float(x))
    except:
        return(np.nan)
```

and did

```
DATE_COL = 'Date'

OPEN_COL = 'Open'
HIGH_COL = 'High'
LOW_COL = 'Low'
CLOSE_COL = 'Close'

VOLUME_COL = 'Volume'

LST_PRICE_COLS = [OPEN_COL, HIGH_COL, LOW_COL, CLOSE_COL]

for col in LST_PRICE_COLS:
    df[col] = df[col].map(to_float)
```

I need to work now on classes DataReaderFamaFrench, DataReaderYahooFinanceOptions, DataReaderWorldBank

I'm comparing my code returns to Pandas DataReader returns using unit tests (nosetests) and that's the reason why I'm catching some bugs around DataReader.

@femtotrader Have you seen #8961 ? 
It seems like you would probably be interested, given your work and ideas you outline above!

Thanks for this link to this issue

I can send what I have done.

I have done a `datareaders` directory (with `__init__.py`) where `base.py` is stored (DataReaderBase)
in this directory there is a file per source (Google Finance, Yahoo Finance, FRED, ...)
and a `tools.py` (everything else)

As every DataReader object inherits from DataReaderBase it's quite easy to decide whether we fetch data using `requests` and so `requests_cache` or `urlopen`.

An other solution to fix this issue is to pass `na_values='-'`to `pd.read_csv`

Code is here https://github.com/femtotrader/pandas_datareaders

The problem is that F has some missing data:

2012-08-01  -   -   -   9.24    0

I'll submit a PR.


Resolving patch:
diff --git a/pandas/io/data.py b/pandas/io/data.py
--- a/pandas/io/data.py
+++ b/pandas/io/data.py
@@ -166,7 +166,7 @@ def _retry_read_url(url, retry_count, pause, name):
             pass
         else:
             rs = read_csv(StringIO(bytes_to_str(lines)), index_col=0,
-                          parse_dates=True)[::-1]
+                          parse_dates=True, na_values='-')[::-1]
             # Yahoo! Finance sometimes does this awesome thing where they
             # return 2 rows for the most recent business day
             if len(rs) > 2 and rs.index[-1] == rs.index[-2]:  # pragma: no cover


Test patch:
diff --git a/pandas/io/tests/test_data.py b/pandas/io/tests/test_data.py
--- a/pandas/io/tests/test_data.py
+++ b/pandas/io/tests/test_data.py
@@ -118,8 +118,8 @@ def test_get_multi2(self):
                 assert_n_failed_equals_n_null_columns(w, result)
 
     def test_dtypes(self):
-        #GH3995
-        data = web.get_data_google('MSFT', 'JAN-01-12', 'JAN-31-12')
+        #GH3995, #GH8980
+        data = web.get_data_google('F', start='JAN-01-10', end='JAN-27-13')
         assert np.issubdtype(data.Open.dtype, np.number)
         assert np.issubdtype(data.Close.dtype, np.number)
         assert np.issubdtype(data.Low.dtype, np.number)