Repository: ranaroussi/yfinance

Issue:
ParserError('Error tokenizing data. ...')
Hello,

I am running Python 3.11.2 and Yfinance 0.2.18.

I am trying to download historical data by executing the following line:

yf.download(tickers="AAPL", start='2021-1-1', auto_adjust=True, threads=True)

I am getting the following output (error):

"- AAPL: ParserError('Error tokenizing data. C error: Expected 2 fields in line 1098, saw 3\n')"

<html>
<body>
<!--StartFragment--><div class="output_container" style="width: 659.2px; color: rgb(204, 204, 204); font-family: &quot;Segoe WPC&quot;, &quot;Segoe UI&quot;, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; position: absolute; overflow: hidden; max-height: 102px; top: 75px; height: 102px;"><div id="640bd5c5-5b99-4d92-980f-bd29d9dc7593" class="output output-stream" style="display: inline-block; width: var(--notebook-output-width); line-height: var(--notebook-cell-output-line-height); font-family: var(--notebook-cell-output-font-family); font-size: var(--notebook-cell-output-font-size); user-select: text; cursor: auto; overflow-wrap: break-word; white-space: pre; margin-left: var(--notebook-output-left-margin); background-color: var(--theme-notebook-output-background); padding-top: var(--notebook-output-node-padding); padding-right: var(--notebook-output-node-padding); padding-bottom: var(--notebook-output-node-padding); padding-left: var(--notebook-output-node-left-padding); box-sizing: border-box; border: 1px solid var(--theme-notebook-output-border); overflow-x: auto; position: absolute; top: 0px; left: 0px;"><div><div output-item-id="640bd5c5-5b99-4d92-980f-bd29d9dc7593"><span><span class=""><span>[*********************100%***********************]  1 of 1 completed
</span><span>
</span><span>1 Failed download:
</span><span>- AAPL: ParserError('Error tokenizing data. C error: Expected 2 fields in line 1098, saw 3\n')
</span></span></span></div></div></div></div><div class="output_container" style="width: 659.2px; color: rgb(204, 204, 204); font-family: &quot;Segoe WPC&quot;, &quot;Segoe UI&quot;, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; position: absolute; overflow: hidden; max-height: 69px; top: 177px; height: 69px;"><div id="47cedc77-d986-401b-8273-4da073ff8c93" class="output" style="font-size: var(--notebook-cell-output-font-size); width: var(--notebook-output-width); margin-left: var(--notebook-output-left-margin); background-color: var(--theme-notebook-output-background); padding-top: var(--notebook-output-node-padding); padding-right: var(--notebook-output-node-padding); padding-bottom: var(--notebook-output-node-padding); padding-left: var(--notebook-output-node-left-padding); box-sizing: border-box; border: 1px solid var(--theme-notebook-output-border); overflow-x: auto; position: absolute; top: 0px; left: 0px;"><div class="output_html"><div>

  | Open | High | Low | Close | Adj Close | Volume
-- | -- | -- | -- | -- | -- | --


</div></div></div></div><!--EndFragment-->
</body>
</html>

Anyone having the same issue? Many thanks!





Discussion:
By the way, I am using Pandas 2.0.1.
Operating system is Windows 10.

Many thanks again! :)
Let's keep the issue thread concise and focused.

Install pre-release via PIP then [enable debug logging](https://github.com/ranaroussi/yfinance/tree/dev#logging) and post output. And post any feedback you have on the logging.
Ok, ValueRaider. I have installed the pre-release version and enabled debug logging. Pandas is 2.0.1, Python 3.11.2, Windows 10. 


Input:
yf.download(tickers="AAPL", start='2021-1-1', auto_adjust=True, threads=True)
_______________________________________________________________________________________________________________________________________

Output:

ERROR 
1 Failed download:
ERROR ['AAPL']: ParserError('Error tokenizing data. C error: Expected 2 fields in line 1098, saw 3\n')
DEBUG ['AAPL']: Traceback (most recent call last):
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\multi.py", line 243, in _download_one_threaded
    data = _download_one(ticker, start, end, auto_adjust, back_adjust, repair,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\multi.py", line 262, in _download_one
    return Ticker(ticker).history(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\base.py", line 158, in history
    tz = self._get_ticker_tz(proxy, timeout)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\base.py", line 1013, in _get_ticker_tz
    cache = utils.get_tz_cache()
            ^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\utils.py", line 978, in get_tz_cache
    _tz_cache = _TzCache()
                ^^^^^^^^^^
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\utils.py", line 902, in __init__
    self._migrate_cache_tkr_tz()
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\utils.py", line 943, in _migrate_cache_tkr_tz
    df = _pd.read_csv(old_cache_file_path, index_col="Ticker")
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\io\parsers\readers.py", line 912, in read_csv
    return _read(filepath_or_buffer, kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\io\parsers\readers.py", line 583, in _read
    return parser.read(nrows)
           ^^^^^^^^^^^^^^^^^^
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\io\parsers\readers.py", line 1704, in read
    ) = self._engine.read(  # type: ignore[attr-defined]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\peter\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\io\parsers\c_parser_wrapper.py", line 234, in read
    chunks = self._reader.read_low_memory(nrows)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pandas\_libs\parsers.pyx", line 812, in pandas._libs.parsers.TextReader.read_low_memory
  File "pandas\_libs\parsers.pyx", line 873, in pandas._libs.parsers.TextReader._read_rows
  File "pandas\_libs\parsers.pyx", line 848, in pandas._libs.parsers.TextReader._tokenize_rows
  File "pandas\_libs\parsers.pyx", line 859, in pandas._libs.parsers.TextReader._check_tokenize_status
  File "pandas\_libs\parsers.pyx", line 2025, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: Expected 2 fields in line 1098, saw 3

	Open	High	Low	Close	Adj Close	Volume
Date			


_______________________________________________________________________________________________________________________________________

A big thank you again, ValueRaider!

Resolving patch:
diff --git a/yfinance/utils.py b/yfinance/utils.py
--- a/yfinance/utils.py
+++ b/yfinance/utils.py
@@ -940,7 +940,7 @@ def _migrate_cache_tkr_tz(self):
         if not _os.path.isfile(old_cache_file_path):
             return None
         try:
-            df = _pd.read_csv(old_cache_file_path, index_col="Ticker")
+            df = _pd.read_csv(old_cache_file_path, index_col="Ticker", on_bad_lines="skip")
         except _pd.errors.EmptyDataError:
             _os.remove(old_cache_file_path)
         except TypeError:


Test patch: