pandas.read_fwf   pandas.read_fwf(filepath_or_buffer, colspecs='infer', widths=None, infer_nrows=100, **kwds)[source]

Read a table of fixed-width formatted lines into DataFrame. Also supports optionally iterating or breaking of the file into chunks. Additional help can be found in the online docs for IO Tools.  Parameters

filepath_or_buffer:str, path object, or file-like object

String, path object (implementing os.PathLike[str]), or file-like object implementing a text read() function.The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.csv.
colspecs:list of tuple (int, int) or ‘infer’. optional

A list of tuples giving the extents of the fixed-width fields of each line as half-open intervals (i.e., [from, to[ ). String value ‘infer’ can be used to instruct the parser to try detecting the column specifications from the first 100 rows of the data which are not being skipped via skiprows (default=’infer’).
widths:list of int, optional

A list of field widths which can be used instead of ‘colspecs’ if the intervals are contiguous.
infer_nrows:int, default 100

The number of rows to consider when letting the parser determine the colspecs.
**kwds:optional

Optional keyword arguments can be passed to TextFileReader.    Returns
 DataFrame or TextFileReader

A comma-separated values (csv) file is returned as two-dimensional data structure with labeled axes.      See also  DataFrame.to_csv

Write DataFrame to a comma-separated values (csv) file.  read_csv

Read a comma-separated values (csv) file into DataFrame.    Examples
>>> pd.read_fwf('data.csv')


pandas.read_gbq   pandas.read_gbq(query, project_id=None, index_col=None, col_order=None, reauth=False, auth_local_webserver=False, dialect=None, location=None, configuration=None, credentials=None, use_bqstorage_api=None, max_results=None, progress_bar_type=None)[source]

Load data from Google BigQuery. This function requires the pandas-gbq package. See the How to authenticate with Google BigQuery guide for authentication instructions.  Parameters

query:str

SQL-Like Query to return data values.
project_id:str, optional

Google BigQuery Account project ID. Optional when available from the environment.
index_col:str, optional

Name of result column to use for index in results DataFrame.
col_order:list(str), optional

List of BigQuery column names in the desired order for results DataFrame.
reauth:bool, default False

Force Google BigQuery to re-authenticate the user. This is useful if multiple accounts are used.
auth_local_webserver:bool, default False

Use the local webserver flow instead of the console flow when getting user credentials. New in version 0.2.0 of pandas-gbq.
dialect:str, default ‘legacy’

Note: The default value is changing to ‘standard’ in a future version. SQL syntax dialect to use. Value can be one of:  'legacy'

Use BigQuery’s legacy SQL dialect. For more information see BigQuery Legacy SQL Reference.  'standard'

Use BigQuery’s standard SQL, which is compliant with the SQL 2011 standard. For more information see BigQuery Standard SQL Reference.
location:str, optional

Location where the query job should run. See the BigQuery locations documentation for a list of available locations. The location must match that of any datasets used in the query. New in version 0.5.0 of pandas-gbq.
configuration:dict, optional

Query config parameters for job processing. For example:
configuration = {‘query’: {‘useQueryCache’: False}}
 For more information see BigQuery REST API Reference.
credentials:google.auth.credentials.Credentials, optional

Credentials for accessing Google APIs. Use this parameter to override default credentials, such as to use Compute Engine google.auth.compute_engine.Credentials or Service Account google.oauth2.service_account.Credentials directly. New in version 0.8.0 of pandas-gbq.
use_bqstorage_api:bool, default False

Use the BigQuery Storage API to download query results quickly, but at an increased cost. To use this API, first enable it in the Cloud Console. You must also have the bigquery.readsessions.create permission on the project you are billing queries to. This feature requires version 0.10.0 or later of the pandas-gbq package. It also requires the google-cloud-bigquery-storage and fastavro packages.  New in version 0.25.0.
max_results:int, optional

If set, limit the maximum number of rows to fetch from the query results. New in version 0.12.0 of pandas-gbq.  New in version 1.1.0.
progress_bar_type:Optional, str

If set, use the tqdm library to display a progress bar while the data downloads. Install the tqdm package to use this feature. Possible values of progress_bar_type include:  None

No progress bar.  'tqdm'

Use the tqdm.tqdm() function to print a progress bar to sys.stderr.  'tqdm_notebook'

Use the tqdm.tqdm_notebook() function to display a progress bar as a Jupyter notebook widget.  'tqdm_gui'

Use the tqdm.tqdm_gui() function to display a progress bar as a graphical dialog box.   Note that this feature requires version 0.12.0 or later of the pandas-gbq package. And it requires the tqdm package. Slightly different than pandas-gbq, here the default is None.  New in version 1.0.0.     Returns
 df: DataFrame

DataFrame representing results of query.      See also  pandas_gbq.read_gbq

This function in the pandas-gbq library.  DataFrame.to_gbq

Write a DataFrame to Google BigQuery.


pandas.read_hdf   pandas.read_hdf(path_or_buf, key=None, mode='r', errors='strict', where=None, start=None, stop=None, columns=None, iterator=False, chunksize=None, **kwargs)[source]

Read from the store, close it if we opened it. Retrieve pandas object stored in file, optionally based on where criteria.  Warning Pandas uses PyTables for reading and writing HDF5 files, which allows serializing object-dtype data with pickle when using the “fixed” format. Loading pickled data received from untrusted sources can be unsafe. See: https://docs.python.org/3/library/pickle.html for more.   Parameters

path_or_buf:str, path object, pandas.HDFStore

Any valid string path is acceptable. Only supports the local file system, remote URLs and file-like objects are not supported. If you want to pass in a path object, pandas accepts any os.PathLike. Alternatively, pandas accepts an open pandas.HDFStore object.
key:object, optional

The group identifier in the store. Can be omitted if the HDF file contains a single pandas object.
mode:{‘r’, ‘r+’, ‘a’}, default ‘r’

Mode to use when opening the file. Ignored if path_or_buf is a pandas.HDFStore. Default is ‘r’.
errors:str, default ‘strict’

Specifies how encoding and decoding errors are to be handled. See the errors argument for open() for a full list of options.
where:list, optional

A list of Term (or convertible) objects.
start:int, optional

Row number to start selection.
stop:int, optional

Row number to stop selection.
columns:list, optional

A list of columns names to return.
iterator:bool, optional

Return an iterator object.
chunksize:int, optional

Number of rows to include in an iteration when using an iterator.  **kwargs

Additional keyword arguments passed to HDFStore.    Returns

item:object

The selected object. Return type depends on the object stored.      See also  DataFrame.to_hdf

Write a HDF file from a DataFrame.  HDFStore

Low-level access to HDF files.    Examples
>>> df = pd.DataFrame([[1, 1.0, 'a']], columns=['x', 'y', 'z'])
>>> df.to_hdf('./store.h5', 'data')
>>> reread = pd.read_hdf('./store.h5')


pandas.read_html   pandas.read_html(io, match='.+', flavor=None, header=None, index_col=None, skiprows=None, attrs=None, parse_dates=False, thousands=',', encoding=None, decimal='.', converters=None, na_values=None, keep_default_na=True, displayed_only=True)[source]

Read HTML tables into a list of DataFrame objects.  Parameters

io:str, path object, or file-like object

String, path object (implementing os.PathLike[str]), or file-like object implementing a string read() function. The string can represent a URL or the HTML itself. Note that lxml only accepts the http, ftp and file url protocols. If you have a URL that starts with 'https' you might try removing the 's'.
match:str or compiled regular expression, optional

The set of tables containing text matching this regex or string will be returned. Unless the HTML is extremely simple you will probably need to pass a non-empty string here. Defaults to ‘.+’ (match any non-empty string). The default value will return all tables contained on a page. This value is converted to a regular expression so that there is consistent behavior between Beautiful Soup and lxml.
flavor:str, optional

The parsing engine to use. ‘bs4’ and ‘html5lib’ are synonymous with each other, they are both there for backwards compatibility. The default of None tries to use lxml to parse and if that fails it falls back on bs4 + html5lib.
header:int or list-like, optional

The row (or list of rows for a MultiIndex) to use to make the columns headers.
index_col:int or list-like, optional

The column (or list of columns) to use to create the index.
skiprows:int, list-like or slice, optional

Number of rows to skip after parsing the column integer. 0-based. If a sequence of integers or a slice is given, will skip the rows indexed by that sequence. Note that a single element sequence means ‘skip the nth row’ whereas an integer means ‘skip n rows’.
attrs:dict, optional

This is a dictionary of attributes that you can pass to use to identify the table in the HTML. These are not checked for validity before being passed to lxml or Beautiful Soup. However, these attributes must be valid HTML table attributes to work correctly. For example,
attrs = {'id': 'table'}
  is a valid attribute dictionary because the ‘id’ HTML tag attribute is a valid HTML attribute for any HTML tag as per this document.
attrs = {'asdf': 'table'}
  is not a valid attribute dictionary because ‘asdf’ is not a valid HTML attribute even if it is a valid XML attribute. Valid HTML 4.01 table attributes can be found here. A working draft of the HTML 5 spec can be found here. It contains the latest information on table attributes for the modern web.
parse_dates:bool, optional

See read_csv() for more details.
thousands:str, optional

Separator to use to parse thousands. Defaults to ','.
encoding:str, optional

The encoding used to decode the web page. Defaults to None.``None`` preserves the previous encoding behavior, which depends on the underlying parser library (e.g., the parser library will try to use the encoding provided by the document).
decimal:str, default ‘.’

Character to recognize as decimal point (e.g. use ‘,’ for European data).
converters:dict, default None

Dict of functions for converting values in certain columns. Keys can either be integers or column labels, values are functions that take one input argument, the cell (not column) content, and return the transformed content.
na_values:iterable, default None

Custom NA values.
keep_default_na:bool, default True

If na_values are specified and keep_default_na is False the default NaN values are overridden, otherwise they’re appended to.
displayed_only:bool, default True

Whether elements with “display: none” should be parsed.    Returns
 dfs

A list of DataFrames.      See also  read_csv

Read a comma-separated values (csv) file into DataFrame.    Notes Before using this function you should read the gotchas about the HTML parsing libraries. Expect to do some cleanup after you call this function. For example, you might need to manually assign column names if the column names are converted to NaN when you pass the header=0 argument. We try to assume as little as possible about the structure of the table and push the idiosyncrasies of the HTML contained in the table to the user. This function searches for <table> elements and only for <tr> and <th> rows and <td> elements within each <tr> or <th> element in the table. <td> stands for “table data”. This function attempts to properly handle colspan and rowspan attributes. If the function has a <thead> argument, it is used to construct the header, otherwise the function attempts to find the header within the body (by putting rows with only <th> elements into the header). Similar to read_csv() the header argument is applied after skiprows is applied. This function will always return a list of DataFrame or it will fail, e.g., it will not return an empty list. Examples See the read_html documentation in the IO section of the docs for some examples of reading in HTML tables.


pandas.read_json   pandas.read_json(path_or_buf=None, orient=None, typ='frame', dtype=None, convert_axes=None, convert_dates=True, keep_default_dates=True, numpy=False, precise_float=False, date_unit=None, encoding=None, encoding_errors='strict', lines=False, chunksize=None, compression='infer', nrows=None, storage_options=None)[source]

Convert a JSON string to pandas object.  Parameters

path_or_buf:a valid JSON str, path object or file-like object

Any valid string path is acceptable. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.json. If you want to pass in a path object, pandas accepts any os.PathLike. By file-like object, we refer to objects with a read() method, such as a file handle (e.g. via builtin open function) or StringIO.
orient:str

Indication of expected JSON string format. Compatible JSON strings can be produced by to_json() with a corresponding orient value. The set of possible orients is:  'split' : dict like {index -> [index], columns -> [columns], data -> [values]} 'records' : list like [{column -> value}, ... , {column -> value}] 'index' : dict like {index -> {column -> value}} 'columns' : dict like {column -> {index -> value}} 'values' : just the values array  The allowed and default values depend on the value of the typ parameter.
when typ == 'series',  allowed orients are {'split','records','index'} default is 'index' The Series index must be unique for orient 'index'.
when typ == 'frame',  allowed orients are {'split','records','index',
'columns','values', 'table'} default is 'columns' The DataFrame index must be unique for orients 'index' and 'columns'. The DataFrame columns must be unique for orients 'index', 'columns', and 'records'.
typ:{‘frame’, ‘series’}, default ‘frame’

The type of object to recover.
dtype:bool or dict, default None

If True, infer dtypes; if a dict of column to dtype, then use those; if False, then don’t infer dtypes at all, applies only to the data. For all orient values except 'table', default is True.  Changed in version 0.25.0: Not applicable for orient='table'.
convert_axes:bool, default None

Try to convert the axes to the proper dtypes. For all orient values except 'table', default is True.  Changed in version 0.25.0: Not applicable for orient='table'.
convert_dates:bool or list of str, default True

If True then default datelike columns may be converted (depending on keep_default_dates). If False, no dates will be converted. If a list of column names, then those columns will be converted and default datelike columns may also be converted (depending on keep_default_dates).
keep_default_dates:bool, default True

If parsing dates (convert_dates is not False), then try to parse the default datelike columns. A column label is datelike if  it ends with '_at', it ends with '_time', it begins with 'timestamp', it is 'modified', or it is 'date'.
numpy:bool, default False

Direct decoding to numpy arrays. Supports numeric data only, but non-numeric column and index labels are supported. Note also that the JSON ordering MUST be the same for each term if numpy=True.  Deprecated since version 1.0.0.
precise_float:bool, default False

Set to enable usage of higher precision (strtod) function when decoding string to double values. Default (False) is to use fast but less precise builtin functionality.
date_unit:str, default None

The timestamp unit to detect if converting dates. The default behaviour is to try and detect the correct precision, but if this is not desired then pass one of ‘s’, ‘ms’, ‘us’ or ‘ns’ to force parsing only seconds, milliseconds, microseconds or nanoseconds respectively.
encoding:str, default is ‘utf-8’

The encoding to use to decode py3 bytes.
encoding_errors:str, optional, default “strict”

How encoding errors are treated. List of possible values .  New in version 1.3.0.
lines:bool, default False

Read the file as a json object per line.
chunksize:int, optional

Return JsonReader object for iteration. See the line-delimited json docs for more information on chunksize. This can only be passed if lines=True. If this is None, the file will be read into memory all at once.  Changed in version 1.2: JsonReader is a context manager.
compression:str or dict, default ‘infer’

For on-the-fly decompression of on-disk data. If ‘infer’ and ‘path_or_buf’ is path-like, then detect compression from the following extensions: ‘.gz’, ‘.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.  Changed in version 1.4.0: Zstandard support.
nrows:int, optional

The number of lines from the line-delimited jsonfile that has to be read. This can only be passed if lines=True. If this is None, all the rows will be returned.  New in version 1.1.
storage_options:dict, optional

Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.  New in version 1.2.0.     Returns
 Series or DataFrame

The type returned depends on the value of typ.      See also  DataFrame.to_json

Convert a DataFrame to a JSON string.  Series.to_json

Convert a Series to a JSON string.  json_normalize

Normalize semi-structured JSON data into a flat table.    Notes Specific to orient='table', if a DataFrame with a literal Index name of index gets written with to_json(), the subsequent read operation will incorrectly set the Index name to None. This is because index is also used by DataFrame.to_json() to denote a missing Index name, and the subsequent read_json() operation cannot distinguish between the two. The same limitation is encountered with a MultiIndex and any names beginning with 'level_'. Examples
>>> df = pd.DataFrame([['a', 'b'], ['c', 'd']],
...                   index=['row 1', 'row 2'],
...                   columns=['col 1', 'col 2'])
  Encoding/decoding a Dataframe using 'split' formatted JSON:
>>> df.to_json(orient='split')
    '{"columns":["col 1","col 2"],"index":["row 1","row 2"],"data":[["a","b"],["c","d"]]}'
>>> pd.read_json(_, orient='split')
      col 1 col 2
row 1     a     b
row 2     c     d
  Encoding/decoding a Dataframe using 'index' formatted JSON:
>>> df.to_json(orient='index')
'{"row 1":{"col 1":"a","col 2":"b"},"row 2":{"col 1":"c","col 2":"d"}}'

>>> pd.read_json(_, orient='index')
      col 1 col 2
row 1     a     b
row 2     c     d
  Encoding/decoding a Dataframe using 'records' formatted JSON. Note that index labels are not preserved with this encoding.
>>> df.to_json(orient='records')
'[{"col 1":"a","col 2":"b"},{"col 1":"c","col 2":"d"}]'
>>> pd.read_json(_, orient='records')
  col 1 col 2
0     a     b
1     c     d
  Encoding with Table Schema
>>> df.to_json(orient='table')
    '{"schema":{"fields":[{"name":"index","type":"string"},{"name":"col 1","type":"string"},{"name":"col 2","type":"string"}],"primaryKey":["index"],"pandas_version":"1.4.0"},"data":[{"index":"row 1","col 1":"a","col 2":"b"},{"index":"row 2","col 1":"c","col 2":"d"}]}'


pandas.read_orc   pandas.read_orc(path, columns=None, **kwargs)[source]

Load an ORC object from the file path, returning a DataFrame.  New in version 1.0.0.   Parameters

path:str, path object, or file-like object

String, path object (implementing os.PathLike[str]), or file-like object implementing a binary read() function. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.orc.
columns:list, default None

If not None, only these columns will be read from the file.  **kwargs

Any additional kwargs are passed to pyarrow.    Returns
 DataFrame
   Notes Before using this function you should read the user guide about ORC and install optional dependencies.


pandas.read_parquet   pandas.read_parquet(path, engine='auto', columns=None, storage_options=None, use_nullable_dtypes=False, **kwargs)[source]

Load a parquet object from the file path, returning a DataFrame.  Parameters

path:str, path object or file-like object

String, path object (implementing os.PathLike[str]), or file-like object implementing a binary read() function. The string could be a URL. Valid URL schemes include http, ftp, s3, gs, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.parquet. A file URL can also be a path to a directory that contains multiple partitioned parquet files. Both pyarrow and fastparquet support paths to directories as well as file URLs. A directory path could be: file://localhost/path/to/tables or s3://bucket/partition_dir.
engine:{‘auto’, ‘pyarrow’, ‘fastparquet’}, default ‘auto’

Parquet library to use. If ‘auto’, then the option io.parquet.engine is used. The default io.parquet.engine behavior is to try ‘pyarrow’, falling back to ‘fastparquet’ if ‘pyarrow’ is unavailable.
columns:list, default=None

If not None, only these columns will be read from the file.
storage_options:dict, optional

Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.  New in version 1.3.0.
use_nullable_dtypes:bool, default False

If True, use dtypes that use pd.NA as missing value indicator for the resulting DataFrame. (only applicable for the pyarrow engine) As new dtypes are added that support pd.NA in the future, the output with this option will change to use those dtypes. Note: this is an experimental option, and behaviour (e.g. additional support dtypes) may change without notice.  New in version 1.2.0.   **kwargs

Any additional kwargs are passed to the engine.    Returns
 DataFrame


pandas.read_pickle   pandas.read_pickle(filepath_or_buffer, compression='infer', storage_options=None)[source]

Load pickled pandas object (or any object) from file.  Warning Loading pickled data received from untrusted sources can be unsafe. See here.   Parameters

filepath_or_buffer:str, path object, or file-like object

String, path object (implementing os.PathLike[str]), or file-like object implementing a binary readlines() function.  Changed in version 1.0.0: Accept URL. URL is not limited to S3 and GCS.
compression:str or dict, default ‘infer’

For on-the-fly decompression of on-disk data. If ‘infer’ and ‘filepath_or_buffer’ is path-like, then detect compression from the following extensions: ‘.gz’, ‘.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.  Changed in version 1.4.0: Zstandard support.
storage_options:dict, optional

Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.  New in version 1.2.0.     Returns

unpickled:same type as object stored in file

    See also  DataFrame.to_pickle

Pickle (serialize) DataFrame object to file.  Series.to_pickle

Pickle (serialize) Series object to file.  read_hdf

Read HDF5 file into a DataFrame.  read_sql

Read SQL query or database table into a DataFrame.  read_parquet

Load a parquet object, returning a DataFrame.    Notes read_pickle is only guaranteed to be backwards compatible to pandas 0.20.3. Examples
>>> original_df = pd.DataFrame({"foo": range(5), "bar": range(5, 10)})
>>> original_df
   foo  bar
0    0    5
1    1    6
2    2    7
3    3    8
4    4    9
>>> pd.to_pickle(original_df, "./dummy.pkl")

>>> unpickled_df = pd.read_pickle("./dummy.pkl")
>>> unpickled_df
   foo  bar
0    0    5
1    1    6
2    2    7
3    3    8
4    4    9


pandas.read_sas   pandas.read_sas(filepath_or_buffer, format=None, index=None, encoding=None, chunksize=None, iterator=False)[source]

Read SAS files stored as either XPORT or SAS7BDAT format files.  Parameters

filepath_or_buffer:str, path object, or file-like object

String, path object (implementing os.PathLike[str]), or file-like object implementing a binary read() function. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.sas.
format:str {‘xport’, ‘sas7bdat’} or None

If None, file format is inferred from file extension. If ‘xport’ or ‘sas7bdat’, uses the corresponding format.
index:identifier of index column, defaults to None

Identifier of column that should be used as index of the DataFrame.
encoding:str, default is None

Encoding for text data. If None, text data are stored as raw bytes.
chunksize:int

Read file chunksize lines at a time, returns iterator.  Changed in version 1.2: TextFileReader is a context manager.
iterator:bool, defaults to False

If True, returns an iterator for reading the file incrementally.  Changed in version 1.2: TextFileReader is a context manager.     Returns
 DataFrame if iterator=False and chunksize=None, else SAS7BDATReader
or XportReader


pandas.read_spss   pandas.read_spss(path, usecols=None, convert_categoricals=True)[source]

Load an SPSS file from the file path, returning a DataFrame.  New in version 0.25.0.   Parameters

path:str or Path

File path.
usecols:list-like, optional

Return a subset of the columns. If None, return all columns.
convert_categoricals:bool, default is True

Convert categorical columns into pd.Categorical.    Returns
 DataFrame


pandas.read_sql   pandas.read_sql(sql, con, index_col=None, coerce_float=True, params=None, parse_dates=None, columns=None, chunksize=None)[source]

Read SQL query or database table into a DataFrame. This function is a convenience wrapper around read_sql_table and read_sql_query (for backward compatibility). It will delegate to the specific function depending on the provided input. A SQL query will be routed to read_sql_query, while a database table name will be routed to read_sql_table. Note that the delegated function might have more specific notes about their functionality not listed here.  Parameters

sql:str or SQLAlchemy Selectable (select or text object)

SQL query to be executed or a table name.
con:SQLAlchemy connectable, str, or sqlite3 connection

Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. The user is responsible for engine disposal and connection closure for the SQLAlchemy connectable; str connections are closed automatically. See here.
index_col:str or list of str, optional, default: None

Column(s) to set as index(MultiIndex).
coerce_float:bool, default True

Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point, useful for SQL result sets.
params:list, tuple or dict, optional, default: None

List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249’s paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={‘name’ : ‘value’}.
parse_dates:list or dict, default: None

 List of column names to parse as dates. Dict of {column_name: format string} where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. Dict of {column_name: arg dict}, where the arg dict corresponds to the keyword arguments of pandas.to_datetime() Especially useful with databases without native Datetime support, such as SQLite.
columns:list, default: None

List of column names to select from SQL table (only used when reading a table).
chunksize:int, default None

If specified, return an iterator where chunksize is the number of rows to include in each chunk.    Returns
 DataFrame or Iterator[DataFrame]
    See also  read_sql_table

Read SQL database table into a DataFrame.  read_sql_query

Read SQL query into a DataFrame.    Examples Read data from SQL via either a SQL query or a SQL tablename. When using a SQLite database only SQL queries are accepted, providing only the SQL tablename will result in an error.
>>> from sqlite3 import connect
>>> conn = connect(':memory:')
>>> df = pd.DataFrame(data=[[0, '10/11/12'], [1, '12/11/10']],
...                   columns=['int_column', 'date_column'])
>>> df.to_sql('test_data', conn)
2

>>> pd.read_sql('SELECT int_column, date_column FROM test_data', conn)
   int_column date_column
0           0    10/11/12
1           1    12/11/10

>>> pd.read_sql('test_data', 'postgres:///db_name')
  Apply date parsing to columns through the parse_dates argument
>>> pd.read_sql('SELECT int_column, date_column FROM test_data',
...             conn,
...             parse_dates=["date_column"])
   int_column date_column
0           0  2012-10-11
1           1  2010-12-11
  The parse_dates argument calls pd.to_datetime on the provided columns. Custom argument values for applying pd.to_datetime on a column are specified via a dictionary format: 1. Ignore errors while parsing the values of “date_column”
>>> pd.read_sql('SELECT int_column, date_column FROM test_data',
...             conn,
...             parse_dates={"date_column": {"errors": "ignore"}})
   int_column date_column
0           0  2012-10-11
1           1  2010-12-11
   Apply a dayfirst date parsing order on the values of “date_column”
>>> pd.read_sql('SELECT int_column, date_column FROM test_data',
...             conn,
...             parse_dates={"date_column": {"dayfirst": True}})
   int_column date_column
0           0  2012-11-10
1           1  2010-11-12
   Apply custom formatting when date parsing the values of “date_column”
>>> pd.read_sql('SELECT int_column, date_column FROM test_data',
...             conn,
...             parse_dates={"date_column": {"format": "%d/%m/%y"}})
   int_column date_column
0           0  2012-11-10
1           1  2010-11-12


pandas.read_sql_query   pandas.read_sql_query(sql, con, index_col=None, coerce_float=True, params=None, parse_dates=None, chunksize=None, dtype=None)[source]

Read SQL query into a DataFrame. Returns a DataFrame corresponding to the result set of the query string. Optionally provide an index_col parameter to use one of the columns as the index, otherwise default integer index will be used.  Parameters

sql:str SQL query or SQLAlchemy Selectable (select or text object)

SQL query to be executed.
con:SQLAlchemy connectable, str, or sqlite3 connection

Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported.
index_col:str or list of str, optional, default: None

Column(s) to set as index(MultiIndex).
coerce_float:bool, default True

Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Useful for SQL result sets.
params:list, tuple or dict, optional, default: None

List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249’s paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={‘name’ : ‘value’}.
parse_dates:list or dict, default: None

 List of column names to parse as dates. Dict of {column_name: format string} where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. Dict of {column_name: arg dict}, where the arg dict corresponds to the keyword arguments of pandas.to_datetime() Especially useful with databases without native Datetime support, such as SQLite.
chunksize:int, default None

If specified, return an iterator where chunksize is the number of rows to include in each chunk.
dtype:Type name or dict of columns

Data type for data or columns. E.g. np.float64 or {‘a’: np.float64, ‘b’: np.int32, ‘c’: ‘Int64’}.  New in version 1.3.0.     Returns
 DataFrame or Iterator[DataFrame]
    See also  read_sql_table

Read SQL database table into a DataFrame.  read_sql

Read SQL query or database table into a DataFrame.    Notes Any datetime values with time zone information parsed via the parse_dates parameter will be converted to UTC.


pandas.read_sql_table   pandas.read_sql_table(table_name, con, schema=None, index_col=None, coerce_float=True, parse_dates=None, columns=None, chunksize=None)[source]

Read SQL database table into a DataFrame. Given a table name and a SQLAlchemy connectable, returns a DataFrame. This function does not support DBAPI connections.  Parameters

table_name:str

Name of SQL table in database.
con:SQLAlchemy connectable or str

A database URI could be provided as str. SQLite DBAPI connection mode not supported.
schema:str, default None

Name of SQL schema in database to query (if database flavor supports this). Uses default schema if None (default).
index_col:str or list of str, optional, default: None

Column(s) to set as index(MultiIndex).
coerce_float:bool, default True

Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Can result in loss of Precision.
parse_dates:list or dict, default None

 List of column names to parse as dates. Dict of {column_name: format string} where format string is strftime compatible in case of parsing string times or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. Dict of {column_name: arg dict}, where the arg dict corresponds to the keyword arguments of pandas.to_datetime() Especially useful with databases without native Datetime support, such as SQLite.
columns:list, default None

List of column names to select from SQL table.
chunksize:int, default None

If specified, returns an iterator where chunksize is the number of rows to include in each chunk.    Returns
 DataFrame or Iterator[DataFrame]

A SQL table is returned as two-dimensional data structure with labeled axes.      See also  read_sql_query

Read SQL query into a DataFrame.  read_sql

Read SQL query or database table into a DataFrame.    Notes Any datetime values with time zone information will be converted to UTC. Examples
>>> pd.read_sql_table('table_name', 'postgres:///db_name')


pandas.read_stata   pandas.read_stata(filepath_or_buffer, convert_dates=True, convert_categoricals=True, index_col=None, convert_missing=False, preserve_dtypes=True, columns=None, order_categoricals=True, chunksize=None, iterator=False, compression='infer', storage_options=None)[source]

Read Stata file into DataFrame.  Parameters

filepath_or_buffer:str, path object or file-like object

Any valid string path is acceptable. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.dta. If you want to pass in a path object, pandas accepts any os.PathLike. By file-like object, we refer to objects with a read() method, such as a file handle (e.g. via builtin open function) or StringIO.
convert_dates:bool, default True

Convert date variables to DataFrame time values.
convert_categoricals:bool, default True

Read value labels and convert columns to Categorical/Factor variables.
index_col:str, optional

Column to set as index.
convert_missing:bool, default False

Flag indicating whether to convert missing values to their Stata representations. If False, missing values are replaced with nan. If True, columns containing missing values are returned with object data types and missing values are represented by StataMissingValue objects.
preserve_dtypes:bool, default True

Preserve Stata datatypes. If False, numeric data are upcast to pandas default types for foreign data (float64 or int64).
columns:list or None

Columns to retain. Columns will be returned in the given order. None returns all columns.
order_categoricals:bool, default True

Flag indicating whether converted categorical data are ordered.
chunksize:int, default None

Return StataReader object for iterations, returns chunks with given number of lines.
iterator:bool, default False

Return StataReader object.
compression:str or dict, default ‘infer’

For on-the-fly decompression of on-disk data. If ‘infer’ and ‘%s’ is path-like, then detect compression from the following extensions: ‘.gz’, ‘.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.
storage_options:dict, optional

Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.    Returns
 DataFrame or StataReader
    See also  io.stata.StataReader

Low-level reader for Stata data files.  DataFrame.to_stata

Export Stata data files.    Notes Categorical variables read through an iterator may not have the same categories and dtype. This occurs when a variable stored in a DTA file is associated to an incomplete set of value labels that only label a strict subset of the values. Examples Creating a dummy stata for this example >>> df = pd.DataFrame({‘animal’: [‘falcon’, ‘parrot’, ‘falcon’, … ‘parrot’], … ‘speed’: [350, 18, 361, 15]}) # doctest: +SKIP >>> df.to_stata(‘animals.dta’) # doctest: +SKIP Read a Stata dta file:
>>> df = pd.read_stata('animals.dta')
  Read a Stata dta file in 10,000 line chunks: >>> values = np.random.randint(0, 10, size=(20_000, 1), dtype=”uint8”) # doctest: +SKIP >>> df = pd.DataFrame(values, columns=[“i”]) # doctest: +SKIP >>> df.to_stata(‘filename.dta’) # doctest: +SKIP
>>> itr = pd.read_stata('filename.dta', chunksize=10000)
>>> for chunk in itr:
...    # Operate on a single chunk, e.g., chunk.mean()
...    pass


pandas.read_table   pandas.read_table(filepath_or_buffer, sep=NoDefault.no_default, delimiter=None, header='infer', names=NoDefault.no_default, index_col=None, usecols=None, squeeze=None, prefix=NoDefault.no_default, mangle_dupe_cols=True, dtype=None, engine=None, converters=None, true_values=None, false_values=None, skipinitialspace=False, skiprows=None, skipfooter=0, nrows=None, na_values=None, keep_default_na=True, na_filter=True, verbose=False, skip_blank_lines=True, parse_dates=False, infer_datetime_format=False, keep_date_col=False, date_parser=None, dayfirst=False, cache_dates=True, iterator=False, chunksize=None, compression='infer', thousands=None, decimal='.', lineterminator=None, quotechar='"', quoting=0, doublequote=True, escapechar=None, comment=None, encoding=None, encoding_errors='strict', dialect=None, error_bad_lines=None, warn_bad_lines=None, on_bad_lines=None, delim_whitespace=False, low_memory=True, memory_map=False, float_precision=None, storage_options=None)[source]

Read general delimited file into DataFrame. Also supports optionally iterating or breaking of the file into chunks. Additional help can be found in the online docs for IO Tools.  Parameters

filepath_or_buffer:str, path object or file-like object

Any valid string path is acceptable. The string could be a URL. Valid URL schemes include http, ftp, s3, gs, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.csv. If you want to pass in a path object, pandas accepts any os.PathLike. By file-like object, we refer to objects with a read() method, such as a file handle (e.g. via builtin open function) or StringIO.
sep:str, default ‘\t’ (tab-stop)

Delimiter to use. If sep is None, the C engine cannot automatically detect the separator, but the Python parsing engine can, meaning the latter will be used and automatically detect the separator by Python’s builtin sniffer tool, csv.Sniffer. In addition, separators longer than 1 character and different from '\s+' will be interpreted as regular expressions and will also force the use of the Python parsing engine. Note that regex delimiters are prone to ignoring quoted data. Regex example: '\r\t'.
delimiter:str, default None

Alias for sep.
header:int, list of int, None, default ‘infer’

Row number(s) to use as the column names, and the start of the data. Default behavior is to infer the column names: if no names are passed the behavior is identical to header=0 and column names are inferred from the first line of the file, if column names are passed explicitly then the behavior is identical to header=None. Explicitly pass header=0 to be able to replace existing names. The header can be a list of integers that specify row locations for a multi-index on the columns e.g. [0,1,3]. Intervening rows that are not specified will be skipped (e.g. 2 in this example is skipped). Note that this parameter ignores commented lines and empty lines if skip_blank_lines=True, so header=0 denotes the first line of data rather than the first line of the file.
names:array-like, optional

List of column names to use. If the file contains a header row, then you should explicitly pass header=0 to override the column names. Duplicates in this list are not allowed.
index_col:int, str, sequence of int / str, or False, optional, default None

Column(s) to use as the row labels of the DataFrame, either given as string name or column index. If a sequence of int / str is given, a MultiIndex is used. Note: index_col=False can be used to force pandas to not use the first column as the index, e.g. when you have a malformed file with delimiters at the end of each line.
usecols:list-like or callable, optional

Return a subset of the columns. If list-like, all elements must either be positional (i.e. integer indices into the document columns) or strings that correspond to column names provided either by the user in names or inferred from the document header row(s). If names are given, the document header row(s) are not taken into account. For example, a valid list-like usecols parameter would be [0, 1, 2] or ['foo', 'bar', 'baz']. Element order is ignored, so usecols=[0, 1] is the same as [1, 0]. To instantiate a DataFrame from data with element order preserved use pd.read_csv(data, usecols=['foo', 'bar'])[['foo', 'bar']] for columns in ['foo', 'bar'] order or pd.read_csv(data, usecols=['foo', 'bar'])[['bar', 'foo']] for ['bar', 'foo'] order. If callable, the callable function will be evaluated against the column names, returning names where the callable function evaluates to True. An example of a valid callable argument would be lambda x: x.upper() in
['AAA', 'BBB', 'DDD']. Using this parameter results in much faster parsing time and lower memory usage.
squeeze:bool, default False

If the parsed data only contains one column then return a Series.  Deprecated since version 1.4.0: Append .squeeze("columns") to the call to read_table to squeeze the data.
prefix:str, optional

Prefix to add to column numbers when no header, e.g. ‘X’ for X0, X1, …  Deprecated since version 1.4.0: Use a list comprehension on the DataFrame’s columns after calling read_csv.
mangle_dupe_cols:bool, default True

Duplicate columns will be specified as ‘X’, ‘X.1’, …’X.N’, rather than ‘X’…’X’. Passing in False will cause data to be overwritten if there are duplicate names in the columns.
dtype:Type name or dict of column -> type, optional

Data type for data or columns. E.g. {‘a’: np.float64, ‘b’: np.int32, ‘c’: ‘Int64’} Use str or object together with suitable na_values settings to preserve and not interpret dtype. If converters are specified, they will be applied INSTEAD of dtype conversion.
engine:{‘c’, ‘python’, ‘pyarrow’}, optional

Parser engine to use. The C and pyarrow engines are faster, while the python engine is currently more feature-complete. Multithreading is currently only supported by the pyarrow engine.  New in version 1.4.0: The “pyarrow” engine was added as an experimental engine, and some features are unsupported, or may not work correctly, with this engine.
converters:dict, optional

Dict of functions for converting values in certain columns. Keys can either be integers or column labels.
true_values:list, optional

Values to consider as True.
false_values:list, optional

Values to consider as False.
skipinitialspace:bool, default False

Skip spaces after delimiter.
skiprows:list-like, int or callable, optional

Line numbers to skip (0-indexed) or number of lines to skip (int) at the start of the file. If callable, the callable function will be evaluated against the row indices, returning True if the row should be skipped and False otherwise. An example of a valid callable argument would be lambda x: x in [0, 2].
skipfooter:int, default 0

Number of lines at bottom of file to skip (Unsupported with engine=’c’).
nrows:int, optional

Number of rows of file to read. Useful for reading pieces of large files.
na_values:scalar, str, list-like, or dict, optional

Additional strings to recognize as NA/NaN. If dict passed, specific per-column NA values. By default the following values are interpreted as NaN: ‘’, ‘#N/A’, ‘#N/A N/A’, ‘#NA’, ‘-1.#IND’, ‘-1.#QNAN’, ‘-NaN’, ‘-nan’, ‘1.#IND’, ‘1.#QNAN’, ‘<NA>’, ‘N/A’, ‘NA’, ‘NULL’, ‘NaN’, ‘n/a’, ‘nan’, ‘null’.
keep_default_na:bool, default True

Whether or not to include the default NaN values when parsing the data. Depending on whether na_values is passed in, the behavior is as follows:  If keep_default_na is True, and na_values are specified, na_values is appended to the default NaN values used for parsing. If keep_default_na is True, and na_values are not specified, only the default NaN values are used for parsing. If keep_default_na is False, and na_values are specified, only the NaN values specified na_values are used for parsing. If keep_default_na is False, and na_values are not specified, no strings will be parsed as NaN.  Note that if na_filter is passed in as False, the keep_default_na and na_values parameters will be ignored.
na_filter:bool, default True

Detect missing value markers (empty strings and the value of na_values). In data without any NAs, passing na_filter=False can improve the performance of reading a large file.
verbose:bool, default False

Indicate number of NA values placed in non-numeric columns.
skip_blank_lines:bool, default True

If True, skip over blank lines rather than interpreting as NaN values.
parse_dates:bool or list of int or names or list of lists or dict, default False

The behavior is as follows:  boolean. If True -> try parsing the index. list of int or names. e.g. If [1, 2, 3] -> try parsing columns 1, 2, 3 each as a separate date column. list of lists. e.g. If [[1, 3]] -> combine columns 1 and 3 and parse as a single date column. dict, e.g. {‘foo’ : [1, 3]} -> parse columns 1, 3 as date and call result ‘foo’  If a column or index cannot be represented as an array of datetimes, say because of an unparsable value or a mixture of timezones, the column or index will be returned unaltered as an object data type. For non-standard datetime parsing, use pd.to_datetime after pd.read_csv. To parse an index or column with a mixture of timezones, specify date_parser to be a partially-applied pandas.to_datetime() with utc=True. See Parsing a CSV with mixed timezones for more. Note: A fast-path exists for iso8601-formatted dates.
infer_datetime_format:bool, default False

If True and parse_dates is enabled, pandas will attempt to infer the format of the datetime strings in the columns, and if it can be inferred, switch to a faster method of parsing them. In some cases this can increase the parsing speed by 5-10x.
keep_date_col:bool, default False

If True and parse_dates specifies combining multiple columns then keep the original columns.
date_parser:function, optional

Function to use for converting a sequence of string columns to an array of datetime instances. The default uses dateutil.parser.parser to do the conversion. Pandas will try to call date_parser in three different ways, advancing to the next if an exception occurs: 1) Pass one or more arrays (as defined by parse_dates) as arguments; 2) concatenate (row-wise) the string values from the columns defined by parse_dates into a single array and pass that; and 3) call date_parser once for each row using one or more strings (corresponding to the columns defined by parse_dates) as arguments.
dayfirst:bool, default False

DD/MM format dates, international and European format.
cache_dates:bool, default True

If True, use a cache of unique, converted dates to apply the datetime conversion. May produce significant speed-up when parsing duplicate date strings, especially ones with timezone offsets.  New in version 0.25.0.
iterator:bool, default False

Return TextFileReader object for iteration or getting chunks with get_chunk().  Changed in version 1.2: TextFileReader is a context manager.
chunksize:int, optional

Return TextFileReader object for iteration. See the IO Tools docs for more information on iterator and chunksize.  Changed in version 1.2: TextFileReader is a context manager.
compression:str or dict, default ‘infer’

For on-the-fly decompression of on-disk data. If ‘infer’ and ‘%s’ is path-like, then detect compression from the following extensions: ‘.gz’, ‘.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.  Changed in version 1.4.0: Zstandard support.
thousands:str, optional

Thousands separator.
decimal:str, default ‘.’

Character to recognize as decimal point (e.g. use ‘,’ for European data).
lineterminator:str (length 1), optional

Character to break file into lines. Only valid with C parser.
quotechar:str (length 1), optional

The character used to denote the start and end of a quoted item. Quoted items can include the delimiter and it will be ignored.
quoting:int or csv.QUOTE_* instance, default 0

Control field quoting behavior per csv.QUOTE_* constants. Use one of QUOTE_MINIMAL (0), QUOTE_ALL (1), QUOTE_NONNUMERIC (2) or QUOTE_NONE (3).
doublequote:bool, default True

When quotechar is specified and quoting is not QUOTE_NONE, indicate whether or not to interpret two consecutive quotechar elements INSIDE a field as a single quotechar element.
escapechar:str (length 1), optional

One-character string used to escape other characters.
comment:str, optional

Indicates remainder of line should not be parsed. If found at the beginning of a line, the line will be ignored altogether. This parameter must be a single character. Like empty lines (as long as skip_blank_lines=True), fully commented lines are ignored by the parameter header but not by skiprows. For example, if comment='#', parsing #empty\na,b,c\n1,2,3 with header=0 will result in ‘a,b,c’ being treated as the header.
encoding:str, optional

Encoding to use for UTF when reading/writing (ex. ‘utf-8’). List of Python standard encodings .  Changed in version 1.2: When encoding is None, errors="replace" is passed to open(). Otherwise, errors="strict" is passed to open(). This behavior was previously only the case for engine="python".   Changed in version 1.3.0: encoding_errors is a new argument. encoding has no longer an influence on how encoding errors are handled.
encoding_errors:str, optional, default “strict”

How encoding errors are treated. List of possible values .  New in version 1.3.0.
dialect:str or csv.Dialect, optional

If provided, this parameter will override values (default or not) for the following parameters: delimiter, doublequote, escapechar, skipinitialspace, quotechar, and quoting. If it is necessary to override values, a ParserWarning will be issued. See csv.Dialect documentation for more details.
error_bad_lines:bool, optional, default None

Lines with too many fields (e.g. a csv line with too many commas) will by default cause an exception to be raised, and no DataFrame will be returned. If False, then these “bad lines” will be dropped from the DataFrame that is returned.  Deprecated since version 1.3.0: The on_bad_lines parameter should be used instead to specify behavior upon encountering a bad line instead.
warn_bad_lines:bool, optional, default None

If error_bad_lines is False, and warn_bad_lines is True, a warning for each “bad line” will be output.  Deprecated since version 1.3.0: The on_bad_lines parameter should be used instead to specify behavior upon encountering a bad line instead.
on_bad_lines:{‘error’, ‘warn’, ‘skip’} or callable, default ‘error’

Specifies what to do upon encountering a bad line (a line with too many fields). Allowed values are :
 ‘error’, raise an Exception when a bad line is encountered. ‘warn’, raise a warning when a bad line is encountered and skip that line. ‘skip’, skip bad lines without raising or warning when they are encountered.
  New in version 1.3.0:   callable, function with signature (bad_line: list[str]) -> list[str] | None that will process a single bad line. bad_line is a list of strings split by the sep. If the function returns None`, the bad line will be ignored.
If the function returns a new list of strings with more elements than
expected, a ``ParserWarning will be emitted while dropping extra elements. Only supported when engine="python"    New in version 1.4.0.
delim_whitespace:bool, default False

Specifies whether or not whitespace (e.g. ' ' or '    ') will be used as the sep. Equivalent to setting sep='\s+'. If this option is set to True, nothing should be passed in for the delimiter parameter.
low_memory:bool, default True

Internally process the file in chunks, resulting in lower memory use while parsing, but possibly mixed type inference. To ensure no mixed types either set False, or specify the type with the dtype parameter. Note that the entire file is read into a single DataFrame regardless, use the chunksize or iterator parameter to return the data in chunks. (Only valid with C parser).
memory_map:bool, default False

If a filepath is provided for filepath_or_buffer, map the file object directly onto memory and access the data directly from there. Using this option can improve performance because there is no longer any I/O overhead.
float_precision:str, optional

Specifies which converter the C engine should use for floating-point values. The options are None or ‘high’ for the ordinary converter, ‘legacy’ for the original lower precision pandas converter, and ‘round_trip’ for the round-trip converter.  Changed in version 1.2.
storage_options:dict, optional

Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.  New in version 1.2.     Returns
 DataFrame or TextParser

A comma-separated values (csv) file is returned as two-dimensional data structure with labeled axes.      See also  DataFrame.to_csv

Write DataFrame to a comma-separated values (csv) file.  read_csv

Read a comma-separated values (csv) file into DataFrame.  read_fwf

Read a table of fixed-width formatted lines into DataFrame.    Examples
>>> pd.read_table('data.csv')


pandas.read_xml   pandas.read_xml(path_or_buffer, xpath='./*', namespaces=None, elems_only=False, attrs_only=False, names=None, encoding='utf-8', parser='lxml', stylesheet=None, compression='infer', storage_options=None)[source]

Read XML document into a DataFrame object.  New in version 1.3.0.   Parameters

path_or_buffer:str, path object, or file-like object

String, path object (implementing os.PathLike[str]), or file-like object implementing a read() function. The string can be any valid XML string or a path. The string can further be a URL. Valid URL schemes include http, ftp, s3, and file.
xpath:str, optional, default ‘./*’

The XPath to parse required set of nodes for migration to DataFrame. XPath should return a collection of elements and not a single element. Note: The etree parser supports limited XPath expressions. For more complex XPath, use lxml which requires installation.
namespaces:dict, optional

The namespaces defined in XML document as dicts with key being namespace prefix and value the URI. There is no need to include all namespaces in XML, only the ones used in xpath expression. Note: if XML document uses default namespace denoted as xmlns=’<URI>’ without a prefix, you must assign any temporary namespace prefix such as ‘doc’ to the URI in order to parse underlying nodes and/or attributes. For example,
namespaces = {"doc": "https://example.com"}

elems_only:bool, optional, default False

Parse only the child elements at the specified xpath. By default, all child elements and non-empty text nodes are returned.
attrs_only:bool, optional, default False

Parse only the attributes at the specified xpath. By default, all attributes are returned.
names:list-like, optional

Column names for DataFrame of parsed XML data. Use this parameter to rename original element names and distinguish same named elements.
encoding:str, optional, default ‘utf-8’

Encoding of XML document.
parser:{‘lxml’,’etree’}, default ‘lxml’

Parser module to use for retrieval of data. Only ‘lxml’ and ‘etree’ are supported. With ‘lxml’ more complex XPath searches and ability to use XSLT stylesheet are supported.
stylesheet:str, path object or file-like object

A URL, file-like object, or a raw string containing an XSLT script. This stylesheet should flatten complex, deeply nested XML documents for easier parsing. To use this feature you must have lxml module installed and specify ‘lxml’ as parser. The xpath must reference nodes of transformed XML document generated after XSLT transformation and not the original XML document. Only XSLT 1.0 scripts and not later versions is currently supported.
compression:str or dict, default ‘infer’

For on-the-fly decompression of on-disk data. If ‘infer’ and ‘path_or_buffer’ is path-like, then detect compression from the following extensions: ‘.gz’, ‘.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.  Changed in version 1.4.0: Zstandard support.
storage_options:dict, optional

Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.    Returns
 df

A DataFrame.      See also  read_json

Convert a JSON string to pandas object.  read_html

Read HTML tables into a list of DataFrame objects.    Notes This method is best designed to import shallow XML documents in following format which is the ideal fit for the two-dimensions of a DataFrame (row by column).
<root>
    <row>
      <column1>data</column1>
      <column2>data</column2>
      <column3>data</column3>
      ...
   </row>
   <row>
      ...
   </row>
   ...
</root>
  As a file format, XML documents can be designed any way including layout of elements and attributes as long as it conforms to W3C specifications. Therefore, this method is a convenience handler for a specific flatter design and not all possible XML structures. However, for more complex XML documents, stylesheet allows you to temporarily redesign original document with XSLT (a special purpose language) for a flatter version for migration to a DataFrame. This function will always return a single DataFrame or raise exceptions due to issues with XML document, xpath, or other parameters. Examples
>>> xml = '''<?xml version='1.0' encoding='utf-8'?>
... <data xmlns="http://example.com">
...  <row>
...    <shape>square</shape>
...    <degrees>360</degrees>
...    <sides>4.0</sides>
...  </row>
...  <row>
...    <shape>circle</shape>
...    <degrees>360</degrees>
...    <sides/>
...  </row>
...  <row>
...    <shape>triangle</shape>
...    <degrees>180</degrees>
...    <sides>3.0</sides>
...  </row>
... </data>'''

>>> df = pd.read_xml(xml)
>>> df
      shape  degrees  sides
0    square      360    4.0
1    circle      360    NaN
2  triangle      180    3.0

>>> xml = '''<?xml version='1.0' encoding='utf-8'?>
... <data>
...   <row shape="square" degrees="360" sides="4.0"/>
...   <row shape="circle" degrees="360"/>
...   <row shape="triangle" degrees="180" sides="3.0"/>
... </data>'''

>>> df = pd.read_xml(xml, xpath=".//row")
>>> df
      shape  degrees  sides
0    square      360    4.0
1    circle      360    NaN
2  triangle      180    3.0

>>> xml = '''<?xml version='1.0' encoding='utf-8'?>
... <doc:data xmlns:doc="https://example.com">
...   <doc:row>
...     <doc:shape>square</doc:shape>
...     <doc:degrees>360</doc:degrees>
...     <doc:sides>4.0</doc:sides>
...   </doc:row>
...   <doc:row>
...     <doc:shape>circle</doc:shape>
...     <doc:degrees>360</doc:degrees>
...     <doc:sides/>
...   </doc:row>
...   <doc:row>
...     <doc:shape>triangle</doc:shape>
...     <doc:degrees>180</doc:degrees>
...     <doc:sides>3.0</doc:sides>
...   </doc:row>
... </doc:data>'''

>>> df = pd.read_xml(xml,
...                  xpath="//doc:row",
...                  namespaces={"doc": "https://example.com"})
>>> df
      shape  degrees  sides
0    square      360    4.0
1    circle      360    NaN
2  triangle      180    3.0


pandas.reset_option   pandas.reset_option(pat)=<pandas._config.config.CallableDynamicDoc object>

Reset one or more options to their default value. Pass “all” as argument to reset all options. Available options:  compute.[use_bottleneck, use_numba, use_numexpr] display.[chop_threshold, colheader_justify, column_space, date_dayfirst, date_yearfirst, encoding, expand_frame_repr, float_format] display.html.[border, table_schema, use_mathjax] display.[large_repr] display.latex.[escape, longtable, multicolumn, multicolumn_format, multirow, repr] display.[max_categories, max_columns, max_colwidth, max_dir_items, max_info_columns, max_info_rows, max_rows, max_seq_items, memory_usage, min_rows, multi_sparse, notebook_repr_html, pprint_nest_depth, precision, show_dimensions] display.unicode.[ambiguous_as_wide, east_asian_width] display.[width] io.excel.ods.[reader, writer] io.excel.xls.[reader, writer] io.excel.xlsb.[reader] io.excel.xlsm.[reader, writer] io.excel.xlsx.[reader, writer] io.hdf.[default_format, dropna_table] io.parquet.[engine] io.sql.[engine] mode.[chained_assignment, data_manager, sim_interactive, string_storage, use_inf_as_na, use_inf_as_null] plotting.[backend] plotting.matplotlib.[register_converters] styler.format.[decimal, escape, formatter, na_rep, precision, thousands] styler.html.[mathjax] styler.latex.[environment, hrules, multicol_align, multirow_align] styler.render.[encoding, max_columns, max_elements, max_rows, repr] styler.sparse.[columns, index]   Parameters

pat:str/regex

If specified only options matching prefix* will be reset. Note: partial matches are supported for convenience, but unless you use the full option name (e.g. x.y.z.option_name), your code may break in future versions if new options with similar names are introduced.    Returns
 None
   Notes The available options with its descriptions:  compute.use_bottleneck:bool

Use the bottleneck library to accelerate if it is installed, the default is True Valid values: False,True [default: True] [currently: True]  compute.use_numba:bool

Use the numba engine option for select operations if it is installed, the default is False Valid values: False,True [default: False] [currently: False]  compute.use_numexpr:bool

Use the numexpr library to accelerate computation if it is installed, the default is True Valid values: False,True [default: True] [currently: True]  display.chop_threshold:float or None

if set to a float value, all float values smaller then the given threshold will be displayed as exactly 0 by repr and friends. [default: None] [currently: None]  display.colheader_justify:‘left’/’right’

Controls the justification of column headers. used by DataFrameFormatter. [default: right] [currently: right]  display.column_space No description available.

[default: 12] [currently: 12]  display.date_dayfirst:boolean

When True, prints and parses dates with the day first, eg 20/01/2005 [default: False] [currently: False]  display.date_yearfirst:boolean

When True, prints and parses dates with the year first, eg 2005/01/20 [default: False] [currently: False]  display.encoding:str/unicode

Defaults to the detected encoding of the console. Specifies the encoding to be used for strings returned by to_string, these are generally strings meant to be displayed on the console. [default: utf-8] [currently: utf-8]  display.expand_frame_repr:boolean

Whether to print out the full DataFrame repr for wide DataFrames across multiple lines, max_columns is still respected, but the output will wrap-around across multiple “pages” if its width exceeds display.width. [default: True] [currently: True]  display.float_format:callable

The callable should accept a floating point number and return a string with the desired format of the number. This is used in some places like SeriesFormatter. See formats.format.EngFormatter for an example. [default: None] [currently: None]  display.html.border:int

A border=value attribute is inserted in the <table> tag for the DataFrame HTML repr. [default: 1] [currently: 1]  display.html.table_schema:boolean

Whether to publish a Table Schema representation for frontends that support it. (default: False) [default: False] [currently: False]  display.html.use_mathjax:boolean

When True, Jupyter notebook will process table contents using MathJax, rendering mathematical expressions enclosed by the dollar symbol. (default: True) [default: True] [currently: True]  display.large_repr:‘truncate’/’info’

For DataFrames exceeding max_rows/max_cols, the repr (and HTML repr) can show a truncated table (the default from 0.13), or switch to the view from df.info() (the behaviour in earlier versions of pandas). [default: truncate] [currently: truncate]  display.latex.escape:bool

This specifies if the to_latex method of a Dataframe uses escapes special characters. Valid values: False,True [default: True] [currently: True]  display.latex.longtable :bool

This specifies if the to_latex method of a Dataframe uses the longtable format. Valid values: False,True [default: False] [currently: False]  display.latex.multicolumn:bool

This specifies if the to_latex method of a Dataframe uses multicolumns to pretty-print MultiIndex columns. Valid values: False,True [default: True] [currently: True]  display.latex.multicolumn_format:bool

This specifies if the to_latex method of a Dataframe uses multicolumns to pretty-print MultiIndex columns. Valid values: False,True [default: l] [currently: l]  display.latex.multirow:bool

This specifies if the to_latex method of a Dataframe uses multirows to pretty-print MultiIndex rows. Valid values: False,True [default: False] [currently: False]  display.latex.repr:boolean

Whether to produce a latex DataFrame representation for jupyter environments that support it. (default: False) [default: False] [currently: False]  display.max_categories:int

This sets the maximum number of categories pandas should output when printing out a Categorical or a Series of dtype “category”. [default: 8] [currently: 8]  display.max_columns:int

If max_cols is exceeded, switch to truncate view. Depending on large_repr, objects are either centrally truncated or printed as a summary view. ‘None’ value means unlimited. In case python/IPython is running in a terminal and large_repr equals ‘truncate’ this can be set to 0 and pandas will auto-detect the width of the terminal and print a truncated object which fits the screen width. The IPython notebook, IPython qtconsole, or IDLE do not run in a terminal and hence it is not possible to do correct auto-detection. [default: 0] [currently: 0]  display.max_colwidth:int or None

The maximum width in characters of a column in the repr of a pandas data structure. When the column overflows, a “…” placeholder is embedded in the output. A ‘None’ value means unlimited. [default: 50] [currently: 50]  display.max_dir_items:int

The number of items that will be added to dir(…). ‘None’ value means unlimited. Because dir is cached, changing this option will not immediately affect already existing dataframes until a column is deleted or added. This is for instance used to suggest columns from a dataframe to tab completion. [default: 100] [currently: 100]  display.max_info_columns:int

max_info_columns is used in DataFrame.info method to decide if per column information will be printed. [default: 100] [currently: 100]  display.max_info_rows:int or None

df.info() will usually show null-counts for each column. For large frames this can be quite slow. max_info_rows and max_info_cols limit this null check only to frames with smaller dimensions than specified. [default: 1690785] [currently: 1690785]  display.max_rows:int

If max_rows is exceeded, switch to truncate view. Depending on large_repr, objects are either centrally truncated or printed as a summary view. ‘None’ value means unlimited. In case python/IPython is running in a terminal and large_repr equals ‘truncate’ this can be set to 0 and pandas will auto-detect the height of the terminal and print a truncated object which fits the screen height. The IPython notebook, IPython qtconsole, or IDLE do not run in a terminal and hence it is not possible to do correct auto-detection. [default: 60] [currently: 60]  display.max_seq_items:int or None

When pretty-printing a long sequence, no more then max_seq_items will be printed. If items are omitted, they will be denoted by the addition of “…” to the resulting string. If set to None, the number of items to be printed is unlimited. [default: 100] [currently: 100]  display.memory_usage:bool, string or None

This specifies if the memory usage of a DataFrame should be displayed when df.info() is called. Valid values True,False,’deep’ [default: True] [currently: True]  display.min_rows:int

The numbers of rows to show in a truncated view (when max_rows is exceeded). Ignored when max_rows is set to None or 0. When set to None, follows the value of max_rows. [default: 10] [currently: 10]  display.multi_sparse:boolean

“sparsify” MultiIndex display (don’t display repeated elements in outer levels within groups) [default: True] [currently: True]  display.notebook_repr_html:boolean

When True, IPython notebook will use html representation for pandas objects (if it is available). [default: True] [currently: True]  display.pprint_nest_depth:int

Controls the number of nested levels to process when pretty-printing [default: 3] [currently: 3]  display.precision:int

Floating point output precision in terms of number of places after the decimal, for regular formatting as well as scientific notation. Similar to precision in numpy.set_printoptions(). [default: 6] [currently: 6]  display.show_dimensions:boolean or ‘truncate’

Whether to print out dimensions at the end of DataFrame repr. If ‘truncate’ is specified, only print out the dimensions if the frame is truncated (e.g. not display all rows and/or columns) [default: truncate] [currently: truncate]  display.unicode.ambiguous_as_wide:boolean

Whether to use the Unicode East Asian Width to calculate the display text width. Enabling this may affect to the performance (default: False) [default: False] [currently: False]  display.unicode.east_asian_width:boolean

Whether to use the Unicode East Asian Width to calculate the display text width. Enabling this may affect to the performance (default: False) [default: False] [currently: False]  display.width:int

Width of the display in characters. In case python/IPython is running in a terminal this can be set to None and pandas will correctly auto-detect the width. Note that the IPython notebook, IPython qtconsole, or IDLE do not run in a terminal and hence it is not possible to correctly detect the width. [default: 80] [currently: 80]  io.excel.ods.reader:string

The default Excel reader engine for ‘ods’ files. Available options: auto, odf. [default: auto] [currently: auto]  io.excel.ods.writer:string

The default Excel writer engine for ‘ods’ files. Available options: auto, odf. [default: auto] [currently: auto]  io.excel.xls.reader:string

The default Excel reader engine for ‘xls’ files. Available options: auto, xlrd. [default: auto] [currently: auto]  io.excel.xls.writer:string

The default Excel writer engine for ‘xls’ files. Available options: auto, xlwt. [default: auto] [currently: auto] (Deprecated, use `` instead.)  io.excel.xlsb.reader:string

The default Excel reader engine for ‘xlsb’ files. Available options: auto, pyxlsb. [default: auto] [currently: auto]  io.excel.xlsm.reader:string

The default Excel reader engine for ‘xlsm’ files. Available options: auto, xlrd, openpyxl. [default: auto] [currently: auto]  io.excel.xlsm.writer:string

The default Excel writer engine for ‘xlsm’ files. Available options: auto, openpyxl. [default: auto] [currently: auto]  io.excel.xlsx.reader:string

The default Excel reader engine for ‘xlsx’ files. Available options: auto, xlrd, openpyxl. [default: auto] [currently: auto]  io.excel.xlsx.writer:string

The default Excel writer engine for ‘xlsx’ files. Available options: auto, openpyxl, xlsxwriter. [default: auto] [currently: auto]  io.hdf.default_format:format

default format writing format, if None, then put will default to ‘fixed’ and append will default to ‘table’ [default: None] [currently: None]  io.hdf.dropna_table:boolean

drop ALL nan rows when appending to a table [default: False] [currently: False]  io.parquet.engine:string

The default parquet reader/writer engine. Available options: ‘auto’, ‘pyarrow’, ‘fastparquet’, the default is ‘auto’ [default: auto] [currently: auto]  io.sql.engine:string

The default sql reader/writer engine. Available options: ‘auto’, ‘sqlalchemy’, the default is ‘auto’ [default: auto] [currently: auto]  mode.chained_assignment:string

Raise an exception, warn, or no action if trying to use chained assignment, The default is warn [default: warn] [currently: warn]  mode.data_manager:string

Internal data manager type; can be “block” or “array”. Defaults to “block”, unless overridden by the ‘PANDAS_DATA_MANAGER’ environment variable (needs to be set before pandas is imported). [default: block] [currently: block]  mode.sim_interactive:boolean

Whether to simulate interactive mode for purposes of testing [default: False] [currently: False]  mode.string_storage:string

The default storage for StringDtype. [default: python] [currently: python]  mode.use_inf_as_na:boolean

True means treat None, NaN, INF, -INF as NA (old way), False means None and NaN are null, but INF, -INF are not NA (new way). [default: False] [currently: False]  mode.use_inf_as_null:boolean

use_inf_as_null had been deprecated and will be removed in a future version. Use use_inf_as_na instead. [default: False] [currently: False] (Deprecated, use mode.use_inf_as_na instead.)  plotting.backend:str

The plotting backend to use. The default value is “matplotlib”, the backend provided with pandas. Other backends can be specified by providing the name of the module that implements the backend. [default: matplotlib] [currently: matplotlib]  plotting.matplotlib.register_converters:bool or ‘auto’.

Whether to register converters with matplotlib’s units registry for dates, times, datetimes, and Periods. Toggling to False will remove the converters, restoring any converters that pandas overwrote. [default: auto] [currently: auto]  styler.format.decimal:str

The character representation for the decimal separator for floats and complex. [default: .] [currently: .]  styler.format.escape:str, optional

Whether to escape certain characters according to the given context; html or latex. [default: None] [currently: None]  styler.format.formatter:str, callable, dict, optional

A formatter object to be used as default within Styler.format. [default: None] [currently: None]  styler.format.na_rep:str, optional

The string representation for values identified as missing. [default: None] [currently: None]  styler.format.precision:int

The precision for floats and complex numbers. [default: 6] [currently: 6]  styler.format.thousands:str, optional

The character representation for thousands separator for floats, int and complex. [default: None] [currently: None]  styler.html.mathjax:bool

If False will render special CSS classes to table attributes that indicate Mathjax will not be used in Jupyter Notebook. [default: True] [currently: True]  styler.latex.environment:str

The environment to replace \begin{table}. If “longtable” is used results in a specific longtable environment format. [default: None] [currently: None]  styler.latex.hrules:bool

Whether to add horizontal rules on top and bottom and below the headers. [default: False] [currently: False]  styler.latex.multicol_align:{“r”, “c”, “l”, “naive-l”, “naive-r”}

The specifier for horizontal alignment of sparsified LaTeX multicolumns. Pipe decorators can also be added to non-naive values to draw vertical rules, e.g. “|r” will draw a rule on the left side of right aligned merged cells. [default: r] [currently: r]  styler.latex.multirow_align:{“c”, “t”, “b”}

The specifier for vertical alignment of sparsified LaTeX multirows. [default: c] [currently: c]  styler.render.encoding:str

The encoding used for output HTML and LaTeX files. [default: utf-8] [currently: utf-8]  styler.render.max_columns:int, optional

The maximum number of columns that will be rendered. May still be reduced to satsify max_elements, which takes precedence. [default: None] [currently: None]  styler.render.max_elements:int

The maximum number of data-cell (<td>) elements that will be rendered before trimming will occur over columns, rows or both if needed. [default: 262144] [currently: 262144]  styler.render.max_rows:int, optional

The maximum number of rows that will be rendered. May still be reduced to satsify max_elements, which takes precedence. [default: None] [currently: None]  styler.render.repr:str

Determine which output to use in Jupyter Notebook in {“html”, “latex”}. [default: html] [currently: html]  styler.sparse.columns:bool

Whether to sparsify the display of hierarchical columns. Setting to False will display each explicit level element in a hierarchical key for each column. [default: True] [currently: True]  styler.sparse.index:bool

Whether to sparsify the display of a hierarchical index. Setting to False will display each explicit level element in a hierarchical key for each row. [default: True] [currently: True]