Convert arrays to MultiIndex.
Each array-like gives one level’s value for each data point. len(arrays) is the number of levels.
Level of sortedness (must be lexicographically sorted by that level).
Names for the levels in the index.
See also
Convert list of tuples to MultiIndex.
Make a MultiIndex from cartesian product of iterables.
Make a MultiIndex from a DataFrame.
Examples


Make a MultiIndex from a DataFrame.
DataFrame to be converted to MultiIndex.
Level of sortedness (must be lexicographically sorted by that level).
If no names are provided, use the column names, or tuple of column names if the columns is a MultiIndex. If a sequence, overwrite names with the given sequence.
The MultiIndex representation of the given DataFrame.
See also
Convert list of arrays to MultiIndex.
Convert list of tuples to MultiIndex.
Make a MultiIndex from cartesian product of iterables.
Examples
Using explicit names, instead of the column names


Make a MultiIndex from the cartesian product of multiple iterables.
Each iterable has unique labels for each level of the index.
Level of sortedness (must be lexicographically sorted by that level).
Names for the levels in the index.
Changed in version 1.0.0: If not explicitly provided, names will be inferred from the elements of iterables if an element has a name attribute
See also
Convert list of arrays to MultiIndex.
Convert list of tuples to MultiIndex.
Make a MultiIndex from a DataFrame.
Examples


Convert list of tuples to MultiIndex.
Each tuple is the index of one row/column.
Level of sortedness (must be lexicographically sorted by that level).
Names for the levels in the index.
See also
Convert list of arrays to MultiIndex.
Make a MultiIndex from cartesian product of iterables.
Make a MultiIndex from a DataFrame.
Examples


Compute indexer and mask for new index given the current index. The indexer should be then used as an input to ndarray.take to align the current data to the new index.
default: exact matches only.
pad / ffill: find the PREVIOUS index value if no exact match.
backfill / bfill: use NEXT index value if no exact match
nearest: use the NEAREST index value if no exact match. Tied distances are broken by preferring the larger index value.
Maximum number of consecutive labels in target to match for inexact matches.
Maximum distance between original and new labels for inexact matches. The values of the index at the matching locations must satisfy the equation abs(index[indexer] - target) <= tolerance.
Tolerance may be a scalar value, which applies the same tolerance to all values, or list-like, which applies variable tolerance per element. List-like includes list, tuple, array, Series, and must be the same size as the index and its dtype must exactly match the index’s type.
Integers from 0 to n - 1 indicating that the index at these positions matches the corresponding target values. Missing values in the target are marked by -1.
Notes
Returns -1 for unmatched values, for further explanation see the example below.
Examples
Notice that the return value is an array of locations in index and x is marked by -1, as it is not in index.


Return vector of label values for requested level.
Length of returned vector is equal to the length of the index.
level is either the integer position of the level in the MultiIndex, or the name of the level.
Values is a level of this MultiIndex converted to a single Index (or subclass thereof).
Notes
If the level contains missing values, the result may be casted to float with missing values specified as NaN. This is because the level is converted to a regular Index.
Examples
Create a MultiIndex:
Get level values by supplying level as either integer or name:
If a level contains missing values, the return type of the level maybe casted to float.


Get location for a label or a tuple of labels.
The location is returned as an integer/slice or boolean mask.
If the key is past the lexsort depth, the return may be a boolean mask array, otherwise it is always a slice or int.
See also
The get_loc method for (single-level) index.
Get slice location given start label(s) and end label(s).
Get location for a label/slice/list/mask or a sequence of such.
Notes
The key cannot be a slice, list of same-level labels, a boolean mask, or a sequence of such. If you want to use those, use MultiIndex.get_locs() instead.
Examples


Get location and sliced index for requested label(s)/level(s).
If False, the resulting index will not drop any level.
Element 0: int, slice object or boolean array Element 1: The resulting sliced multiindex/index. If the key contains all levels, this will be None.
See also
Get location for a label or a tuple of labels.
Get location for a label/slice/list/mask or a sequence of such.
Examples


Get location for a sequence of labels.
You should use one of the above for each level. If a level should not be used, set it to slice(None).
NumPy array of integers suitable for passing to iloc.
See also
Get location for a label or a tuple of labels.
Get slice location given start label(s) and end label(s).
Examples


A tuple with the length of each level.
Examples


Integer number of levels in this MultiIndex.
Examples


Create new MultiIndex from current that removes unused levels.
Unused level(s) means levels that are not expressed in the labels. The resulting MultiIndex will have the same outward appearance, meaning the same .values and ordering. It will also be .equals() to the original.
Examples
The 0 from the first level is not represented and can be removed


Rearrange levels using input order. May not drop or duplicate levels.
List representing new level order. Reference level by number (position) or by key (label).
Examples


Set new codes on MultiIndex. Defaults to returning new index.
New codes to apply.
Level(s) to set (None for all levels).
If True, mutates in place.
Deprecated since version 1.2.0.
If True, checks that levels and codes are compatible.
The same type as the caller or None if inplace=True.
Examples


Set new levels on MultiIndex. Defaults to returning new index.
New level(s) to apply.
Level(s) to set (None for all levels).
If True, mutates in place.
Deprecated since version 1.2.0.
If True, checks that levels and codes are compatible.
The same type as the caller or None if inplace=True.
Examples
If any of the levels passed to set_levels() exceeds the existing length, all of the values from that argument will be stored in the MultiIndex levels, though the values will be truncated in the MultiIndex output.


Sort MultiIndex at the requested level.
The result will respect the original ordering of the associated factor at that level.
If a string is given, must be a name of the level. If list-like must be names or ints of levels.
False to sort in descending order. Can also be a list to specify a directed ordering.
Resulting index.
Indices of output values in original index.
Examples


Swap level i with level j.
Calling this method does not change the ordering of the values.
First level of index to be swapped. Can pass level name as string. Type of parameters can be mixed.
Second level of index to be swapped. Can pass level name as string. Type of parameters can be mixed.
A new MultiIndex.
See also
Swap levels i and j in a MultiIndex.
Swap levels i and j in a MultiIndex on a particular axis.
Examples


Convert a MultiIndex to an Index of Tuples containing the level values.
Index with the MultiIndex data represented in Tuples.
See also
Convert flat index back to MultiIndex.
Notes
This method will simply return the caller if called by anything other than a MultiIndex.
Examples


Create a DataFrame with the levels of the MultiIndex as columns.
Column ordering is determined by the DataFrame constructor with data as a dict.
Set the index of the returned DataFrame as the original MultiIndex.
The passed names should substitute index level names.
See also
Two-dimensional, size-mutable, potentially heterogeneous tabular data.
Examples


Detect non-missing values for an array-like object.
This function takes a scalar or array-like object and indicates whether values are valid (not missing, which is NaN in numeric arrays, None or NaN in object arrays, NaT in datetimelike).
Object to check for not null or non-missing values.
For scalar input, returns a scalar boolean. For array input, returns an array of boolean indicating whether each corresponding element is valid.
See also
Boolean inverse of pandas.notna.
Detect valid values in a Series.
Detect valid values in a DataFrame.
Detect valid values in an Index.
Examples
Scalar arguments (including strings) result in a scalar boolean.
ndarrays result in an ndarray of booleans.
For indexes, an ndarray of booleans is returned.
For Series and DataFrame, the same type is returned, containing booleans.


Context manager to temporarily set options in the with statement context.
You need to invoke as option_context(pat, val, [(pat, val), ...]).
Examples
Methods
__call__(func)
Call self as a function.


Represents a period of time.
The time period represented (e.g., ‘4Q2005’).
One of pandas period strings or corresponding objects.
The period offset from the proleptic Gregorian epoch.
Year value of the period.
Month value of the period.
Quarter value of the period.
Day value of the period.
Hour value of the period.
Minute value of the period.
Second value of the period.
Attributes
day
Get day of the month that a Period falls on.
day_of_week
Day of the week the period lies in, with Monday=0 and Sunday=6.
day_of_year
Return the day of the year.
dayofweek
Day of the week the period lies in, with Monday=0 and Sunday=6.
dayofyear
Return the day of the year.
days_in_month
Get the total number of days in the month that this period falls on.
daysinmonth
Get the total number of days of the month that the Period falls in.
end_time
Get the Timestamp for the end of the period.
freqstr
Return a string representation of the frequency.
hour
Get the hour of the day component of the Period.
is_leap_year
Return True if the period's year is in a leap year.
minute
Get minute of the hour component of the Period.
month
Return the month this Period falls on.
quarter
Return the quarter this Period falls on.
qyear
Fiscal year the Period lies in according to its starting-quarter.
second
Get the second component of the Period.
start_time
Get the Timestamp for the start of the period.
week
Get the week of the year on the given Period.
weekday
Day of the week the period lies in, with Monday=0 and Sunday=6.
weekofyear
Get the week of the year on the given Period.
year
Return the year this Period falls on.
freq
ordinal
Methods
asfreq
Convert Period to desired frequency, at the start or end of the interval.
now
Return the period of now's date.
strftime
Returns the string representation of the Period, depending on the selected fmt.
to_timestamp
Return the Timestamp representation of the Period.


Convert Period to desired frequency, at the start or end of the interval.
The desired frequency.
Start or end of the timespan.


Get day of the month that a Period falls on.
See also
Get the day of the week.
Get the day of the year.
Examples


Day of the week the period lies in, with Monday=0 and Sunday=6.
If the period frequency is lower than daily (e.g. hourly), and the period spans over multiple days, the day at the start of the period is used.
If the frequency is higher than daily (e.g. monthly), the last day of the period is used.
Day of the week.
See also
Day of the week the period lies in.
Alias of Period.day_of_week.
Day of the month.
Day of the year.
Examples
For periods that span over multiple days, the day at the beginning of the period is returned.
For periods with a frequency higher than days, the last day of the period is returned.


Return the day of the year.
This attribute returns the day of the year on which the particular date occurs. The return value ranges between 1 to 365 for regular years and 1 to 366 for leap years.
The day of year.
See also
Return the day of the month.
Return the day of week.
Return the day of year of all indexes.
Examples


Get the total number of days in the month that this period falls on.
See also
Gets the number of days in the month.
Gets the number of days in the month.
Returns a tuple containing weekday (0-6 ~ Mon-Sun) and number of days (28-31).
Examples
Handles the leap year case as well:


Get the total number of days of the month that the Period falls in.
See also
Return the days of the month.
Return the day of the year.
Examples


Get the Timestamp for the end of the period.
See also
Return the start Timestamp.
Return the day of year.
Return the days in that month.
Return the day of the week.


Return a string representation of the frequency.


Get the hour of the day component of the Period.
The hour as an integer, between 0 and 23.
See also
Get the second component of the Period.
Get the minute component of the Period.
Examples
Period longer than a day


Return True if the period’s year is in a leap year.


Get minute of the hour component of the Period.
The minute as an integer, between 0 and 59.
See also
Get the hour component of the Period.
Get the second component of the Period.
Examples


Return the quarter this Period falls on.


Fiscal year the Period lies in according to its starting-quarter.
The year and the qyear of the period will be the same if the fiscal and calendar years are the same. When they are not, the fiscal year can be different from the calendar year of the period.
The fiscal year of the period.
See also
Return the calendar year of the period.
Examples
If the natural and fiscal year are the same, qyear and year will be the same.
If the fiscal year starts in April (Q-MAR), the first quarter of 2018 will start in April 2017. year will then be 2018, but qyear will be the fiscal year, 2018.


Get the second component of the Period.
The second of the Period (ranges from 0 to 59).
See also
Get the hour component of the Period.
Get the minute component of the Period.
Examples


Get the Timestamp for the start of the period.
See also
Return the end Timestamp.
Return the day of year.
Return the days in that month.
Return the day of the week.
Examples


Returns the string representation of the Period, depending on the selected fmt. fmt must be a string containing one or several directives. The method recognizes the same directives as the time.strftime() function of the standard Python distribution, as well as the specific additional directives %f, %F, %q. (formatting & docs originally from scikits.timeries).
Directive
Meaning
Notes
%a
Locale’s abbreviated weekday name.
%A
Locale’s full weekday name.
%b
Locale’s abbreviated month name.
%B
Locale’s full month name.
%c
Locale’s appropriate date and time representation.
%d
Day of the month as a decimal number [01,31].
%f
‘Fiscal’ year without a century as a decimal number [00,99]
(1)
%F
‘Fiscal’ year with a century as a decimal number
(2)
%H
Hour (24-hour clock) as a decimal number [00,23].
%I
Hour (12-hour clock) as a decimal number [01,12].
%j
Day of the year as a decimal number [001,366].
%m
Month as a decimal number [01,12].
%M
Minute as a decimal number [00,59].
%p
Locale’s equivalent of either AM or PM.
(3)
%q
Quarter as a decimal number [01,04]
%S
Second as a decimal number [00,61].
(4)
%U
Week number of the year (Sunday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Sunday are considered to be in week 0.
(5)
%w
Weekday as a decimal number [0(Sunday),6].
%W
Week number of the year (Monday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Monday are considered to be in week 0.
(5)
%x
Locale’s appropriate date representation.
%X
Locale’s appropriate time representation.
%y
Year without century as a decimal number [00,99].
%Y
Year with century as a decimal number.
%Z
Time zone name (no characters if no time zone exists).
%%
A literal '%' character.
Notes
The %f directive is the same as %y if the frequency is not quarterly. Otherwise, it corresponds to the ‘fiscal’ year, as defined by the qyear attribute.
The %F directive is the same as %Y if the frequency is not quarterly. Otherwise, it corresponds to the ‘fiscal’ year, as defined by the qyear attribute.
The %p directive only affects the output hour field if the %I directive is used to parse the hour.
The range really is 0 to 61; this accounts for leap seconds and the (very rare) double leap seconds.
The %U and %W directives are only used in calculations when the day of the week and the year are specified.
Examples


Return the Timestamp representation of the Period.
Uses the target frequency specified at the part of the period specified by how, which is either Start or Finish.
Target frequency. Default is ‘D’ if self.freq is week or longer and ‘S’ otherwise.
One of ‘S’, ‘E’. Can be aliased as case insensitive ‘Start’, ‘Finish’, ‘Begin’, ‘End’.


Get the week of the year on the given Period.
See also
Get the day component of the Period.
Get the day component of the Period.
Examples


Day of the week the period lies in, with Monday=0 and Sunday=6.
If the period frequency is lower than daily (e.g. hourly), and the period spans over multiple days, the day at the start of the period is used.
If the frequency is higher than daily (e.g. monthly), the last day of the period is used.
Day of the week.
See also
Day of the week the period lies in.
Alias of Period.dayofweek.
Day of the month.
Day of the year.
Examples
For periods that span over multiple days, the day at the beginning of the period is returned.
For periods with a frequency higher than days, the last day of the period is returned.


Return a fixed frequency PeriodIndex.
The day (calendar) is the default frequency.
Left bound for generating periods.
Right bound for generating periods.
Number of periods to generate.
Frequency alias. By default the freq is taken from start or end if those are Period objects. Otherwise, the default is "D" for daily frequency.
Name of the resulting PeriodIndex.
Notes
Of the three parameters: start, end, and periods, exactly two must be specified.
To learn more about the frequency strings, please see this link.
Examples
If start or end are Period objects, they will be used as anchor endpoints for a PeriodIndex with frequency matching that of the period_range constructor.


An ExtensionDtype for Period data.
This is not an actual numpy dtype, but a duck type.
The frequency of this PeriodDtype.
Examples
Attributes
freq
The frequency object of this PeriodDtype.
Methods
None


The frequency object of this PeriodDtype.


Immutable ndarray holding ordinal values indicating regular periods in time.
Index keys are boxed to Period objects which carries the metadata (eg, frequency information).
Optional period-like data to construct index with.
Make a copy of input ndarray.
One of pandas period strings or corresponding objects.
See also
The base pandas Index type.
Represents a period of time.
Index with datetime64 data.
Index of timedelta64 data.
Create a fixed-frequency PeriodIndex.
Examples
Attributes
day
The days of the period.
dayofweek
The day of the week with Monday=0, Sunday=6.
day_of_week
The day of the week with Monday=0, Sunday=6.
dayofyear
The ordinal day of the year.
day_of_year
The ordinal day of the year.
days_in_month
The number of days in the month.
daysinmonth
The number of days in the month.
freq
Return the frequency object if it is set, otherwise None.
freqstr
Return the frequency object as a string if its set, otherwise None.
hour
The hour of the period.
is_leap_year
Logical indicating if the date belongs to a leap year.
minute
The minute of the period.
month
The month as January=1, December=12.
quarter
The quarter of the date.
second
The second of the period.
week
The week ordinal of the year.
weekday
The day of the week with Monday=0, Sunday=6.
weekofyear
The week ordinal of the year.
year
The year of the period.
end_time
qyear
start_time
Methods
asfreq([freq, how])
Convert the PeriodArray to the specified frequency freq.
strftime(*args, **kwargs)
Convert to Index using specified date_format.
to_timestamp([freq, how])
Cast to DatetimeArray/Index.


Convert the PeriodArray to the specified frequency freq.
Equivalent to applying pandas.Period.asfreq() with the given arguments to each Period in this PeriodArray.
A frequency.
Whether the elements should be aligned to the end or start within pa period.
‘E’, ‘END’, or ‘FINISH’ for end,
‘S’, ‘START’, or ‘BEGIN’ for start.
January 31st (‘END’) vs. January 1st (‘START’) for example.
The transformed PeriodArray with the new frequency.
See also
Convert each Period in a PeriodArray to the given frequency.
Convert a Period object to the given frequency.
Examples


The day of the week with Monday=0, Sunday=6.


Return the frequency object if it is set, otherwise None.


Return the frequency object as a string if its set, otherwise None.


Logical indicating if the date belongs to a leap year.


Convert to Index using specified date_format.
Return an Index of formatted strings specified by date_format, which supports the same string format as the python standard library. Details of the string format can be found in python string format doc.
Date format string (e.g. “%Y-%m-%d”).
NumPy ndarray of formatted strings.
See also
Convert the given argument to datetime.
Return DatetimeIndex with times to midnight.
Round the DatetimeIndex to the specified freq.
Floor the DatetimeIndex to the specified freq.
Examples


Cast to DatetimeArray/Index.
Target frequency. The default is ‘D’ for week or longer, ‘S’ otherwise.
Whether to use the start or end of the time period being converted.


Return reshaped DataFrame organized by given index / column values.
Reshape data (produce a “pivot” table) based on column values. Uses unique values from specified index / columns to form axes of the resulting DataFrame. This function does not support data aggregation, multiple values will result in a MultiIndex in the columns. See the User Guide for more on reshaping.
Column to use to make new frame’s index. If None, uses existing index.
Changed in version 1.1.0: Also accept list of index names.
Column to use to make new frame’s columns.
Changed in version 1.1.0: Also accept list of columns names.
Column(s) to use for populating new frame’s values. If not specified, all remaining columns will be used and the result will have hierarchically indexed columns.
Returns reshaped DataFrame.
When there are any index, columns combinations with multiple values. DataFrame.pivot_table when you need to aggregate.
See also
Generalization of pivot that can handle duplicate values for one index/column pair.
Pivot based on the index values instead of a column.
Wide panel to long format. Less flexible but more user-friendly than melt.
Notes
For finer-tuned control, see hierarchical indexing documentation along with the related stack/unstack methods.
Examples
You could also assign a list of column names or a list of index names.
A ValueError is raised if there are any duplicates.
Notice that the first two rows are the same for our index and columns arguments.


Create a spreadsheet-style pivot table as a DataFrame.
The levels in the pivot table will be stored in MultiIndex objects (hierarchical indexes) on the index and columns of the result DataFrame.
If an array is passed, it must be the same length as the data. The list can contain any of the other types (except list). Keys to group by on the pivot table index. If an array is passed, it is being used as the same manner as column values.
If an array is passed, it must be the same length as the data. The list can contain any of the other types (except list). Keys to group by on the pivot table column. If an array is passed, it is being used as the same manner as column values.
If list of functions passed, the resulting pivot table will have hierarchical columns whose top level are the function names (inferred from the function objects themselves) If dict is passed, the key is column to aggregate and value is function or list of functions.
Value to replace missing values with (in the resulting pivot table, after aggregation).
Add all row / columns (e.g. for subtotal / grand totals).
Do not include columns whose entries are all NaN.
Name of the row / column that will contain the totals when margins is True.
This only applies if any of the groupers are Categoricals. If True: only show observed values for categorical groupers. If False: show all values for categorical groupers.
Changed in version 0.25.0.
Specifies if the result should be sorted.
New in version 1.3.0.
An Excel style pivot table.
See also
Pivot without aggregation that can handle non-numeric data.
Unpivot a DataFrame from wide to long format, optionally leaving identifiers set.
Wide panel to long format. Less flexible but more user-friendly than melt.
Examples
This first example aggregates values by taking the sum.
We can also fill missing values using the fill_value parameter.
The next example aggregates by taking the mean across multiple columns.
We can also calculate multiple types of aggregations for any given value column.


Generate a matplotlib plot of Andrews curves, for visualising clusters of multivariate data.
Andrews curves have the functional form:
x_4 sin(2t) + x_5 cos(2t) + …
Where x coefficients correspond to the values of each dimension and t is linearly spaced between -pi and +pi. Each row of frame then corresponds to a single curve.
Data to be plotted, preferably normalized to (0.0, 1.0).
Colors to use for the different classes.
Colormap to select colors from. If string, load colormap with that name from matplotlib.
Options to pass to matplotlib plotting method.
Examples


Autocorrelation plot for time series.
Options to pass to matplotlib plotting method.
Examples
The horizontal lines in the plot correspond to 95% and 99% confidence bands.
The dashed line is 99% confidence band.


Bootstrap plot on mean, median and mid-range statistics.
The bootstrap plot is used to estimate the uncertainty of a statistic by relaying on random sampling with replacement [1]. This function will generate bootstrapping plots for mean, median and mid-range statistics for the given number of samples of the given size.
“Bootstrapping (statistics)” in https://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29
Series from where to get the samplings for the bootstrapping.
If given, it will use the fig reference for plotting instead of creating a new one with default parameters.
Number of data points to consider during each sampling. It must be less than or equal to the length of the series.
Number of times the bootstrap procedure is performed.
Options to pass to matplotlib plotting method.
Matplotlib figure.
See also
Basic plotting for DataFrame objects.
Basic plotting for Series objects.
Examples
This example draws a basic bootstrap plot for a Series.


Make a box plot from DataFrame columns.
Make a box-and-whisker plot from DataFrame columns, optionally grouped by some other columns. A box plot is a method for graphically depicting groups of numerical data through their quartiles. The box extends from the Q1 to Q3 quartile values of the data, with a line at the median (Q2). The whiskers extend from the edges of box to show the range of the data. By default, they extend no more than 1.5 * IQR (IQR = Q3 - Q1) from the edges of the box, ending at the farthest data point within that interval. Outliers are plotted as separate dots.
For further details see Wikipedia’s entry for boxplot.
Column name or list of names, or vector. Can be any valid input to pandas.DataFrame.groupby().
Column in the DataFrame to pandas.DataFrame.groupby(). One box-plot will be done per value of columns in by.
The matplotlib axes to be used by boxplot.
Tick label font size in points or as a string (e.g., large).
The rotation angle of labels (in degrees) with respect to the screen coordinate system.
Setting this to True will show the grid.
The size of the figure to create in matplotlib.
For example, (3, 5) will display the subplots using 3 columns and 5 rows, starting from the top-left.
The kind of object to return. The default is axes.
‘axes’ returns the matplotlib axes the boxplot is drawn on.
‘dict’ returns a dictionary whose values are the matplotlib Lines of the boxplot.
‘both’ returns a namedtuple with the axes and dict.
when grouping with by, a Series mapping columns to return_type is returned.
If return_type is None, a NumPy array of axes with the same shape as layout is returned.
All other plotting keyword arguments to be passed to matplotlib.pyplot.boxplot().
See Notes.
See also
Make a histogram.
Matplotlib equivalent plot.
Notes
The return type depends on the return_type parameter:
‘axes’ : object of class matplotlib.axes.Axes
‘dict’ : dict of matplotlib.lines.Line2D objects
‘both’ : a namedtuple with structure (ax, lines)
For data grouped with by, return a Series of the above or a numpy array:
Series
array (for return_type = None)
Use return_type='dict' when you want to tweak the appearance of the lines after plotting. In this case a dict containing the Lines making up the boxes, caps, fliers, medians, and whiskers is returned.
Examples
Boxplots can be created for every column in the dataframe by df.boxplot() or indicating the columns to be used:
Boxplots of variables distributions grouped by the values of a third variable can be created using the option by. For instance:
A list of strings (i.e. ['X', 'Y']) can be passed to boxplot in order to group the data by combination of the variables in the x-axis:
The layout of boxplot can be adjusted giving a tuple to layout:
Additional formatting can be done to the boxplot, like suppressing the grid (grid=False), rotating the labels in the x-axis (i.e. rot=45) or changing the fontsize (i.e. fontsize=15):
The parameter return_type can be used to select the type of element returned by boxplot. When return_type='axes' is selected, the matplotlib axes on which the boxplot is drawn are returned:
When grouping with by, a Series mapping columns to return_type is returned:
If return_type is None, a NumPy array of axes with the same shape as layout is returned:


Remove pandas formatters and converters.
Removes the custom converters added by register(). This attempts to set the state of the registry back to the state before pandas registered its own units. Converters for pandas’ own types like Timestamp and Period are removed completely. Converters for types pandas overwrites, like datetime.datetime, are restored to their original value.
See also
Register pandas formatters and converters with matplotlib.


Lag plot for time series.
Matplotlib scatter method keyword arguments.
Examples
Lag plots are most commonly used to look for patterns in time series data.
Given the following time series
A lag plot with lag=1 returns


Parallel coordinates plotting.
Column name containing class names.
A list of column names to use.
Matplotlib axis object.
Colors to use for the different classes.
If true, columns will be used as xticks.
A list of values to use for xticks.
Colormap to use for line colors.
If true, vertical lines will be added at each xtick.
Options to be passed to axvline method for vertical lines.
Sort class_column labels, useful when assigning colors.
Options to pass to matplotlib plotting method.
Examples


Stores pandas plotting options.
Allows for parameter aliasing so you can just use parameter names that are the same as the plot function parameters, but is stored in a canonical format that makes it easy to breakdown into groups later.


Plot a multidimensional dataset in 2D.
Each Series in the DataFrame is represented as a evenly distributed slice on a circle. Each data point is rendered in the circle according to the value on each Series. Highly correlated Series in the DataFrame are placed closer on the unit circle.
RadViz allow to project a N-dimensional data set into a 2D space where the influence of each dimension can be interpreted as a balance between the influence of all dimensions.
More info available at the original article describing RadViz.
Object holding the data.
Column name containing the name of the data point category.
A plot instance to which to add the information.
Assign a color to each category. Example: [‘blue’, ‘green’].
Colormap to select colors from. If string, load colormap with that name from matplotlib.
Options to pass to matplotlib scatter plotting method.
See also
Plot clustering visualization.
Examples


Register pandas formatters and converters with matplotlib.
This function modifies the global matplotlib.units.registry dictionary. pandas adds custom converters for
pd.Timestamp
pd.Period
np.datetime64
datetime.datetime
datetime.date
datetime.time
See also
Remove pandas formatters and converters.


Draw a matrix of scatter plots.
Amount of transparency applied.
A tuple (width, height) in inches.
Setting this to True will show the grid.
Pick between ‘kde’ and ‘hist’ for either Kernel Density Estimation or Histogram plot in the diagonal.
Matplotlib marker type, default ‘.’.
Keyword arguments to be passed to kernel density estimate plot.
Keyword arguments to be passed to hist function.
Relative extension of axis range in x and y with respect to (x_max - x_min) or (y_max - y_min).
Keyword arguments to be passed to scatter function.
A matrix of scatter plots.
Examples


Helper function to convert DataFrame and Series to matplotlib.table.
Data for table contents.
Keyword arguments to be passed to matplotlib.table.table. If rowLabels or colLabels is not specified, data index or column name will be used.


Quantile-based discretization function.
Discretize variable into equal-sized buckets based on rank or based on sample quantiles. For example 1000 values for 10 quantiles would produce a Categorical object indicating quantile membership for each data point.
Number of quantiles. 10 for deciles, 4 for quartiles, etc. Alternately array of quantiles, e.g. [0, .25, .5, .75, 1.] for quartiles.
Used as labels for the resulting bins. Must be of the same length as the resulting bins. If False, return only integer indicators of the bins. If True, raises an error.
Whether to return the (bins, labels) or not. Can be useful if bins is given as a scalar.
The precision at which to store and display the bins labels.
If bin edges are not unique, raise ValueError or drop non-uniques.
The return type (Categorical or Series) depends on the input: a Series of type category if input is a Series else Categorical. Bins are represented as categories when categorical data is returned.
Returned only if retbins is True.
Notes
Out of bounds values will be NA in the resulting Categorical object
Examples


Immutable Index implementing a monotonic integer range.
RangeIndex is a memory-saving special case of Int64Index limited to representing monotonic ranges. Using RangeIndex may in some instances improve computing speed.
This is the default index type used by DataFrame and Series when no explicit index is provided by the user.
If int and “stop” is not given, interpreted as “stop” instead.
Unused, accepted for homogeneity with other index types.
Unused, accepted for homogeneity with other index types.
Name to be stored in the index.
See also
The base pandas Index type.
Index of int64 data.
Attributes
start
The value of the start parameter (0 if this was not supplied).
stop
The value of the stop parameter.
step
The value of the step parameter (1 if this was not supplied).
Methods
from_range(data[, name, dtype])
Create RangeIndex from a range object.


The value of the start parameter (0 if this was not supplied).


The value of the step parameter (1 if this was not supplied).


Read text from clipboard and pass to read_csv.
A string or regex delimiter. The default of ‘s+’ denotes one or more whitespace characters.
See read_csv for the full argument list.
A parsed DataFrame object.


Read a comma-separated values (csv) file into DataFrame.
Also supports optionally iterating or breaking of the file into chunks.
Additional help can be found in the online docs for IO Tools.
Any valid string path is acceptable. The string could be a URL. Valid URL schemes include http, ftp, s3, gs, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.csv.
If you want to pass in a path object, pandas accepts any os.PathLike.
By file-like object, we refer to objects with a read() method, such as a file handle (e.g. via builtin open function) or StringIO.
Delimiter to use. If sep is None, the C engine cannot automatically detect the separator, but the Python parsing engine can, meaning the latter will be used and automatically detect the separator by Python’s builtin sniffer tool, csv.Sniffer. In addition, separators longer than 1 character and different from '\s+' will be interpreted as regular expressions and will also force the use of the Python parsing engine. Note that regex delimiters are prone to ignoring quoted data. Regex example: '\r\t'.
Alias for sep.
Row number(s) to use as the column names, and the start of the data. Default behavior is to infer the column names: if no names are passed the behavior is identical to header=0 and column names are inferred from the first line of the file, if column names are passed explicitly then the behavior is identical to header=None. Explicitly pass header=0 to be able to replace existing names. The header can be a list of integers that specify row locations for a multi-index on the columns e.g. [0,1,3]. Intervening rows that are not specified will be skipped (e.g. 2 in this example is skipped). Note that this parameter ignores commented lines and empty lines if skip_blank_lines=True, so header=0 denotes the first line of data rather than the first line of the file.
List of column names to use. If the file contains a header row, then you should explicitly pass header=0 to override the column names. Duplicates in this list are not allowed.
Column(s) to use as the row labels of the DataFrame, either given as string name or column index. If a sequence of int / str is given, a MultiIndex is used.
Note: index_col=False can be used to force pandas to not use the first column as the index, e.g. when you have a malformed file with delimiters at the end of each line.
Return a subset of the columns. If list-like, all elements must either be positional (i.e. integer indices into the document columns) or strings that correspond to column names provided either by the user in names or inferred from the document header row(s). If names are given, the document header row(s) are not taken into account. For example, a valid list-like usecols parameter would be [0, 1, 2] or ['foo', 'bar', 'baz']. Element order is ignored, so usecols=[0, 1] is the same as [1, 0]. To instantiate a DataFrame from data with element order preserved use pd.read_csv(data, usecols=['foo', 'bar'])[['foo', 'bar']] for columns in ['foo', 'bar'] order or pd.read_csv(data, usecols=['foo', 'bar'])[['bar', 'foo']] for ['bar', 'foo'] order.
If callable, the callable function will be evaluated against the column names, returning names where the callable function evaluates to True. An example of a valid callable argument would be lambda x: x.upper() in
['AAA', 'BBB', 'DDD']. Using this parameter results in much faster parsing time and lower memory usage.
If the parsed data only contains one column then return a Series.
Deprecated since version 1.4.0: Append .squeeze("columns") to the call to read_csv to squeeze the data.
Prefix to add to column numbers when no header, e.g. ‘X’ for X0, X1, …
Deprecated since version 1.4.0: Use a list comprehension on the DataFrame’s columns after calling read_csv.
Duplicate columns will be specified as ‘X’, ‘X.1’, …’X.N’, rather than ‘X’…’X’. Passing in False will cause data to be overwritten if there are duplicate names in the columns.
Data type for data or columns. E.g. {‘a’: np.float64, ‘b’: np.int32, ‘c’: ‘Int64’} Use str or object together with suitable na_values settings to preserve and not interpret dtype. If converters are specified, they will be applied INSTEAD of dtype conversion.
Parser engine to use. The C and pyarrow engines are faster, while the python engine is currently more feature-complete. Multithreading is currently only supported by the pyarrow engine.
New in version 1.4.0: The “pyarrow” engine was added as an experimental engine, and some features are unsupported, or may not work correctly, with this engine.
Dict of functions for converting values in certain columns. Keys can either be integers or column labels.
Values to consider as True.
Values to consider as False.
Skip spaces after delimiter.
Line numbers to skip (0-indexed) or number of lines to skip (int) at the start of the file.
If callable, the callable function will be evaluated against the row indices, returning True if the row should be skipped and False otherwise. An example of a valid callable argument would be lambda x: x in [0, 2].
Number of lines at bottom of file to skip (Unsupported with engine=’c’).
Number of rows of file to read. Useful for reading pieces of large files.
Additional strings to recognize as NA/NaN. If dict passed, specific per-column NA values. By default the following values are interpreted as NaN: ‘’, ‘#N/A’, ‘#N/A N/A’, ‘#NA’, ‘-1.#IND’, ‘-1.#QNAN’, ‘-NaN’, ‘-nan’, ‘1.#IND’, ‘1.#QNAN’, ‘<NA>’, ‘N/A’, ‘NA’, ‘NULL’, ‘NaN’, ‘n/a’, ‘nan’, ‘null’.
Whether or not to include the default NaN values when parsing the data. Depending on whether na_values is passed in, the behavior is as follows:
If keep_default_na is True, and na_values are specified, na_values is appended to the default NaN values used for parsing.
If keep_default_na is True, and na_values are not specified, only the default NaN values are used for parsing.
If keep_default_na is False, and na_values are specified, only the NaN values specified na_values are used for parsing.
If keep_default_na is False, and na_values are not specified, no strings will be parsed as NaN.
Note that if na_filter is passed in as False, the keep_default_na and na_values parameters will be ignored.
Detect missing value markers (empty strings and the value of na_values). In data without any NAs, passing na_filter=False can improve the performance of reading a large file.
Indicate number of NA values placed in non-numeric columns.
If True, skip over blank lines rather than interpreting as NaN values.
The behavior is as follows:
boolean. If True -> try parsing the index.
list of int or names. e.g. If [1, 2, 3] -> try parsing columns 1, 2, 3 each as a separate date column.
list of lists. e.g. If [[1, 3]] -> combine columns 1 and 3 and parse as a single date column.
dict, e.g. {‘foo’ : [1, 3]} -> parse columns 1, 3 as date and call result ‘foo’
If a column or index cannot be represented as an array of datetimes, say because of an unparsable value or a mixture of timezones, the column or index will be returned unaltered as an object data type. For non-standard datetime parsing, use pd.to_datetime after pd.read_csv. To parse an index or column with a mixture of timezones, specify date_parser to be a partially-applied pandas.to_datetime() with utc=True. See Parsing a CSV with mixed timezones for more.
Note: A fast-path exists for iso8601-formatted dates.
If True and parse_dates is enabled, pandas will attempt to infer the format of the datetime strings in the columns, and if it can be inferred, switch to a faster method of parsing them. In some cases this can increase the parsing speed by 5-10x.
If True and parse_dates specifies combining multiple columns then keep the original columns.
Function to use for converting a sequence of string columns to an array of datetime instances. The default uses dateutil.parser.parser to do the conversion. Pandas will try to call date_parser in three different ways, advancing to the next if an exception occurs: 1) Pass one or more arrays (as defined by parse_dates) as arguments; 2) concatenate (row-wise) the string values from the columns defined by parse_dates into a single array and pass that; and 3) call date_parser once for each row using one or more strings (corresponding to the columns defined by parse_dates) as arguments.
DD/MM format dates, international and European format.
If True, use a cache of unique, converted dates to apply the datetime conversion. May produce significant speed-up when parsing duplicate date strings, especially ones with timezone offsets.
New in version 0.25.0.
Return TextFileReader object for iteration or getting chunks with get_chunk().
Changed in version 1.2: TextFileReader is a context manager.
Return TextFileReader object for iteration. See the IO Tools docs for more information on iterator and chunksize.
Changed in version 1.2: TextFileReader is a context manager.
For on-the-fly decompression of on-disk data. If ‘infer’ and ‘%s’ is path-like, then detect compression from the following extensions: ‘.gz’, ‘.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.
Changed in version 1.4.0: Zstandard support.
Thousands separator.
Character to recognize as decimal point (e.g. use ‘,’ for European data).
Character to break file into lines. Only valid with C parser.
The character used to denote the start and end of a quoted item. Quoted items can include the delimiter and it will be ignored.
Control field quoting behavior per csv.QUOTE_* constants. Use one of QUOTE_MINIMAL (0), QUOTE_ALL (1), QUOTE_NONNUMERIC (2) or QUOTE_NONE (3).
When quotechar is specified and quoting is not QUOTE_NONE, indicate whether or not to interpret two consecutive quotechar elements INSIDE a field as a single quotechar element.
One-character string used to escape other characters.
Indicates remainder of line should not be parsed. If found at the beginning of a line, the line will be ignored altogether. This parameter must be a single character. Like empty lines (as long as skip_blank_lines=True), fully commented lines are ignored by the parameter header but not by skiprows. For example, if comment='#', parsing #empty\na,b,c\n1,2,3 with header=0 will result in ‘a,b,c’ being treated as the header.
Encoding to use for UTF when reading/writing (ex. ‘utf-8’). List of Python standard encodings .
Changed in version 1.2: When encoding is None, errors="replace" is passed to open(). Otherwise, errors="strict" is passed to open(). This behavior was previously only the case for engine="python".
Changed in version 1.3.0: encoding_errors is a new argument. encoding has no longer an influence on how encoding errors are handled.
How encoding errors are treated. List of possible values .
New in version 1.3.0.
If provided, this parameter will override values (default or not) for the following parameters: delimiter, doublequote, escapechar, skipinitialspace, quotechar, and quoting. If it is necessary to override values, a ParserWarning will be issued. See csv.Dialect documentation for more details.
Lines with too many fields (e.g. a csv line with too many commas) will by default cause an exception to be raised, and no DataFrame will be returned. If False, then these “bad lines” will be dropped from the DataFrame that is returned.
Deprecated since version 1.3.0: The on_bad_lines parameter should be used instead to specify behavior upon encountering a bad line instead.
If error_bad_lines is False, and warn_bad_lines is True, a warning for each “bad line” will be output.
Deprecated since version 1.3.0: The on_bad_lines parameter should be used instead to specify behavior upon encountering a bad line instead.
Specifies what to do upon encountering a bad line (a line with too many fields). Allowed values are :
‘error’, raise an Exception when a bad line is encountered.
‘warn’, raise a warning when a bad line is encountered and skip that line.
‘skip’, skip bad lines without raising or warning when they are encountered.
New in version 1.3.0:
callable, function with signature (bad_line: list[str]) -> list[str] | None that will process a single bad line. bad_line is a list of strings split by the sep. If the function returns None`, the bad line will be ignored.
If the function returns a new list of strings with more elements than
expected, a ``ParserWarning will be emitted while dropping extra elements. Only supported when engine="python"
New in version 1.4.0.
Specifies whether or not whitespace (e.g. ' ' or '    ') will be used as the sep. Equivalent to setting sep='\s+'. If this option is set to True, nothing should be passed in for the delimiter parameter.
Internally process the file in chunks, resulting in lower memory use while parsing, but possibly mixed type inference. To ensure no mixed types either set False, or specify the type with the dtype parameter. Note that the entire file is read into a single DataFrame regardless, use the chunksize or iterator parameter to return the data in chunks. (Only valid with C parser).
If a filepath is provided for filepath_or_buffer, map the file object directly onto memory and access the data directly from there. Using this option can improve performance because there is no longer any I/O overhead.
Specifies which converter the C engine should use for floating-point values. The options are None or ‘high’ for the ordinary converter, ‘legacy’ for the original lower precision pandas converter, and ‘round_trip’ for the round-trip converter.
Changed in version 1.2.
Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.
New in version 1.2.
A comma-separated values (csv) file is returned as two-dimensional data structure with labeled axes.
See also
Write DataFrame to a comma-separated values (csv) file.
Read a comma-separated values (csv) file into DataFrame.
Read a table of fixed-width formatted lines into DataFrame.
Examples


Read an Excel file into a pandas DataFrame.
Supports xls, xlsx, xlsm, xlsb, odf, ods and odt file extensions read from a local filesystem or URL. Supports an option to read a single sheet or a list of sheets.
Any valid string path is acceptable. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.xlsx.
If you want to pass in a path object, pandas accepts any os.PathLike.
By file-like object, we refer to objects with a read() method, such as a file handle (e.g. via builtin open function) or StringIO.
Strings are used for sheet names. Integers are used in zero-indexed sheet positions (chart sheets do not count as a sheet position). Lists of strings/integers are used to request multiple sheets. Specify None to get all worksheets.
Available cases:
Defaults to 0: 1st sheet as a DataFrame
1: 2nd sheet as a DataFrame
"Sheet1": Load sheet with name “Sheet1”
[0, 1, "Sheet5"]: Load first, second and sheet named “Sheet5” as a dict of DataFrame
None: All worksheets.
Row (0-indexed) to use for the column labels of the parsed DataFrame. If a list of integers is passed those row positions will be combined into a MultiIndex. Use None if there is no header.
List of column names to use. If file contains no header row, then you should explicitly pass header=None.
Column (0-indexed) to use as the row labels of the DataFrame. Pass None if there is no such column. If a list is passed, those columns will be combined into a MultiIndex. If a subset of data is selected with usecols, index_col is based on the subset.
If None, then parse all columns.
If str, then indicates comma separated list of Excel column letters and column ranges (e.g. “A:E” or “A,C,E:F”). Ranges are inclusive of both sides.
If list of int, then indicates list of column numbers to be parsed.
If list of string, then indicates list of column names to be parsed.
If callable, then evaluate each column name against it and parse the column if the callable returns True.
Returns a subset of the columns according to behavior above.
If the parsed data only contains one column then return a Series.
Deprecated since version 1.4.0: Append .squeeze("columns") to the call to read_excel to squeeze the data.
Data type for data or columns. E.g. {‘a’: np.float64, ‘b’: np.int32} Use object to preserve data as stored in Excel and not interpret dtype. If converters are specified, they will be applied INSTEAD of dtype conversion.
If io is not a buffer or path, this must be set to identify io. Supported engines: “xlrd”, “openpyxl”, “odf”, “pyxlsb”. Engine compatibility :
“xlrd” supports old-style Excel files (.xls).
“openpyxl” supports newer Excel file formats.
“odf” supports OpenDocument file formats (.odf, .ods, .odt).
“pyxlsb” supports Binary Excel files.
Changed in version 1.2.0: The engine xlrd now only supports old-style .xls files. When engine=None, the following logic will be used to determine the engine:
If path_or_buffer is an OpenDocument format (.odf, .ods, .odt), then odf will be used.
Otherwise if path_or_buffer is an xls format, xlrd will be used.
Otherwise if path_or_buffer is in xlsb format, pyxlsb will be used.
New in version 1.3.0.
Otherwise openpyxl will be used.
Changed in version 1.3.0.
Dict of functions for converting values in certain columns. Keys can either be integers or column labels, values are functions that take one input argument, the Excel cell content, and return the transformed content.
Values to consider as True.
Values to consider as False.
Line numbers to skip (0-indexed) or number of lines to skip (int) at the start of the file. If callable, the callable function will be evaluated against the row indices, returning True if the row should be skipped and False otherwise. An example of a valid callable argument would be lambda
x: x in [0, 2].
Number of rows to parse.
Additional strings to recognize as NA/NaN. If dict passed, specific per-column NA values. By default the following values are interpreted as NaN: ‘’, ‘#N/A’, ‘#N/A N/A’, ‘#NA’, ‘-1.#IND’, ‘-1.#QNAN’, ‘-NaN’, ‘-nan’, ‘1.#IND’, ‘1.#QNAN’, ‘<NA>’, ‘N/A’, ‘NA’, ‘NULL’, ‘NaN’, ‘n/a’, ‘nan’, ‘null’.
Whether or not to include the default NaN values when parsing the data. Depending on whether na_values is passed in, the behavior is as follows:
If keep_default_na is True, and na_values are specified, na_values is appended to the default NaN values used for parsing.
If keep_default_na is True, and na_values are not specified, only the default NaN values are used for parsing.
If keep_default_na is False, and na_values are specified, only the NaN values specified na_values are used for parsing.
If keep_default_na is False, and na_values are not specified, no strings will be parsed as NaN.
Note that if na_filter is passed in as False, the keep_default_na and na_values parameters will be ignored.
Detect missing value markers (empty strings and the value of na_values). In data without any NAs, passing na_filter=False can improve the performance of reading a large file.
Indicate number of NA values placed in non-numeric columns.
The behavior is as follows:
bool. If True -> try parsing the index.
list of int or names. e.g. If [1, 2, 3] -> try parsing columns 1, 2, 3 each as a separate date column.
list of lists. e.g. If [[1, 3]] -> combine columns 1 and 3 and parse as a single date column.
dict, e.g. {‘foo’ : [1, 3]} -> parse columns 1, 3 as date and call result ‘foo’
If a column or index contains an unparsable date, the entire column or index will be returned unaltered as an object data type. If you don`t want to parse some cells as date just change their type in Excel to “Text”. For non-standard datetime parsing, use pd.to_datetime after pd.read_excel.
Note: A fast-path exists for iso8601-formatted dates.
Function to use for converting a sequence of string columns to an array of datetime instances. The default uses dateutil.parser.parser to do the conversion. Pandas will try to call date_parser in three different ways, advancing to the next if an exception occurs: 1) Pass one or more arrays (as defined by parse_dates) as arguments; 2) concatenate (row-wise) the string values from the columns defined by parse_dates into a single array and pass that; and 3) call date_parser once for each row using one or more strings (corresponding to the columns defined by parse_dates) as arguments.
Thousands separator for parsing string columns to numeric. Note that this parameter is only necessary for columns stored as TEXT in Excel, any numeric columns will automatically be parsed, regardless of display format.
Character to recognize as decimal point for parsing string columns to numeric. Note that this parameter is only necessary for columns stored as TEXT in Excel, any numeric columns will automatically be parsed, regardless of display format.(e.g. use ‘,’ for European data).
New in version 1.4.0.
Comments out remainder of line. Pass a character or characters to this argument to indicate comments in the input file. Any data between the comment string and the end of the current line is ignored.
Rows at the end to skip (0-indexed).
Convert integral floats to int (i.e., 1.0 –> 1). If False, all numeric data will be read in as floats: Excel stores all numbers as floats internally.
Deprecated since version 1.3.0: convert_float will be removed in a future version
Duplicate columns will be specified as ‘X’, ‘X.1’, …’X.N’, rather than ‘X’…’X’. Passing in False will cause data to be overwritten if there are duplicate names in the columns.
Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc., if using a URL that will be parsed by fsspec, e.g., starting “s3://”, “gcs://”. An error will be raised if providing this argument with a local path or a file-like buffer. See the fsspec and backend storage implementation docs for the set of allowed keys and values.
New in version 1.2.0.
DataFrame from the passed in Excel file. See notes in sheet_name argument for more information on when a dict of DataFrames is returned.
See also
Write DataFrame to an Excel file.
Write DataFrame to a comma-separated values (csv) file.
Read a comma-separated values (csv) file into DataFrame.
Read a table of fixed-width formatted lines into DataFrame.
Examples
The file can be read using the file name as string or an open file object:
Index and header can be specified via the index_col and header arguments
Column types are inferred but can be explicitly specified
True, False, and NA values, and thousands separators have defaults, but can be explicitly specified, too. Supply the values you would like as strings or lists of strings!
Comment lines in the excel input file can be skipped using the comment kwarg


Load a feather-format object from the file path.
String, path object (implementing os.PathLike[str]), or file-like object implementing a binary read() function. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.feather.
If not provided, all columns are read.
Whether to parallelize reading using multiple threads.
Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.
New in version 1.2.0.


Read a table of fixed-width formatted lines into DataFrame.
Also supports optionally iterating or breaking of the file into chunks.
Additional help can be found in the online docs for IO Tools.
String, path object (implementing os.PathLike[str]), or file-like object implementing a text read() function.The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.csv.
A list of tuples giving the extents of the fixed-width fields of each line as half-open intervals (i.e., [from, to[ ). String value ‘infer’ can be used to instruct the parser to try detecting the column specifications from the first 100 rows of the data which are not being skipped via skiprows (default=’infer’).
A list of field widths which can be used instead of ‘colspecs’ if the intervals are contiguous.
The number of rows to consider when letting the parser determine the colspecs.
Optional keyword arguments can be passed to TextFileReader.
A comma-separated values (csv) file is returned as two-dimensional data structure with labeled axes.
See also
Write DataFrame to a comma-separated values (csv) file.
Read a comma-separated values (csv) file into DataFrame.
Examples


Load data from Google BigQuery.
This function requires the pandas-gbq package.
See the How to authenticate with Google BigQuery guide for authentication instructions.
SQL-Like Query to return data values.
Google BigQuery Account project ID. Optional when available from the environment.
Name of result column to use for index in results DataFrame.
List of BigQuery column names in the desired order for results DataFrame.
Force Google BigQuery to re-authenticate the user. This is useful if multiple accounts are used.
Use the local webserver flow instead of the console flow when getting user credentials.
New in version 0.2.0 of pandas-gbq.
Note: The default value is changing to ‘standard’ in a future version.
SQL syntax dialect to use. Value can be one of:
Use BigQuery’s legacy SQL dialect. For more information see BigQuery Legacy SQL Reference.
Use BigQuery’s standard SQL, which is compliant with the SQL 2011 standard. For more information see BigQuery Standard SQL Reference.
Location where the query job should run. See the BigQuery locations documentation for a list of available locations. The location must match that of any datasets used in the query.
New in version 0.5.0 of pandas-gbq.
Query config parameters for job processing. For example:
configuration = {‘query’: {‘useQueryCache’: False}}
For more information see BigQuery REST API Reference.
Credentials for accessing Google APIs. Use this parameter to override default credentials, such as to use Compute Engine google.auth.compute_engine.Credentials or Service Account google.oauth2.service_account.Credentials directly.
New in version 0.8.0 of pandas-gbq.
Use the BigQuery Storage API to download query results quickly, but at an increased cost. To use this API, first enable it in the Cloud Console. You must also have the bigquery.readsessions.create permission on the project you are billing queries to.
This feature requires version 0.10.0 or later of the pandas-gbq package. It also requires the google-cloud-bigquery-storage and fastavro packages.
New in version 0.25.0.
If set, limit the maximum number of rows to fetch from the query results.
New in version 0.12.0 of pandas-gbq.
New in version 1.1.0.
If set, use the tqdm library to display a progress bar while the data downloads. Install the tqdm package to use this feature.
Possible values of progress_bar_type include:
No progress bar.
Use the tqdm.tqdm() function to print a progress bar to sys.stderr.
Use the tqdm.tqdm_notebook() function to display a progress bar as a Jupyter notebook widget.
Use the tqdm.tqdm_gui() function to display a progress bar as a graphical dialog box.
Note that this feature requires version 0.12.0 or later of the pandas-gbq package. And it requires the tqdm package. Slightly different than pandas-gbq, here the default is None.
New in version 1.0.0.
DataFrame representing results of query.
See also
This function in the pandas-gbq library.
Write a DataFrame to Google BigQuery.


Read from the store, close it if we opened it.
Retrieve pandas object stored in file, optionally based on where criteria.
Warning
Pandas uses PyTables for reading and writing HDF5 files, which allows serializing object-dtype data with pickle when using the “fixed” format. Loading pickled data received from untrusted sources can be unsafe.
See: https://docs.python.org/3/library/pickle.html for more.
Any valid string path is acceptable. Only supports the local file system, remote URLs and file-like objects are not supported.
If you want to pass in a path object, pandas accepts any os.PathLike.
Alternatively, pandas accepts an open pandas.HDFStore object.
The group identifier in the store. Can be omitted if the HDF file contains a single pandas object.
Mode to use when opening the file. Ignored if path_or_buf is a pandas.HDFStore. Default is ‘r’.
Specifies how encoding and decoding errors are to be handled. See the errors argument for open() for a full list of options.
A list of Term (or convertible) objects.
Row number to start selection.
Row number to stop selection.
A list of columns names to return.
Return an iterator object.
Number of rows to include in an iteration when using an iterator.
Additional keyword arguments passed to HDFStore.
The selected object. Return type depends on the object stored.
See also
Write a HDF file from a DataFrame.
Low-level access to HDF files.
Examples


Read HTML tables into a list of DataFrame objects.
String, path object (implementing os.PathLike[str]), or file-like object implementing a string read() function. The string can represent a URL or the HTML itself. Note that lxml only accepts the http, ftp and file url protocols. If you have a URL that starts with 'https' you might try removing the 's'.
The set of tables containing text matching this regex or string will be returned. Unless the HTML is extremely simple you will probably need to pass a non-empty string here. Defaults to ‘.+’ (match any non-empty string). The default value will return all tables contained on a page. This value is converted to a regular expression so that there is consistent behavior between Beautiful Soup and lxml.
The parsing engine to use. ‘bs4’ and ‘html5lib’ are synonymous with each other, they are both there for backwards compatibility. The default of None tries to use lxml to parse and if that fails it falls back on bs4 + html5lib.
The row (or list of rows for a MultiIndex) to use to make the columns headers.
The column (or list of columns) to use to create the index.
Number of rows to skip after parsing the column integer. 0-based. If a sequence of integers or a slice is given, will skip the rows indexed by that sequence. Note that a single element sequence means ‘skip the nth row’ whereas an integer means ‘skip n rows’.
This is a dictionary of attributes that you can pass to use to identify the table in the HTML. These are not checked for validity before being passed to lxml or Beautiful Soup. However, these attributes must be valid HTML table attributes to work correctly. For example,
is a valid attribute dictionary because the ‘id’ HTML tag attribute is a valid HTML attribute for any HTML tag as per this document.
is not a valid attribute dictionary because ‘asdf’ is not a valid HTML attribute even if it is a valid XML attribute. Valid HTML 4.01 table attributes can be found here. A working draft of the HTML 5 spec can be found here. It contains the latest information on table attributes for the modern web.
See read_csv() for more details.
Separator to use to parse thousands. Defaults to ','.
The encoding used to decode the web page. Defaults to None.``None`` preserves the previous encoding behavior, which depends on the underlying parser library (e.g., the parser library will try to use the encoding provided by the document).
Character to recognize as decimal point (e.g. use ‘,’ for European data).
Dict of functions for converting values in certain columns. Keys can either be integers or column labels, values are functions that take one input argument, the cell (not column) content, and return the transformed content.
Custom NA values.
If na_values are specified and keep_default_na is False the default NaN values are overridden, otherwise they’re appended to.
Whether elements with “display: none” should be parsed.
A list of DataFrames.
See also
Read a comma-separated values (csv) file into DataFrame.
Notes
Before using this function you should read the gotchas about the HTML parsing libraries.
Expect to do some cleanup after you call this function. For example, you might need to manually assign column names if the column names are converted to NaN when you pass the header=0 argument. We try to assume as little as possible about the structure of the table and push the idiosyncrasies of the HTML contained in the table to the user.
This function searches for <table> elements and only for <tr> and <th> rows and <td> elements within each <tr> or <th> element in the table. <td> stands for “table data”. This function attempts to properly handle colspan and rowspan attributes. If the function has a <thead> argument, it is used to construct the header, otherwise the function attempts to find the header within the body (by putting rows with only <th> elements into the header).
Similar to read_csv() the header argument is applied after skiprows is applied.
This function will always return a list of DataFrame or it will fail, e.g., it will not return an empty list.
Examples
See the read_html documentation in the IO section of the docs for some examples of reading in HTML tables.


Convert a JSON string to pandas object.
Any valid string path is acceptable. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.json.
If you want to pass in a path object, pandas accepts any os.PathLike.
By file-like object, we refer to objects with a read() method, such as a file handle (e.g. via builtin open function) or StringIO.
Indication of expected JSON string format. Compatible JSON strings can be produced by to_json() with a corresponding orient value. The set of possible orients is:
'split' : dict like {index -> [index], columns -> [columns], data -> [values]}
'records' : list like [{column -> value}, ... , {column -> value}]
'index' : dict like {index -> {column -> value}}
'columns' : dict like {column -> {index -> value}}
'values' : just the values array
The allowed and default values depend on the value of the typ parameter.
when typ == 'series',
allowed orients are {'split','records','index'}
default is 'index'
The Series index must be unique for orient 'index'.
when typ == 'frame',
allowed orients are {'split','records','index',
'columns','values', 'table'}
default is 'columns'
The DataFrame index must be unique for orients 'index' and 'columns'.
The DataFrame columns must be unique for orients 'index', 'columns', and 'records'.
The type of object to recover.
If True, infer dtypes; if a dict of column to dtype, then use those; if False, then don’t infer dtypes at all, applies only to the data.
For all orient values except 'table', default is True.
Changed in version 0.25.0: Not applicable for orient='table'.
Try to convert the axes to the proper dtypes.
For all orient values except 'table', default is True.
Changed in version 0.25.0: Not applicable for orient='table'.
If True then default datelike columns may be converted (depending on keep_default_dates). If False, no dates will be converted. If a list of column names, then those columns will be converted and default datelike columns may also be converted (depending on keep_default_dates).
If parsing dates (convert_dates is not False), then try to parse the default datelike columns. A column label is datelike if
it ends with '_at',
it ends with '_time',
it begins with 'timestamp',
it is 'modified', or
it is 'date'.
Direct decoding to numpy arrays. Supports numeric data only, but non-numeric column and index labels are supported. Note also that the JSON ordering MUST be the same for each term if numpy=True.
Deprecated since version 1.0.0.
Set to enable usage of higher precision (strtod) function when decoding string to double values. Default (False) is to use fast but less precise builtin functionality.
The timestamp unit to detect if converting dates. The default behaviour is to try and detect the correct precision, but if this is not desired then pass one of ‘s’, ‘ms’, ‘us’ or ‘ns’ to force parsing only seconds, milliseconds, microseconds or nanoseconds respectively.
The encoding to use to decode py3 bytes.
How encoding errors are treated. List of possible values .
New in version 1.3.0.
Read the file as a json object per line.
Return JsonReader object for iteration. See the line-delimited json docs for more information on chunksize. This can only be passed if lines=True. If this is None, the file will be read into memory all at once.
Changed in version 1.2: JsonReader is a context manager.
For on-the-fly decompression of on-disk data. If ‘infer’ and ‘path_or_buf’ is path-like, then detect compression from the following extensions: ‘.gz’, ‘.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.
Changed in version 1.4.0: Zstandard support.
The number of lines from the line-delimited jsonfile that has to be read. This can only be passed if lines=True. If this is None, all the rows will be returned.
New in version 1.1.
Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.
New in version 1.2.0.
The type returned depends on the value of typ.
See also
Convert a DataFrame to a JSON string.
Convert a Series to a JSON string.
Normalize semi-structured JSON data into a flat table.
Notes
Specific to orient='table', if a DataFrame with a literal Index name of index gets written with to_json(), the subsequent read operation will incorrectly set the Index name to None. This is because index is also used by DataFrame.to_json() to denote a missing Index name, and the subsequent read_json() operation cannot distinguish between the two. The same limitation is encountered with a MultiIndex and any names beginning with 'level_'.
Examples
Encoding/decoding a Dataframe using 'split' formatted JSON:
Encoding/decoding a Dataframe using 'index' formatted JSON:
Encoding/decoding a Dataframe using 'records' formatted JSON. Note that index labels are not preserved with this encoding.
Encoding with Table Schema


Load an ORC object from the file path, returning a DataFrame.
New in version 1.0.0.
String, path object (implementing os.PathLike[str]), or file-like object implementing a binary read() function. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.orc.
If not None, only these columns will be read from the file.
Any additional kwargs are passed to pyarrow.
Notes
Before using this function you should read the user guide about ORC and install optional dependencies.


Load a parquet object from the file path, returning a DataFrame.
String, path object (implementing os.PathLike[str]), or file-like object implementing a binary read() function. The string could be a URL. Valid URL schemes include http, ftp, s3, gs, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.parquet. A file URL can also be a path to a directory that contains multiple partitioned parquet files. Both pyarrow and fastparquet support paths to directories as well as file URLs. A directory path could be: file://localhost/path/to/tables or s3://bucket/partition_dir.
Parquet library to use. If ‘auto’, then the option io.parquet.engine is used. The default io.parquet.engine behavior is to try ‘pyarrow’, falling back to ‘fastparquet’ if ‘pyarrow’ is unavailable.
If not None, only these columns will be read from the file.
Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.
New in version 1.3.0.
If True, use dtypes that use pd.NA as missing value indicator for the resulting DataFrame. (only applicable for the pyarrow engine) As new dtypes are added that support pd.NA in the future, the output with this option will change to use those dtypes. Note: this is an experimental option, and behaviour (e.g. additional support dtypes) may change without notice.
New in version 1.2.0.
Any additional kwargs are passed to the engine.


Load pickled pandas object (or any object) from file.
Warning
Loading pickled data received from untrusted sources can be unsafe. See here.
String, path object (implementing os.PathLike[str]), or file-like object implementing a binary readlines() function.
Changed in version 1.0.0: Accept URL. URL is not limited to S3 and GCS.
For on-the-fly decompression of on-disk data. If ‘infer’ and ‘filepath_or_buffer’ is path-like, then detect compression from the following extensions: ‘.gz’, ‘.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.
Changed in version 1.4.0: Zstandard support.
Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.
New in version 1.2.0.
See also
Pickle (serialize) DataFrame object to file.
Pickle (serialize) Series object to file.
Read HDF5 file into a DataFrame.
Read SQL query or database table into a DataFrame.
Load a parquet object, returning a DataFrame.
Notes
read_pickle is only guaranteed to be backwards compatible to pandas 0.20.3.
Examples


Read SAS files stored as either XPORT or SAS7BDAT format files.
String, path object (implementing os.PathLike[str]), or file-like object implementing a binary read() function. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.sas.
If None, file format is inferred from file extension. If ‘xport’ or ‘sas7bdat’, uses the corresponding format.
Identifier of column that should be used as index of the DataFrame.
Encoding for text data. If None, text data are stored as raw bytes.
Read file chunksize lines at a time, returns iterator.
Changed in version 1.2: TextFileReader is a context manager.
If True, returns an iterator for reading the file incrementally.
Changed in version 1.2: TextFileReader is a context manager.


Load an SPSS file from the file path, returning a DataFrame.
New in version 0.25.0.
File path.
Return a subset of the columns. If None, return all columns.
Convert categorical columns into pd.Categorical.


Read SQL query or database table into a DataFrame.
This function is a convenience wrapper around read_sql_table and read_sql_query (for backward compatibility). It will delegate to the specific function depending on the provided input. A SQL query will be routed to read_sql_query, while a database table name will be routed to read_sql_table. Note that the delegated function might have more specific notes about their functionality not listed here.
SQL query to be executed or a table name.
Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. The user is responsible for engine disposal and connection closure for the SQLAlchemy connectable; str connections are closed automatically. See here.
Column(s) to set as index(MultiIndex).
Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point, useful for SQL result sets.
List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249’s paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={‘name’ : ‘value’}.
List of column names to parse as dates.
Dict of {column_name: format string} where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps.
Dict of {column_name: arg dict}, where the arg dict corresponds to the keyword arguments of pandas.to_datetime() Especially useful with databases without native Datetime support, such as SQLite.
List of column names to select from SQL table (only used when reading a table).
If specified, return an iterator where chunksize is the number of rows to include in each chunk.
See also
Read SQL database table into a DataFrame.
Read SQL query into a DataFrame.
Examples
Read data from SQL via either a SQL query or a SQL tablename. When using a SQLite database only SQL queries are accepted, providing only the SQL tablename will result in an error.
Apply date parsing to columns through the parse_dates argument
The parse_dates argument calls pd.to_datetime on the provided columns. Custom argument values for applying pd.to_datetime on a column are specified via a dictionary format: 1. Ignore errors while parsing the values of “date_column”
Apply a dayfirst date parsing order on the values of “date_column”
Apply custom formatting when date parsing the values of “date_column”


Read SQL query into a DataFrame.
Returns a DataFrame corresponding to the result set of the query string. Optionally provide an index_col parameter to use one of the columns as the index, otherwise default integer index will be used.
SQL query to be executed.
Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported.
Column(s) to set as index(MultiIndex).
Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Useful for SQL result sets.
List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249’s paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={‘name’ : ‘value’}.
List of column names to parse as dates.
Dict of {column_name: format string} where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps.
Dict of {column_name: arg dict}, where the arg dict corresponds to the keyword arguments of pandas.to_datetime() Especially useful with databases without native Datetime support, such as SQLite.
If specified, return an iterator where chunksize is the number of rows to include in each chunk.
Data type for data or columns. E.g. np.float64 or {‘a’: np.float64, ‘b’: np.int32, ‘c’: ‘Int64’}.
New in version 1.3.0.
See also
Read SQL database table into a DataFrame.
Read SQL query or database table into a DataFrame.
Notes
Any datetime values with time zone information parsed via the parse_dates parameter will be converted to UTC.


Read SQL database table into a DataFrame.
Given a table name and a SQLAlchemy connectable, returns a DataFrame. This function does not support DBAPI connections.
Name of SQL table in database.
A database URI could be provided as str. SQLite DBAPI connection mode not supported.
Name of SQL schema in database to query (if database flavor supports this). Uses default schema if None (default).
Column(s) to set as index(MultiIndex).
Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Can result in loss of Precision.
List of column names to parse as dates.
Dict of {column_name: format string} where format string is strftime compatible in case of parsing string times or is one of (D, s, ns, ms, us) in case of parsing integer timestamps.
Dict of {column_name: arg dict}, where the arg dict corresponds to the keyword arguments of pandas.to_datetime() Especially useful with databases without native Datetime support, such as SQLite.
List of column names to select from SQL table.
If specified, returns an iterator where chunksize is the number of rows to include in each chunk.
A SQL table is returned as two-dimensional data structure with labeled axes.
See also
Read SQL query into a DataFrame.
Read SQL query or database table into a DataFrame.
Notes
Any datetime values with time zone information will be converted to UTC.
Examples


Read Stata file into DataFrame.
Any valid string path is acceptable. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. A local file could be: file://localhost/path/to/table.dta.
If you want to pass in a path object, pandas accepts any os.PathLike.
By file-like object, we refer to objects with a read() method, such as a file handle (e.g. via builtin open function) or StringIO.
Convert date variables to DataFrame time values.
Read value labels and convert columns to Categorical/Factor variables.
Column to set as index.
Flag indicating whether to convert missing values to their Stata representations. If False, missing values are replaced with nan. If True, columns containing missing values are returned with object data types and missing values are represented by StataMissingValue objects.
Preserve Stata datatypes. If False, numeric data are upcast to pandas default types for foreign data (float64 or int64).
Columns to retain. Columns will be returned in the given order. None returns all columns.
Flag indicating whether converted categorical data are ordered.
Return StataReader object for iterations, returns chunks with given number of lines.
Return StataReader object.
For on-the-fly decompression of on-disk data. If ‘infer’ and ‘%s’ is path-like, then detect compression from the following extensions: ‘.gz’, ‘.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.
Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.
See also
Low-level reader for Stata data files.
Export Stata data files.
Notes
Categorical variables read through an iterator may not have the same categories and dtype. This occurs when a variable stored in a DTA file is associated to an incomplete set of value labels that only label a strict subset of the values.
Examples
Creating a dummy stata for this example >>> df = pd.DataFrame({‘animal’: [‘falcon’, ‘parrot’, ‘falcon’, … ‘parrot’], … ‘speed’: [350, 18, 361, 15]}) # doctest: +SKIP >>> df.to_stata(‘animals.dta’) # doctest: +SKIP
Read a Stata dta file:
Read a Stata dta file in 10,000 line chunks: >>> values = np.random.randint(0, 10, size=(20_000, 1), dtype=”uint8”) # doctest: +SKIP >>> df = pd.DataFrame(values, columns=[“i”]) # doctest: +SKIP >>> df.to_stata(‘filename.dta’) # doctest: +SKIP