GroupBy objects are returned by groupby calls: pandas.DataFrame.groupby(), pandas.Series.groupby(), etc.
GroupBy.__iter__()
Groupby iterator.
GroupBy.groups
Dict {group name -> group labels}.
GroupBy.indices
Dict {group name -> group indices}.
GroupBy.get_group(name[, obj])
Construct DataFrame from group with provided name.
Grouper(*args, **kwargs)
A Grouper allows the user to specify a groupby instruction for an object.
GroupBy.apply(func, *args, **kwargs)
Apply function func group-wise and combine the results together.
GroupBy.agg(func, *args, **kwargs)
SeriesGroupBy.aggregate([func, engine, ...])
Aggregate using one or more operations over the specified axis.
DataFrameGroupBy.aggregate([func, engine, ...])
Aggregate using one or more operations over the specified axis.
SeriesGroupBy.transform(func, *args[, ...])
Call function producing a like-indexed Series on each group and return a Series having the same indexes as the original object filled with the transformed values.
DataFrameGroupBy.transform(func, *args[, ...])
Call function producing a like-indexed DataFrame on each group and return a DataFrame having the same indexes as the original object filled with the transformed values.
GroupBy.pipe(func, *args, **kwargs)
Apply a function func with arguments to this GroupBy object and return the function's result.
GroupBy.all([skipna])
Return True if all values in the group are truthful, else False.
GroupBy.any([skipna])
Return True if any value in the group is truthful, else False.
GroupBy.bfill([limit])
Backward fill the values.
GroupBy.backfill([limit])
Backward fill the values.
GroupBy.count()
Compute count of group, excluding missing values.
GroupBy.cumcount([ascending])
Number each item in each group from 0 to the length of that group - 1.
GroupBy.cummax([axis])
Cumulative max for each group.
GroupBy.cummin([axis])
Cumulative min for each group.
GroupBy.cumprod([axis])
Cumulative product for each group.
GroupBy.cumsum([axis])
Cumulative sum for each group.
GroupBy.ffill([limit])
Forward fill the values.
GroupBy.first([numeric_only, min_count])
Compute first of group values.
GroupBy.head([n])
Return first n rows of each group.
GroupBy.last([numeric_only, min_count])
Compute last of group values.
GroupBy.max([numeric_only, min_count])
Compute max of group values.
GroupBy.mean([numeric_only, engine, ...])
Compute mean of groups, excluding missing values.
GroupBy.median([numeric_only])
Compute median of groups, excluding missing values.
GroupBy.min([numeric_only, min_count])
Compute min of group values.
GroupBy.ngroup([ascending])
Number each group from 0 to the number of groups - 1.
GroupBy.nth(n[, dropna])
Take the nth row from each group if n is an int, otherwise a subset of rows.
GroupBy.ohlc()
Compute open, high, low and close values of a group, excluding missing values.
GroupBy.pad([limit])
Forward fill the values.
GroupBy.prod([numeric_only, min_count])
Compute prod of group values.
GroupBy.rank([method, ascending, na_option, ...])
Provide the rank of values within each group.
GroupBy.pct_change([periods, fill_method, ...])
Calculate pct_change of each value to previous entry in group.
GroupBy.size()
Compute group sizes.
GroupBy.sem([ddof])
Compute standard error of the mean of groups, excluding missing values.
GroupBy.std([ddof, engine, engine_kwargs])
Compute standard deviation of groups, excluding missing values.
GroupBy.sum([numeric_only, min_count, ...])
Compute sum of group values.
GroupBy.var([ddof, engine, engine_kwargs])
Compute variance of groups, excluding missing values.
GroupBy.tail([n])
Return last n rows of each group.
The following methods are available in both SeriesGroupBy and DataFrameGroupBy objects, but may differ slightly, usually in that the DataFrameGroupBy version usually permits the specification of an axis argument, and often an argument indicating whether to restrict application to columns of a specific data type.
DataFrameGroupBy.all([skipna])
Return True if all values in the group are truthful, else False.
DataFrameGroupBy.any([skipna])
Return True if any value in the group is truthful, else False.
DataFrameGroupBy.backfill([limit])
Backward fill the values.
DataFrameGroupBy.bfill([limit])
Backward fill the values.
DataFrameGroupBy.corr
Compute pairwise correlation of columns, excluding NA/null values.
DataFrameGroupBy.count()
Compute count of group, excluding missing values.
DataFrameGroupBy.cov
Compute pairwise covariance of columns, excluding NA/null values.
DataFrameGroupBy.cumcount([ascending])
Number each item in each group from 0 to the length of that group - 1.
DataFrameGroupBy.cummax([axis])
Cumulative max for each group.
DataFrameGroupBy.cummin([axis])
Cumulative min for each group.
DataFrameGroupBy.cumprod([axis])
Cumulative product for each group.
DataFrameGroupBy.cumsum([axis])
Cumulative sum for each group.
DataFrameGroupBy.describe(**kwargs)
Generate descriptive statistics.
DataFrameGroupBy.diff
First discrete difference of element.
DataFrameGroupBy.ffill([limit])
Forward fill the values.
DataFrameGroupBy.fillna
Fill NA/NaN values using the specified method.
DataFrameGroupBy.filter(func[, dropna])
Return a copy of a DataFrame excluding filtered elements.
DataFrameGroupBy.hist
Make a histogram of the DataFrame's columns.
DataFrameGroupBy.idxmax([axis, skipna])
Return index of first occurrence of maximum over requested axis.
DataFrameGroupBy.idxmin([axis, skipna])
Return index of first occurrence of minimum over requested axis.
DataFrameGroupBy.mad
Return the mean absolute deviation of the values over the requested axis.
DataFrameGroupBy.nunique([dropna])
Return DataFrame with counts of unique elements in each position.
DataFrameGroupBy.pad([limit])
Forward fill the values.
DataFrameGroupBy.pct_change([periods, ...])
Calculate pct_change of each value to previous entry in group.
DataFrameGroupBy.plot
Class implementing the .plot attribute for groupby objects.
DataFrameGroupBy.quantile([q, interpolation])
Return group values at the given quantile, a la numpy.percentile.
DataFrameGroupBy.rank([method, ascending, ...])
Provide the rank of values within each group.
DataFrameGroupBy.resample(rule, *args, **kwargs)
Provide resampling when using a TimeGrouper.
DataFrameGroupBy.sample([n, frac, replace, ...])
Return a random sample of items from each group.
DataFrameGroupBy.shift([periods, freq, ...])
Shift each group by periods observations.
DataFrameGroupBy.size()
Compute group sizes.
DataFrameGroupBy.skew
Return unbiased skew over requested axis.
DataFrameGroupBy.take
Return the elements in the given positional indices along an axis.
DataFrameGroupBy.tshift
(DEPRECATED) Shift the time index, using the index's frequency if available.
DataFrameGroupBy.value_counts([subset, ...])
Return a Series or DataFrame containing counts of unique rows.
The following methods are available only for SeriesGroupBy objects.
SeriesGroupBy.hist
Draw histogram of the input series using matplotlib.
SeriesGroupBy.nlargest([n, keep])
Return the largest n elements.
SeriesGroupBy.nsmallest([n, keep])
Return the smallest n elements.
SeriesGroupBy.nunique([dropna])
Return number of unique elements in the group.
SeriesGroupBy.unique
Return unique values of Series object.
SeriesGroupBy.value_counts([normalize, ...])
SeriesGroupBy.is_monotonic_increasing
Alias for is_monotonic.
SeriesGroupBy.is_monotonic_decreasing
Return boolean if values in the object are monotonic_decreasing.
The following methods are available only for DataFrameGroupBy objects.
DataFrameGroupBy.corrwith
Compute pairwise correlation.
DataFrameGroupBy.boxplot([subplots, column, ...])
Make box plots from DataFrameGroupBy data.


read_pickle(filepath_or_buffer[, ...])
Load pickled pandas object (or any object) from file.
DataFrame.to_pickle(path[, compression, ...])
Pickle (serialize) object to file.
read_table(filepath_or_buffer[, sep, ...])
Read general delimited file into DataFrame.
read_csv(filepath_or_buffer[, sep, ...])
Read a comma-separated values (csv) file into DataFrame.
DataFrame.to_csv([path_or_buf, sep, na_rep, ...])
Write object to a comma-separated values (csv) file.
read_fwf(filepath_or_buffer[, colspecs, ...])
Read a table of fixed-width formatted lines into DataFrame.
read_clipboard([sep])
Read text from clipboard and pass to read_csv.
DataFrame.to_clipboard([excel, sep])
Copy object to the system clipboard.
read_excel(io[, sheet_name, header, names, ...])
Read an Excel file into a pandas DataFrame.
DataFrame.to_excel(excel_writer[, ...])
Write object to an Excel sheet.
ExcelFile.parse([sheet_name, header, names, ...])
Parse specified sheet(s) into a DataFrame.
Styler.to_excel(excel_writer[, sheet_name, ...])
Write Styler to an Excel sheet.
ExcelWriter(path[, engine, date_format, ...])
Class for writing DataFrame objects into excel sheets.
read_json([path_or_buf, orient, typ, dtype, ...])
Convert a JSON string to pandas object.
json_normalize(data[, record_path, meta, ...])
Normalize semi-structured JSON data into a flat table.
DataFrame.to_json([path_or_buf, orient, ...])
Convert the object to a JSON string.
build_table_schema(data[, index, ...])
Create a Table schema from data.
read_html(io[, match, flavor, header, ...])
Read HTML tables into a list of DataFrame objects.
DataFrame.to_html([buf, columns, col_space, ...])
Render a DataFrame as an HTML table.
Styler.to_html([buf, table_uuid, ...])
Write Styler to a file, buffer or string in HTML-CSS format.
read_xml(path_or_buffer[, xpath, ...])
Read XML document into a DataFrame object.
DataFrame.to_xml([path_or_buffer, index, ...])
Render a DataFrame to an XML document.
DataFrame.to_latex([buf, columns, ...])
Render object to a LaTeX tabular, longtable, or nested table.
Styler.to_latex([buf, column_format, ...])
Write Styler to a file, buffer or string in LaTeX format.
read_hdf(path_or_buf[, key, mode, errors, ...])
Read from the store, close it if we opened it.
HDFStore.put(key, value[, format, index, ...])
Store object in HDFStore.
HDFStore.append(key, value[, format, axes, ...])
Append to Table in file.
HDFStore.get(key)
Retrieve pandas object stored in file.
HDFStore.select(key[, where, start, stop, ...])
Retrieve pandas object stored in file, optionally based on where criteria.
HDFStore.info()
Print detailed information on the store.
HDFStore.keys([include])
Return a list of keys corresponding to objects stored in HDFStore.
HDFStore.groups()
Return a list of all the top-level nodes.
HDFStore.walk([where])
Walk the pytables group hierarchy for pandas objects.
Warning
One can store a subclass of DataFrame or Series to HDF5, but the type of the subclass is lost upon storing.
read_feather(path[, columns, use_threads, ...])
Load a feather-format object from the file path.
DataFrame.to_feather(path, **kwargs)
Write a DataFrame to the binary Feather format.
read_parquet(path[, engine, columns, ...])
Load a parquet object from the file path, returning a DataFrame.
DataFrame.to_parquet([path, engine, ...])
Write a DataFrame to the binary parquet format.
read_orc(path[, columns])
Load an ORC object from the file path, returning a DataFrame.
read_sas(filepath_or_buffer[, format, ...])
Read SAS files stored as either XPORT or SAS7BDAT format files.
read_spss(path[, usecols, convert_categoricals])
Load an SPSS file from the file path, returning a DataFrame.
read_sql_table(table_name, con[, schema, ...])
Read SQL database table into a DataFrame.
read_sql_query(sql, con[, index_col, ...])
Read SQL query into a DataFrame.
read_sql(sql, con[, index_col, ...])
Read SQL query or database table into a DataFrame.
DataFrame.to_sql(name, con[, schema, ...])
Write records stored in a DataFrame to a SQL database.
read_gbq(query[, project_id, index_col, ...])
Load data from Google BigQuery.
read_stata(filepath_or_buffer[, ...])
Read Stata file into DataFrame.
DataFrame.to_stata(path[, convert_dates, ...])
Export DataFrame object to Stata dta format.
StataReader.data_label
Return data label of Stata file.
StataReader.value_labels()
Return a dict, associating each variable name a dict, associating each value its corresponding label.
StataReader.variable_labels()
Return variable labels as a dict, associating each variable name with corresponding label.
StataWriter.write_file()
Export DataFrame object to Stata dta format.


Abstract base class for custom 1-D array types.
pandas will recognize instances of this class as proper arrays with a custom type and will not attempt to coerce them to objects. They may be stored directly inside a DataFrame or Series.
Notes
The interface includes the following abstract methods that must be implemented by subclasses:
_from_sequence
_from_factorized
__getitem__
__len__
__eq__
dtype
nbytes
isna
take
copy
_concat_same_type
A default repr displaying the type, (truncated) data, length, and dtype is provided. It can be customized or replaced by by overriding:
__repr__ : A default repr for the ExtensionArray.
_formatter : Print scalars inside a Series or DataFrame.
Some methods require casting the ExtensionArray to an ndarray of Python objects with self.astype(object), which may be expensive. When performance is a concern, we highly recommend overriding the following methods:
fillna
dropna
unique
factorize / _values_for_factorize
argsort / _values_for_argsort
searchsorted
The remaining methods implemented on this class should be performant, as they only compose abstract methods. Still, a more efficient implementation may be available, and these methods can be overridden.
One can implement methods to handle array reductions.
_reduce
One can implement methods to handle parsing from strings that will be used in methods such as pandas.io.parsers.read_csv.
_from_sequence_of_strings
This class does not inherit from ‘abc.ABCMeta’ for performance reasons. Methods and properties required by the interface raise pandas.errors.AbstractMethodError and no register method is provided for registering virtual subclasses.
ExtensionArrays are limited to 1 dimension.
They may be backed by none, one, or many NumPy arrays. For example, pandas.Categorical is an extension array backed by two arrays, one for codes and one for categories. An array of IPv6 address may be backed by a NumPy structured array with two fields, one for the lower 64 bits and one for the upper 64 bits. Or they may be backed by some other storage type, like Python lists. Pandas makes no assumptions on how the data are stored, just that it can be converted to a NumPy array. The ExtensionArray interface does not impose any rules on how this data is stored. However, currently, the backing data cannot be stored in attributes called .values or ._values to ensure full compatibility with pandas internals. But other names as .data, ._data, ._items, … can be freely used.
If implementing NumPy’s __array_ufunc__ interface, pandas expects that
You defer by returning NotImplemented when any Series are present in inputs. Pandas will extract the arrays and call the ufunc again.
You define a _HANDLED_TYPES tuple as an attribute on the class. Pandas inspect this to determine whether the ufunc is valid for the types present.
See NumPy universal functions for more.
By default, ExtensionArrays are not hashable. Immutable subclasses may override this behavior.
Attributes
dtype
An instance of 'ExtensionDtype'.
nbytes
The number of bytes needed to store this object in memory.
ndim
Extension Arrays are only allowed to be 1-dimensional.
shape
Return a tuple of the array dimensions.
Methods
argsort([ascending, kind, na_position])
Return the indices that would sort this array.
astype(dtype[, copy])
Cast to a NumPy array or ExtensionArray with 'dtype'.
copy()
Return a copy of the array.
dropna()
Return ExtensionArray without NA values.
factorize([na_sentinel])
Encode the extension array as an enumerated type.
fillna([value, method, limit])
Fill NA/NaN values using the specified method.
equals(other)
Return if another array is equivalent to this array.
insert(loc, item)
Insert an item at the given position.
isin(values)
Pointwise comparison for set containment in the given values.
isna()
A 1-D array indicating if each value is missing.
ravel([order])
Return a flattened view on this array.
repeat(repeats[, axis])
Repeat elements of a ExtensionArray.
searchsorted(value[, side, sorter])
Find indices where elements should be inserted to maintain order.
shift([periods, fill_value])
Shift values by desired number.
take(indices, *[, allow_fill, fill_value])
Take elements from an array.
tolist()
Return a list of the values.
unique()
Compute the ExtensionArray of unique values.
view([dtype])
Return a view on the array.
_concat_same_type(to_concat)
Concatenate multiple array of this dtype.
_formatter([boxed])
Formatting function for scalar values.
_from_factorized(values, original)
Reconstruct an ExtensionArray after factorization.
_from_sequence(scalars, *[, dtype, copy])
Construct a new ExtensionArray from a sequence of scalars.
_from_sequence_of_strings(strings, *[, ...])
Construct a new ExtensionArray from a sequence of strings.
_reduce(name, *[, skipna])
Return a scalar result of performing the reduction operation.
_values_for_argsort()
Return values for sorting.
_values_for_factorize()
Return an array and missing value suitable for factorization.


Concatenate multiple array of this dtype.


Formatting function for scalar values.
This is used in the default ‘__repr__’. The returned formatting function receives instances of your scalar type.
An indicated for whether or not your array is being printed within a Series, DataFrame, or Index (True), or just by itself (False). This may be useful if you want scalar values to appear differently within a Series versus on its own (e.g. quoted or not).
A callable that gets instances of the scalar type and returns a string. By default, repr() is used when boxed=False and str() is used when boxed=True.


Reconstruct an ExtensionArray after factorization.
An integer ndarray with the factorized values.
The original ExtensionArray that factorize was called on.
See also
Top-level factorize method that dispatches here.
Encode the extension array as an enumerated type.


Construct a new ExtensionArray from a sequence of scalars.
Each element will be an instance of the scalar type for this array, cls.dtype.type or be converted into this type in this method.
Construct for this particular dtype. This should be a Dtype compatible with the ExtensionArray.
If True, copy the underlying data.


Construct a new ExtensionArray from a sequence of strings.
Each element will be an instance of the scalar type for this array, cls.dtype.type.
Construct for this particular dtype. This should be a Dtype compatible with the ExtensionArray.
If True, copy the underlying data.


Return a scalar result of performing the reduction operation.
Name of the function, supported values are: { any, all, min, max, sum, mean, median, prod, std, var, sem, kurt, skew }.
If True, skip NaN values.
Additional keyword arguments passed to the reduction function. Currently, ddof is the only supported kwarg.


Return values for sorting.
The transformed values should maintain the ordering between values within the array.
See also
Return the indices that would sort this array.


Return an array and missing value suitable for factorization.
An array suitable for factorization. This should maintain order and be a supported dtype (Float64, Int64, UInt64, String, Object). By default, the extension array is cast to object dtype.
The value in values to consider missing. This will be treated as NA in the factorization routines, so it will be coded as na_sentinel and not included in uniques. By default, np.nan is used.
Notes
The values returned by this method are also used in pandas.util.hash_pandas_object().


Return the indices that would sort this array.
Whether the indices should result in an ascending or descending sort.
Sorting algorithm.
Passed through to numpy.argsort().
Array of indices that sort self. If NaN values are contained, NaN values are placed at the end.
See also
Sorting implementation used internally.


Cast to a NumPy array or ExtensionArray with ‘dtype’.
Typecode or data-type to which the array is cast.
Whether to copy the data, even if not necessary. If False, a copy is made only if the old dtype does not match the new dtype.
An ExtensionArray if dtype is ExtensionDtype, Otherwise a NumPy ndarray with ‘dtype’ for its dtype.


Return ExtensionArray without NA values.


Return if another array is equivalent to this array.
Equivalent means that both arrays have the same shape and dtype, and all values compare equal. Missing values in the same location are considered equal (in contrast with normal equality).
Array to compare to this Array.
Whether the arrays are equivalent.


Encode the extension array as an enumerated type.
Value to use in the codes array to indicate missing values.
An integer NumPy array that’s an indexer into the original ExtensionArray.
An ExtensionArray containing the unique values of self.
Note
uniques will not contain an entry for the NA value of the ExtensionArray if there are any missing values present in self.
See also
Top-level factorize method that dispatches here.
Notes
pandas.factorize() offers a sort keyword as well.


Fill NA/NaN values using the specified method.
If a scalar value is passed it is used to fill all missing values. Alternatively, an array-like ‘value’ can be given. It’s expected that the array-like have the same length as ‘self’.
Method to use for filling holes in reindexed Series pad / ffill: propagate last valid observation forward to next valid backfill / bfill: use NEXT valid observation to fill gap.
If method is specified, this is the maximum number of consecutive NaN values to forward/backward fill. In other words, if there is a gap with more than this number of consecutive NaNs, it will only be partially filled. If method is not specified, this is the maximum number of entries along the entire axis where NaNs will be filled.
With NA/NaN filled.


Insert an item at the given position.
Notes
This method should be both type and dtype-preserving. If the item cannot be held in an array of this type/dtype, either ValueError or TypeError should be raised.
The default implementation relies on _from_sequence to raise on invalid items.


Pointwise comparison for set containment in the given values.
Roughly equivalent to np.array([x in values for x in self])


A 1-D array indicating if each value is missing.
In most cases, this should return a NumPy ndarray. For exceptional cases like SparseArray, where returning an ndarray would be expensive, an ExtensionArray may be returned.
Notes
If returning an ExtensionArray, then
na_values._is_boolean should be True
na_values should implement ExtensionArray._reduce()
na_values.any and na_values.all should be implemented


The number of bytes needed to store this object in memory.


Extension Arrays are only allowed to be 1-dimensional.


Return a flattened view on this array.
Notes
Because ExtensionArrays are 1D-only, this is a no-op.
The “order” argument is ignored, is for compatibility with NumPy.


Repeat elements of a ExtensionArray.
Returns a new ExtensionArray where each element of the current ExtensionArray is repeated consecutively a given number of times.
The number of repetitions for each element. This should be a non-negative integer. Repeating 0 times will return an empty ExtensionArray.
Must be None. Has no effect but is accepted for compatibility with numpy.
Newly created ExtensionArray with repeated elements.
See also
Equivalent function for Series.
Equivalent function for Index.
Similar method for numpy.ndarray.
Take arbitrary positions.
Examples


Find indices where elements should be inserted to maintain order.
Find the indices into a sorted array self (a) such that, if the corresponding elements in value were inserted before the indices, the order of self would be preserved.
Assuming that self is sorted:
side
returned index i satisfies
left
self[i-1] < value <= self[i]
right
self[i-1] <= value < self[i]
Value(s) to insert into self.
If ‘left’, the index of the first suitable location found is given. If ‘right’, return the last such index. If there is no suitable index, return either 0 or N (where N is the length of self).
Optional array of integer indices that sort array a into ascending order. They are typically the result of argsort.
If value is array-like, array of insertion points. If value is scalar, a single integer.
See also
Similar method from NumPy.


Shift values by desired number.
Newly introduced missing values are filled with self.dtype.na_value.
The number of periods to shift. Negative values are allowed for shifting backwards.
The scalar value to use for newly introduced missing values. The default is self.dtype.na_value.
Shifted.
Notes
If self is empty or periods is 0, a copy of self is returned.
If periods > len(self), then an array of size len(self) is returned, with all values filled with self.dtype.na_value.


Take elements from an array.
Indices to be taken.
How to handle negative values in indices.
False: negative values in indices indicate positional indices from the right (the default). This is similar to numpy.take().
True: negative values in indices indicate missing values. These values are set to fill_value. Any other other negative values raise a ValueError.
Fill value to use for NA-indices when allow_fill is True. This may be None, in which case the default NA value for the type, self.dtype.na_value, is used.
For many ExtensionArrays, there will be two representations of fill_value: a user-facing “boxed” scalar, and a low-level physical NA value. fill_value should be the user-facing version, and the implementation should handle translating that to the physical version for processing the take if necessary.
When the indices are out of bounds for the array.
When indices contains negative values other than -1 and allow_fill is True.
See also
Take elements from an array along an axis.
Take elements from an array.
Notes
ExtensionArray.take is called by Series.__getitem__, .loc, iloc, when indices is a sequence of values. Additionally, it’s called by Series.reindex(), or any other method that causes realignment, with a fill_value.
Examples
Here’s an example implementation, which relies on casting the extension array to object dtype. This uses the helper method pandas.api.extensions.take().


Return a list of the values.
These are each a scalar type, which is a Python scalar (for str, int, float) or a pandas scalar (for Timestamp/Timedelta/Interval/Period)


Compute the ExtensionArray of unique values.


Return a view on the array.
Default None.
A view on the ExtensionArray’s data.


A custom data type, to be paired with an ExtensionArray.
See also
Register an ExtensionType with pandas as class decorator.
Abstract base class for custom 1-D array types.
Notes
The interface includes the following abstract methods that must be implemented by subclasses:
type
name
construct_array_type
The following attributes and methods influence the behavior of the dtype in pandas operations
_is_numeric
_is_boolean
_get_common_dtype
The na_value class attribute can be used to set the default NA value for this type. numpy.nan is used by default.
ExtensionDtypes are required to be hashable. The base class provides a default implementation, which relies on the _metadata class attribute. _metadata should be a tuple containing the strings that define your data type. For example, with PeriodDtype that’s the freq attribute.
If you have a parametrized dtype you should set the ``_metadata`` class property.
Ideally, the attributes in _metadata will match the parameters to your ExtensionDtype.__init__ (if any). If any of the attributes in _metadata don’t implement the standard __eq__ or __hash__, the default implementations here will not work.
For interaction with Apache Arrow (pyarrow), a __from_arrow__ method can be implemented: this method receives a pyarrow Array or ChunkedArray as only argument and is expected to return the appropriate pandas ExtensionArray for this dtype and the passed values:
This class does not inherit from ‘abc.ABCMeta’ for performance reasons. Methods and properties required by the interface raise pandas.errors.AbstractMethodError and no register method is provided for registering virtual subclasses.
Attributes
kind
A character code (one of 'biufcmMOSUV'), default 'O'
na_value
Default NA value to use for this type.
name
A string identifying the data type.
names
Ordered list of field names, or None if there are no fields.
type
The scalar type for the array, e.g.
Methods
construct_array_type()
Return the array type associated with this dtype.
construct_from_string(string)
Construct this type from a string.
empty(shape)
Construct an ExtensionArray of this dtype with the given shape.
is_dtype(dtype)
Check if we match 'dtype'.


Return the array type associated with this dtype.


Construct this type from a string.
This is useful mainly for data types that accept parameters. For example, a period dtype accepts a frequency parameter that can be set as period[H] (where H means hourly frequency).
By default, in the abstract class, just the name of the type is expected. But subclasses can overwrite this method to accept parameters.
The name of the type, for example category.
Instance of the dtype.
If a class cannot be constructed from this ‘string’.
Examples
For extension dtypes with arguments the following may be an adequate implementation.


Construct an ExtensionArray of this dtype with the given shape.
Analogous to numpy.empty.


Check if we match ‘dtype’.
The object to check.
Notes
The default implementation is True if
cls.construct_from_string(dtype) is an instance of cls.
dtype is an object and is an instance of cls
dtype has a dtype attribute, and any of the above conditions is true for dtype.dtype.


A character code (one of ‘biufcmMOSUV’), default ‘O’
This should match the NumPy dtype used when the array is converted to an ndarray, which is probably ‘O’ for object if the extension type cannot be represented as a built-in NumPy type.
See also


Default NA value to use for this type.
This is used in e.g. ExtensionArray.take. This should be the user-facing “boxed” version of the NA value, not the physical NA value for storage. e.g. for JSONArray, this is an empty dictionary.


A string identifying the data type.
Will be used for display in, e.g. Series.dtype


Ordered list of field names, or None if there are no fields.
This is for compatibility with NumPy arrays, and may be removed in the future.


The scalar type for the array, e.g. int
It’s expected ExtensionArray[item] returns an instance of ExtensionDtype.type for scalar item, assuming that value is valid (not NA). NA values do not need to be instances of type.


Register a custom accessor on DataFrame objects.
Name under which the accessor should be registered. A warning is issued if this name conflicts with a preexisting attribute.
A class decorator.
See also
Register a custom accessor on DataFrame objects.
Register a custom accessor on Series objects.
Register a custom accessor on Index objects.
Notes
When accessed, your accessor will be initialized with the pandas object the user is interacting with. So the signature must be
For consistency with pandas methods, you should raise an AttributeError if the data passed to your accessor has an incorrect dtype.
Examples
In your library code:
Back in an interactive IPython session:


Register an ExtensionType with pandas as class decorator.
This enables operations like .astype(name) for the name of the ExtensionDtype.
A class decorator.
Examples


Register a custom accessor on Index objects.
Name under which the accessor should be registered. A warning is issued if this name conflicts with a preexisting attribute.
A class decorator.
See also
Register a custom accessor on DataFrame objects.
Register a custom accessor on Series objects.
Register a custom accessor on Index objects.
Notes
When accessed, your accessor will be initialized with the pandas object the user is interacting with. So the signature must be
For consistency with pandas methods, you should raise an AttributeError if the data passed to your accessor has an incorrect dtype.
Examples
In your library code:
Back in an interactive IPython session:


Register a custom accessor on Series objects.
Name under which the accessor should be registered. A warning is issued if this name conflicts with a preexisting attribute.
A class decorator.
See also
Register a custom accessor on DataFrame objects.
Register a custom accessor on Series objects.
Register a custom accessor on Index objects.
Notes
When accessed, your accessor will be initialized with the pandas object the user is interacting with. So the signature must be
For consistency with pandas methods, you should raise an AttributeError if the data passed to your accessor has an incorrect dtype.
Examples
In your library code:
Back in an interactive IPython session:


Base class for window bounds calculations.
Methods
get_window_bounds([num_values, min_periods, ...])
Computes the bounds of a window.


Computes the bounds of a window.
number of values that will be aggregated over
the number of rows in a window
min_periods passed from the top level rolling API
center passed from the top level rolling API
closed passed from the top level rolling API
win_type passed from the top level rolling API


Check if indexer is a valid array indexer for array.
For a boolean mask, array and indexer are checked to have the same length. The dtype is validated, and if it is an integer or boolean ExtensionArray, it is checked if there are missing values present, and it is converted to the appropriate numpy array. Other dtypes will raise an error.
Non-array indexers (integer, slice, Ellipsis, tuples, ..) are passed through as is.
New in version 1.0.0.
The array that is being indexed (only used for the length).
The array-like that’s used to index. List-like input that is not yet a numpy array or an ExtensionArray is converted to one. Other input types are passed through as is.
The validated indexer as a numpy array that can be used to index.
When the lengths don’t match.
When indexer cannot be converted to a numpy ndarray to index (e.g. presence of missing values).
See also
Check if key is of boolean dtype.
Examples
When checking a boolean mask, a boolean ndarray is returned when the arguments are all valid.
An IndexError is raised when the lengths don’t match.
NA values in a boolean array are treated as False.
A numpy boolean mask will get passed through (if the length is correct):
Similarly for integer indexers, an integer ndarray is returned when it is a valid indexer, otherwise an error is (for integer indexers, a matching length is not required):
For non-integer/boolean dtypes, an appropriate error is raised:


Creates window boundaries for fixed-length windows that include the current row.
Examples
Methods
get_window_bounds([num_values, min_periods, ...])
Computes the bounds of a window.


Calculate window boundaries based on a non-fixed offset such as a BusinessDay.
Methods
get_window_bounds([num_values, min_periods, ...])
Computes the bounds of a window.


Efficiently infer the type of a passed val, or list-like array of values. Return a string describing the type.
Ignore NaN values when inferring the type.
Describing the common type of the input data.
If ndarray-like but cannot infer the dtype
Notes
‘mixed’ is the catchall for anything that is not otherwise specialized
‘mixed-integer-float’ are floats and integers
‘mixed-integer’ are integers mixed with non-integers
‘unknown-array’ is the catchall for something that is an array (has a dtype attribute), but has a dtype unknown to pandas (e.g. external extension array)
Examples


Check whether the provided array or dtype is of a boolean dtype.
The array or dtype to check.
Whether or not the array or dtype is of a boolean dtype.
Notes
An ExtensionArray is considered boolean when the _is_boolean attribute is set to True.
Examples


Check whether an array-like is a Categorical instance.
The array-like to check.
Whether or not the array-like is of a Categorical instance.
Examples
Categoricals, Series Categoricals, and CategoricalIndex will return True.


Check whether an array-like or dtype is of the Categorical dtype.
The array-like or dtype to check.
Whether or not the array-like or dtype is of the Categorical dtype.
Examples


Check whether the provided array or dtype is of a complex dtype.
The array or dtype to check.
Whether or not the array or dtype is of a complex dtype.
Examples


Check whether the provided array or dtype is of the datetime64 dtype.
The array or dtype to check.
Whether or not the array or dtype is of the datetime64 dtype.
Examples


Check whether an array-like or dtype is of the datetime64 dtype.
The array-like or dtype to check.
Whether or not the array-like or dtype is of the datetime64 dtype.
Examples


Check whether the provided array or dtype is of the datetime64[ns] dtype.
The array or dtype to check.
Whether or not the array or dtype is of the datetime64[ns] dtype.
Examples


Check whether an array-like or dtype is of a DatetimeTZDtype dtype.
The array-like or dtype to check.
Whether or not the array-like or dtype is of a DatetimeTZDtype dtype.
Examples


Check if the object is dict-like.
Whether obj has dict-like properties.
Examples


Check if an object is a pandas extension array type.
See the Use Guide for more.
For array-like input, the .dtype attribute will be extracted.
Whether the arr_or_dtype is an extension array type.
Notes
This checks whether an object implements the pandas extension array interface. In pandas, this includes:
Categorical
Sparse
Interval
Period
DatetimeArray
TimedeltaArray
Third-party libraries may implement arrays or types satisfying this interface as well.
Examples


Check whether an array-like is of a pandas extension class instance.
Deprecated since version 1.0.0: Use is_extension_array_dtype instead.
Extension classes include categoricals, pandas sparse objects (i.e. classes represented within the pandas library and not ones external to it like scipy sparse matrices), and datetime-like arrays.
The array-like to check.
Whether or not the array-like is of a pandas extension class instance.
Examples


Check if the object is a file-like object.
For objects to be considered file-like, they must be an iterator AND have either a read and/or write method as an attribute.
Note: file-like objects must be iterable, but iterable objects need not be file-like.
Whether obj has file-like properties.
Examples


Check whether the provided array or dtype is of a float dtype.
This function is internal and should not be exposed in the public API.
The array or dtype to check.
Whether or not the array or dtype is of a float dtype.
Examples


Return True if hash(obj) will succeed, False otherwise.
Some types will pass a test against collections.abc.Hashable but fail when they are actually hashed with hash().
Distinguish between these and other types by trying the call to hash() and seeing if they raise TypeError.
Examples


Check whether the provided array or dtype is of the int64 dtype.
The array or dtype to check.
Whether or not the array or dtype is of the int64 dtype.
Notes
Depending on system architecture, the return value of is_int64_dtype( int) will be True if the OS uses 64-bit integers and False if the OS uses 32-bit integers.
Examples


Check whether the provided array or dtype is of an integer dtype.
Unlike in is_any_int_dtype, timedelta64 instances will return False.
The nullable Integer dtypes (e.g. pandas.Int64Dtype) are also considered as integer by this function.
The array or dtype to check.
Whether or not the array or dtype is of an integer dtype and not an instance of timedelta64.
Examples


Check whether an array-like or dtype is of the Interval dtype.
The array-like or dtype to check.
Whether or not the array-like or dtype is of the Interval dtype.
Examples


Check if the object is an iterator.
This is intended for generators, not list-like objects.
Whether obj is an iterator.
Examples


Check if the object is list-like.
Objects that are considered list-like are for example Python lists, tuples, sets, NumPy arrays, and Pandas Series.
Strings and datetime objects, however, are not considered list-like.
Object to check.
If this parameter is False, sets will not be considered list-like.
Whether obj has list-like properties.
Examples


Check if the object is a named tuple.
Whether obj is a named tuple.
Examples


Check if the object is a number.
Returns True when the object is a number, and False if is not.
The object to check if is a number.
Whether obj is a number or not.
See also
Checks a subgroup of numbers.
Examples
Booleans are valid because they are int subclass.


Check whether the provided array or dtype is of a numeric dtype.
The array or dtype to check.
Whether or not the array or dtype is of a numeric dtype.
Examples


Check whether an array-like or dtype is of the object dtype.
The array-like or dtype to check.
Whether or not the array-like or dtype is of the object dtype.
Examples


Check whether an array-like or dtype is of the Period dtype.
The array-like or dtype to check.
Whether or not the array-like or dtype is of the Period dtype.
Examples


Check if the object is a regex pattern instance.
Whether obj is a regex pattern.
Examples


Check if the object can be compiled into a regex pattern instance.
Whether obj can be compiled as a regex pattern.
Examples


Return True if given object is scalar.
This includes:
numpy array scalar (e.g. np.int64)
Python builtin numerics
Python builtin byte arrays and strings
None
datetime.datetime
datetime.timedelta
Period
decimal.Decimal
Interval
DateOffset
Fraction
Number.
Return True if given object is scalar.
Examples
pandas supports PEP 3141 numbers:


Check whether the provided array or dtype is of a signed integer dtype.
Unlike in is_any_int_dtype, timedelta64 instances will return False.
The nullable Integer dtypes (e.g. pandas.Int64Dtype) are also considered as integer by this function.
The array or dtype to check.
Whether or not the array or dtype is of a signed integer dtype and not an instance of timedelta64.
Examples


Check whether an array-like is a 1-D pandas sparse array.
Check that the one-dimensional array-like is a pandas sparse array. Returns True if it is a pandas sparse array, not another type of sparse array.
Array-like to check.
Whether or not the array-like is a pandas sparse array.
Examples
Returns True if the parameter is a 1-D pandas sparse array.
Returns False if the parameter is not sparse.
Returns False if the parameter is not a pandas sparse array.
Returns False if the parameter has more than one dimension.


Check whether the provided array or dtype is of the string dtype.
The array or dtype to check.
Whether or not the array or dtype is of the string dtype.
Examples


Check whether an array-like or dtype is of the timedelta64 dtype.
The array-like or dtype to check.
Whether or not the array-like or dtype is of the timedelta64 dtype.
Examples


Check whether the provided array or dtype is of the timedelta64[ns] dtype.
This is a very specific dtype, so generic ones like np.timedelta64 will return False if passed into this function.
The array or dtype to check.
Whether or not the array or dtype is of the timedelta64[ns] dtype.
Examples


Check whether the provided array or dtype is of an unsigned integer dtype.
The nullable Integer dtypes (e.g. pandas.UInt64Dtype) are also considered as integer by this function.
The array or dtype to check.
Whether or not the array or dtype is of an unsigned integer dtype.
Examples


Convert input into a pandas only dtype object or a numpy dtype object.


Combine list-like of Categorical-like, unioning categories.
All categories must have the same dtype.
Categorical, CategoricalIndex, or Series with dtype=’category’.
If true, resulting categories will be lexsorted, otherwise they will be ordered as they appear in the data.
If true, the ordered attribute of the Categoricals will be ignored. Results in an unordered categorical.
all inputs do not have the same dtype
all inputs do not have the same ordered property
all inputs are ordered and their categories are not identical
sort_categories=True and Categoricals are ordered
Empty list of categoricals passed
Notes
To learn more about categories, see link
Examples
If you want to combine categoricals that do not necessarily have the same categories, union_categoricals will combine a list-like of categoricals. The new categories will be the union of the categories being combined.
By default, the resulting categories will be ordered as they appear in the categories of the data. If you want the categories to be lexsorted, use sort_categories=True argument.
union_categoricals also works with the case of combining two categoricals of the same categories and order information (e.g. what you could also append for).
Raises TypeError because the categories are ordered and not identical.
New in version 0.20.0
Ordered categoricals with different categories or orderings can be combined by using the ignore_ordered=True argument.
union_categoricals also works with a CategoricalIndex, or Series containing categorical data, but note that the resulting array will always be a plain Categorical


Create an array.
The scalars inside data should be instances of the scalar type for dtype. It’s expected that data represents a 1-dimensional array of data.
When data is an Index or Series, the underlying array will be extracted from data.
The dtype to use for the array. This may be a NumPy dtype or an extension type registered with pandas using pandas.api.extensions.register_extension_dtype().
If not specified, there are two possibilities:
When data is a Series, Index, or ExtensionArray, the dtype will be taken from the data.
Otherwise, pandas will attempt to infer the dtype from the data.
Note that when data is a NumPy array, data.dtype is not used for inferring the array type. This is because NumPy cannot represent all the types of data that can be held in extension arrays.
Currently, pandas will infer an extension dtype for sequences of
Scalar Type
Array Type
pandas.Interval
pandas.arrays.IntervalArray
pandas.Period
pandas.arrays.PeriodArray
datetime.datetime
pandas.arrays.DatetimeArray
datetime.timedelta
pandas.arrays.TimedeltaArray
int
pandas.arrays.IntegerArray
float
pandas.arrays.FloatingArray
str
pandas.arrays.StringArray or pandas.arrays.ArrowStringArray
bool
pandas.arrays.BooleanArray
The ExtensionArray created when the scalar type is str is determined by pd.options.mode.string_storage if the dtype is not explicitly given.
For all other cases, NumPy’s usual inference rules will be used.
Changed in version 1.0.0: Pandas infers nullable-integer dtype for integer data, string dtype for string data, and nullable-boolean dtype for boolean data.
Changed in version 1.2.0: Pandas now also infers nullable-floating dtype for float-like input data
Whether to copy the data, even if not necessary. Depending on the type of data, creating the new array may require copying data, even if copy=False.
The newly created array.
When data is not 1-dimensional.
See also
Construct a NumPy array.
Construct a pandas Series.
Construct a pandas Index.
ExtensionArray wrapping a NumPy array.
Extract the array stored within a Series.
Notes
Omitting the dtype argument means pandas will attempt to infer the best array type from the values in the data. As new array types are added by pandas and 3rd party libraries, the “best” array type may change. We recommend specifying dtype to ensure that
the correct array type for the data is returned
the returned array type doesn’t change as new extension types are added by pandas and third-party libraries
Additionally, if the underlying memory representation of the returned array matters, we recommend specifying the dtype as a concrete object rather than a string alias or allowing it to be inferred. For example, a future version of pandas or a 3rd-party library may include a dedicated ExtensionArray for string data. In this event, the following would no longer return a arrays.PandasArray backed by a NumPy array.
This would instead return the new ExtensionArray dedicated for string data. If you really need the new array to be backed by a NumPy array, specify that in the dtype.
Finally, Pandas has arrays that mostly overlap with NumPy
arrays.DatetimeArray
arrays.TimedeltaArray
When data with a datetime64[ns] or timedelta64[ns] dtype is passed, pandas will always return a DatetimeArray or TimedeltaArray rather than a PandasArray. This is for symmetry with the case of timezone-aware data, which NumPy does not natively support.
Examples
If a dtype is not specified, pandas will infer the best dtype from the values. See the description of dtype for the types pandas infers for.
You can use the string alias for dtype
Or specify the actual dtype
If pandas does not infer a dedicated extension type a arrays.PandasArray is returned.
As mentioned in the “Notes” section, new extension types may be added in the future (by pandas or 3rd party libraries), causing the return value to no longer be a arrays.PandasArray. Specify the dtype as a NumPy dtype if you need to ensure there’s no future change in behavior.
data must be 1-dimensional. A ValueError is raised when the input has the wrong dimensionality.


Extension array for string data in a pyarrow.ChunkedArray.
New in version 1.2.0.
Warning
ArrowStringArray is considered experimental. The implementation and parts of the API may change without warning.
The array of data.
See also
The recommended function for creating a ArrowStringArray.
The string methods are available on Series backed by a ArrowStringArray.
Notes
ArrowStringArray returns a BooleanArray for comparison methods.
Examples
Attributes
None
Methods
None


Array of boolean (True/False) data with missing values.
This is a pandas Extension array for boolean data, under the hood represented by 2 numpy arrays: a boolean array with the data and a boolean array with the mask (True indicating missing).
BooleanArray implements Kleene logic (sometimes called three-value logic) for logical operations. See Kleene logical operations for more.
To construct an BooleanArray from generic array-like input, use pandas.array() specifying dtype="boolean" (see examples below).
New in version 1.0.0.
Warning
BooleanArray is considered experimental. The implementation and parts of the API may change without warning.
A 1-d boolean-dtype array with the data.
A 1-d boolean-dtype array indicating missing values (True indicates missing).
Whether to copy the values and mask arrays.
Examples
Create an BooleanArray with pandas.array():
Attributes
None
Methods
None


Pandas ExtensionArray for tz-naive or tz-aware datetime data.
Warning
DatetimeArray is currently experimental, and its API may change without warning. In particular, DatetimeArray.dtype is expected to change to always be an instance of an ExtensionDtype subclass.
The datetime data.
For DatetimeArray values (or a Series or Index boxing one), dtype and freq will be extracted from values.
Note that the only NumPy dtype allowed is ‘datetime64[ns]’.
The frequency.
Whether to copy the underlying array of values.
Attributes
None
Methods
None


Array of integer (optional missing) values.
Changed in version 1.0.0: Now uses pandas.NA as the missing value rather than numpy.nan.
Warning
IntegerArray is currently experimental, and its API or internal implementation may change without warning.
We represent an IntegerArray with 2 numpy arrays:
data: contains a numpy integer array of the appropriate dtype
mask: a boolean array holding a mask on the data, True is missing
To construct an IntegerArray from generic array-like input, use pandas.array() with one of the integer dtypes (see examples).
See Nullable integer data type for more.
A 1-d integer-dtype array.
A 1-d boolean-dtype array indicating missing values.
Whether to copy the values and mask.
Examples
Create an IntegerArray with pandas.array().
String aliases for the dtypes are also available. They are capitalized.
Attributes
None
Methods
None


Pandas array for interval data that are closed on the same side.
New in version 0.24.0.
Array-like containing Interval objects from which to build the IntervalArray.
Whether the intervals are closed on the left-side, right-side, both or neither.
If None, dtype will be inferred.
Copy the input data.
Verify that the IntervalArray is valid.
See also
The base pandas Index type.
A bounded slice-like interval; the elements of an IntervalArray.
Function to create a fixed frequency IntervalIndex.
Bin values into discrete Intervals.
Bin values into equal-sized Intervals based on rank or sample quantiles.
Notes
See the user guide for more.
Examples
A new IntervalArray can be constructed directly from an array-like of Interval objects:
It may also be constructed using one of the constructor methods: IntervalArray.from_arrays(), IntervalArray.from_breaks(), and IntervalArray.from_tuples().
Attributes
left
Return the left endpoints of each Interval in the IntervalArray as an Index.
right
Return the right endpoints of each Interval in the IntervalArray as an Index.
closed
Whether the intervals are closed on the left-side, right-side, both or neither.
mid
Return the midpoint of each Interval in the IntervalArray as an Index.
length
Return an Index with entries denoting the length of each Interval in the IntervalArray.
is_empty
Indicates if an interval is empty, meaning it contains no points.
is_non_overlapping_monotonic
Return True if the IntervalArray is non-overlapping (no Intervals share points) and is either monotonic increasing or monotonic decreasing, else False.
Methods
from_arrays(left, right[, closed, copy, dtype])
Construct from two arrays defining the left and right bounds.
from_tuples(data[, closed, copy, dtype])
Construct an IntervalArray from an array-like of tuples.
from_breaks(breaks[, closed, copy, dtype])
Construct an IntervalArray from an array of splits.
contains(other)
Check elementwise if the Intervals contain the value.
overlaps(other)
Check elementwise if an Interval overlaps the values in the IntervalArray.
set_closed(closed)
Return an IntervalArray identical to the current one, but closed on the specified side.
to_tuples([na_tuple])
Return an ndarray of tuples of the form (left, right).


Whether the intervals are closed on the left-side, right-side, both or neither.


Check elementwise if the Intervals contain the value.
Return a boolean mask whether the value is contained in the Intervals of the IntervalArray.
New in version 0.25.0.
The value to check whether it is contained in the Intervals.
See also
Check whether Interval object contains value.
Check if an Interval overlaps the values in the IntervalArray.
Examples


Construct from two arrays defining the left and right bounds.
Left bounds for each interval.
Right bounds for each interval.
Whether the intervals are closed on the left-side, right-side, both or neither.
Copy the data.
If None, dtype will be inferred.
When a value is missing in only one of left or right. When a value in left is greater than the corresponding value in right.
See also
Function to create a fixed frequency IntervalIndex.
Construct an IntervalArray from an array of splits.
Construct an IntervalArray from an array-like of tuples.
Notes
Each element of left must be less than or equal to the right element at the same position. If an element is missing, it must be missing in both left and right. A TypeError is raised when using an unsupported type for left or right. At the moment, ‘category’, ‘object’, and ‘string’ subtypes are not supported.


Construct an IntervalArray from an array of splits.
Left and right bounds for each interval.
Whether the intervals are closed on the left-side, right-side, both or neither.
Copy the data.
If None, dtype will be inferred.
See also
Function to create a fixed frequency IntervalIndex.
Construct from a left and right array.
Construct from a sequence of tuples.
Examples


Construct an IntervalArray from an array-like of tuples.
Array of tuples.
Whether the intervals are closed on the left-side, right-side, both or neither.
By-default copy the data, this is compat only and ignored.
If None, dtype will be inferred.
See also
Function to create a fixed frequency IntervalIndex.
Construct an IntervalArray from a left and right array.
Construct an IntervalArray from an array of splits.
Examples


Indicates if an interval is empty, meaning it contains no points.
New in version 0.25.0.
A boolean indicating if a scalar Interval is empty, or a boolean ndarray positionally indicating if an Interval in an IntervalArray or IntervalIndex is empty.
Examples
An Interval that contains points is not empty:
An Interval that does not contain any points is empty:
An Interval that contains a single point is not empty:
An IntervalArray or IntervalIndex returns a boolean ndarray positionally indicating if an Interval is empty:
Missing values are not considered empty:


Return True if the IntervalArray is non-overlapping (no Intervals share points) and is either monotonic increasing or monotonic decreasing, else False.


Return the left endpoints of each Interval in the IntervalArray as an Index.


Return an Index with entries denoting the length of each Interval in the IntervalArray.


Return the midpoint of each Interval in the IntervalArray as an Index.


Check elementwise if an Interval overlaps the values in the IntervalArray.
Two intervals overlap if they share a common point, including closed endpoints. Intervals that only have an open endpoint in common do not overlap.
Interval to check against for an overlap.
Boolean array positionally indicating where an overlap occurs.
See also
Check whether two Interval objects overlap.
Examples
Intervals that share closed endpoints overlap:
Intervals that only have an open endpoint in common do not overlap:


Return the right endpoints of each Interval in the IntervalArray as an Index.


Return an IntervalArray identical to the current one, but closed on the specified side.
Whether the intervals are closed on the left-side, right-side, both or neither.
Examples


Return an ndarray of tuples of the form (left, right).
Returns NA as a tuple if True, (nan, nan), or just as the NA value itself if False, nan.


A pandas ExtensionArray for NumPy data.
This is mostly for internal compatibility, and is not especially useful on its own.
The NumPy ndarray to wrap. Must be 1-dimensional.
Whether to copy values.
Attributes
None
Methods
None


Pandas ExtensionArray for storing Period data.
Users should use period_array() to create new instances. Alternatively, array() can be used to create new instances from a sequence of Period scalars.
The data to store. These should be arrays that can be directly converted to ordinals without inference or copy (PeriodArray, ndarray[int64]), or a box around such an array (Series[period], PeriodIndex).
A PeriodDtype instance from which to extract a freq. If both freq and dtype are specified, then the frequencies must match.
The freq to use for the array. Mostly applicable when values is an ndarray of integers, when freq is required. When values is a PeriodArray (or box around), it’s checked that values.freq matches freq.
Whether to copy the ordinals before storing.
See also
Represents a period of time.
Immutable Index for period data.
Create a fixed-frequency PeriodArray.
Construct a pandas array.
Notes
There are two components to a PeriodArray
ordinals : integer ndarray
freq : pd.tseries.offsets.Offset
The values are physically stored as a 1-D ndarray of integers. These are called “ordinals” and represent some kind of offset from a base.
The freq indicates the span covered by each element of the array. All elements in the PeriodArray have the same freq.
Attributes
None
Methods
None


An ExtensionArray for storing sparse data.
A dense array of values to store in the SparseArray. This may contain fill_value.
Deprecated since version 1.4.0: Use a function like np.full to construct an array with the desired repeats of the scalar value instead.
Elements in data that are fill_value are not stored in the SparseArray. For memory savings, this should be the most common value in data. By default, fill_value depends on the dtype of data:
data.dtype
na_value
float
np.nan
int
0
bool
False
datetime64
pd.NaT
timedelta64
pd.NaT
The fill value is potentially specified in three ways. In order of precedence, these are
The fill_value argument
dtype.fill_value if fill_value is None and dtype is a SparseDtype
data.dtype.fill_value if fill_value is None and dtype is not a SparseDtype and data is a SparseArray.
Can be ‘integer’ or ‘block’, default is ‘integer’. The type of storage for sparse locations.
‘block’: Stores a block and block_length for each contiguous span of sparse values. This is best when sparse data tends to be clumped together, with large regions of fill-value values between sparse values.
‘integer’: uses an integer to store the location of each sparse value.
The dtype to use for the SparseArray. For numpy dtypes, this determines the dtype of self.sp_values. For SparseDtype, this determines self.sp_values and self.fill_value.
Whether to explicitly copy the incoming data array.
Examples
Attributes
None
Methods
None


Extension array for string data.
New in version 1.0.0.
Warning
StringArray is considered experimental. The implementation and parts of the API may change without warning.
The array of data.
Warning
Currently, this expects an object-dtype ndarray where the elements are Python strings or pandas.NA. This may change without warning in the future. Use pandas.array() with dtype="string" for a stable way of creating a StringArray from any sequence.
Whether to copy the array of data.
See also
The recommended function for creating a StringArray.
The string methods are available on Series backed by a StringArray.
Notes
StringArray returns a BooleanArray for comparison methods.
Examples
Unlike arrays instantiated with dtype="object", StringArray will convert the values to strings.
However, instantiating StringArrays directly with non-strings will raise an error.
For comparison methods, StringArray returns a pandas.BooleanArray:
Attributes
None
Methods
None


Pandas ExtensionArray for timedelta data.
Warning
TimedeltaArray is currently experimental, and its API may change without warning. In particular, TimedeltaArray.dtype is expected to change to be an instance of an ExtensionDtype subclass.
The timedelta data.
Currently, only numpy.dtype("timedelta64[ns]") is accepted.
Whether to copy the underlying array of data.
Attributes
None
Methods
None


Return a fixed frequency DatetimeIndex, with business day as the default frequency.
Left bound for generating dates.
Right bound for generating dates.
Number of periods to generate.
Frequency strings can have multiples, e.g. ‘5H’.
Time zone name for returning localized DatetimeIndex, for example Asia/Beijing.
Normalize start/end dates to midnight before generating date range.
Name of the resulting DatetimeIndex.
Weekmask of valid business days, passed to numpy.busdaycalendar, only used when custom frequency strings are passed. The default value None is equivalent to ‘Mon Tue Wed Thu Fri’.
Dates to exclude from the set of valid business days, passed to numpy.busdaycalendar, only used when custom frequency strings are passed.
Make the interval closed with respect to the given frequency to the ‘left’, ‘right’, or both sides (None).
Deprecated since version 1.4.0: Argument closed has been deprecated to standardize boundary inputs. Use inclusive instead, to set each bound as closed or open.
Include boundaries; Whether to set each bound as closed or open.
New in version 1.4.0.
For compatibility. Has no effect on the result.
Notes
Of the four parameters: start, end, periods, and freq, exactly three must be specified. Specifying freq is a requirement for bdate_range. Use date_range if specifying freq is not desired.
To learn more about the frequency strings, please see this link.
Examples
Note how the two weekend days are skipped in the result.


Extension dtype for boolean data.
New in version 1.0.0.
Warning
BooleanDtype is considered experimental. The implementation and parts of the API may change without warning.
Examples
Attributes
None
Methods
None


Represent a categorical variable in classic R / S-plus fashion.
Categoricals can only take on only a limited, and usually fixed, number of possible values (categories). In contrast to statistical categorical variables, a Categorical might have an order, but numerical operations (additions, divisions, …) are not possible.
All values of the Categorical are either in categories or np.nan. Assigning values outside of categories will raise a ValueError. Order is defined by the order of the categories, not lexical order of the values.
The values of the categorical. If categories are given, values not in categories will be replaced with NaN.
The unique categories for this categorical. If not given, the categories are assumed to be the unique values of values (sorted, if possible, otherwise in the order in which they appear).
Whether or not this categorical is treated as a ordered categorical. If True, the resulting categorical will be ordered. An ordered categorical respects, when sorted, the order of its categories attribute (which in turn is the categories argument, if provided).
An instance of CategoricalDtype to use for this categorical.
If the categories do not validate.
If an explicit ordered=True is given but no categories and the values are not sortable.
See also
Type for categorical data.
An Index with an underlying Categorical.
Notes
See the user guide for more.
Examples
Missing values are not included as a category.
However, their presence is indicated in the codes attribute by code -1.
Ordered Categoricals can be sorted according to the custom order of the categories and can have a min and max value.
Attributes
categories
The categories of this categorical.
codes
The category codes of this categorical.
ordered
Whether the categories have an ordered relationship.
dtype
The CategoricalDtype for this instance.
Methods
from_codes(codes[, categories, ordered, dtype])
Make a Categorical type from codes and categories or dtype.
__array__([dtype])
The numpy array interface.


The numpy array interface.
A numpy array of either the specified dtype or, if dtype==None (default), the same dtype as categorical.categories.dtype.


The categories of this categorical.
Setting assigns new values to each category (effectively a rename of each individual category).
The assigned value has to be a list-like object. All items must be unique and the number of items in the new categories must be the same as the number of items in the old categories.
Assigning to categories is a inplace operation!
If the new categories do not validate as categories or if the number of new categories is unequal the number of old categories
See also
Rename categories.
Reorder categories.
Add new categories.
Remove the specified categories.
Remove categories which are not used.
Set the categories to the specified ones.


The category codes of this categorical.
Codes are an array of integers which are the positions of the actual values in the categories array.
There is no setter, use the other categorical methods and the normal item setter to change values in the categorical.
A non-writable view of the codes array.


Make a Categorical type from codes and categories or dtype.
This constructor is useful if you already have codes and categories/dtype and so do not need the (computation intensive) factorization step, which is usually done on the constructor.
If your data does not follow this convention, please use the normal constructor.
An integer array, where each integer points to a category in categories or dtype.categories, or else is -1 for NaN.
The categories for the categorical. Items need to be unique. If the categories are not given here, then they must be provided in dtype.
Whether or not this categorical is treated as an ordered categorical. If not given here or in dtype, the resulting categorical will be unordered.
If CategoricalDtype, cannot be used together with categories or ordered.
Examples


Whether the categories have an ordered relationship.


Type for categorical data with the categories and orderedness.
Must be unique, and must not contain any nulls. The categories are stored in an Index, and if an index is provided the dtype of that index will be used.
Whether or not this categorical is treated as a ordered categorical. None can be used to maintain the ordered value of existing categoricals when used in operations that combine categoricals, e.g. astype, and will resolve to False if there is no existing ordered to maintain.
See also
Represent a categorical variable in classic R / S-plus fashion.
Notes
This class is useful for specifying the type of a Categorical independent of the values. See CategoricalDtype for more.
Examples
An empty CategoricalDtype with a specific dtype can be created by providing an empty index. As follows,
Attributes
categories
An Index containing the unique categories allowed.
ordered
Whether the categories have an ordered relationship.
Methods
None


An Index containing the unique categories allowed.


Index based on an underlying Categorical.
CategoricalIndex, like Categorical, can only take on a limited, and usually fixed, number of possible values (categories). Also, like Categorical, it might have an order, but numerical operations (additions, divisions, …) are not possible.
The values of the categorical. If categories are given, values not in categories will be replaced with NaN.
The categories for the categorical. Items need to be unique. If the categories are not given here (and also not in dtype), they will be inferred from the data.
Whether or not this categorical is treated as an ordered categorical. If not given here or in dtype, the resulting categorical will be unordered.
If CategoricalDtype, cannot be used together with categories or ordered.
Make a copy of input ndarray.
Name to be stored in the index.
If the categories do not validate.
If an explicit ordered=True is given but no categories and the values are not sortable.
See also
The base pandas Index type.
A categorical array.
Type for categorical data.
Notes
See the user guide for more.
Examples
CategoricalIndex can also be instantiated from a Categorical:
Ordered CategoricalIndex can have a min and max value.
Attributes
codes
The category codes of this categorical.
categories
The categories of this categorical.
ordered
Whether the categories have an ordered relationship.
Methods
rename_categories(*args, **kwargs)
Rename categories.
reorder_categories(*args, **kwargs)
Reorder categories as specified in new_categories.
add_categories(*args, **kwargs)
Add new categories.
remove_categories(*args, **kwargs)
Remove the specified categories.
remove_unused_categories(*args, **kwargs)
Remove categories which are not used.
set_categories(*args, **kwargs)
Set the categories to the specified new_categories.
as_ordered(*args, **kwargs)
Set the Categorical to be ordered.
as_unordered(*args, **kwargs)
Set the Categorical to be unordered.
map(mapper)
Map values using input an input mapping or function.


Add new categories.
new_categories will be included at the last/highest place in the categories and will be unused directly after this call.
The new categories to be included.
Whether or not to add the categories inplace or return a copy of this categorical with added categories.
Deprecated since version 1.3.0.
Categorical with new categories added or None if inplace=True.
If the new categories include old categories or do not validate as categories
See also
Rename categories.
Reorder categories.
Remove the specified categories.
Remove categories which are not used.
Set the categories to the specified ones.
Examples


Set the Categorical to be ordered.
Whether or not to set the ordered attribute in-place or return a copy of this categorical with ordered set to True.
Ordered Categorical or None if inplace=True.


Set the Categorical to be unordered.
Whether or not to set the ordered attribute in-place or return a copy of this categorical with ordered set to False.
Unordered Categorical or None if inplace=True.


Determine if two CategoricalIndex objects contain the same elements.
If two CategoricalIndex objects have equal elements True, otherwise False.


Map values using input an input mapping or function.
Maps the values (their categories, not the codes) of the index to new categories. If the mapping correspondence is one-to-one the result is a CategoricalIndex which has the same order property as the original, otherwise an Index is returned.
If a dict or Series is used any unmapped category is mapped to NaN. Note that if this happens an Index will be returned.
Mapping correspondence.
Mapped index.
See also
Apply a mapping correspondence on an Index.
Apply a mapping correspondence on a Series.
Apply more complex functions on a Series.
Examples
If the mapping is one-to-one the ordering of the categories is preserved:
If the mapping is not one-to-one an Index is returned:
If a dict is used, all unmapped categories are mapped to NaN and the result is an Index:


Remove the specified categories.
removals must be included in the old categories. Values which were in the removed categories will be set to NaN
The categories which should be removed.
Whether or not to remove the categories inplace or return a copy of this categorical with removed categories.
Deprecated since version 1.3.0.
Categorical with removed categories or None if inplace=True.
If the removals are not contained in the categories
See also
Rename categories.
Reorder categories.
Add new categories.
Remove categories which are not used.
Set the categories to the specified ones.
Examples


Remove categories which are not used.
Whether or not to drop unused categories inplace or return a copy of this categorical with unused categories dropped.
Deprecated since version 1.2.0.
Categorical with unused categories dropped or None if inplace=True.
See also
Rename categories.
Reorder categories.
Add new categories.
Remove the specified categories.
Set the categories to the specified ones.
Examples


Rename categories.
New categories which will replace old categories.
list-like: all items must be unique and the number of items in the new categories must match the existing number of categories.
dict-like: specifies a mapping from old categories to new. Categories not contained in the mapping are passed through and extra categories in the mapping are ignored.
callable : a callable that is called on all items in the old categories and whose return values comprise the new categories.
Whether or not to rename the categories inplace or return a copy of this categorical with renamed categories.
Deprecated since version 1.3.0.
Categorical with removed categories or None if inplace=True.
If new categories are list-like and do not have the same number of items than the current categories or do not validate as categories
See also
Reorder categories.
Add new categories.
Remove the specified categories.
Remove categories which are not used.
Set the categories to the specified ones.
Examples
For dict-like new_categories, extra keys are ignored and categories not in the dictionary are passed through
You may also provide a callable to create the new categories


Reorder categories as specified in new_categories.
new_categories need to include all old categories and no new category items.
The categories in new order.
Whether or not the categorical is treated as a ordered categorical. If not given, do not change the ordered information.
Whether or not to reorder the categories inplace or return a copy of this categorical with reordered categories.
Deprecated since version 1.3.0.
Categorical with removed categories or None if inplace=True.
If the new categories do not contain all old category items or any new ones
See also
Rename categories.
Add new categories.
Remove the specified categories.
Remove categories which are not used.
Set the categories to the specified ones.


Set the categories to the specified new_categories.
new_categories can include new categories (which will result in unused categories) or remove old categories (which results in values set to NaN). If rename==True, the categories will simple be renamed (less or more items than in old categories will result in values set to NaN or in unused categories respectively).
This method can be used to perform more than one action of adding, removing, and reordering simultaneously and is therefore faster than performing the individual steps via the more specialised methods.
On the other hand this methods does not do checks (e.g., whether the old categories are included in the new categories on a reorder), which can result in surprising changes, for example when using special string dtypes, which does not considers a S1 string equal to a single char python string.
The categories in new order.
Whether or not the categorical is treated as a ordered categorical. If not given, do not change the ordered information.
Whether or not the new_categories should be considered as a rename of the old categories or as reordered categories.
Whether or not to reorder the categories in-place or return a copy of this categorical with reordered categories.
Deprecated since version 1.3.0.
If new_categories does not validate as categories
See also
Rename categories.
Reorder categories.
Add new categories.
Remove the specified categories.
Remove categories which are not used.


Concatenate pandas objects along a particular axis with optional set logic along the other axes.
Can also add a layer of hierarchical indexing on the concatenation axis, which may be useful if the labels are the same (or overlapping) on the passed axis number.
If a mapping is passed, the sorted keys will be used as the keys argument, unless it is passed, in which case the values will be selected (see below). Any None objects will be dropped silently unless they are all None in which case a ValueError will be raised.
The axis to concatenate along.
How to handle indexes on other axis (or axes).
If True, do not use the index values along the concatenation axis. The resulting axis will be labeled 0, …, n - 1. This is useful if you are concatenating objects where the concatenation axis does not have meaningful indexing information. Note the index values on the other axes are still respected in the join.
If multiple levels passed, should contain tuples. Construct hierarchical index using the passed keys as the outermost level.
Specific levels (unique values) to use for constructing a MultiIndex. Otherwise they will be inferred from the keys.
Names for the levels in the resulting hierarchical index.
Check whether the new concatenated axis contains duplicates. This can be very expensive relative to the actual data concatenation.
Sort non-concatenation axis if it is not already aligned when join is ‘outer’. This has no effect when join='inner', which already preserves the order of the non-concatenation axis.
Changed in version 1.0.0: Changed to not sort by default.
If False, do not copy data unnecessarily.
When concatenating all Series along the index (axis=0), a Series is returned. When objs contains at least one DataFrame, a DataFrame is returned. When concatenating along the columns (axis=1), a DataFrame is returned.
See also
Concatenate Series.
Concatenate DataFrames.
Join DataFrames using indexes.
Merge DataFrames by indexes or columns.
Notes
The keys, levels, and names arguments are all optional.
A walkthrough of how this method fits in with other tools for combining pandas objects can be found here.
Examples
Combine two Series.
Clear the existing index and reset it in the result by setting the ignore_index option to True.
Add a hierarchical index at the outermost level of the data with the keys option.
Label the index keys you create with the names option.
Combine two DataFrame objects with identical columns.
Combine DataFrame objects with overlapping columns and return everything. Columns outside the intersection will be filled with NaN values.
Combine DataFrame objects with overlapping columns and return only those that are shared by passing inner to the join keyword argument.
Combine DataFrame objects horizontally along the x axis by passing in axis=1.
Prevent the result from including duplicate index values with the verify_integrity option.


Aggregate using one or more operations over the specified axis.
Function to use for aggregating the data. If a function, must either work when passed a DataFrame or when passed to DataFrame.apply.
Accepted combinations are:
function
string function name
list of functions and/or function names, e.g. [np.sum, 'mean']
dict of axis labels -> functions, function names or list of such.
Can also accept a Numba JIT function with engine='numba' specified. Only passing a single function is supported with this engine.
If the 'numba' engine is chosen, the function must be a user defined function with values and index as the first and second arguments respectively in the function signature. Each group’s index will be passed to the user defined function and optionally available for use.
Changed in version 1.1.0.
Positional arguments to pass to func.
'cython' : Runs the function through C-extensions from cython.
'numba' : Runs the function through JIT compiled code from numba.
None : Defaults to 'cython' or globally setting compute.use_numba
New in version 1.1.0.
For 'cython' engine, there are no accepted engine_kwargs
For 'numba' engine, the engine can accept nopython, nogil and parallel dictionary keys. The values must either be True or False. The default engine_kwargs for the 'numba' engine is {'nopython': True, 'nogil': False, 'parallel': False} and will be applied to the function
New in version 1.1.0.
Keyword arguments to be passed into func.
See also
Apply function func group-wise and combine the results together.
Aggregate using one or more operations over the specified axis.
Transforms the Series on each group based on the given function.
Notes
When using engine='numba', there will be no “fall back” behavior internally. The group data and group index will be passed as numpy arrays to the JITed user defined function, and no alternative execution attempts will be tried.
Functions that mutate the passed object can produce unexpected behavior or errors and are not supported. See Mutating with User Defined Function (UDF) methods for more details.
Changed in version 1.3.0: The resulting dtype will reflect the return value of the passed func, see the examples below.
Examples
The aggregation is for each column.
Multiple aggregations
Select a column for aggregation
Different aggregations per column
To control the output names with different aggregations per column, pandas supports “named aggregation”
The keywords are the output column names
The values are tuples whose first element is the column to select and the second element is the aggregation to apply to that column. Pandas provides the pandas.NamedAgg namedtuple with the fields ['column', 'aggfunc'] to make it clearer what the arguments are. As usual, the aggregation can be a callable or a string alias.
See Named aggregation for more.
Changed in version 1.3.0: The resulting dtype will reflect the return value of the aggregating function.


Return True if all values in the group are truthful, else False.
Flag to ignore nan values during truth testing.
DataFrame or Series of boolean values, where a value is True if all elements are True within its respective group, False otherwise.
See also
Apply a function groupby to a Series.
Apply a function groupby to each row or column of a DataFrame.


Return True if any value in the group is truthful, else False.
Flag to ignore nan values during truth testing.
DataFrame or Series of boolean values, where a value is True if any element is True within its respective group, False otherwise.
See also
Apply a function groupby to a Series.
Apply a function groupby to each row or column of a DataFrame.


Backward fill the values.
Limit of how many values to fill.
Object with missing values filled.
See also
Backward fill the missing values in the dataset.
Backward fill the missing values in the dataset.
Fill NaN values of a Series.
Fill NaN values of a DataFrame.


Make box plots from DataFrameGroupBy data.
False - no subplots will be used
True - create a subplot for each group.
Can be any valid input to groupby.
The layout of the plot: (rows, columns).
Whether x-axes will be shared among subplots.
Whether y-axes will be shared among subplots.
Backend to use instead of the backend specified in the option plotting.backend. For instance, ‘matplotlib’. Alternatively, to specify the plotting.backend for the whole session, set pd.options.plotting.backend.
New in version 1.0.0.
All other plotting keyword arguments to be passed to matplotlib’s boxplot function.
Examples
You can create boxplots for grouped data and show them as separate subplots:
The subplots=False option shows the boxplots in a single figure.


Compute pairwise correlation of columns, excluding NA/null values.
Method of correlation:
pearson : standard correlation coefficient
kendall : Kendall Tau correlation coefficient
spearman : Spearman rank correlation
and returning a float. Note that the returned matrix from corr will have 1 along the diagonals and will be symmetric regardless of the callable’s behavior.
Minimum number of observations required per pair of columns to have a valid result. Currently only available for Pearson and Spearman correlation.
Correlation matrix.
See also
Compute pairwise correlation with another DataFrame or Series.
Compute the correlation between two Series.
Examples


Compute pairwise correlation.
Pairwise correlation is computed between rows or columns of DataFrame with rows or columns of Series or DataFrame. DataFrames are first aligned along both axes before computing the correlations.
Object with which to compute correlations.
The axis to use. 0 or ‘index’ to compute column-wise, 1 or ‘columns’ for row-wise.
Drop missing indices from result.
Method of correlation:
pearson : standard correlation coefficient
kendall : Kendall Tau correlation coefficient
spearman : Spearman rank correlation
and returning a float.
Pairwise correlations.
See also
Compute pairwise correlation of columns.


Compute count of group, excluding missing values.
Count of values within each group.
See also
Apply a function groupby to a Series.
Apply a function groupby to each row or column of a DataFrame.


Compute pairwise covariance of columns, excluding NA/null values.
Compute the pairwise covariance among the series of a DataFrame. The returned data frame is the covariance matrix of the columns of the DataFrame.
Both NA and null values are automatically excluded from the calculation. (See the note below about bias from missing values.) A threshold can be set for the minimum number of observations for each value created. Comparisons with observations below this threshold will be returned as NaN.
This method is generally used for the analysis of time series data to understand the relationship between different measures across time.
Minimum number of observations required per pair of columns to have a valid result.
Delta degrees of freedom. The divisor used in calculations is N - ddof, where N represents the number of elements.
New in version 1.1.0.
The covariance matrix of the series of the DataFrame.
See also
Compute covariance with another Series.
Exponential weighted sample covariance.
Expanding sample covariance.
Rolling sample covariance.
Notes
Returns the covariance matrix of the DataFrame’s time series. The covariance is normalized by N-ddof.
For DataFrames that have Series that are missing data (assuming that data is missing at random) the returned covariance matrix will be an unbiased estimate of the variance and covariance between the member Series.
However, for many applications this estimate may not be acceptable because the estimate covariance matrix is not guaranteed to be positive semi-definite. This could lead to estimate correlations having absolute values which are greater than one, and/or a non-invertible covariance matrix. See Estimation of covariance matrices for more details.
Examples
Minimum number of periods
This method also supports an optional min_periods keyword that specifies the required minimum number of non-NA observations for each column pair in order to have a valid result:


Number each item in each group from 0 to the length of that group - 1.
Essentially this is equivalent to
If False, number in reverse, from length of group - 1 to 0.
Sequence number of each element within each group.
See also
Number the groups themselves.
Examples


Cumulative max for each group.
See also
Apply a function groupby to a Series.
Apply a function groupby to each row or column of a DataFrame.


Cumulative min for each group.
See also
Apply a function groupby to a Series.
Apply a function groupby to each row or column of a DataFrame.


Cumulative product for each group.
See also
Apply a function groupby to a Series.
Apply a function groupby to each row or column of a DataFrame.


Cumulative sum for each group.
See also
Apply a function groupby to a Series.
Apply a function groupby to each row or column of a DataFrame.


Generate descriptive statistics.
Descriptive statistics include those that summarize the central tendency, dispersion and shape of a dataset’s distribution, excluding NaN values.
Analyzes both numeric and object series, as well as DataFrame column sets of mixed data types. The output will vary depending on what is provided. Refer to the notes below for more detail.
The percentiles to include in the output. All should fall between 0 and 1. The default is [.25, .5, .75], which returns the 25th, 50th, and 75th percentiles.
A white list of data types to include in the result. Ignored for Series. Here are the options:
‘all’ : All columns of the input will be included in the output.
A list-like of dtypes : Limits the results to the provided data types. To limit the result to numeric types submit numpy.number. To limit it instead to object columns submit the numpy.object data type. Strings can also be used in the style of select_dtypes (e.g. df.describe(include=['O'])). To select pandas categorical columns, use 'category'
None (default) : The result will include all numeric columns.
A black list of data types to omit from the result. Ignored for Series. Here are the options:
A list-like of dtypes : Excludes the provided data types from the result. To exclude numeric types submit numpy.number. To exclude object columns submit the data type numpy.object. Strings can also be used in the style of select_dtypes (e.g. df.describe(exclude=['O'])). To exclude pandas categorical columns, use 'category'
None (default) : The result will exclude nothing.
Whether to treat datetime dtypes as numeric. This affects statistics calculated for the column. For DataFrame input, this also controls whether datetime columns are included by default.
New in version 1.1.0.
Summary statistics of the Series or Dataframe provided.
See also
Count number of non-NA/null observations.
Maximum of the values in the object.
Minimum of the values in the object.
Mean of the values.
Standard deviation of the observations.
Subset of a DataFrame including/excluding columns based on their dtype.
Notes
For numeric data, the result’s index will include count, mean, std, min, max as well as lower, 50 and upper percentiles. By default the lower percentile is 25 and the upper percentile is 75. The 50 percentile is the same as the median.
For object data (e.g. strings or timestamps), the result’s index will include count, unique, top, and freq. The top is the most common value. The freq is the most common value’s frequency. Timestamps also include the first and last items.
If multiple object values have the highest count, then the count and top results will be arbitrarily chosen from among those with the highest count.
For mixed data types provided via a DataFrame, the default is to return only an analysis of numeric columns. If the dataframe consists only of object and categorical data without any numeric columns, the default is to return an analysis of both the object and categorical columns. If include='all' is provided as an option, the result will include a union of attributes of each type.
The include and exclude parameters can be used to limit which columns in a DataFrame are analyzed for the output. The parameters are ignored when analyzing a Series.
Examples
Describing a numeric Series.
Describing a categorical Series.
Describing a timestamp Series.
Describing a DataFrame. By default only numeric fields are returned.
Describing all columns of a DataFrame regardless of data type.
Describing a column from a DataFrame by accessing it as an attribute.
Including only numeric columns in a DataFrame description.
Including only string columns in a DataFrame description.
Including only categorical columns from a DataFrame description.
Excluding numeric columns from a DataFrame description.
Excluding object columns from a DataFrame description.


First discrete difference of element.
Calculates the difference of a Dataframe element compared with another element in the Dataframe (default is element in previous row).
Periods to shift for calculating difference, accepts negative values.
Take difference over rows (0) or columns (1).
First differences of the Series.
See also
Percent change over given number of periods.
Shift index by desired number of periods with an optional time freq.
First discrete difference of object.
Notes
For boolean dtypes, this uses operator.xor() rather than operator.sub(). The result is calculated according to current dtype in Dataframe, however dtype of the result is always float64.
Examples
Difference with previous row
Difference with previous column
Difference with 3rd previous row
Difference with following row
Overflow in input dtype


Forward fill the values.
Limit of how many values to fill.
Object with missing values filled.
See also
Returns Series with minimum number of char in object.
Object with missing values filled or None if inplace=True.
Fill NaN values of a Series.
Fill NaN values of a DataFrame.


Fill NA/NaN values using the specified method.
Value to use to fill holes (e.g. 0), alternately a dict/Series/DataFrame of values specifying which value to use for each index (for a Series) or column (for a DataFrame). Values not in the dict/Series/DataFrame will not be filled. This value cannot be a list.
Method to use for filling holes in reindexed Series pad / ffill: propagate last valid observation forward to next valid backfill / bfill: use next valid observation to fill gap.
Axis along which to fill missing values.
If True, fill in-place. Note: this will modify any other views on this object (e.g., a no-copy slice for a column in a DataFrame).
If method is specified, this is the maximum number of consecutive NaN values to forward/backward fill. In other words, if there is a gap with more than this number of consecutive NaNs, it will only be partially filled. If method is not specified, this is the maximum number of entries along the entire axis where NaNs will be filled. Must be greater than 0 if not None.
A dict of item->dtype of what to downcast if possible, or the string ‘infer’ which will try to downcast to an appropriate equal type (e.g. float64 to int64 if possible).
Object with missing values filled or None if inplace=True.
See also
Fill NaN values using interpolation.
Conform object to new index.
Convert TimeSeries to specified frequency.
Examples
Replace all NaN elements with 0s.
We can also propagate non-null values forward or backward.
Replace all NaN elements in column ‘A’, ‘B’, ‘C’, and ‘D’, with 0, 1, 2, and 3 respectively.
Only replace the first NaN element.
When filling using a DataFrame, replacement happens along the same column names and same indices
Note that column D is not affected since it is not present in df2.


Return a copy of a DataFrame excluding filtered elements.
Elements from groups are filtered if they do not satisfy the boolean criterion specified by func.
Function to apply to each subframe. Should return True or False.
If False, groups that evaluate False are filled with NaNs.
Notes
Each subframe is endowed the attribute ‘name’ in case you need to know which group you are working on.
Functions that mutate the passed object can produce unexpected behavior or errors and are not supported. See Mutating with User Defined Function (UDF) methods for more details.
Examples


Make a histogram of the DataFrame’s columns.
A histogram is a representation of the distribution of data. This function calls matplotlib.pyplot.hist(), on each series in the DataFrame, resulting in one histogram per column.
The pandas object holding the data.
If passed, will be used to limit data to a subset of columns.
If passed, then used to form histograms for separate groups.
Whether to show axis grid lines.
If specified changes the x-axis label size.
Rotation of x axis labels. For example, a value of 90 displays the x labels rotated 90 degrees clockwise.
If specified changes the y-axis label size.
Rotation of y axis labels. For example, a value of 90 displays the y labels rotated 90 degrees clockwise.
The axes to plot the histogram on.
In case subplots=True, share x axis and set some x axis labels to invisible; defaults to True if ax is None otherwise False if an ax is passed in. Note that passing in both an ax and sharex=True will alter all x axis labels for all subplots in a figure.
In case subplots=True, share y axis and set some y axis labels to invisible.
The size in inches of the figure to create. Uses the value in matplotlib.rcParams by default.
Tuple of (rows, columns) for the layout of the histograms.
Number of histogram bins to be used. If an integer is given, bins + 1 bin edges are calculated and returned. If bins is a sequence, gives bin edges, including left edge of first bin and right edge of last bin. In this case, bins is returned unmodified.
Backend to use instead of the backend specified in the option plotting.backend. For instance, ‘matplotlib’. Alternatively, to specify the plotting.backend for the whole session, set pd.options.plotting.backend.
New in version 1.0.0.
Whether to show the legend.
New in version 1.1.0.
All other plotting keyword arguments to be passed to matplotlib.pyplot.hist().
See also
Plot a histogram using matplotlib.
Examples
This example draws a histogram based on the length and width of some animals, displayed in three bins


Return index of first occurrence of maximum over requested axis.
NA/null values are excluded.
The axis to use. 0 or ‘index’ for row-wise, 1 or ‘columns’ for column-wise.
Exclude NA/null values. If an entire row/column is NA, the result will be NA.
Indexes of maxima along the specified axis.
If the row/column is empty
See also
Return index of the maximum element.
Notes
This method is the DataFrame version of ndarray.argmax.
Examples
Consider a dataset containing food consumption in Argentina.
By default, it returns the index for the maximum value in each column.
To return the index for the maximum value in each row, use axis="columns".