Saravanan666/pypi-org huggingface dataset:
Package: pandas-plots
pandas-plotsusageinstall / update packagepipinstallpandas-plots-Uinclude in pythonfrompandas_plotsimporttbl,pls,venexample# load sample dataset from seabornimportseabornassbdf=sb.load_dataset('taxis')pls.plot_box(df['fare'],height=400,violin=True)why use pandas-plotspandas-plotsis a package to help you examine and visualize data that are organized in a pandas DataFrame. It provides a high level api to pandas / plotly with some selected functions.It is subdivided into:tblutilities for table descriptionsdescribe_df()an alternative version of pandasdescribe()functionpivot_df()gets a pivot table of a 3 column dataframe🆕show_num_df()displays a table as styled version with additional informationplsfor plotly visualizationsplot_box()auto annotated boxplot w/ violin optionplot_boxes()multiple boxplots(annotation is experimental)plots_bars()a standardized bar plotplot_stacked_bars()shortcut to stacked bars 😄plot_quadrants()quickly shows a 2x2 heatmapvenoffers functions forvenn diagramsshow_venn2()displays a venn diagram for 2 setsshow_venn3()displays a venn diagram for 3 setssqlis added as convienience wrapper for retrieving data from sql databasesconnect_sqlget data from['mssql', 'sqlite','postgres']txtincludes some text based utilitieswrapformats strings or lists to a given width to fit nicely on the screennote: theming can be controlled through all functions by setting the environment variableTHEMEto either light or darkmore examples# quick and exhaustive description of any tabletbl.describe_df(df,'taxis',top_n_uniques=5)# show pivoted values for selected columnstbl.pivot_df(df[['color','payment','fare']])# show venn diagram for 3 setsfrompandas_plotsimportvenset_a={'ford','ferrari','mercedes','bmw'}set_b={'opel','bmw','bentley','audi'}set_c={'ferrari','bmw','chrysler','renault','peugeot','fiat'}_df,_details=ven.show_venn3(title="taxis",a_set=set_a,a_label="cars1",b_set=set_b,b_label="cars2",c_set=set_c,c_label="cars3",verbose=0,size=8,)dependencies
Package: pandas-profiling
pandas-profiling⚠️pandas-profilingpackage naming was changed. To continue profiling data useydata-profilinginstead!This repo implements the brownout strategy for deprecating the pandas-profiling package on PyPI.⚠️🎊 New year, new face, more functionalities!Thank you for using and followingpandas-profilingdevelopments. Yet, we have a new exciting feature - we are now thrilled to announce
that Spark is now part of the Data Profiling family from version 4.0.0 onwardsWith its introduction, there was also the need for a new naming, one that will allow to decouple the concept of profiling from the Pandas Dataframes -ydata-profiling!But fear not,pip install pandas-profilingwill still be a valid for a while, and we will keep investing in growing the best open-source for data profiling, so you can use it for even more use cases.How to fix the error for the main use casesusepip install ydata-profilingrather thanpip install pandas-profilingreplacepandas-profilingbyydata-profilingin your pip requirements files (requirements.txt, setup.py, setup.cfg, Pipfile, etc ...)if thepandas-profilingpackage is used by one of your dependencies it would be great if you take some time to track which package usespandas_profilinginstead ofydata_profilingfor the importsSchedule for deprecationydata-profilingwas launched in February 1st.pip install pandas-profilingwill still be supported untilApril 1st, but a warning will be thrown.from pandas_profiling import ProfileReportwill be supported until April 1st.After April 1st, an error will be thrown ifpip install pandas-profilingis used. Usepip install ydata-profilinginstead.After April 1st, an error will be thrown iffrom pandas_profiling import ProfileReportis used. Usefrom ydata_profiling import ProfileReportinstead.About pandas-profilingpandas-profilingprimary goal is to provide a one-line Exploratory Data Analysis (EDA) experience in a consistent and fast solution. Like pandasdf.describe()function, that is so handy, pandas-profiling delivers an extended analysis of a DataFrame while alllowing the data analysis to be exported in different formats such ashtmlandjson.The package outputs a simple and digested analysis of a dataset, includingtime-seriesandtext.Documentation|Discord|Stack Overflow|Latest changelogDo you like this project? Show us your love andgive feedback!
Package: pandaspublisher
Pandas PublisherPandas Publisher allows you to generate HTML and PDF reports using Jinja2 templates and Pandas dataframes. The generated reports can be saved as HTML, PDF, or returned as an HTML string.DependenciespandasJinja2WeasyPrintTo install run the following after cloning this repo:pipinstall-e.Usageimportpandasaspdfrompandaspublisherimportgenerate# Create some sample DataFramesdf1=pd.DataFrame({'A':[1,2],'B':[3,4]})df2=pd.DataFrame({'C':[5,6],'D':[7,8]})# Use a dictionary to pass the DataFrames and their variable namesdataframes={'dataframe1':df1,'dataframe2':df2}# Use the full path to the template filetemplate_path="/Users/jasonflittner/code/PandasPublisher/examples/templates/modern_template.html"output_file='output.html'output_format='html'# The desired output format for the report, either 'html', 'pdf', or 'html_string'.context={"title":"Sample Report","content":"This is a sample report generated using Pandas Publisher."}generate.report(template_path=template_path,dataframes=dataframes,output_format=output_format,output_filename=output_file,context=context)The output_format parameter accepts the following values:"html": Save the report as an HTML file."pdf": Save the report as a PDF file."html_string": Return the rendered HTML content as a string.Template CreationCreate your Jinja2 templates using standard HTML and Jinja2 syntax. Use the keys from your dataframes dictionary to reference the respective dataframes in the template. For example:<!DOCTYPE html><html><head><style>table{border-collapse:collapse;width:100%;}th,td{border:1pxsolidblack;padding:8px;text-align:left;}th{background-color:#f2f2f2;}</style></head><body><h1>{{ title }}</h1><p>{{ content }}</p><h2>DataFrame 1</h2><table><thead><tr>{% for column in dataframe1.columns %}<th>{{ column }}</th>{% endfor %}</tr></thead><tbody>{% for _, row in dataframe1.iterrows() %}<tr>{% for cell in row %}<td>{{ cell }}</td>{% endfor %}</tr>{% endfor %}</tbody></table><h2>DataFrame 2</h2><table><thead><tr>{% for column in dataframe2.columns %}<th>{{ column }}</th>{% endfor %}</tr></thead><tbody>{% for _, row in dataframe2.iterrows() %}<tr>{% for cell in row %}<td>{{ cell }}</td>{% endfor %}</tr>{% endfor %}</tbody></table></body></html>
Package: pandasql
pandasqlallows you to querypandasDataFrames using SQL syntax.
It works similarly tosqldfin R.pandasqlseeks to provide a
more familiar way of manipulating and cleaning data for people new to
Python orpandas.Installation$ pip install -U pandasqlBasicsThe main function used in pandasql issqldf.sqldfaccepts 2
parametrs - a sql query string - an set of session/environment variables
(locals()orglobals())Specifyinglocals()orglobals()can get tedious. You can
defined a short helper function to fix this.from pandasql import sqldf
pysqldf = lambda q: sqldf(q, globals())QueryingpandasqlusesSQLite syntax.
Anypandasdataframes will be automatically detected bypandasql. You can query them as you would any regular SQL table.$ python
>>> from pandasql import sqldf, load_meat, load_births
>>> pysqldf = lambda q: sqldf(q, globals())
>>> meat = load_meat()
>>> births = load_births()
>>> print pysqldf("SELECT * FROM meat LIMIT 10;").head()
                  date  beef  veal  pork  lamb_and_mutton broilers other_chicken turkey
0  1944-01-01 00:00:00   751    85  1280               89     None          None   None
1  1944-02-01 00:00:00   713    77  1169               72     None          None   None
2  1944-03-01 00:00:00   741    90  1128               75     None          None   None
3  1944-04-01 00:00:00   650    89   978               66     None          None   None
4  1944-05-01 00:00:00   681   106  1029               78     None          None   Nonejoins and aggregations are also supported>>> q = """SELECT
        m.date, m.beef, b.births
     FROM
        meats m
     INNER JOIN
        births b
           ON m.date = b.date;"""
>>> joined = pyqldf(q)
>>> print joined.head()
                    date    beef  births
403  2012-07-01 00:00:00  2200.8  368450
404  2012-08-01 00:00:00  2367.5  359554
405  2012-09-01 00:00:00  2016.0  361922
406  2012-10-01 00:00:00  2343.7  347625
407  2012-11-01 00:00:00  2206.6  320195

>>> q = "select
           strftime('%Y', date) as year
           , SUM(beef) as beef_total
           FROM
              meat
           GROUP BY
              year;"
>>> print pysqldf(q).head()
   year  beef_total
0  1944        8801
1  1945        9936
2  1946        9010
3  1947       10096
4  1948        8766More information and code samples available in theexamplesfolder or onour
blog.
Package: pandasql3
pandasqlThis is a fork of the original pandasql, with support of multiple SQL
backends and more convenient interface. See below for more info.pandasqlallows you to querypandasDataFrames using SQL syntax.
It works similarly tosqldfin R.pandasqlseeks to provide a
more familiar way of manipulating and cleaning data for people new to
Python orpandas.Installation$ pip install -U pandasqlBasicsIn addition to the original pandasql’ssqldffunction this fork has
a classPandaSQL, which new users are encouraged to use.sqldffunctionThe main function used in original pandasql issqldf.sqldfaccepts one three parameters: - sql query string, - dict of environment
variables (optional, if not specified assumed to be{**locals(),**globals()}) - database URI in the same format as in
SQLAlchemy (optional, by default use in-memory SQLite database)PandaSQLclassThe class is more convenient when you need to perform multiple queries
(almost always): - first create the class, specifying db_uri if not
default:pdsql = PandaSQL(db_uri)- to execute queries just callpdsql(query)(environment can also be specified expicitly)QueryingAnypandasdataframes will be automatically detected bypandasqland you can query them as you would any regular SQL table.$ python
>>> from pandasql import PandaSQL, load_meat, load_births
>>> meat = load_meat()
>>> births = load_births()
>>> pdsql = PandaSQL()
>>> print pdsql("SELECT * FROM meat LIMIT 10;").head()
                  date  beef  veal  pork  lamb_and_mutton broilers other_chicken turkey
0  1944-01-01 00:00:00   751    85  1280               89     None          None   None
1  1944-02-01 00:00:00   713    77  1169               72     None          None   None
2  1944-03-01 00:00:00   741    90  1128               75     None          None   None
3  1944-04-01 00:00:00   650    89   978               66     None          None   None
4  1944-05-01 00:00:00   681   106  1029               78     None          None   Nonejoins and aggregations are also supported>>> q = """SELECT
        m.date, m.beef, b.births
     FROM
        meats m
     INNER JOIN
        births b
           ON m.date = b.date;"""
>>> joined = pdsql(q)
>>> print joined.head()
                    date    beef  births
403  2012-07-01 00:00:00  2200.8  368450
404  2012-08-01 00:00:00  2367.5  359554
405  2012-09-01 00:00:00  2016.0  361922
406  2012-10-01 00:00:00  2343.7  347625
407  2012-11-01 00:00:00  2206.6  320195

>>> q = "select
           strftime('%Y', date) as year
           , SUM(beef) as beef_total
           FROM
              meat
           GROUP BY
              year;"
>>> print pdsql(q).head()
   year  beef_total
0  1944        8801
1  1945        9936
2  1946        9010
3  1947       10096
4  1948        8766More information and code samples (by the author of the original
version) available in theexamplesfolder or onhis
blog.
Package: pandas-quant-data-provider
Pandas ML Quant Data ProviderAn independent module used to fetch time series and other data used for quant finance studies.InstallationData can always be fetched instant without any local data base. However if you need to screen across assets
for matching criteria then a local database is needed. Developing models often depend on some premise. Like
for a crash detection model you would want to use instruments which indeed suffered a crash. Therefore some
data is stored into a data base using dolthub. Querying this data requires you to installdoltby
following theirinstallation instructions.pipinstallpandas-quant-data-provider!! NOTE !!We need to keep data libraries like yfincance up do date alsmost every minute. Whenever there is a change in the api
the library has to react. This happens more often as we whish, especially with yahoo finance. This is why this library
getspip installed -Uon every import. This means for server based solutions on every restart. Make sure you use
all pandas-ml* libraries inside a virtual env.Example:# monkey patch pandasfrompandas_quant_data_providerimportpd,YAHOO,INVESTING,CRYPTO_COMPARE# fetch data from various data sources#   * fetches all available dates#   * caches data for 10 minutesdf=pd.fetch_timeseries({YAHOO:["SPY","DIA"],INVESTING:["index::NYSE Tick Index::united states","bond::U.S. 30Y::united states"],CRYPTO_COMPARE:["BTC"]})df.tail()PS If you are not familiar with pandas MultiIndex, you can watch this video:How do I use the MultiIndex in pandas?InstallationFollow the instructions onhttps://github.com/KIC/pandas-ml-quantDocumentationCheck out the notebooks athttps://github.com/KIC/pandas-ml-quant/blob/master/notebooks
Package: pandas-redistrict
# pandas-redistrictUses data on redistricting to apply redistricting to older datasets to represent the districts in their current state.Supports merging and splitting of districts:- Merged districts are summed up under new identifier- Split districts are distributed by population-based ratio.Data on redistricting is in `data/` directory. Currently only available for German *Kreise* (containing reforms in NRW, Sachsen, Sachsen-Anhalt and Mecklenburg-Vorpommern).Install like this:pip install pandas-redistrict## Usage``` python>>> df  # Values indexed by German district identifiersvalue1  value2AGS05354       4       505313       5       605334       6       715154       8       915159      10      1115151      12      1315082      13      14>>> # Port old identifiers to new versions. Sum and distribute values on the way>>> from redistrict import redistrict>>> redistrict(df, 'de/kreise', drop=True, splits=True)value1  value2AGS05334   15.00   18.0015001    2.40    2.6015082   35.44   38.8115086    0.96    1.0415091    4.20    4.55```When you want to preserve groups inside districts, you can use ``redistrict_grouped``:``` python>>> # Specify district column (e.g. AGS)>>> # Also specify groups to preserve, in this case year>>> dfAGS  year  value1  value20  05354  2008       4       51  05313  2008       5       62  05334  2011       6       73  15154  2005       8       94  15159  2005      10      115  15151  2005      12      136  15082  2013      13      14>>> from redistrict import redistrict_groupedredistrict_grouped(df, 'de/kreise', ['year'],district_col='AGS',value_cols=['value1', 'value2'],drop=True)AGS  value1  value2  year0  15001    2.40    2.60  20051  15082   22.44   24.81  20052  15086    0.96    1.04  20053  15091    4.20    4.55  20050  05334    9.00   11.00  20080  05334    6.00    7.00  20110  15082   13.00   14.00  2013```
Package: pandas-refract
pandas-refractis an MIT licensed Python package with a simple function that allows users to divide their
dataframes by the ‘Truthy’ and ‘Falseyness’ of a provided array.Because this library deals with splitting dataframes by its values, it uses imagery of light - refract and disperse, like light entering a prism.
Eventually, the goal of this package is additional features to the Pandas library that allows users to .pop rows
from a dataframe where a condition is met, etc. As far as I can tell this is not possible like the below example.Ideal case would be:target_df = df.pop(df['target_column'] == 'target_value')
non_target_df = dfWhat is required now is:target_df = df[df['target_column'] == 'target_value']
non_target_df = df[df['target_column'] != 'target_value']With pandas-refract this becomes:target_df, non_target_df = refract(df, df['target_column'] == 'target_value']]Obviously, this package is not providing anything not currently possible in the current Pandas library. It does,
however, add a layer of convenience for more complex slicing where you need to separate, not remove, rows by conditions.ExamplesSimplest example of current Pandas requires:df1 = df[df.column.notnull()].reset_index(drop=True)
df2 = df[df.column.isnull()].reset_index(drop=True)or:df1 = df[df.column == 'test_string'].reset_index(drop=True)
df2 = df[df.column != 'test_string'].reset_index(drop=True)With pandas-refract this becomes:df1, df2 = refract(df, df.column.notnull(), True]and:df1, df2 = refract(df, df.column == test_string', True]But you don’t have to pass it explicit boolean arrays:data = {'a': ['', 'truthy', '', 'truthy'],
        'b': [0, 1, 2, 3]
        }

df = pd.DataFrame(data)

truthy_df, falsey_df = refract(df, df.a)More complex examples:(where ‘a’ is Falsey and ‘b’ is an odd number)df1, df2 = refract(df, ((~df.a) & (df.b % 2 == 1)))
Package: pandas-rose
Pandas RoseThis python package adds a custom Pandas accessor to generate polar wind rose plots from a Pandas dataframe.I don't mean to compete with the wonderfulwindrosepackage already available, but that package has a little too much complexity for what I wanted. This package is meant to provide a minimal, simple interface to making wind rose plots. This is done by using Pandas methodspd.cutanddf.groupbyand using Matplotlib regular polar axes.InstallInstall with pip. The requirements are only pandas, numpy, and matplotlib.pipinstallpandas-roseUsagePandas-rose is simple.importpandasaspdimportrose# df is a pandas dataframe with columns# "wind_speed" and "wind_direction"df=pd.DataFrame({"wind_speed":[1,2,3,4],"wind_direction":[20,10,190,300]})# Display a polar wind plot of the datadf.rose.plot()You can specify the pandas column to use for wind direction and wind speed. You may also change the number of sectors to bin the wind direction .df.rose.plot(var_column="A",# name of variable columndir_column="B",# name of direction columnsectors=8,# number of sectors (direction bins)bins=range(0,30,5)# specify variable binsnormed=False# If True, values as percentage instead of countscolors='Blues'# Name of matplotlib colormap or list of colors)There are two other accessors that give some information.# Display a dataframe of the binned valuesdf.rose.table(sectors=8)# Display the binned data as bar graph on regular axes.df.rose.bar()
Package: pandas-rs
[![Circle CI](https://circleci.com/gh/SamuraiT/pandas-rs.svg?style=svg)](https://circleci.com/gh/SamuraiT/pandas-rs)# pandas-rspandas-rs is oirginally designed for RedShift butalso works for PostgreSQL. Inspired by [pandas-td](https://github.com/treasure-data/pandas-td)I may should have made the package name as pandas-pg, since It also works forPostgreSQL.# requirementTo connect PostgreSQL and RedShift, you have to have the PostgreSQL client.### Mac~~~brew updatebrew install postgresql~~~# installation~~~pip install pandas-rs~~~# UsageAs I mentioned, above pandas-rs works for PostgreSQL as well.First export password via shell(recommended)~~~shellexport REDSHIFT_OR_POSTGRESQL_PASSWORD='password'~~~~~~pyimport pandas_rs as rsimport os # use only if you will access password through environment variablesrs.create_engine(dbname='dev',user='test',password=os.environ['REDSHIFT_OR_POSTGRESQL_PASSWORD'],host='foobar.redshift.exmple',port='5439')print(rs.read_sql("""select 'hello PostgreSQL or redshift' greeting"""))~~~result~~~pygreeting0  hello PostgreSQL or redshift~~~
Package: pandasrw
pandas的I/O相对各类包一直偏慢且存在易用性问题，特别是对于大文件的读写，瓶颈非常明显。pandasio 库通过将各类库进一步封装，提高了pandas 读写excel、csv等文件的性能和易用性。安装pip install pandasiogithub 项目https://github.com/stormtozero/pandasio一、常用API1、加载表df=load(file_path, col_name=None,sheetname='Sheet1',engine="polars")示例：输入路径读取Sheet1表的全部列，生产pandas的DataFrame 默认使用polars引擎。该表可以是xlsx、xlsx、csv和pkl格式df=load(file_path)2、写入表dump(df,file_path,sheetname='Sheet1',engine="polars")示例：输入路径，将pandas的DataFrame写入Sheet1表，默认使用polars引擎。该表可以是xlsx、xlsx、csv和pkl格式dump(df,file_path)3、查看表可以在excel中打开DataFrame和文件路径进行查看，方便在jupyter等交互环境中使用。输入参数f既可以是文件路径也可以是DataFrame。view(f)4、写入增强4.1、追加写
当参数mode="a"时能够追加写。dump(df, file_path, mode="a", sheetname='Sheet1',cell='A1')支持csv和excel的追加写。pandas对csv的追加写支持较好，但是对excel的追加写比较繁琐。本库通过分别通过pandas和xlwings实现了追加写，其中小数据集通过pandas较为快速方便，对于大数据集或者需要指定写入文件单元格的情况使用xlwings库。
注意：
1、追加写时文档必须关闭，否则使用pandas引擎会报错，xlwings不会报错但是无法追加写入
2、需要指定写入文件单元格的情况引擎必须使用xlwings库，既engine == "xlwings"dump(df, file_path, mode="a", sheetname='Sheet1',engine="xlwings", cell='A1')4.1、后缀增加时间戳
当参数 time=False 时能够在文件后缀上自动添加写入时间的功能，时间格式为  -年月日_时分dump(df, file_path, sheetname='Sheet1', time=False)5、流式加载表file_path是路径， row_count是每次读取的行。load_stream_row(file_path, row_count,col_name=None)生成一个pandas.io.parsers.readers.TextFileReader对象。对于该迭代器对象，通过遍历迭代器分块运算5.1、遍历迭代器5.2、对于迭代器中的每个DataFrame进行运算5.3、采用追加写（功能mode="a"或者mode="a+"）的方式写入csv。注意：file_result_csv和上文函数中的file_path_csv绝对不能相同，即读取的csv和存入的csv不能同路径。否则会不停的迭代下去，不能退出循环。原因：使用chunksize分块读取后，pandas并没有真正的将csv的内容加载入内存，只是解析了csv的内容和建立了连接（类似浅拷贝），在调用迭代器时再从csv中加载。所以再使用追加写的时候，会一边写入csv，一边再从csv中读取，形成死循环。for df in df_iter:
    df=运算结果
    df.to_csv(file_result_csv,mode="a", index=False,encoding='UTF-8',header=False)示例3：输入路径读取Sheet1表的全部列，生产pandas的DataFrame 默认使用polars引擎。该表可以是xlsx、xlsx、csv和pkl格式df_iter=load_stream_row(file_path, row_count）6、将csv转化为utf8编码encode_to_utf8(filename, des_encode):7、将xlsx转换为csv会在file_path文件夹下生成一个同名csv文件xlsxtocsv(file_path)二、功能特点1、性能提升通过封装polars、pandas、xlwings库实现性能的提升和保证兼容性。通过参数engine来选择采用的读写引擎，默认采用polars库。对于polars和xlwings不支持的格式后端自动选择pandas进行兼容。各类引擎的特点如下。polars库：一个rust写的高性能库。读写快，支持xlsx和csv文件类型，其中xlsx为通过xlsx2csv库转换为csv来实现。兼容性较好，但是不如pandas。xlsx可以比pandas快2倍，csv可以快3倍.建议：推荐使用。截至2023年3月该库的star已经超过1.3k，性能和质量已经经过检验。pandas库：支持xlsx、xls、csv、pkl四种文件类型，读写较慢，但是兼容性最好。建议：备选。xlwings库：支持xlsx和xls两种文件类型，该库本身支持csv，但是在测试时性能较差未采用。读写xlsx文件速度最快，可以达到pandas的3倍，但是在数据类型推断上较为粗糙，数据类型只能区分为object和float64两种类型，不能进一步区分int64等类型。本库通过调用 win32实现对excel的操作，需要使用windows操作协同且安装了excel软件。建议：在excel数据较大一般超过100M时使用，且对细分数据类型不敏感或者采用手动修改数据类型的场景。2、易用性提升易用性方面主要做了以下三个方面的提升2.1、对csv格式的编码自动修改为utf-8解决了 “'utf-8' codec can't decode”等编码类报错。2.2、对各类后缀进行了自适应，无需在手动指定excel、csv、pickle等。2.3、方便的实现csv、excel的追加写和在后缀加写入时间。三、转为pickle读写如果一个较大的表会多次使用，请转pickle存储，后续读写pickle文件会大大加快读写性能。本库的读写函数支持pkl格式文件的读写。