Saravanan666/pypi-org huggingface dataset:
Package: pandas-gpt
pandas-gptPower up your data science workflow with ChatGPT.pandas-gptis a Python library for doing almost anything with apandasDataFrame using ChatGPT prompts.Installationpipinstallpandas-gptSet theOPENAI_API_KEYenvironment variable to yourOpenAI API key, or use the following code snippet:importopenaiopenai.api_key='<API Key>'ExamplesSetup and usage examples are available in thisGoogle Colab notebook.importpandasaspdimportpandas_gptdf=pd.DataFrame('https://gist.githubusercontent.com/bluecoconut/9ce2135aafb5c6ab2dc1d60ac595646e/raw/c93c3500a1f7fae469cba716f09358cfddea6343/sales_demo_with_pii_and_all_states.csv')# Data transformationdf=df.ask('drop purchases from Laurenchester, NY')df=df.ask('add a new Category column with values "cheap", "regular", or "expensive"')# Queriesweekday=df.ask('which day of the week had the largest number of orders?')top_10=df.ask('what are the top 10 most popular products, as a table')# Plottingdf.ask('plot monthly and hourly sales')top_10.ask('horizontal bar plot with pastel colors')# Allow changes to original datasetdf.ask('do something interesting',mutable=True)# Show source code before runningdf.ask('convert prices from USD to GBP',verbose=True)Other HostsIf you want to use a different API host such asAzure OpenAI Service:importopenaiopenai.api_type='azure'openai.api_base='<Endpoint>'openai.api_version='<Version>'openai.api_key='<API Key>'importpandas_gpt# pandas_gpt.model = '<Model>' # Default is 'gpt-3.5-turbo'pandas_gpt.completion_config={'engine':'<Engine>',# 'deployment_id': '<Deployment ID>',}AlternativesGitHub Copilot: General-purpose code completion (paid subscription)Sketch: AI-powered data summarization and code suggestions (works without an API key)DisclaimerPlease note that thelimitationsof ChatGPT also apply to this library. I would recommend usingpandas-gptin a sandboxed environment such asGoogle Colab,Kaggle, orGitPod.
Package: pandasgui
AboutDemoInstallationUsageFeaturesMore InfoAboutPandasGUI is a GUI for viewing, plotting and analyzing Pandas DataFrames.DemoInstallationInstall latest release from PyPi:pipinstallpandasguiInstall directly from Github for the latest unreleased changes:pipinstallgit+https://github.com/adamerose/pandasgui.gitUsageCreate and view a simple DataFrameimportpandasaspdfrompandasguiimportshowdf=pd.DataFrame({'a':[1,2,3],'b':[4,5,6],'c':[7,8,9]})show(df)PandasGUI comes with sample datasets that will download on first use. You can also importall_datasetswhich is a dictionary of all sample datasets likeDict[str, DataFrame]frompandasguiimportshowfrompandasgui.datasetsimportpokemon,titanic,all_datasetsshow(pokemon,titanic)show(**all_datasets)FeaturesView DataFrames and Series (with MultiIndex support)Interactive plottingFilteringStatistics summaryData editing and copy / pasteImport CSV files with drag & dropSearch toolbarMore InfoIssues, feedback and pull requests are welcome.This project is still in version 0.x.y and subject to breaking changes. The latest changes will be on thedevelopbranch, and will be occasionally merged tomasteras a release with a tag indicating the version number and published to PyPi.If you like this project remember to leave a ⭐! And if you want to give more support you canBuy Me A Coffee.
Package: pandas-highcharts
What is itpandas-highchartsis a Python package which allows you to easily
buildHighcharts plotswithpandas.DataFrameobjects.Motivationpandas is the best tool to handle data in Pythonpandas is able to produce matplotlib plots. They work pretty well but
have two major drawbacksNot very web friendlyPretty uglyHighcharts produce nice, interactive plot in your browser and is very
completeFeaturesSame interface as DataFrame.plotFollowing parameters are handleddataxykindfigsizeuse_indextitlegridlegendstylelogxlogyloglogxticksyticksxlimylimrotfontsizepositionstackedsort_columnssecondary_ymark_rightFollowing parameters are not handled (yet) :axaysubplotssharexshareylayoutcolormapcolorbarlayouttableyerrxerrkwdsYou can specify those specific highcharts parameters:tooltipStatic files (highcharts.js) are not embeddedInstallationInstall the package using pippipinstallpandas-highchartsUsageImport it in your viewsimportpandas_highchartsdf=...# create your dataframe herechart=pandas_highcharts.serialize(df,render_to='my-chart',output_type='json')In your templates<divid="my-chart"></div><scripttype="text/javascript">newHighcharts.Chart({{chart|safe}});</script>ContributingSee CONTRIBUTING.rst for information on how to contribute to pandas-highcharts.More examplesSome examples are available onnbviewer.Please read the doc forDataFrame.plot.For example, with the following dataset:importpandasaspdfrompandas_highcharts.coreimportserializefrompandas.compatimportStringIOdat="""ts;A;B;C
2015-01-01 00:00:00;27451873;29956800;113
2015-01-01 01:00:00;20259882;17906600;76
2015-01-01 02:00:00;11592256;12311600;48
2015-01-01 03:00:00;11795562;11750100;50
2015-01-01 04:00:00;9396718;10203900;43
2015-01-01 05:00:00;14902826;14341100;53"""df=pd.read_csv(StringIO(dat),sep=';',index_col='ts',parse_dates='ts')# Basic line plotchart=serialize(df,render_to="my-chart",title="My Chart")# Basic column plotchart=serialize(df,render_to="my-chart",title="Test",kind="bar")# Basic column plotchart=serialize(df,render_to="my-chart",title="Test",kind="barh")# Plot C on secondary axischart=serialize(df,render_to="my-chart",title="Test",secondary_y=["C"])# Plot on a 1000x700 divchart=serialize(df,render_to="my-chart",title="Test",figsize=(1000,700))
Package: pandas-historical
pandas_historicalAwesome pandas_historical created by dvvolynkinInstall it from PyPIpipinstallpandas_historicalUsageimportpandasaspdcurrencies_scraping=pd.DataFrame([{'date':'2022-02-21','key':'DOLLAR','value':78,'scraping_id':123},{'date':'2022-02-21','key':'EURO','value':87,'scraping_id':123},{'date':'2022-02-28','key':'DOLLAR','value':105,'scraping_id':124},{'date':'2022-03-07','key':'EURO','value':139,'scraping_id':125},{'date':'2022-03-07','key':'EURO','value':148,'scraping_id':125}])currencies_scrapingdatekeyvaluescraping_id02022-02-21DOLLAR7812312022-02-21EURO8712322022-02-28DOLLAR10512432022-03-07EURO13912542022-03-07EURO148125frompandas_historicalimportmake_historical_dfhistorical_df=make_historical_df(currencies_scraping)historical_dfdatekeyvaluescraping_id02022-02-21DOLLAR7812312022-02-28DOLLAR10512422022-02-21EURO8712332022-03-07EURO13912542022-03-07EURO148125frompandas_historicalimportupdate_historical_dfnew_values=pd.DataFrame([{'date':'2022-03-11','key':'DOLLAR','value':113,'scraping_id':127},{'date':'2022-03-11','key':'EURO','value':144,'scraping_id':127}])historical_df=update_historical_df(historical_df,new_values)historical_dfdatekeyvaluescraping_id02022-02-21DOLLAR7812312022-02-28DOLLAR10512422022-03-11DOLLAR11312732022-02-21EURO8712342022-03-07EURO13912552022-03-07EURO14812562022-03-11EURO144127frompandas_historicalimportget_history_stateget_history_state(historical_df)datekeyvaluescraping_id22022-03-11 00:00:00DOLLAR11312762022-03-11 00:00:00EURO144127get_history_state(historical_df,state_date='2022-03-07')datekeyvaluescraping_id12022-02-28 00:00:00DOLLAR10512442022-03-07 00:00:00EURO139125DevelopmentRead theCONTRIBUTING.mdfile.
Package: pandas-hug
pandas hugsometimes you need to embrace your data and get a little, or a lot, more of it into memory.your column data types are rarely space efficient. most of the time this is because they were chosen by someone else, but sometimes its just a hassle to find the most space efficient types.pandas-hugis here to help crush your data to fit in memory.installationpipinstallpandas-hugusageimportpandasaspdimportpandas_hugS=pd.Series([2**8])A=pd.Series([f'a{i}'foriinrange(100)])M=pd.Series([42])E=pd.Series(['a','b','c']*15)df=pd.DataFrame({'S':S,'A':A,'M':M,'E':E})df.info()<class'pandas.core.frame.DataFrame'>RangeIndex:100entries,0to99Datacolumns(total4columns):#   Column  Non-Null Count  Dtype----------------------------0S1non-nullfloat641A100non-nullobject2M1non-nullfloat643E45non-nullobjectdtypes:float64(2),object(2)memoryusage:3.2+KBdf.convert_dtypes().hug().info()<class'pandas.core.frame.DataFrame'>RangeIndex:100entries,0to99Datacolumns(total4columns):#   Column  Non-Null Count  Dtype----------------------------0S1non-nullUInt161A100non-nullstring2M1non-nullUInt83E45non-nullcategorydtypes:UInt16(1),UInt8(1),category(1),string(1)memoryusage:1.6KBpandas-hugmonkey-patchespandas.DataFrameandpandas.Seriesto add thehug()method.you should callconvert_dtypes()before hugging your data. this does useful things like convertingfloattoint(pandas >=1.2.0, dec 2020) andobjecttostringwhere appropriate.
Package: pandas-illustrated
п»ї# pandas-illustratedThis repo contains code for a number of helper functions mentioned in thePandas Illustratedguide.Installation:pip install pandas-illustratedContentsBasic operations:find(s, x, pos=False)findall(s, x, pos=False)insert(dst, pos, value, label, axis=0, ignore_index = False,  order=None, allow_duplicates=False, inplace=False)append(dst, value, label = lib.no_default, axis=0, ignore_index = False, order=None, allow_duplicates: bool = False, inplace=False)drop(obj, items=None, like=None, regex=None, axis=None)move(obj, pos, label=None, column=None, index=None, axis=None, reset_index=False)join(dfs, on=None, how="left", suffixes=None)Visualization improvements:patch_series_repr(footer=True)unpatch_series_repr()sidebyside(*dfs, names=[], index=True, valign="top")sbs = sidebysideMultiIndex helpers:patch_mi_co()from_dict(d)from_kw(**kwargs)Locking columns order:locked(obj, level=None, axis=None, categories=None, inplace=False)lock = locked with inplace=Truevis_lock(obj, checkmark="вњ“")vis_patch()vis_unpatch()from_product(iterables, sortorder=None, names=lib.no_default, lock=True)MultiIndex manipulations:get_level(obj, level_id, axis=None)set_level(obj, level_id, labels, name=lib.no_default, axis=None, inplace=False)move_level(obj, src, dst, axis=None, inplace=False, sort=False)insert_level(obj, pos, labels, name=lib.no_default, axis=None, inplace=False, sort=False)drop_level(obj, level_id, axis=None, inplace=False)swap_levels(obj, i: Axis = -2, j: Axis = -1, axis: Axis = None, inplace=False, sort=False)join_levels(obj, name=None, sep="_", axis=None, inplace=False)split_level(obj, names=None, sep="_", axis=None, inplace=False)rename_level(obj, mapping, level_id=None, axis=None, inplace=False)Usagefind and findallBy defaultfind(series, value)looks for the first occurrence of the givenvaluein aseriesand returns the corresponsing index label.>>>importpandasaspd>>>importpdi>>>s=pd.Series([4,2,4,6],index=['cat','penguin','dog','butterfly'])>>>pdi.find(s,2)'penguin'>>>pdi.find(s,4)'cat'When the value is not found raises aValueError.findall(series, value)returns a (possibly empty) index of all matching occurrences:>>>pdi.findall(s,4)Index(['cat','dog'],dtype='object')Withpos=Truekeyword argumentfind()andfindall()return the positional index instead:>>>pdi.find(s,2,pos=True)1>>>pdi.find(s,4,pos=True)0There is a number of ways to find index label for a given value. The most efficient of them are:вЂ”s.index[s.tolist().index(x)]# faster for Series with less than 1000 elementsвЂ”s.index[np.where(s==x)[0][0]]# faster for Series with over 1000 elementsfind()chooses optimal implementation depending on the series size;findall()always uses thewhereimplementation.Improving Series RepresentationRunpdi.patch_series_repr()to make Series look better:If you want to display several Series from one cell, calldisplay(s)for each.Displaying several Pandas objects side vy sideTo display several dataframes, series or indices side by side runpdi.sidebyside(s1, s2, ...)TestingRunpytestin the project root.
Package: pandas-indexing
Summarypandas-indexingis a helpers package to make interacting with pandas multi-indices less
painful. It contains functions, thatfilter easily into multi indices:isin,ismatchadd or update levels in a multiindex:assignlevelselect one or multiple specific levels:projectlevelUsageGiven you have a time-series like dataframe with several multi index levels, likemodel,scenario,variable:
then you can select a subset with:df.loc[isin(model="m1",scenario=["s1","s2"])]or with shell like glob-patterns:df.loc[ismatch(model="REMIND*",variable="Emissions|**",unit=["Mt CO2/yr","kt N2O/yr"])]You can overwrite index levels:assignlevel(df,selected=1)or project your data to only a few desired index levels:projectlevel(df,["model","scenario"])All commands are described in detail in the API reference indocumentationand are
togehter with introductions for installing and using this package, but they are mostly
bare-bones atm.Issues and DiscussionsAs usual for any GitHub-based project, raise anissueif you find any bug or
want to suggest an improvement, or open a discussion if you want to discuss.
Package: pandas-jalali
pandas-jalaliPackage for converting dates from Gregorian to A Persian Date (aka:
Jalali Calendar) and back.This package is port of packagekhayyamto use with Pandas
dataframes.InstallationTo install from package manager:pip install pandas-jalaliUsageTo convert from Jalali dates to Gregorian:importpandasaspdfrompandas_jalali.converterimportget_gregorian_date_from_jalali_datedf=pd.DataFrame([[1346,1,1],[1346,10,20]],columns=['year','month','day'])df['year_gregorian'],df['month_gregorian'],df['day_gregorian']=get_gregorian_date_from_jalali_date(df['year'],df['month'],df['day'])printdfyear  month   day  year_gregorian  month_gregorian  day_gregorian
0  1346.0    1.0   1.0          1967.0              3.0           21.0
1  1346.0   10.0  20.0          1968.0              1.0           10.0To convert from Gregorian dates to Jalali:Not yet implementedTestsAt first make sure that you are in virtualenv.Install all dependencies:make setupTo run tests:make testLicenseMIT licence
Package: pandas-label-encoder
Pandas-powered LabelEncoderPerformance benchmarkFrom the test, compare to sklearn's LabelEncoder.Total rows: 24,123,464
Scikit-learn's LabelEncoder - 13.35 seconds
Pandas-powered LabelEncoder - 2.44 secondsUsageInstallationpipinstallpandas-label-encoderInitiation and fittingimportpandas_label_encoderasecfrompandas_label_encoderimportEncoderCategoryErrorcategories=['Cat','Dog','Bird']# can be pd.Series, np.array, list# Fit at inititationanimal_encoder=ec.Encoder(categories)# Fit lateranimal_encoder=ec.Encoder()animal_encoder.fit(categories)animal_encoder.categories# ['Cat', 'Dog', 'Bird'], read-only# Trying to use functions before assign appropiate categories will raise EncoderCategoryErrorec.Encoder().transform()# Raise EncoderCategoryErrorec.Encoder().inverse_transform()# Raise EncoderCategoryErrorTransformUnknown categories would be parsed as -1If you want to raise an error, there are 2 validation options.validation=all-- Raise EncoderError if any result is -1validation=any-- Raise EncoderError if all of them are -1frompandas_label_encoderimportEncoderValidationErroranimal_encoder.transform(['Cat'])# [2]animal_encoder.transform(['Fish'])# [-1]animal_encoder.transform(['Fish'],validation='all')# Raise EncoderValidationErroranimal_encoder.transform(['Fish'],validation='any')# Raise EncoderValidationErrortry:animal_encoder.transform(['Fish','Cat'],validation='all')# Raise EncoderValidationErrorexceptEncoderError:print('There is an unknown animal.')animal_encoder.transform(['Fish','Cat'],validation='any')# [-1, 2]Inverse transformUnknown categories would be parsed as NaNIf you want to raise an error, there are 2 validation options.validation=all-- Raise EncoderError if any result is NaNvalidation=any-- Raise EncoderError if all of them are NaNfrompandas_label_encoderimportEncoderValidationErroranimal_encoder.inverse_transform([2])# ['Cat']animal_encoder.inverse_transform([9])# [NaN]animal_encoder.inverse_transform([9],validation='all')# Raise EncoderValidationErroranimal_encoder.inverse_transform([9],validation='any')# Raise EncoderValidationErrortry:animal_encoder.inverse_transform([9,2],validation='all')# Raise EncoderValidationErrorexceptEncoderError:print('There is an unknown animal.')animal_encoder.inverse_transform([9,2],validation='any')# [NaN, 'Cat']Save and load the encoderThe load_encoder and encoder.Encoder.load methods will load the encoder and check for the encoder version.Different encoder version may have some changes that cause errors.To check current encoder version, useencoder.Encoder.__version__.frompandas_label_encoderimportsave_encoder,load_encoder# Save or load other encoder directly from the encoder itselfanimal_encoder.save(path)# save current encoderanimal_encoder.load(path)# load other encoder and assign to current encoder# Save or load other encoder by using functionsanimal_encoder=load_encoder(path)save_encoder(path)
Package: pandaslearn
pandaslearnpandaslearnis a small wrapper on top ofscikit-learnto automate common modeling tasks.CreateTrainerinstance withDatasetandModelinstances,__init__()inTrainerinstance should populateDatasetandModelinstance'sloggerattributes. Methods onDatasetandModelshould be called after that, so that everything gets logged appropriately.TODOTODO: visualization: add barcharts (plotnine)TODO: visualization: add histograms (plotnine)TODO: visualization: add scatterplots (plotnine)TODO: visualization: add lineplots (plotnine)TODO: visualization: add boxplots (plotnine)TODO: visualization: add violin plots (plotnine)TODO: visualization: add function to change theme (xkcd, ?)TODO: add ageonamespace (+ feature engineering, plots)TODO: add tests against a few standard fixtures (precompute values and test against them)TODO: integrate missingno package: functions to only compute/sort nullityTODO: integrate missingno package: plotnine functions matching missingno plot(including geo)TODO: integrate missingno package: timeseries nullity plots (just plot all timelines with gaps)TODO: pandas-profiling has a lot of useful analysis useful for ml. Integrate those (provide textual outcomes like dicts or dfs instead of plot)TODO: future integration targets:https://compose.alteryx.com/en/stable/TODO: future integration targets:https://featuretools.alteryx.com/en/stable/TODO: future integration targets:https://evalml.alteryx.com/en/stable/
Package: pandas-lineage
pandas-lineageBEWARE: This project is in very early stages (as of 2022-09-12)pandas-lineage is intended to extend the functionality of I/O and standard transform operations on a pandas dataframe to emit OpenLineage RunEvents. I am starting just with read/write operations emiting RunEvents with schema facets.Badges:Installationpip install pandas-lineageDevelopment Documentationdependency-managementpublishingtestingExamples:marquez-examplescontains getting started code and a script for running Marquez locally in Dockermock-api-examplecontains getting started code and a simple Flask API for sending lineage events to which will just always return a 200 status codeReferences::green_heart:Marquez:green_heart::green_heart:OpenLineage:green_heart::green_heart:Pandas:green_heart:Contributing:IssuesI have not created any sort of contribution guide yet, but I don't want that to stop anyone!
If you are interested in contributing, fork this repository and open a PR. As this becomes more feature-rich/useful, we will establish a contributors workflow. For now, please just use the pre-commit hooks.Notes:The pandas-lineage directory structure (for now) will mirror the directory structure of pandas for the components that it is extending.
Package: pandas-linker
# pandas-linker`pandas-linker` runs comparison windows over different sortings of a pandas DataFrame and links the rows via assigned UUIDs. This library does not actually do any duplicate detection. Instead it provides a harness to run your own comparison functions on your data.This library is meant for datasets of a size where comparing every row with every other is undesirable. Instead you can decide on a sorting order of the DataFrame and only compare every row with every other inside a sliding window.## Install```pip install pandas-linker```## ExampleLet's say you have a DataFrame like this:[ix] | name | country------|------|--------0  | Pete | Spain1  | Mary | USA2  | Bart | US3  | Mary | USand you want to detect similar rows and mark them as such. Here's how to do that:```pythonfrom pandas_linker import get_linkerdef compare_rows(a, b):''' Example function that decides if two rows represent same entity.'''return a['name'] in b['name'] or b['name'] in a['name']# df is a pandas.DataFrame with a unique indexwith get_linker(df, field='uid') as linker:print('Comparing in 10 row window sorted by name')linker(sort_cols=['name'], window_size=10, cmp=compare_rows)print('Comparing in 15 row window sorted by country')linker(sort_cols=['country'], window_size=15, cmp=compare_rows)```After running the linker the process is complete[ix] | name | country | uid------|------|---------|----0  | Pete | Spain   | 7509781940fc471cad5dc32944652d701  | Mary | USA     | 8f8dccd91568472daf740e9160349d6c2  | Bart | US      | 12b55fbe80f64d378193acd727b0e0513  | Mary | US      | 8f8dccd91568472daf740e9160349d6cNote that both "Mary" rows in the DataFrame have been identified as representingthe same entity and were assigned the same UUID.
Package: pandas-lite
pandas_liteA simpler alternative to pandasMain GoalsA very minimal set of featuresBe as explicit as possibleThere should be one– and preferably only one –obvious way to do it.Data StructuresOnly DataFramesNo SeriesData TypesOnly primitive types - int, float, boolean, numpy.unicodeNo object data typesRow and Column LabelsNo index, meaning no row labelsNo hierarchical indexColumn names must be stringsColumn names must be uniqueColumns stored in a numpy arraySubset SelectionOnly one way to select data -[ ]Subset selection will be explicit and necessitate both rows and
columnsRows will be selected only by integer locationColumns will be selected by either label or integer location. Since
columns must be strings, this will not be amibguousColumn names cannot be duplicatedAll selections and operations copyAll selections and operations provide new copies of the dataThis will avoid any chained indexing confusionDevelopmentMust use type hintsMust use 3.6 - fstringsMust have numpy, bottleneck, numexprSmall feature setImplement as few attributes and methods as possibleFocus on good idiomatic cookbook examples for doing more complex
tasksOnly Scalar Data TypesNo complex Python data types - [x] bool - always 8 bits, not-null - [x]
int - always 64 bits, not-null - [x] float - always 64 bits, nulls
allowed - [x] str - A python unicode object, nulls allowed - [ ]
categorical - [ ] datetime - [ ] timedeltaAttributes to implement[x] size[x] shape[x] values[x] dtypesMay not implement any of the binary operators as methods (add, sub, mul,
etc…)MethodsStats- [x] abs - [x] all - [x] any - [x] argmax - [x] argmin - [x]
clip - [ ] corr - [x] count - [ ] cov - [x] cummax - [x] cummin - [ ]
cumprod - [x] cumsum - [ ] describe - [x] max - [x] min - [x] median -
[x] mean - [ ] mode - [ ] nlargest - [ ] nsmallest - [ ] quantile - [ ]
rank - [x] std - [x] sum - [x] var - [ ] unique - [ ] nuniqueSelection- [ ] drop - [ ] drop_duplicates - [x] head - [ ] isin -
[ ] sample - [x] select_dtypes - [x] tail - [ ] whereMissing Data- [ ] isna - [ ] dropna - [ ] fillna - [ ] interpolateOther- [ ] append - [ ] apply - [ ] assign - [x] astype - [ ]
groupby - [ ] info - [ ] melt - [ ] memory_usage - [ ] merge - [ ]
pivot - [ ] replace - [ ] rolling - [ ] sort_valuesFunctions- [ ] read_csv - [ ] read_sql - [ ] concat