Saravanan666/pypi-org huggingface dataset:
Package: pandas-little-helper
Pandas Little HelperAs part of a data sciense course I took, I've created a nice list of tools, that I found usefull. I think you can enjoy them too. I've decided that the right thing to do is publish them to help you with your data sciense journyWarningThis module is still in alpha stages, and should not be used in production. Expect no backward
compatibility at this pointLicenseThis package is released with GPL 3 for private and commertial use.How to installI don't know. This is the first package I'm trying to build. I promise I'll get back to you when I canHow to contributeAll contributions must follow the following guidelines:All module code Must pass with 100% mypy.All "type: ignore" comments have to be precided with a comment explaining itAll module code Must pass pylint with 10/10All functionalities must have a pytest unittest. Test can ignore pylint and mypy rulesAll tests must passTo contribute Just follow these instructions.
(https://www.dataschool.io/how-to-contribute-on-github/)How to I talk to you?guruyaya@gmail.com
Package: pandas-lookup
pandas-lookupA port ofagate-lookupthat provides the lookup in Pandas DataFrames instead of Agate Tables.This uses lookup tables from Wireservice'slookupproject.Installationpip install pandas-lookupLook up a column from a lookup tableWhen the key in your data is the same as the key in the lookup table:>>> import pandaslookup
>>> import pandas as pd
>>> df = pd.DataFrame({'usps': ['CT', 'NY', 'NJ']})
>>> print(df)
  state_abbr
0         CT
1         NY
2         NJ
>>> df.pipe(pandaslookup.lookup, 'usps', 'state')
  usps        state
0   CT  Connecticut
1   NY     New York
2   NJ   New JerseyWhen the key in your data is different than the key in the lookup table:>>> import pandaslookup
>>> import pandas as pd
>>> df = pd.DataFrame({'state_abbr': ['CT', 'NY', 'NJ']})
>>> print(df)
  state_abbr
0         CT
1         NY
2         NJ
>>> df.pipe(pandaslookup.lookup, 'state_abbr', 'state', lookup_key='usps')
  usps        state
0   CT  Connecticut
1   NY     New York
2   NJ   New JerseyRetrieve a table without joining>>> import pandaslookup
>>> table = pandaslookup.from_lookup(['usps', 'year'], 'population')
>>> print(table.head())
  usps  year  population
0   AL  1970     3454557
1   AL  1971     3497349
2   AL  1972     3540003
3   AL  1973     3580759
4   AL  1974     3627778Installing for developmentpipenv install --dev -e .Running testspipenv run python -m unittestPrior artharbolkn/pandas-lookup: This is the same idea, but it relies on theagate-lookuppackage, which createds a needless dependency on Agate. Also, I don't think I like monkey-patchingDataFrame.
Package: pandas_mate
Welcome to pandas_mate Documentationpandas_mateprovides lots of utility method for pandas.pandas_mate.csv_io: provide advance csv I/O interface.pandas_mate.sql_io: provide database I/O interface.pandas_mate.transform: convert pandas’s data type into varieties of form.Quick LinksGitHub HomepageOnline DocumentationPyPI downloadInstallIssue submit and feature requestAPI reference and source codeInstallpandas_mateis released on PyPI, so all you need is:$pipinstallpandas_mateTo upgrade to latest version:$pipinstall--upgradepandas_mate
Package: pandas-md-summary
# pandas_markdown_summary
Markdown summaries of pandas DataFrame and Series### Installation -` pip install pandas_md_summary `### Usage
To generate markdown summary for a pandas DataFrame -` from pandas_md_summary import gen_markdown_for_df gen_markdown_for_df(df, 'therapy',out_dir='out,top=100) # df is the DataFrame # 'therapy' is a label for the table name. The generated table summary and folder will be named using this variable. # out_dir - the path where you would want the markdown summary generated # top - the number of most frequent values to include in each column summary of the DataFrame. `Refer Example.ipynb for a full example
Package: pandasmemuc
Automating memu emulator with pandaspip install pandasmemucTested against Windows 10 / Python 3.10 / AnacondaMemu Player and at least one Android instance must be installedhttps://www.memuplay.com/Most of the command line stuff already implemented, documentation soonhttps://www.memuplay.com/blog/memucommand-reference-manual.htmlhttps://www.memuplay.com/blog/wp-content/uploads/2016/02/MEmu-Command-Line-Management-Interface-Version-4.3.20_OSE.pdfExample:frompandasmemucimportMeMuc,phoneconfigimportpandasaspdphoneconfig.phone_dataframe=pd.read_pickle("c:\\allphonedata.pkl")# optionalphoneconfig.mac_address_prefix="52:54:00"df=MeMuc()# df.create_vm_96().update_status()df.import_vm(path=r"C:\ProgramData\anaconda3\envs\dfdir\memuimage.ova",name="mem38")df.iloc[-1].bb_start()df.iloc[-1].bb_installapp(r"C:\Users\hansc\Downloads\spotify-8-8-50-466.apk")df.iloc[-1].bb_startapp('com.spotify.music')# spoti = df.get_ui_automator_df(2)# spoti = spoti.loc[(spoti.bb_resource_id == 'com.spotify.music:id/bottom_navigation_item_icon') & (spoti.bb_content_desc == 'Buscar')].iloc[0].ff_bb_tap_exact_center# spoti = df.get_ui_automator_df(2)# spoti.loc[spoti.bb_resource_id == 'com.spotify.music:id/find_search_field'].ff_bb_tap_exact_center.iloc[0]()# df.bb_input_natural.iloc[0]('AC/DC')# allbois.reindex(allbois.bb_text.str.len().sort_values().index).iloc[0].ff_bb_tap_exact_center()# df.bb_adbtools.iloc[-1].aa_disable_notifications()# df.bb_adbtools.iloc[-1].aa_force_stop('com.microvirt.memuime')# df = m.get_ui_automator_df(0, save_screenshot=True) # check out https://github.com/hansalemaos/adbkit
Package: pandas-missing
Pandas missingInstallpipinstallpandas-missingorcondainstall-cjvelezmagicpandas_missingHow to useimportpandasaspdimportpandas_missingdf=pd.DataFrame.from_dict({"number":range(0,15)})df.iloc[3:6,:]=Nonedf<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>number00.011.022.03NaN4NaN5NaN66.077.088.099.01010.01111.01212.01313.01414.0df.missing.number_missing()3df.missing.number_complete()12
Package: pandas-ml
Overviewpandas,scikit-learnandxgboostintegration.Installation$ pip install pandas_mlDocumentationhttp://pandas-ml.readthedocs.org/en/stable/Example>>>importpandas_mlaspdml>>>importsklearn.datasetsasdatasets# create ModelFrame instance from sklearn.datasets>>>df=pdml.ModelFrame(datasets.load_digits())>>>type(df)<class'pandas_ml.core.frame.ModelFrame'># binarize data (features), not touching target>>>df.data=df.data.preprocessing.binarize()>>>df.head().target012345678...5455565758596061626300001111000...000011100011000111000...000001110022000111000...100001111033001111000...100011110044000110000...0000011100[5rowsx65columns]# split to training and test data>>>train_df,test_df=df.model_selection.train_test_split()# create estimator (accessor is mapped to sklearn namespace)>>>estimator=df.svm.LinearSVC()# fit to training data>>>train_df.fit(estimator)# predict test data>>>test_df.predict(estimator)041227...44854498Length:450,dtype:int64# Evaluate the result>>>test_df.metrics.confusion_matrix()Predicted0123456789Target052000000000103710010033202481000110311044010031410004301000501000390000601001035000700002004210802101000331902120000138Supported Packagesscikit-learnpatsyxgboost
Package: pandas-ml-common
The pandas ml common moduleThis module holds all common extensions and utilities for the pandas ml quant stack.
Feel free to study theexamplesas well.easy joining of data frames with multi indexesfrompandas_ml_commonimportpd,npdf1=pd.DataFrame({"a":np.random.random(10),"b":np.random.random(10)})print(df1.inner_join(df1,prefix_left='A',prefix='B',force_multi_index=True).to_markdown())('A', 'a')('A', 'b')('B', 'a')('B', 'b')00.9078920.7269130.9078920.72691310.6022750.1342780.6022750.13427820.2643990.2074290.2643990.20742930.5597510.8167590.5597510.81675940.9511720.7975240.9511720.79752450.5043320.519960.5043320.5199660.7652350.179080.7652350.1790870.3886910.6441030.3886910.64410380.6636360.6788790.6636360.67887990.2916030.01646270.2916030.0164627access columns with regexdf4=pd.DataFrame({"a_22_a":np.random.random(1),"b_21_b":np.random.random(1)})df4._[r'.*\d+_.']a_22_ab_21_b00.220390.0374084easy access multi level indexdf1.unique_level_columns(0)['A','B']df1.add_multi_index('Z',axis=1)data splitting, sampling and folding (aka cross validation)frompandas_ml_commonimportSampler,XYWeight,random_splitterdf2=pd.DataFrame({"c":np.random.random(10)})sampler=Sampler(XYWeight(df1,df2),splitter=random_splitter(0.5))forbatchesinsampler.sample_for_training():forbatchinbatches:print(batch)access to nested numpy arrays in data frame columns (df._.values)df3=pd.DataFrame({"a":[[1,2],[3,4],[5,6]]})df3._.valuesarray([[1,2],[3,4],[5,6]])dynamic method call providing suitable *args and **kwargs (dependency injection)frompandas_ml_commonimportcall_callable_dynamic_argsdefadder(a,b):returna+bcall_callable_dynamic_args(adder,a=12,b=10,c='illegal')22numpy utilsfrompandas_ml_commonimportnp_nansnp_nans((3,3))array([[nan,nan,nan],[nan,nan,nan],[nan,nan,nan]])frompandas_ml_commonimporttemp_seedwithtemp_seed(42):print(np.random.random(2))np.random.random(2)[0.374540120.95071431]array([0.69373278,0.69790163])serialization utilsfrompandas_ml_commonimportserializeb,deserializebdeserializeb(serializeb(np.array([1,2,3])))array([1,2,3])re-scalingsfrompandas_ml_commonimportReScalerx=np.arange(0,1,.1)rescaler=ReScaler((0,1),(5,-5))rescaler(x)array([5.,4.,3.,2.,1.,0.,-1.,-2.,-3.,-4.])
Package: pandas-ml-quant-data-provider
Pandas ML Quant Data ProviderAn independent module used to fetch time series data for quant finance studies.Example:# monkey patch pandasfrompandas_ml_quant_data_providerimportpd,YAHOO,INVESTING,CRYPTO_COMPARE# fetch data from various data sources#   * fetches all available dates#   * caches data for 10 minutesdf=pd.fetch_timeseries({YAHOO:["SPY","DIA"],INVESTING:["index::NYSE Tick Index::united states","bond::U.S. 30Y::united states"],CRYPTO_COMPARE:["BTC"]})df.tail()PS If you are not familiar with pandas MultiIndex, you can watch this video:How do I use the MultiIndex in pandas?InstallationFollow the instructions onhttps://github.com/KIC/pandas-ml-quantDocumentationCheck out the notebooks athttps://github.com/KIC/pandas-ml-quant/blob/master/notebooks
Package: pandas-ml-utils
Pandas ML UtilsPandas Machine Learning Utilities is part of a bigger set of libraries for a convenient experience. Usually exploring
statistical models start with a pandasDataFrame.But soon enough you will find yourself converting your data frames to numpy, splitting arrays, applying min
max scalers, lagging and concatenating columns etc. As a result your notebook looks messy and became and
unreadable beast. Yet the mess becomes only worse once you start to deploy your research into a productive
application. The untested hard coded data pipelines need be be maintained at two places.The aim of this library is to conveniently operate with data frames without and abstract away the ugly unreproducible
data pipelines. The only thing you need is the original unprocessed data frame where you started.
The data pipeline becomes a part of your model and gets saved that way. Going into production is as easy as
this:importpandasaspdimportpandas_ml_utils# monkey patch the `DataFrame`frompandas_ml_utilsimportModel# alternatively as a one liner `from pandas_ml_utils import pd, Model`model=Model.load('your_saved.model')df=pd.read_csv('your_raw_data.csv')df_prediction=df.model.predict(model)# do something with your predictiondf_prediction.plot()is intended to help you through your journey of statistical or machine learning models,
while you never need to leave the world of pandas.InstallationThe basic implementation supportsscikit learnclassifiers and regressors.pipinstallpandas-ml-utilsAdditional machine learning libraries are available as an add on:pipinstallpandas-ml-utils-torch# pytorch implementationpipinstallpandas-ml-utils-keras# keras + tensorflow 1.x implementationNote that the keras/tensorflow version is currently stalled as I focus on pytorch recently. This might change
with PyMC4 and tensorflow probabilityExampleYou will find some demo projects in theexamplesdirectory. But It might also be worth it to check
the unit tests and theintegration tests. Here is how classification challenge
might look like:
Package: pandas-ml-utils-torch
pytorch extension forpandas-ml-utilsAdds aPytorchModelto the pandas ml utils suite. While a regular class extendingnn.Moduleis sufficient,
there is also a special classPytorchNNwhich can be extended as well. Using this class has the following
advantages:allows to use L1, L2 regularization ->exampleallows different forward path for training and prediction (useful i.e. for reparameterization trick) ->exampleallows to implement auto-encoders easily by just providing the encode/decode functionsadded loss functions likeSoftDTW(fit time series) loss orHeteroscedasticityLoss(fit Normal Distribution) ->example
Package: pandas-money
Pandas MoneyA Pandas ArrayExtension for handling currency with int64 performanceGetting startedAdd to your projectpoetry add pandas-moneyor with pip/pipenvpipenv install pandas-moneyExamplefromdecimalimportDecimalimportpandasaspdimportpandas_moneyaspms=pd.Series(["1.11","2.22","3.33"],dtype="Money64")print(s)print(s+s)print(s*Decimal("1.5"))print(s/2)print(s.iloc[0],type(s.iloc[0]))outputs0    $1.11
1    $2.22
2    $3.33
dtype: Money64
0    $2.22
1    $4.44
2    $6.66
dtype: Money64
0    $1.66
1    $3.33
2    $5.00
dtype: Money64
0    $0.56
1    $1.11
2    $1.66
dtype: Money64
$1.11 <class 'moneyed.classes.Money'>SetupClone repopoetry installRun testsmaketestFormat and lint before commitmakepre-commit
Package: pandas-montecarlo
pandas-montecarlois a lightweight Python library for running simpleMonte Carlo Simulationson Pandas Series data.Changelog »Quick StartLet’s run a monte carlo simulation on the returns ofSPY(S&P 500 Spider ETF).First, let’s download SPY’s data and calculate the daily returns.frompandas_datareaderimportdatadf=data.get_data_yahoo("SPY")df['return']=df['Adj Close'].pct_change().fillna(0)Next, we’ll importpandas_montecarloand run monte carlo simulation
with 10 simulations (for demo simplifications) and bust/max drawdown set to-10.0%and goal threshhold set to+100.0%(defaults is>=0%):importpandas_montecarlomc=df['return'].montecarlo(sims=10,bust=-0.1,goal=1)Plot simulationsmc.plot(title="SPY Returns Monte Carlo Simulations")# optional: , figsize=(x, y)Show test statsprint(mc.stats)# prints{'min':0.98088401987146789,'max':0.98088401987146934,'mean':0.98088401987146911,'median':0.98088401987146911,'std':4.0792198665315552e-16,'maxdd':-0.17221175099828012,# max drawdown'bust':0.2,# probability of going bust'goal':0.0# probability of reaching 100% goal}Show bust / max drawdown statsprint(mc.maxdd)# prints{'min':-0.27743285515585991,'max':-0.00031922711279186444,'mean':-0.07888087155686732,'median':-0.06010335858432081,'std':0.062172124557467685}Access raw simulations’ DataFrameprint(mc.data.head())original          1          2          3          4  ...       10
0   0.000000   0.017745  -0.002586  -0.005346  -0.042107  ...  0.00139
1   0.002647   0.000050   0.000188   0.010141   0.007443  ...  0.00108
2   0.000704   0.002916   0.005324   0.000073  -0.003238  ...  0.00071
3   0.004221   0.008564   0.001397   0.007950  -0.006392  ...  0.00902
4   0.003328  -0.000511   0.005123   0.013491  -0.005105  ...  0.00252InstallationInstallpandas_montecarlousingpip:$pipinstallpandas_montecarlo--upgrade--no-cache-dirRequirementsPython>=3.4Pandas(tested to work with >=0.18.1)Matplotlib(tested to work with >=1.5.3)Legal Stuffpandas-montecarlois distributed under theGNU Lesser General Public License v3.0. See theLICENSE.txtfile in the release for details.P.S.Please drop me an note with any feedback you have.Ran Aroussi
Package: pandasmore
pandasmoreThe full documentation site ishere, and the GitHub page ishere.Here is a short description of some of the main functions (more details
below and in thedocumentation):setup_tseries:
cleans up dates and sets them as the indexsetup_panel:
cleans up dates and panel id’s and sets them as the index (panel id,
period date)lag: robust
lagging that accounts for panel structure, unsorted or duplicate
dates, or gaps in the time-seriesInstallpipinstallpandasmoreHow to useFirst, we set up an example dataset to showcase the functions in this
module.importpandasaspdimportnumpyasnpimportpandasmoreaspdmraw=pd.DataFrame(np.random.rand(15,2),columns=list('AB'),index=pd.MultiIndex.from_product([[1,2,np.nan],[np.nan,'2010-01','2010-02','2010-02','2010-04']],names=['firm_id','date'])).reset_index()rawfirm_iddateAB01.0NaN0.2493700.92633511.02010-010.2825010.51385921.02010-020.8042780.30717131.02010-020.8288950.74678941.02010-040.5690990.33181452.0NaN0.5339770.82345762.02010-010.2075580.40137872.02010-020.0860010.95937182.02010-020.0542300.99398092.02010-040.0625250.20027210NaNNaN0.0910120.63540911NaN2010-010.8663690.97239412NaN2010-020.4320870.83759713NaN2010-020.8782190.14800914NaN2010-040.8203860.834821df=pdm.setup_tseries(raw.query('firm_id==1'),time_var='date',time_var_format="%Y-%m",freq='M')dfdatedtdatefirm_idABMdate2010-012010-012010-01-011.00.2825010.5138592010-022010-022010-02-011.00.8288950.7467892010-042010-042010-04-011.00.5690990.331814df=pdm.setup_panel(raw,panel_ids='firm_id',time_var='date',time_var_format="%Y-%m",freq='M')dfdatedtdateABfirm_idMdate12010-012010-012010-01-010.2825010.5138592010-022010-022010-02-010.8288950.7467892010-042010-042010-04-010.5690990.33181422010-012010-012010-01-010.2075580.4013782010-022010-022010-02-010.0542300.9939802010-042010-042010-04-010.0625250.200272pdm.lag(df['A'])firm_id  Mdate  
1        2010-01         NaN
         2010-02    0.282501
         2010-04         NaN
2        2010-01         NaN
         2010-02    0.207558
         2010-04         NaN
Name: A_lag1, dtype: float64
Package: pandas-msgpack
pandas-msgpackis a package providing an interface to msgpack from pandasInstallationInstall latest release version via conda# not enabled YET$condainstallpandas-msgpack--channelconda-forgeInstall latest release version via pip$pipinstallpandas-msgpackInstall latest development version$pipinstallgit+https://github.com/pydata/pandas-msgpack.gitUsageSee thepandas-msgpack documentationfor more details.
Package: pandas-multi
pandas_multi============Simple loop for reading multiple csv files (matching a certain pattern) as a``pandas.DataFrame``.I'm aware this need can be solved in even one line of Python, but loadingmultiple similar csv's is just something that should be as easy as loadingone csv. If you don't want to add a new dependency to your project, googlewhat ``os.listdir`` and ``glob`` can do for you.Installation can be done by typing::pip install pandas_multiUsage of ``pandas_multi.read_csvs`` has been kept as similar as possible to``pandas.read_csv``::import pandas_multi# <sarcasm>#   Note that dataframes only work if you give them the non-descriptive name df# </sarcasm>df = pandas_multi.read_csvs('./20180728*.csv')# if you provide it with a path to a folder and nothing else, it will assume# everything in the folder is a comma-separated filedf = pandas_multi.read_csvs('./data/')# if this is not the case, do this:df = pandas_multi.read_csvs('./data/*.csv')All options that are available to ``pandas.read_csv`` or ``pandas.concat``can be passed into ``pandas_multi.read_csvs`` and will be redirected to theappropriate underlying functions.If you wish to maintain a trace back to the original data, you can run thefunction with the keyword argument ``filenames_as_keys=True``. Note that youshould no longer use the keyword argument ``keys``. This will be ignored.Please note that the API for concatenation of Excel sheets is not yet stable.
Package: pandasnet
pandasnetpandasnetis a python package build on top ofpythonnet.
It provides additional data conversions forpandas,numpyanddatetimePrerequisitespython 3.6 or higher.dotnet.dotnet also providesscriptsto proceed the installation by command line.Installationpip install pandasnetFeaturesTo load the converter you need to import the package once in your python environment.
If the dotnet clr isn't started yet through the pytonnet package the import will.importpandasnetWe construct a simple C# function to test conversionusingSystem;usingSystem.Collections.Generic;namespaceLibForTests{publicclassPandasNet{publicstaticDictionary<string,Array>BasicDataFrame(Dictionary<string,Array>df)=>df;}}We build this function into a library namedLibForTests.dll.
We load this library into our python environment then use it.importclrimportpandasnet# Load the convertersimportpandasaspdfromdatetimeimportdatetime# Load your dllclr.AddReference('LibForTests.dll')fromLibForTestsimportPandasNetaspdnetx=pd.DataFrame({'A':[1,2,3],'B':[1.23,1.24,1.22],'C':['foo','bar','other'],'D':[datetime(2021,1,22),datetime(2021,1,23),datetime(2021,1,24)]})y=pdnet.BasicDataFrame(x)print(y)Below an exhausitve list of supported data convertions.Python -> .NetPython.Netdatetime.datetimeDateTimedatetime.dateDateTimedatetime.timedeltaTimeSpandatetime.timeTimeSpannumpy.ndarray(dtype=bool_)bool[]numpy.ndarray(dtype=int8)sbyte[]numpy.ndarray(dtype=int16)short[]numpy.ndarray(dtype=int32)int[]numpy.ndarray(dtype=int64)long[]numpy.ndarray(dtype=uint8)byte[]numpy.ndarray(dtype=uint16)ushort[]numpy.ndarray(dtype=uint32)uint[]numpy.ndarray(dtype=uint64)ulong[]numpy.ndarray(dtype=float32)float[]numpy.ndarray(dtype=float64)double[]numpy.ndarray(dtype=datetime64)DateTime[]numpy.ndarray(dtype=timedelta64)TimeSpan[]numpy.ndarray(dtype=str)string[]pandas._libs.tslibs.timestamps.TimestampDateTimepandas._libs.tslibs.timedeltas.TimeDeltaTimeSpanpandas.core.series.SeriesArraypandas.core.frame.DataFrameDictionary[string, Array].Net -> Python.NetPythonDateTimedatetime.datetimeTimeSpandatetime.timedeltabool[]numpy.ndarray(dtype=bool_)sbyte[]numpy.ndarray(dtype=int8)short[]numpy.ndarray(dtype=int16)int[]numpy.ndarray(dtype=int32)long[]numpy.ndarray(dtype=int64)byte[]numpy.ndarray(dtype=uint8)ushort[]numpy.ndarray(dtype=uint16)uint[]numpy.ndarray(dtype=uint32)ulong[]numpy.ndarray(dtype=uint64)float[]numpy.ndarray(dtype=float32)double[]numpy.ndarray(dtype=float64)DateTime[]numpy.ndarray(dtype=datetime64)TimeSpan[]numpy.ndarray(dtype=timedelta64)Dictionary[string, Array]pandas.core.frame.DataFrameContributingIssue tracker:https://github.com/fdieulle/pandasnet/issuesIf you want to checkout the project and propose your own contribution, you will need to setup it following few steps:Create a virtual environment:python -m venv venvActivate your virtual environment:venv/Scripts/activateInstall package dependenciespip install -r requirements.txtLicenseThis project is open source under theMIT license.