Saravanan666/pypi-org huggingface dataset:
Package: pandas-sacct
Pandas sacctProvides anSacctclass interface to theSlurm.In general options tosacct(seeman sacct) can passed by keyword toSacct.UsageSacct(flag=True)corresponds tosacct --flag.Sacct(key=value)corresponds tosacct --key=value.starttime and endtime options support both string and types that
support.isoformat()such aspd.Timestamp().examplesimportsaccts=sacct.Sacct(starttime="2020-08-30T00:00:00",endtime="2020-08-31T00:00:00",allocations=True)df=s.execute()print(s)print(df.head())sacct --format=jobidraw,start,end,nodelist --noheader --parsable2 --truncate --allocations --starttime=2020-08-30T00:00:00 --endtime=2020-08-31T00:00:00
   jobidraw               start                 end    nodelist
0  33891896 2020-08-30 03:01:58 2020-08-30 03:15:14  [nid01454]
1  33891898 2020-08-30 03:01:58 2020-08-30 03:04:59  [nid01565]
2  33891899 2020-08-30 03:07:13 2020-08-30 03:10:10  [nid01163]
3  33891900 2020-08-30 03:01:58 2020-08-30 03:15:56  [nid01998]
4  33891902 2020-08-30 03:01:58 2020-08-30 03:04:54  [nid01716]importsacctimportpandasaspdt0=pd.Timestamp("2020-08-30T00:00:00")t1=t0+pd.Timedelta(days=1)s=sacct.Sacct(starttime=t0,endtime=t1,allocations=True,format=['jobidraw','jobname'],allusers=True)df=s.execute()print(s)print(df.head())sacct --format=jobidraw,jobname --noheader --parsable2 --truncate --allocations --starttime=2020-08-30T00:00:00 --endtime=2020-08-31T00:00:00 --allusers
   jobidraw   jobname
0  27731125  JT_D1_16
1  27731129  JT_D1_16
2  27731131  JT_D1_16
3  27731136  JT_D1_16
4  27731165  JT_D2_16importsacctimportpandasaspdt0=pd.Timestamp("2020-08-30T00:00:00")t1=t0+pd.Timedelta(days=1)s=sacct.Sacct(starttime=t0,endtime=t1,format=['jobidraw','jobname'],allusers=True)df=s.execute()print(s)print(df.head())sacct --format=jobidraw,jobname --noheader --parsable2 --truncate --starttime=2020-08-30T00:00:00 --endtime=2020-08-31T00:00:00 --allusers
   jobidraw    step   jobname
0  29056157   batch     batch
1  29056157  extern    extern
2  29056157       0  lmp_cori
3  29056168   batch     batch
4  29056168  extern    externSpecificsacctcommand (optional):s = sacct.Sacct(sacct_cmd="/usr/bin/sacct", starttime="2020-08-30T00:00:00", endtime="2020-08-31T00:00:00", allocations=True)Copywrite NoticePandas-sacct Copyright (c) 2020, The Regents of the University of California,
through Lawrence Berkeley National Laboratory (subject to receipt of
any required approvals from the U.S. Dept. of Energy). All rights reserved.If you have questions about your rights to use or distribute this software,
please contact Berkeley Lab's Intellectual Property Office atIPO@lbl.gov.NOTICE.  This Software was developed under funding from the U.S. Department
of Energy and the U.S. Government consequently retains certain rights.  As
such, the U.S. Government has been granted for itself and others acting on
its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the
Software to reproduce, distribute copies to the public, prepare derivative
works, and  perform publicly and display publicly, and to permit others to do so.
Package: pandas-sans-lambdas
Pandas Sans LambdasPandas method chaining usingassignorlocusually means using lambdas. These get repetitive and reduce readability. Pandas Sans Lambdas is a solution to this. Here's an example:importpandasaspdfrompandas_sans_lambdasimportcoldf=pd.DataFrame({"a":[1,2,3],"b":[4,5,6]})# The old waydf=df.assign(with_lambdas=lambdaDF:DF["a"]**2/DF["b"])# The new waydf=df.assign(sans_lambdas=col("a")**2/col("b"))print(df)Hopefully you agree that this is more readable!Current project statusThis package is under development. Specifically, unit test coverage is currently incomplete. Get in touch if you want to contribute to test coverage, or if you find any bugs!
Package: pandas-scd2
pandas_scdexecuting slowly changing dimension type 2 on pandas dataframesgiven pandas df of the source table, and pandas df of the target table, returning pandas df with the entire new target after scd2Installationbasic installtion :pip install pandas-scd2Getting startedfrom pandas_scd import scd2
import pandas as df

tgt = pd.DataFrame.from_dict({'first_name': ["Chris"], 'last_name': ['Paul'], 'team': ["Clippers"], "start_ts": [datetime(2012, 1, 14, 3, 21, 34)], "end_ts": [None], "is_active": [True]})

src = pd.DataFrame.from_dict({'first_name': ["Chris"], 'last_name': ['Paul'], 'team': ['Suns']})

final_df = scd2(src, tgt)tgt:first_namelast_nameteamstart_tsend_tsis_activeChrisPaulClippers2012-01-14 03:21:34Truesrc:first_namelast_nameteamChrisPaulClippersfinal_df:first_namelast_nameteamstart_tsend_tsis_activeChrisPaulClippers2012-01-14 03:21:342018-01-01 00:00:00FalseChrisPaulSuns2018-01-01 00:00:00Truesrc:pandas dataframe with the source of the SCDtgt:pandas dataframe with the target of the SCD (target can be empty)cols_to_track:list of columns to track changes (default is all columns from the source table)tz:pytz time zone to use on start_ts and end_ts, default is None (will use local time)
Package: pandas-schema
For the full documentation, refer to theGithub Pages Website.PandasSchema is a module for validating tabulated data, such as CSVs
(Comma Separated Value files), and TSVs (Tab Separated Value files).
It uses the incredibly powerful data analysis tool Pandas to do so
quickly and efficiently.For example, say your code expects a CSV that looks a bit like this:Given Name,Family Name,Age,Sex,Customer ID
Gerald,Hampton,82,Male,2582GABK
Yuuwa,Miyake,27,Male,7951WVLW
Edyta,Majewska,50,Female,7758NSIDNow you want to be able to ensure that the data in your CSV is in the
correct format:importpandasaspdfromioimportStringIOfrompandas_schemaimportColumn,Schemafrompandas_schema.validationimportLeadingWhitespaceValidation,TrailingWhitespaceValidation,CanConvertValidation,MatchesPatternValidation,InRangeValidation,InListValidationschema=Schema([Column('Given Name',[LeadingWhitespaceValidation(),TrailingWhitespaceValidation()]),Column('Family Name',[LeadingWhitespaceValidation(),TrailingWhitespaceValidation()]),Column('Age',[InRangeValidation(0,120)]),Column('Sex',[InListValidation(['Male','Female','Other'])]),Column('Customer ID',[MatchesPatternValidation(r'\d{4}[A-Z]{4}')])])test_data=pd.read_csv(StringIO('''Given Name,Family Name,Age,Sex,Customer ID
Gerald ,Hampton,82,Male,2582GABK
Yuuwa,Miyake,270,male,7951WVLW
Edyta,Majewska ,50,Female,775ANSID
'''))errors=schema.validate(test_data)forerrorinerrors:print(error)PandasSchema would then output{row: 0, column: "Given Name"}: "Gerald " contains trailing whitespace
{row: 1, column: "Age"}: "270" was not in the range [0, 120)
{row: 1, column: "Sex"}: "male" is not in the list of legal options (Male, Female, Other)
{row: 2, column: "Family Name"}: "Majewska " contains trailing whitespace
{row: 2, column: "Customer ID"}: "775ANSID" does not match the pattern "\d{4}[A-Z]{4}"
Package: pandas-select
pandas-selectis a collection of DataFrame selectors that facilitates indexing
and selecting data, fully compatible with pandas vanilla indexing.The selector functions can choose variables based on theirname,data type,arbitrary conditions,
or anycombination of these.pandas-selectis inspired by the excellent R librarytidyselect.Installationpandas-selectis a Python-only packagehosted on PyPI.
It can be installed viapip:pip install pandas-selectDesign goalsFully compatible with thepandas.DataFrame[]operator and thepandas.DataFrame.locaccessor.Emphasise readability and conciseness by cutting boilerplate:# pandas-selectdf[AllNumeric()]# vanilladf.select_dtypes("number").columns# pandas-selectdf[StartsWith("Type")|"Legendary"]# vanilladf.loc[:,df.columns.str.startswith("Type")|(df.columns=="Legendary")]Ease the challenges ofindexing with hierarchical indexand offers an alternative toslicerswhen the labels cannot be listed manually.# pandas-selectdf_mi.loc[Contains("Jeff",axis="index",level="Name")]# vanilladf_mi.loc[df_mi.index.get_level_values("Name").str.contains("Jeff")]Play well with machine learning applications.Respect the columns order.Allowdeferred selectionwhen the DataFrame’s columns are not known in advance,
for example in automated machine learning applications.Offer integration withsklearn.frompandas_selectimportAnyOf,AllBool,AllNominal,AllNumeric,ColumnSelectorfromsklearn.composeimportmake_column_transformerfromsklearn.preprocessingimportOneHotEncoder,StandardScalerct=make_column_transformer((StandardScaler(),ColumnSelector(AllNumeric()&~AnyOf("Generation"))),(OneHotEncoder(),ColumnSelector(AllNominal()|AllBool()|"Generation")),)ct.fit_transform(df)Project Informationpandas-selectis released under theBS3license,
its documentation lives atRead the Docs,
the code onGitHub,
and the latest release onPyPI.
It is tested on Python 3.6+.
Package: pandas-selectable
pandas-selectableWhat Is It?pandas-selectableadds aselectaccessor to pandas DataFrames and Series. It's likequerybut with the niceties of tab-completion.QuickstartIn[1]:importnumpyasnpIn[2]:importpandasaspdIn[3]:importpandas_selectable# magicIn[4]:dta=pd.DataFrame.from_dict({...:'X':['A','B','C']*5,...:'Y':np.arange(1,16),...:'Z':pd.date_range('2020-01-01',periods=15)...:})In[5]:dta.head()Out[5]:XYZ0A12020-01-011B22020-01-022C32020-01-033A42020-01-044B52020-01-05In[6]:dta.select.X=='B'Out[6]:XYZ1B22020-01-024B52020-01-057B82020-01-0810B112020-01-1113B142020-01-14In[7]:dta.select.Z>='2020-01-03'Out[7]:XYZ2C32020-01-033A42020-01-044B52020-01-055C62020-01-066A72020-01-077B82020-01-088C92020-01-099A102020-01-1010B112020-01-1111C122020-01-1212A132020-01-1313B142020-01-1414C152020-01-15In[8]:dta.select.X.str.contains('A')Out[8]:XYZ0A12020-01-013A42020-01-046A72020-01-079A102020-01-1012A132020-01-13In[9]:dta.select.Z.dt.is_month_startOut[9]:XYZ0A12020-01-01It also works for Series.In[10]:dta.X.select=='A'Out[10]:0A3A6A9A12AName:X,dtype:objectThough the string and datetime accessor APIs are slightly inconsistent. They're available via the select accessor now.In[11]:dta.X.select.str.contains('B')Out[11]:1B4B7B10B13BName:X,dtype:objectRequirementspandas>= 1.1Installationpipinstallpandas-selectable
Package: pandas-sets
Pandas Sets: Set-oriented Operations in PandasIf you store standard Pythonsets orfrozensets in yourSeriesorDataFrameobjects, you'll find this useful.Thepandas_setspackage adds a.setaccessor to any pandasSeriesobject;
it's like.dtfordatetimeor.strforstring, but forset.It exposes all public methods available in the standardset.Installationpipinstallpandas-setsJust import thepandas_setspackage and it will register a.setaccessor to anySeriesobject.importpandas_setsExamplesimportpandas_setsimportpandasaspddf=pd.DataFrame({'post':[1,2,3,4],'tags':[{'python','pandas'},{'philosophy','strategy'},{'scikit-learn'},{'pandas'}]})pandas_posts=df[df.tags.set.contains('pandas')]pandas_posts.tags.set.add('data')pandas_posts.tags.set.update({'data','analysis'})pandas_posts.tags.set.len()NotesThe implementation is primitive for now. It's based heavily on the pandas' coreStringMethodsimplementation.The public API has been tested for most expected scenarios.The API will need to be extended to handleNAvalues appropriately.
Package: pandas-shortcuts
pandas-shortcutsWhy even wait for autocompletion when you can usepandas_shortcuts?How to useSimply importpandas_shortcutstogether withpandas.importpandasaspdimportpandas_shortcutsEverypd.DataFrameandpd.Seriesobjects will have:shortcuts (full listbelow)# shortcut for `df.head()`df.h()# shortcut for df.columnsdf.c# shortcut for df["col"].unique()df["col"].u()new methods (full listbelow)# view up to `r` rows and `c` columns of a dataframe, overiding pandas' default limitdf.v()# default r=50, c=50# view up to `r` rows of a series, overiding pandas' default limitdf["col"].v(100)# stylize a dataframe's numeric columns as heatmap or bars# view up to `r` rows and `c` of a dataframe, overiding pandas' default limitdf.sh()# style=heatmapdf.sb()# style=bar# call `dtale.show`, refer to dtale docs for detailsdf.dt()# call `pandas_profiling.ProfileReport`, refer to pandas_profiling docs for detailsdf.pp()Available Shortcuts and Methods# Heads or tailsdf.h(...)# df.head(...)df.t(...)# df.tail(...)df["col"].h(...)# df["col"].head(...)df["col"].t(...)# df["col"].tail(...)# Sortdf.si(...)# df.sort_index(...)df["col"].si(...)# df["col"].sort_index(...)df.sv(...)# df.sort_values(...)df["col"].sv(...)# df["col"].sort_values(...)# Indexdf.sx(...)# df.set_index(...)df.rx(...)# df.reset_index(...)df["col"].rx(...)# df["col"].reset_index(...)# Groupbydf.gb(...)# df.groupby(...)df["col"].gb(...)# df["col"].groupby(...)# Duplicatesdf.dd(...)# df.drop_duplicates(...)df["col"].dd(...)# df["col"].drop_duplicates(...)df.dup(...)# df.duplicated(...)df["col"].dup(...)# df["col"].duplicated(...)df["col"].u(...)# df["col"].unique(...)# Propertiesdf.c# df.columnsdf.i# df.indexdf["col"].i# df["col"].index# Methodsdf.v(...)df["col"].v(...)df.sh(...)df.sb(...)df.dt(...)df.pp(...)NoteSome dependencies (pandas_profiling'snumbaandllvmlite) do not support Python 3.9 as ofpandas_shortcutsv0.0.1df.v()directly generatesIPython.core.display.HTMLobject under the hood, thus completely bypassing anypd.set_option("display.max_rows", ...)andpd.set_option("display.max_columns", ...)that the user may have specified.
Package: pandas-should
pandas-shouldpandas-should is pandas extension for asserting.InstallInstall and update released version using pip:$pipinstallpandas-shouldIf you want development version:$pipinstallgit+https://github.com/momijiame/pandas-should.gitQuickstartYou can just to importpandas_shouldto use:importpandas_shouldDataFrameLength (rows)Before:assertlen(df1)==len(df2)After:assertdf1.should.have_same_length(df2)Orassertdf1.should.have_length(len(df2))Before (multiple DataFrame):assertlen(df1)==(len(df2)+len(df3))After (multiple DataFrame)assertdf1.should.have_same_length(df2,df3)Width (columns)Before:assertlen(df1.columns)==len(df2.columns)After:assertdf1.should.have_same_width(df2)Orassertdf1.should.have_width(len(df2.columns))Before (multiple DataFrame):assertlen(df1.columns)==(len(df2.columns))+len(df3.columns)))After (multiple DataFrame)assertdf1.should.have_same_width(df2,df3)EqualityBefore:frompandas.util.testingimportassert_frame_equaldefequal(a,b):try:assert_frame_equal(a,b)exceptAssertionError:returnFalsereturnTrueassertequal(df1,df2)After:assertdf1.should.equal(df2)Null inclusionBefore:assertdf.isnull().any(axis=None)After:assertdf.should.have_null()Or expect not to be included:assertdf.should.have_not_null()assertnotdf.should.have_null()ShapeBefore:assertdf1.shape==df2.shapeAfter:assertdf1.should.be_shaped_like(df2)Orassertdf1.should.be_shaped_like(df2.shape)assertdf1.should.be_shaped_like(df2.shape[0],df2.shape[1])Value rangeBefore:assert(df>=range_min).any(axis=None)and(df<=range_max).any(axis=None)After:assertdf.should.fall_within_range(range_min,range_max)Greater than only:assertdf.should.greater_than(range_min)assertdf.should.greater_than_or_equal(range_min)Less than only:assertdf.should.less_than(range_min)assertdf.should.less_than_or_equal(range_min)SeriesLengthBefore:assertlen(s1)==len(s2)After:asserts1.should.have_same_length(s2)Or:asserts1.should.have_length(len(s2))Before (multiple Series):assertlen(s1)==(len(s2)+len(s3))After (multiple Series):asserts1.should.have_same_length(s2,s3)EqualityBefore:frompandas.util.testingimportassert_series_equaldefequal(a,b):try:assert_series_equal(a,b)exceptAssertionError:returnFalsereturnTrueassertequal(s1,s2)After:asserts1.should.equal(s2)Null inclusionBefore:asserts.isnull().any()After:asserts.should.have_null()Or expect not to be included:asserts.should.have_not_null()assertnots.should.have_null()Value rangeBefore:assert(s>=range_min).any()and(s<=range_max).any()After:asserts.should.fall_within_range(range_min,range_max)Greater than only:asserts.should.greater_than(range_min)asserts.should.greater_than_or_equal(range_min)Less than only:asserts.should.less_than(range_min)asserts.should.less_than_or_equal(range_min)Value varietyBefore:assertlen(s.unique())==expect_sizeAfter:asserts.should.have_number_of_unique_values(expect_size)
Package: pandas-show
pandas_showDisplay a long Pandas DataFrame or Series in a wide format.Installpip install pandas_showfrompandas_show.coreimportdisplay_in_colsHow to useGet some DataFrame that does not fit in the height of your screen, for example the list of Japan prefectures!pip install jp_preffromjp_pref.prefectureimportdfasdf_jp_prefdf_jp_pref<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>nameshort_namecode1北海道北海2青森県青森3岩手県岩手4宮城県宮城5秋田県秋田6山形県山形7福島県福島8茨城県茨城9栃木県栃木10群馬県群馬11埼玉県埼玉12千葉県千葉13東京都東京14神奈川県神奈川15新潟県新潟16富山県富山17石川県石川18福井県福井19山梨県山梨20長野県長野21岐阜県岐阜22静岡県静岡23愛知県愛知24三重県三重25滋賀県滋賀26京都府京都27大阪府大阪28兵庫県兵庫29奈良県奈良30和歌山県和歌山31鳥取県鳥取32島根県島根33岡山県岡山34広島県広島35山口県山口36徳島県徳島37香川県香川38愛媛県愛媛39高知県高知40福岡県福岡41佐賀県佐賀42長崎県長崎43熊本県熊本44大分県大分45宮崎県宮崎46鹿児島県鹿児島47沖縄県沖縄Let display the dataframe in a wide format.display_in_cols(df_jp_pref,3)nameshort_namecode1北海道北海2青森県青森3岩手県岩手4宮城県宮城5秋田県秋田6山形県山形7福島県福島8茨城県茨城9栃木県栃木10群馬県群馬11埼玉県埼玉12千葉県千葉13東京都東京14神奈川県神奈川15新潟県新潟16富山県富山nameshort_namecode17石川県石川18福井県福井19山梨県山梨20長野県長野21岐阜県岐阜22静岡県静岡23愛知県愛知24三重県三重25滋賀県滋賀26京都府京都27大阪府大阪28兵庫県兵庫29奈良県奈良30和歌山県和歌山31鳥取県鳥取32島根県島根nameshort_namecode33岡山県岡山34広島県広島35山口県山口36徳島県徳島37香川県香川38愛媛県愛媛39高知県高知40福岡県福岡41佐賀県佐賀42長崎県長崎43熊本県熊本44大分県大分45宮崎県宮崎46鹿児島県鹿児島47沖縄県沖縄Now let widely display the series of Japan prefecture names.display_in_cols(df_jp_pref.name,5)namecode1北海道2青森県3岩手県4宮城県5秋田県6山形県7福島県8茨城県9栃木県10群馬県namecode11埼玉県12千葉県13東京都14神奈川県15新潟県16富山県17石川県18福井県19山梨県20長野県namecode21岐阜県22静岡県23愛知県24三重県25滋賀県26京都府27大阪府28兵庫県29奈良県30和歌山県namecode31鳥取県32島根県33岡山県34広島県35山口県36徳島県37香川県38愛媛県39高知県40福岡県namecode41佐賀県42長崎県43熊本県44大分県45宮崎県46鹿児島県47沖縄県
Package: pandas-sigproc
Pandas Signal Processing (pandas-sigproc)Useful pandas extensions for detailed signal processing.This package contains extensions for thepandaslibrary. It is a Python package and can be installed via pip:pipinstallpandas-sigprocOnce installed, the package can be referenced as such:importpandas_sigproc.extensionThere are two extensions included in the package:freqdomain: analyze pandas series in the frequency domaintimedomain: analyze pandas series in the time domainThe code should be referenced for a more thorough understanding of what functions are available. However, given a pandas (time) series called my_series which has index time and values sound pressure, here is an example of how one would apply an a-weighted filter to that data:my_series_aweighted=my_series.timedomain.filt_a()
Package: pandas-sphinx
pandas-sphinxRender apandasDataFrame as an opinionated table forSphinx.Documentation:https://joaopalmeiro.github.io/pandas-sphinx/Source code:https://github.com/joaopalmeiro/pandas-sphinxPython package:https://pypi.org/project/pandas-sphinx/Referenceshttps://github.com/yehoshuadimarsky/bcpandas#quickstarthttps://github.com/yehoshuadimarsky/bcpandas/blob/master/bcpandas/main.py#L309https://github.com/Zsailer/pandas_flavorSphinx:https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#tableshttps://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#hyperlinkshttps://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-code-blockhttps://pygments.org/docs/lexers/#lexers-for-various-shellshttps://raw.githubusercontent.com/readthedocs/sphinx_rtd_theme/master/docs/index.rsthttps://github.com/astanin/python-tabulate#table-formathttps://en.wikipedia.org/wiki/Ellipsishttps://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_markdown.htmlhttps://github.com/pandas-dev/pandas/blob/v1.1.0/pandas/core/frame.py#L2240https://pandas.pydata.org/pandas-docs/version/1.3/reference/api/pandas.DataFrame.to_markdown.htmlhttps://github.com/koaning/scikit-bloom/blob/main/setup.pyhttps://github.com/scikit-hep/cookieDocumentation:https://numpydoc.readthedocs.io/en/stable/format.html+https://numpydoc.readthedocs.io/en/stable/example.htmlhttps://cjolowicz.github.io/posts/hypermodern-python-05-documentation/https://cjolowicz.github.io/posts/hypermodern-python-06-ci-cd/https://nashpy.readthedocs.io/en/stable/contributing/discussion/darglint/index.htmlNoteshttps://hatch.pypa.io/latest/install/:brew update+brew install hatch+hatch --versionTabulate:The minimum column size is 2 (+ 2 padding spaces):https://github.com/gregbanks/python-tabulate/blob/master/tabulate.py#L791https://github.com/gregbanks/python-tabulate/blob/master/tabulate.py#L132Hatch:hatch new --inithatch env create(defaultenvironment) +hatch shell+pip show pandas-sphinx+pip listhatch env removeorhatch env remove formathatch env create formathatch run format:i+hatch run format:bhatch env prunehatch env showhatch shell+python dev.py+exitorhatch run python dev.pyhatch version minororhatch version patchhatch versionhatch run lint:i+hatch run lint:ib+hatch run lint:dhatch -e lint run darglint --helprm -rf dist/+hatch buildhatch publish --helphatch env create docshatch run docs:checkhatch -e format run rstfmt --helphatch run docs:build+hatch run docs:openhatch run docs:cleanhatch run docs:clean && hatch run docs:build && hatch run docs:openhttps://pycqa.github.io/isort/docs/configuration/black_compatibility.htmlCheck if docstrings exist withinterrogate, check their format withdarglint
Package: pandas-sqlite3
Pandas Sqlite3This project is to simplify the joining of pandas dataframes using Sqlite3.
In the world of data science there are often two camps 1)Pandas and 2)SQL.
We want to bring these worlds together by making it easier for those more
familiar with SQL to manipulate Pandas dataframes within python. With Pandas Sqlite3
one can simply pass a list of Pandas dataframes, their names, and a Sqlite3 statement
to be executed. Enjoy :)AuthorsHamilton NoelInstallationInstall my-project with pippipinstallpandas_sqlite3Example Useimportpandasaspdfrompandas_sqlite3.pandas_queryimportpandas_query# create dataframessample_df=pd.read_csv('https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv')long_petal_df=sample_df.loc[sample_df['petal_length']>5].copy()# write sql querysql_query="""SELECTS.*FROM sample_df sJOIN long_petal_df USING ('sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species')"""# pass dataframes, their names, and SQL query to pandas_query functionfinal_df=pandas_query(dfs=[sample_df,long_petal_df],df_names=['sample_df','long_petal_df'],sql=sql_query)ContributingContributions are always welcome!LicenseMITAcknowledgementsAwesome Readme TemplatesPython Package to PyPI Using Poetry
Package: pandas-streaming
pandas-streamingaims at processing big files withpandas,
too big to hold in memory, too small to be parallelized with a significant gain.
The module replicates a subset ofpandasAPI
and implements other functionalities for machine learning.frompandas_streaming.dfimportStreamingDataFramesdf=StreamingDataFrame.read_csv("filename",sep="\t",encoding="utf-8")fordfinsdf:# process this chunk of data# df is a dataframeprint(df)The module can also stream an existing dataframe.importpandasdf=pandas.DataFrame([dict(cf=0,cint=0,cstr="0"),dict(cf=1,cint=1,cstr="1"),dict(cf=3,cint=3,cstr="3")])frompandas_streaming.dfimportStreamingDataFramesdf=StreamingDataFrame.read_df(df)fordfinsdf:# process this chunk of data# df is a dataframeprint(df)It contains other helpers to split datasets into
train and test with some weird constraints.