Saravanan666/pypi-org huggingface dataset:
Package: dfagent
dfagent 🤖The dfagent is a package for the handling of Dialogflow agents. You can for example retrieve training examples and save into a preferred format, or you can use it to update an intent by simply feeding it training examples you stored in a preferred format.Overviewdfagent/contains all the core code to extend dfagent.InstallTo install the dfagent package you only need to runinstall.shscript.UsageOnce dfagent is installed you can simply import it in your code.Save training phrases [remote]The following snippet illustrates a simple example to get and save training examples from an online Dialogflow agent.To create a Dialogflow agent you only need thatimportdfagentagent=dfagent.DialogFlowAgent(local_path_or_url='my_gcp_project_id',service_account='path/to/sa.json',content_type='json',output_format='default')Then you can get a list of dialogflow examples for saving as followsexamples=agent.get_training_examples()agent.save_training_examples(examples,output_dir='path/to/dir')Update intent with new training phrases [remote]In the following is a snippet that illustrates an example to update a remote Dialogflow agent using training phrases you stored as a raw text file. Remember that dfagent can be extended to support any input or output file format.Once you instante a df agentimportdfagentagent=dfagent.DialogFlowAgent(local_path_or_url='my_gcp_project_id',service_account='path/to/sa.json',input_format='default',)You can update your remote Dialoflow agent in that wayresponse,raw_examples,df_examples=agent.add_training_examples(intent_name='help.cooking',input_dir_or_file='path/to/phrases.train',lang='en')From local or zipIn case you already have exported your Dialogflow on your local computer, you can give as local_path_or_url the path to the zip or unzipped exported agent.importdfagentagent=dfagent.DialogFlowAgent(local_path_or_url='path/to/myagent.zip',...)
Package: dfainductor
DFA-Inductor-pyA python tool for solving minDFA problem.Requirementspython 3.7+PySATclickInstallationYou can install it via pip orpipx.pip(x?)installdfainductorUsageFor a list of options checkdfainductor --help.TODO: add detailsSAT solversAll the work with SAT solvers is done byPySAT toolkit.One can check the list of available optionshere.
Package: dfa-mutate
DFA MutatePrimitives and utilities for manipulating deterministic finite automata (DFA) represented using thedfalibrary.Table of ContentsInstallationUsageInstallationIf you just need to usedfa_mutate, you can just run:$ pip install dfa_mutateFor developers, note that this project uses thepoetrypython package/dependency
management tool. Please familarize yourself with it and then
run:$ poetry installUsageExample Usage:fromdfaimportDFAimportdfa_mutatedfa1=DFA(start=0,inputs={0,1},label=lambdas:(s%4)==3,transition=lambdas,c:(s+c)%4,)# Pick a specific mutation.dfa2=dfa_mutate.add_state(dfa1)dfa3=dfa_mutate.change_start(dfa1)dfa4=dfa_mutate.change_transition(dfa1)dfa5=dfa_mutate.relabel_state(dfa1)# Infinite Generator mutations round-robin (will repeat).dfas=dfa_mutate.generate_mutations(orig)# Sample DFA using softmax over a scoring function (default constant).# Uses first n dfas generated by above generator.# NOTE: requires the optional numpy dependency.dfa6=dfa_mutate.sample_mutation(dfa1,n=20,score=lambdad:len(d.states()))# All functions support passing in random number generator.importrandomdfa7=dfa_mutate.relabel_state(dfa1,random.Random(0))
Package: dfanalyzer
DFAnalyzerDFAnalyzer Python is a Python package for data analysis, built on top of the popular DFAnalyzer for Excel. It provides a powerful set of tools for importing, exploring, cleaning, transforming, and visualizing data. It also offers features such as filtering, sorting, grouping, and performing calculations on data. DFAnalyzer Python is designed to enable users to quickly and easily analyze large amounts of data andextract meaningful insights.Find details & insight about each columns.Easy to perform cycles over pyspark.Percentage stats around NaN , Blank Values, Null Values.Describes datatypes of Pyspark Dataframe.Help in POC of data.Who Should use DFAnalyserDevelopers working with bigdataDevelopers using pyspark in the Data exploration.Developers who needs to do poc over raw data.UsagePySparkYou can install the DFAnalyzer package using the pip command. To install DFAnalyzer, open a terminal window and type: pip install dfanalyzer. Once the installation is complete, you can start using DFAnalyzer with Python.Install the preset:pipinstalldfanalyzerImport it:import DFAnalyzer as dfaUse it on existing pyspark dataframe:#[isHavingNullData,%NullData,isHavingNanValues,%NanValues,isHavingBlankValues,%BlankValues,DataType]options=[1,1,1,1,1,1,1]#flags of what all kind of analysis you needdfa.analyze(df,options)More is about to come. Stay tuned.
Package: df-and-order
🗄️ df-and-orderYeah, it's just like Law & Order, but Dataframe & Order!pip install df_and_orderUsingdf-and-orderyour interactions with dataframes become very clean and predictable.Tired of absolute file paths to data in shared notebooks in your repository?Can't remember how your datasets were generated?Want to have safe and reproducible data transformations?Like declarative config-based solutions?Good news for you!How it looks in code?Imagine the world where all you need to do for reading some dataframe you need just a few lines:reader=MagicDfReader()df=reader.read(df_id='user_activity_may_2020')Maybe you are interested in some transformed version of that dataframe? No problem!reader=MagicDfReader()# ready to fit a model on!model_input_df=reader.read(df_id='user_activity_may_2020',transform_id='model_input')Wow. Is it really magic?df-and-orderworks with yaml configs. Every config contains metadata about a dataset as well as all desired transfomations.
Here's an example:df_id:user_activity_may_2020# here's the dataframe identifierinitial_df_format:csvmetadata:# this section contains some useful information about the datasetauthor:Data Mandata_collection_date:2020-05-01transforms:model_input:# here's the transform identifierdf_format:csvin_memory:# means we want to perform transformations in memory every time we calling it, permanent transforms are supported as well-module_path:df_and_order.steps.pd.DropColsTransformStep# file where to find class describing some transformation. this one drops columnsparams:# init params for the transformation classcols:-redundant_col-module_path:df_and_order.steps.DatesTransformStep# another transformation that converts str to datetimeparams:cols:-date_colOkay, what exactly is adf-and-order's transform?Every transformation is about changing an initial dataset in any way.A transformation is made of one or many steps. Each step represents some operation.
Here are examples of such operations:dropping colsadding colstransforming existing colsetcdf-and-orderuses subclasses ofDfTransformStepConfigto describe a step. It's possible and highly recommended to declare init parameters for any step in config.
Using Single Responsibility principle we achieve a granular control over our entire transformation.Just by looking at the config you can say how the transformed dataframe was created.Take a look at the more detailed overview to find more exciting stuff.I also wrote an article to describe the benefits, check it out! There are lemurs and stuff.Hope the lib will help somebody to boost the productivity.
Package: dfasat
# dfasat python package: flexible state-merging for pythonThis package brings a scikit-learn like interface for the dfasat tool. dfasat is a C++ program implementing a red-blue state-merging approach with flexibly exchangable merge heuristics.## usageCreate a dfasat.DFASATEstimator object, call fit() and predict() on it. plot() requires graphviz and will output a rendering of the automaton.
A short introduction using ipython notebooks is available on automatonlearning.net.## buildingdfasat is a container package that currently does not include the binary, but the toolchain to build it. The setup.py file automatically clones and and builds the latest version from the current dfasat repository.
For this to work, it requires all the dependencies of dfasat (libgsl, libpopt) as well as lib-boost-python and a working castxml installation.## copyright and thanksdfasat was originally written by Marijn Heule and Sicco Verwer. The Python wrapper was written by Benjamin Loos under supervision of Chris Hammerschmidt.
Package: dfauditor
still in an early development stage and undergoing significant changes regularlydataframe-auditorA dataframe auditor that computes a number characteristics of the data.SummaryInstallationTestingUsageContributionsSummaryData profilingis important in data analysis and analytics, as well as in determining characteristics of data pipelines.
This repository aims to provide a means to extract a selection of attributes from data.It is currently focused on processingpandasdataframes, but this functionality is being
extended tosparkdataframes too.Given a pandas dataframe, the extracted values are (whereobjectandcategorytypes are mapped tostring, and all numerical types tonumeric):TypeMeasureString & NumericPercentage nullStringDistinct countsMost frequent categoriesNumericMeanStandard deviationVarianceMin valueMax valueRangeKurtosisSkewnessKullback-Liebler divergenceMean absolute deviationMedianInterquartile rangePercentage zero valuesPercentage nan valuesNaturally, many of these characteristics are not independent of one another, but some may be excluded as suits the application.The result of auditing a dataframe using this library is that a dictionary of these measures is returned for each column in the dataframe.
For example, if a dataframe consists of a single column, namedtrivial, where all values are1, then[{"attr":"trivial","type":"NUMERIC","median":1.0,"variance":0.0,"std":0.0,"max":1,"min":1,"mad":0.0,"p_zeros":0.0,"kurtosis":0,"skewness":0,"iqr":0.0,"range":0,"p_nan":0.0,"mean":1.0}]For a dataframe with columns["trivial", "non-trivial"], a list of dictionaries is returned:[{"attr":"trivial"},{"attr":"non-trivial"}]InstallationDependencies are contained inrequirements.txt:pipinstall-rrequirements.txtAlternatively, if you wish to install directly from github, you may use:pipinstallgit+https://github.com/jackdotwa/dataframe-auditor.gitTestingUnittests may be run via:python -m unittest discover testsCode coverage may be determined via:coveragerun-munittestdiscovertests&&coveragereportUsageMany examples of using this package is:importpandasaspdimportdfauditornumeric_data={'x':[50,50,-10,0,0,5,15,-3,None,0],'y':[0.00001,256.128,None,16.32,2048,-3.1415926535,111,2.4,4.8,0.0],'trivial':[1]*10}numeric_df=pd.DataFrame(numeric_data)result_dict=dfauditor.audit_dataframe(numeric_df,nr_processes=3)ContributionsPull requests are always welcome.
Package: dfault
A simple lightweight class that allows the user to fallback on default
valuesInstallationpip3install--userdfaultUsageBasic Examplefromdfault.objectsimportDfaultexpected_values={"search_engine":"duckduckgo","website":"http://www.duckduckgo.com"}d=Dfault(expected_values)d.get("search_engine","google")# returns "duckduckgo" as the value existsd.get("favourite_team","leafs")# returns "leafs" as there are no values for "favourite_team"With Configimportconfigparserfromdfault.objectsimportDfaultconfig=configparser.ConfigParser()# .config:# [USER]# team=leafs#config.read(".config")default_config=config["user"]d=Dfault(default_config)d.get("team","raptors")# returns "leafs" as the value exiss in the configd.get("username","joeyism")# returns "joeyism" as there is no "username" value in the config fileVersions1.0.xbug fixes1.0.0First Publish
Package: df-autosk
Deep Forest for Auto-SklearnExtension of AutoSklearnClassifier with DF21 -CascadeForestClassifier- a Deep Forest implementantion. Based on anexample extensionfrom Auto-Sklearn documentation.RequirementsLinux operating system (requirement of Auto-Sklearn)numpy version <= 1.19installation of CascadeForestClassifierinstallation ofAuto-Sklearninput variables have to be converted to numeric without missing valuesoutput variables also should be converted to numeric values (you can use sklearn LabelEncoder)Example use# import libraries
import sklearn.metrics
import autosklearn.classification
import autosklearn.pipeline.components.classification


# import DFClassifier
from df_autosk.df_autosk import DFClassifier

# add DFClassifier to autosklearn classifier
autosklearn.pipeline.components.classification.add_classifier(DFClassifier)

# initialize and get hyperparameter search space
cs = DFClassifier.get_hyperparameter_search_space()
print(cs)

# load the dataset
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# use the classifier
clf = autosklearn.classification.AutoSklearnClassifier(
        	time_left_for_this_task=5400,
        	include={"classifier": ['DFClassifier']},
        	initial_configurations_via_metalearning=0,
        	memory_limit = 102400,
                # Not recommended for a real implementation
               	smac_scenario_args={"runcount_limit": 2}
                )

clf.fit(X_train, y_train)

#get result
y_pred = clf.predict(X_test)
print("accuracy: ", sklearn.metrics.accuracy_score(y_pred, y_test))Example based on anexample extensionfrom Auto-Sklearn documentation.
Package: dfba
This project provides an object-oriented software package for dynamic
flux-balance analysis (DFBA) simulations using implementations of the direct
method or Algorithm 1 described in the paperHarwood et al., 2016. The main
algorithms for solving embedded LP problems are written inC++and use the GNU
Linear Programming Kit (GLPK) and the Suite of Nonlinear and
Differential/Algebraic Equation Solvers (SUNDIALS) CVODE or IDA. Extension
modules tocobrapyare provided for easy generation and simulation of DFBA
models.InstallationCurrently, we do not provide Python wheels for this package and thereforeInstalling from
sourceis a bit more involved. The quickest way to run the software
is from the providedDockerimage:docker run --rm -it davidtourigny/dfba:latestInstalling from sourceCurrently this package is compatible with most UNIX-like operating systems.
Provided the followingDependenciesare installed, the module
can be installed from the root of the repository using the command:pip install .DependenciesA version ofPython3.6 or higher is requiredYou needcmakefor the build processYou will needgitto clone this repository to access
the scripts and build filesYou need a working compiler with C++11 support, for example, by installingbuild-essentialon Debian-derived Linux systemsGLPKversion 4.65 is required or can be installed usingbuild_glpk.shSUNDIALSversion 5.0.0 or higher is required or can be installed usingbuild_sundials.shpybind11is required or can be installed usingbuild_pybind11.shBe aware that some of these packages have their own dependencies that must
therefore be installed also (e.g.GLPKdepends onGMPandpybind11requirespytest).Alternatively, aDockerfileis provided for building aDockerimage to run the software from an interactive container. TheDockerimage can be
built in one step by issuing the command:make buildfrom the root of this repository. It can then be started using:make runDocumentationDocumentation for dfba is provided atreadthedocsAuthorsDavid S. TourignyMoritz E. BeberAdditional contributorsJorge Carrasco Muriel (visualization and documentation)FundingDavid S. Tourigny is a Simons Foundation Fellow of the Life Sciences Research
Foundation.CopyrightCopyright © 2018,2019 Columbia University Irving Medical Center, New York, USACopyright © 2019 Novo Nordisk Foundation Center for Biosustainability,
Technical University of DenmarkFree software distributed under theGNU General Public License v3 or later
(GPLv3+).
Package: dFBApy
Reaction kinetics data can be parameterized and simulated in a genome-scale metabolic model (GEM) through dynamic Flux Balance Analysis (FBA). ThedFBApymodule supports this method of computational biology through concise and efficient functionality aroundCOBRApy, which is detailed in the following sections. Examples Notebook are available in the examples directory of thedFBApy GitHub repository. Please submit errors, inquiries, or suggestions asGitHub issueswhere they can be addressed.InstallationdFBApy is installed in a command prompt, Powershell, Terminal, or Anaconda Command Prompt viapip:pip install dfbapyThe full documentation is available onReadTheDocs.
Package: dfbgn
DFBGN is a Python package for  nonlinear least-squares minimization, where derivatives are not available.
It is particularly useful when evaluations of the objective are expensive and/or noisy, and the number of variables to be optimized is large.DFBGN is based onDFO-LS, but is better-suited when there are many variables to be optimized (so the linear algebra in DFO-LS is too slow).
Unlike DFO-LS, DFBGN does not currently support bound constraints on the variables.If you are interested in solving general optimization problems (without a least-squares structure), you may wish to tryPy-BOBYQA.RequirementsDFBGN requires the following software to be installed:Python 2.7 or Python 3 (http://www.python.org/)Additionally, the following python packages should be installed (these will be installed automatically if usingpip, seeInstallation using pip):NumPy 1.11 or higher (http://www.numpy.org/)SciPy 0.18 or higher (http://www.scipy.org/)Pandas 0.17 or higher (http://pandas.pydata.org/)Installation using pipFor easy installation, usepipas root:$[sudo]pipinstalldfbgnor alternativelyeasy_install:$[sudo]easy_installdfbgnIf you do not have root privileges or you want to install DFBGN for your private use, you can use:$pipinstall--userdfbgnwhich will install DFBGN in your home directory.Note that if an older install of DFBGN is present on your system you can use:$[sudo]pipinstall--upgradedfbgnto upgrade DFBGN to the latest version.Manual installationAlternatively, you can download the source code fromGithuband unpack as follows:$gitclonehttps://github.com/numericalalgorithmsgroup/dfbgn$cddfbgnDFBGN is written in pure Python and requires no compilation. It can be installed using:$[sudo]pipinstall.If you do not have root privileges or you want to install DFBGN for your private use, you can use:$pipinstall--user.instead.To upgrade DFBGN to the latest version, navigate to the top-level directory (i.e. the one containingsetup.py) and rerun the installation usingpip, as above:$gitpull$[sudo]pipinstall.# with admin privilegesTestingIf you installed DFBGN manually, you can test your installation by running:$pythonsetup.pytestAlternatively, the HTML documentation provides some simple examples of how to run DFBGN.ExamplesExamples of how to run DFBGN are given in thedocumentation, and theexamplesdirectory in Github.UninstallationIf DFBGN was installed usingpipyou can uninstall as follows:$[sudo]pipuninstalldfbgnIf DFBGN was installed manually you have to remove the installed files by hand (located in your python site-packages directory).BugsPlease report any bugs using GitHub’s issue tracker.LicenseThis algorithm is released under the GNU GPL license. Pleasecontact NAGfor alternative licensing.
Package: dfbridge
dfbridgeA Schematized dataframe formatter.We often have need to reformat a base dataframe to create a dataframe following a schema, applying a combination of renaming some columns, applying functions to others, and doing groupby/transform operations.
These steps introduce a lot of boilerplate, but here we can assign it as a dictionary schema.
The original dataframe is unchanged, and all of the operations take place only on the original dataframe.Let's say we want the output dataframe to have columnsfinal_name1,final_name2, andfinal_name3, with one of them a simple rename from an input dataframe, one the result of some fucntion applied to the input dataframe, and one some groupby transform operation.
We can even remap values to other values in the process.
Settingfill_missingto True lets one add the column and set it as full of pandas NA values.The schema to do this looks like:schema={"final_name1`":{"type":"rename","from":"original_name","fill_missing":True,"column_type":None,'remap_dict':{'orig_val':'new_val'},# Remaps elements with original val to new val. Set to None or ignore to not use.'strict_remap':True,# If True, values not in the remap_dict are made pd.NA, else are passed through intact.},"final_name2":{"type":"apply","func":function,# Expects the whole row of the original dataframe, so use row['col] style arguments."fill_missing":True,"column_type":None,'remap_dict':None,# Remaps elements with original val to new val. Set to None or ignore to not use.},"final_name3":{"type":"transform","groupby":"groupby_column","column":"return_column","action`":"mean",# (or anything that works in df.groupby().transform())"fill_missing":True,"column_type":None,},}
Package: dfbrowse
dfbrowsePython library that provides spreadsheet-style browsing of a Pandas
dataframe within a terminal/curses environment.It was borne out of my love for curses-style/modal interfaces, and a
frustration with how difficult it is to really navigate/browse/inspect
a large dataframe while you're operating on it within IPython.Installationpip install dfbrowseGetting starteddfbrowse is generally intended to be usedinsideIPython. Start by
launching IPython, then import the IPython magics:import dfbrowse.ipython_mainIf you want to jump straight into browsing, load a parquet or CSV directly:%csv ~/work/project1/employees.csv emp
%pq ~/work/project2/my_data.parquet dataIf you've already loaded a dataframe, browse it like so:import pandas as pd

emp = pd.read_parquet('~/work/project2/my_data.parquet')

%show empInside the browserOnce inside the browser, you can navigate with arrow keys and the mouse.Press?to find out about some basic browsing commands. You can
resize columns, hide them, reorder them, and even search the row
contents, with a few keystrokes.There are many other commands that you can discover inkeybindings.py.There are even more commands that can be discovered by typing/and
then pressing TAB. These are pre-registered functions, such asstr_matchandquery, that perform some operation on a
dataframe. The returned dataframe will be rendered in the browser.In theory it is possible to write and register your own dataframe
functions as well, but this is a topic for a much larger document and
I'm not sure how well it's working anyway. If you want to do complex
transforms on your dataframe, you're probably better off to drop back
into the IPython 'shell', do your work, and then jump back into the
browser with one of the options above.quitting and reenteringYou can useqon your keyboard to go back to the shell/ipython session.Re-enter the browser via%show <df_name>, or%fgto foreground the
most recent dataframe.bugsThere are lots. I even know about some of them. Please feel free to
report any you find on GitHub.
Package: df-compare
# Informative Comparison of DataFrame DifferencesThis is a tool that is used to compare and test Pandas DataFrames. It has three parts.### Core function
At it’s core is a function that compares a number of aspects of two dataframes.Number of rowsColumn namesdtypesIndexIntegersFloatsDatetimesObjects (typically strings)NaNsBooleansFor each dimension, if differences are found, a key is included in the diffs dictionary that is returned.
The value of that key (e.g. ‘nrows’) would be a string description (e.g. ‘observed contains 5 rows. expected 10.)### PyTest IntegrationIf one is testing functionality that produces a dataframe, and one can create one to compare it to…pytest_kit.pyautomatically injects N separate pytests into a test module (test_<name>.py)
simply by doing the following 2 things.Create two pytest fixtures:df_observed, anddf_expectedAdding the import statementfrom df_compare.pytest_kit import *That’s it, when you run$ pytest test_<name>.py, it will discover and run N separate tests!### Shell scriptThis comparison can be used to compare two files, or even a directory of files containing tables.###### Details to be added[![Build Status](https://travis-ci.org/caseyclements/pennies.svg?branch=master)](https://travis-ci.org/caseyclements/pennies)