Saravanan666/pypi-org huggingface dataset:
Package: dframe-utils
# dframeutils![dframeutils](assets/dframeutils.png)<p align=”center”><img src=”https://img.shields.io/badge/stars-1-blue.svg?style=flat”/>
<img src=”https://img.shields.io/badge/forks-0-blue.svg?style=flat”/>
<img src=”https://img.shields.io/badge/contributions-welcome-blue.svg?style=flat”>
<img src=”https://img.shields.io/github/license/mashape/apistatus.svg”>
<img src=”https://img.shields.io/travis/bharathgs/dframeutils.svg”>
<a href=”https://codecov.io/gh/bharathgs/dframeutils”><img src=”https://codecov.io/gh/bharathgs/dframeutils/branch/master/graph/badge.svg” /></a></p>simple utility tools for dataframes in PythonTable of Contents[Installation](#installation)[License](#license)## Installationdframeutils is distributed on [PyPI](https://pypi.org) as a universal
wheel and is available on Linux/macOS and Windows and supports
Python 3.6+.`bash $ pip install dframe_utils `## Licensedframeutils is distributed under the terms of the
[MIT License](https://choosealicense.com/licenses/mit).
Package: dfreduce
dfreduceDescription: Automatically reduce the memory size of your pandas dataframeInstallationpip install dfreduceUsagefrom dfreduce.core import DFReduce
import pandas as pd

df = pd.read_csv('https://ed-public-download.app.cloud.gov/downloads/Most-Recent-Cohorts-Scorecard-Elements.csv')
df_reduced = DFReduce(df).reduce()

# Alternatively, you can also use the inplace parameter to overwrite input dataframe
DFReduce(df, inplace=True).reduce()Uninstallationpip uninstall dfreduce
Package: dfrnt
dfrnt is a library for comparing PNG screenshots during automated testing and producing visual diffs.dfrnt will compare images with identical names from automation and produce a result “diff” image that red-highlights regions
in the image that differ.dfrnt requires a minimum of 3 folders to be specified:run_dir is the directory of screenshots from your run, aka ‘actual screenshots’
gold_dir directory of screenshots that are compared against, aka ‘expected screenshots’
diff_dir is the directory to output the visual diff images intoInvoking:You have a directory of images from your latest test run in “test/run” (test.png)You have a directory of the expected images in “test/gold” (test.png)You want the output of visual diffs to be placed in directory “test/diff”from dfrnt import dfrntvisual_diffs = dfrnt(run_dir=”test/run”, gold_dir=”test/gold”, diff_dir=”test/diff”)visual_diffs.diff()Output:
Diff failed! test.png is not identical enoughA red-lined image showing the differences between run/test.png and gold/test.png is generated in diff/test.pngOptional Features:
mask_dir is the directory to contain images with highlighted areas you wish to ignore in the diff.
This is useful for ignoring portions of an image that always change, like date / time.
These mask images should be transparent background PNGsfuzzy is a “fuzziness” factor of how different images can be and still pass.
If you experience jitter you should experimentally determine the lowest acceptable number you can use.
Default is None, starting your binary search around ‘40’ is recommendedvisual_diffs = visual_diffs = dfrnt(run_dir=”run”, gold_dir=”gold”, diff_dir=”diff”, mask_dir=”mask”, fuzzy=40)Why is this project called ‘dfrnt’?
To mock the modern practice of dropping vowels in software names to sound unique. Be glad I didn’t name it dfrntzlyr.
Package: dfrun
DFRUNgadget for dl experimentsWhat it does:create a folder to copy all the code in the current directory to freeze one training processswitch to the working path to the folder created, and run the commandif gpu resources is needed, nvidia-ml-py package is needed.
In python 2.x,pip install nvidia-ml-py, while in
python 3.x,pip install nvidia-ml-py3Dependencies:​	Code is tested on python 3.4 and it's supposed to run in python 3 environment.Install:pipinstalldfrunor clone this repository and run:python setup.py installUsage:dfrun-d[experimentfolder]-n[experimentname]-g[NUMxMEM][command]What does the project folder look like:Proj|__ [CODE FILES]|__ .dfignore (contains the files/folders ignored in the copy process)|__ Experiment Folder (--exp-dir -d)​                         |__ Experiment Name (--exp-name -n)​                         |                     |__ [CODE FILES COPIED]​                         | Other Exp Names....Contact:mayeoliver@163.com
Package: df-runner
Dialog Flow RunnerDialog Flow Runneris python module add-on forDialog Flow Framework, a free and open-source software stack for creating chatbots, released under the terms of Apache License 2.0.Dialog Flow Runnerallows you to ...Quick StartInstallationpipinstalldf_runnerBasic exampleTo get more advanced examples, take a look atexampleson GitHub.Contributing to the Dialog Flow RunnerPlease refer toCONTRIBUTING.md.
Package: dfsaas-blog
Blog is a Django app to run blog.Quick startAdd “dfsaas_blog.apps.DFSAASBlogAppConfig” to your INSTALLED_APPS setting like this, along withckeditorandtaggit:INSTALLED_APPS = [
    ...

    'ckeditor',
    'ckeditor_uploader',
    'taggit',
    'dfsaas_blog.apps.DFSAASBlogAppConfig',
]Include the polls URLconf in your project urls.py like this:path('blog/', include('dfsaas_blog.urls')),Runpython manage.py migrateto create the polls models.Start the development server and visithttp://127.0.0.1:8000/admin/to create a poll (you’ll need the Admin app enabled).Visithttp://127.0.0.1:8000/blog/to participate in the poll.
Package: dfschema
DFS (aka Dataframe_Schema)DFSis a lightweight validator forpandas.DataFrame. You can think of it as ajsonschemafor dataframe.Key features:Lightweight: only dependent onpandasandpydantic(which depends only ontyping_extensions)Explicit: inspired byJsonSchema, all schemas are stored as json (or yaml) files and can be generated or changed on the fly.Simple: Easy to use, no need to change your workflow and dive into the implementation details.Comprehensive: Summarizes all errors in a single summary exception, checks for distributions, works on subsets of the dataframeRapid: base schemas can be generated from given dataframe or sql query (usingpd.read_sql).Handy: Supports command line interface (with[cli]extra).Extendable: Core idea is to validatedataframesof any type. While now supports only pandas, we'll add abstractions to run same checks on different types of dataframes (CuDF, Dask, SparkDF, etc )QuickStart1. Validate DataFrameVia wrapperimportpandasaspdimportdfschemaasdfsdf=pd.DataFrame({"a":[1,2,3,4,5,6,7,8,9,10],"b":[1,2,3,4,5,6,7,8,9,10]})schema_pass={"shape":{"min_rows":10}}schema_raise={"shape":{"min_rows":20}}dfs.validate(df,schema_pass)# won't raise any issuesdfs.validate(df,schema_raise)# will Raise DataFrameSchemaErrorAlternatively (v2 optional), you can use the root class,DfSchema:dfs.DfSchema.from_dict(schema_pass).validate(df)# won't raise any issuesdfs.DfSchema.from_dict(schema_raise).validate(df)# will Raise DataFrameSchemaError2. Generate Schemadfs.DfSchema.from_df(df)3. Read and Write Schemasschema=dfs.DfSchema.from_file('schema.json')schema.to_file("schema.yml")4. Using CLINote: requires [cli] extra as relies onTyperandclickValidate via CLIdfschemavalidate--read_kwargs_json'{delimiter="|"}'FILEPATHSCHEMA_FILEPATHSupportscsvxlsxparquetfeatherGenerate via CLIdfsgenerate--format'yaml'DATA_PATH>schema.yamlInstallationWIPAlternativesTableSchemeGreatExpectations. Large and complex package with Html reports, Airflow Operator, connectors, etc. an work on out-of-memory data, SQL databases, parquet, etcPandera- awesome package, great and suitable for type hinting, compatible withhypothesisgreat talkTensorflow validateDTF expectationsChanges[[changelog]]RoadmapAdd tutorial NotebookSupport tableschemaSupport Modin modelsSupport SQLAlchemy ORM modelsBuilt-in Airflow Operator?Interactive CLI/jupyter for schema generation
Package: df-script-parser
Dialog Flow Script ParserDialog Flow Script Parseris python module add-on forDialog Flow Framework, a free and open-source software stack for creating chatbots, released under the terms of Apache License 2.0.Dialog Flow Script Parserallows you to parse python files in order to extract inputs and dictionaries.Quick StartInstallationpipinstalldf_script_parserpy2yamldf_script_parser.py2yaml--helpusage: df_script_parser.py2yaml [-h] [--requirements REQUIREMENTS] ROOT_FILE PROJECT_ROOT_DIR OUTPUT_FILE

Compress a dff project into a yaml file by parsing files inside PROJECT_ROOT_DIR starting with ROOT_FILE.
Extract imports, assignments of dictionaries and function calls from each file.
Recursively parse imported local modules.
Collect non-local modules as project requirements

positional arguments:
ROOT_FILE             Python file to start parsing with
PROJECT_ROOT_DIR      Directory that contains all the local files required to run ROOT_FILE
OUTPUT_FILE           Yaml file to store parser output in

optional arguments:
-h, --help            show this help message and exit
--requirements REQUIREMENTS
File with project requirements to override those collected by parserNOTE:Usepy2yamlparser in the same python environment that is used to launch the script otherwise site packages will not be found.NOTE:Any assignments of function calls in which the function being called isdf_engine.core.Actorwill be checked for correctness of the arguments passed to the function.File formatsTheOUTPUT_FILEwill contain:requirementskey which points to a list of project requirements collected by parser.
If optional argumentREQUIREMENTSwas specified replace the list with a list of requirements from that filenamespaceskey which points to a dictionary with all the parsed files. Keys are the names of those
modules that could be used to import it fromPROJECT_ROOT_DIRor its parent directory ifPROJECT_ROOT_DIRcontains a__init__.pyfile. The values are dictionaries in which keys are the names of the objects inside the module
while values are their definitions.yaml2pydf_script_parser.yaml2py--helpusage: df_script_parser.yaml2py [-h] YAML_FILE EXTRACT_TO_DIRECTORY

Extract project from a yaml file to a directory

positional arguments:
YAML_FILE             Yaml file to load
EXTRACT_TO_DIRECTORY  Path to the directory to extract project to

optional arguments:
-h, --help            show this help message and exitExamplesTo get more advanced examples, take a look atexamples.Contributing to the Dialog Flow Script ParserPlease refer toCONTRIBUTING.md.
Package: dfsjson
DFS JSONA Depth First Search powered JSON decoding library. For the remaining operations (dump, dumps), it uses standard python JSON library.Installationpip install dfsjsonUsage# Create a complex JSON object
example_json = """{
    "key1": "value1'
    "key2": [1 2, 3]
    "key3":  {
        "subkey1": "subvalue1"
        "subkey2": "subvalue2"
    ,
}"""

dj = DFSJson(max_depth = 100, max_diff = 10)
dj.loads(example_json)DiscussionHey I know this. Why not have you used Breadth First Search?Technically you are correct but keep in mind that Breadth First Search keeps the previous states in its list which
scales with number of iterations you are making and it can be a huge strain on the memory.However, Depth First search requires only O(log(n)) memory complexity in which we can search a lot more possibilities.ContributionPlease don't hesitate to create issues and pull requests since this is developed overnight.
Package: df-slots
Dialog Flow SlotsDialog Flow Slotsis python module add-on forDialog Flow Framework, a free and open-source software stack for creating chatbots, released under the terms of Apache License 2.0.Dialog Flow Slotsallows you to ...Quick StartInstallationpipinstallDialogFlowSlotsBasic exampleTo get more advanced examples, take a look atexampleson GitHub.Contributing to the Dialog Flow SlotsPlease refer toCONTRIBUTING.md.
Package: dfsmpy
Deterministic, Finite State MachineContentsInstallationState MachineblueprintstatecontextinitialacceptedfinalUsageInstallationpython setyp.py installStateMachineMembersblueprintThis is a propery with a getter and setter that defines the state machine.
Setting the blueprint will also reset the state machine.{"initialState":state,"initialContext":dict(),"alphabet":set(),"validStates":set(),"acceptedStates":set(),"finalStates":set(),"transition":lambdastate,context,event:new_state,"lifecycles":dict()}Initial ContextAn optional dictionary, which can be used to share information between states
and updated during state transitions.AlphabetA set of events, which are used to drive state transitions.Initial StateThe starting state. Must be a valid state.Valid StatesA set of valid states.Accepted StatesA set of accepted states.Final StatesA set of final states, once reached, new transitions will raiseStopMachine.TransitionA function, which takes state, context and event as parameters
and returns the next state.  The event must be a member of the alphabet
and the new state must be a valid state.LifecyclesLifecycle actions can be ran before or after specific events.EventsA set of events, which are used to drive lifecycle actions.ActionsA list of functions, which take state, context and event as parameters.{"lifecycles":{"before":[{"events":{0,1,2,3},"actions":[before_any]}],"after":[{events:{3},"actions":[after_three]},{"events":{2,3},"actions":[action1,action2]}]}}before_any executes before events 0, 1, 2 & 3 transitions are executedafter_three executres after event 3 transition is executedaction1, then action2 executes after events 2 & 3 transitions are executedstateThe current state of the state machine.contextThe current context of the state machine.initialTrue or False if the current state is the initial state.acceptedTrue or False if the current state is an accepted state.finalTrue or False if the current state is a final state.Methodsreset()Resets the state machine's state and context to their initial values defined
in the blueprint. The initial state must be a valid state.ValueError- Invalid initial stateset_state(state, context)Set the state machine's state and context. The state must be a valid state.ValueError- Invalid statetransition(event)Transitions the state machine to the next state by executing the transition
defined in the blueprint. The event must be a valid member of the alphabet
defined in the blueprint. The state must be a valid state.StopMachine- Current state is finalValueError- Invalid eventis_initial(state) -> True | Falseis_valid(state) -> True | Falseis_accepted(state) -> True | Falseis_final(state) -> True | Falseis_event(event) -> True | FalseUsageCreate a state machine with a blueprint and transition
from the initial state1to accepted, final state2.fromdfsmpyimportStateMachinemachine=StateMachine({"alphabet":{1,2},"initialState":1,"validStates":{1,2},"acceptedStates":{2},"finalStates":{2},"transition":lambdaa,c,e:e})machine.transition(2)
Package: dfspy
PyDFSDistributed File System written in Python.Installationpip install dfspyUsagePyDFS is Centralized Distributed File System, which means there is amasterandslavenodes.
The current implementation assumes that the system has only one master node and many slave nodes.PyDFS supportscommand line interface(CLI) to interact with it.
There are 2 groups of the commands (like Docker Management Commands):pydfs init- to initialize, manage and sync nodespydfs dfs- to interact with DFS itself (put/get data to/from it)Let's take a closer look at these commands.init commandsWithpydfs initcommand you can initialize master and slave nodes - it's pretty simple:pydfs init masterpydfs init slave --master_ip [IP]dfs commandspydfs dfs put --path [PATH] --master_ip [IP]pydfs dfs get --path [PATH] --master_ip [IP]other commandspydfs --versionpydfs --infoDockerYou can also usedocker-composeto run multi-container application with:1 master node2 slave nodes2 user nodesdocker-compose up --buildRequirementsPython >= 3.7CitationIf you usePyDFSin a scientific publication, we would appreciate references to the following BibTex entry:@misc{silkwayai2022pydfs,author={Dani El-Ayyass and Artem Fomin},title={Distributed File System written in Python},howpublished={\url{https://github.com/silkway-ai/pydfs}},year={2022}}
Package: df-squeezer
DF_SqueezerDF_Squeezer is a Python package designed to help reduce the memory usage of pandas DataFrames. By analyzing the data
types of DataFrame columns, DF-Squeezer suggests and can automatically apply data type conversions that are more
memory-efficient without losing information.FeaturesAnalysis of DataFrame Columns: Identifies columns that can be converted to more memory-efficient data types.Memory Usage Reporting: Calculates and reports the potential memory savings from suggested conversions.Automatic Conversion Application: Optionally applies suggested conversions directly to the DataFrame.InstallationTo install DF-Squeezer, run the following command in your terminal:pipinstalldf_squeezerEnsure you have pandas installed in your environment as it is a required dependency.Usagefrom df_squeezer import df_squeezer
import pandas as pd

# Sample DataFrame
df = pd.DataFrame(
    {
        "a": range(1000),
        "b": [float(i) for i in range(1000)],
        "c": ["category" + str(i % 3) for i in range(1000)],
    }
)

# Use df_squeeze to analyze and report potential dtype conversions
optimized_df = df_squeezer(df, report=True, edit=False)

# To directly apply the suggested conversions
optimized_df = df_squeezer(df, report=True, edit=True)Parametersdf (pd.DataFrame): The input DataFrame to analyze and optimize.
report (bool, optional): If True (default), prints suggested changes for data type conversions along with potential
memory savings.
edit (bool, optional): If False (default), the function only reports potential optimizations without applying them. If
True, the function applies the suggested data type conversions directly to the DataFrame.
Returns
pd.DataFrame: The original DataFrame with applied changes if edit=True. Otherwise, returns the original DataFrame
unmodified.
Contributing
Contributions to DF-Squeezer are welcome! Please refer to the contribution guidelines for more information.License
This project is licensed under the MIT License - see the LICENSE file for details.
Package: dfs-tool
dfs_toolIt is a HDFS cli tool. You can use it to manage your HDFS file system.It calls theWebHDFSAPI.ConfigurationYou need to put a config file. By default, the config file is at~/.dfs_tool/config.json, however you can change it's location by setting environment variableDFS_TOOL_CFGThe configuration looks like below:{"api_base_uri":"https://my_hdfs_cluster.com/gateway/ui/webhdfs/v1/","username":"superman","password":"mypassword","io_chunk_size":16777216}In some case, server uses certificate to authenticate client, then you can config like below:{"api_base_uri":"https://my_hdfs_cluster.com/gateway/ui/webhdfs/v1/","auth_cert":"/Users/shizhong/.dfs_tool/sso_client.cer","auth_key":"/Users/shizhong/.dfs_tool/sso_client.key","ca_cert":"/Users/shizhong/.dfs_tool/CombinedDigicertCA.cer"}api_base_url: You need to put your WebHDFS endpoint hereusername: You need to specify your HDFS account usernamepassword: You need to specify your HDFS account passwordio_chunk_size: optional, if not set, the default value is 1048576. It is the chunk size for downloading data from HDFS or uploading data to HDFS, you may want to bump this value if your bandwidth is highCommand supporteddfs_tool ls [-R]          <remote_path>                            -- list directory or file
dfs_tool download [-R]    <remote_filename> <local_path>           -- download file
dfs_tool cat              <remote_filename>                        -- cat a file
dfs_tool mkdir            <remote_dir_name>                        -- make a directory
dfs_tool rm [-RM]         <remote_path>                            -- remove a file or directory
dfs_tool upload [-R] [-F] <local_filename> <remote_path>           -- upload file
dfs_tool mv               <source_location> <destination_location> -- move file or directoryOptionsSome command support options, here are options:-RIt means "recursive"-FIt means "force", in upload command, when-Fis specified, it will override the file already exist there.-MIn "rm" command, you can specify a pattern to match the files you want to delete, for example:dfs_tool rm -M "/tmp/*.parquet"
Package: dfsummarizer
dfsummarizerThis is an application to summarize the variables in a data frame.
It will accept a CSV, TSV or XLS file and produce a table summarizing
all columns individually.This was motivated by the fact that the summary function for a pandas
data frame ignores all non-numeric columns, and does not contain multiple
common analytical considerations: how many unique values, how many missing
values, min and max dates, min, mean and max string lengths.Output can be generated as either Latex or Markdown.Released and distributed via setuptools/PyPI/pip for Python 3.Additional detail available in thecompanion blog postNotesInitial implementation can handle larger files by chunking data and iteratively
building statistics. All statistics are robust except for estimation of the proportion
of unique values. We have used a simple implementation of the Flajolet Martin algorithm
based on the implementation byJavia JinkalThisreview article by Phillip Gibbonsgives a great overview of the alternatives.UsageYou can use this application multiple waysUse the runner:./dfsummarizer-runner.py markdown data/test.csv > markdown_test.mdWhich was used to generate the markdownoutput test fileInvoke the directory as a package:python -m dfsummarizer markdown data/test.csvOr simply install the package and use the command line application directlyInstallationInstallation from the source tree:python setup.py install(or via pip from PyPI):pip install dfsummarizerNow, thedfsummarizercommand is available::dfsummarizer markdown test.csvThis will produce a markdown table summarizing the contents of the CSV
file test.csvAcknowledgementsPython package built using thebootstrap cmdline templatebyjgehrckeNameTypeUnique ValsNullsModeMinMeanMaxidChar60.0%S00144.04openingDate60.0%2019-01-01 00:00:002019-01-012019-04-182019-07-12firstBool216.7%NO0.00.41.0lastBool250.0%NaN00.3331stateChar316.7%NSW3.03.03.0balanceFloat50.0%500.0200.01093.554230.9durationFloat333.3%24.012.021.024.0yearsInt30.0%223.04flagFloat266.7%NaN1.01.01.0commentsChar60.0%Combined savings account921.16735
Package: dft
TBD
Followinghttps://packaging.python.org/en/latest/tutorials/packaging-projects/andhttps://dzone.com/articles/executable-package-pip-installto create pip package for df_plotpython3 setup.py bdist_wheel
pip uninstall dist/dft-0.1-py3-none-any.whl
pip install dist/dft-0.1-py3-none-any.whl
python3 -m build
python3 -m twine upload --repository testpypi dist/*
python3 -m twine upload --repository pypi dist/*
Package: dft2kp
DFT2kpA numerical framework to explicitly calculate kp matrix elements fromab-initiodata.Documentationavailable atdft2kp.gitlab.io/dft2kpCode repositoryavailable atgitlab.com/dft2kp/dft2kpCurrent featuresCalculates kp matrix elements using the DFT eigenstates fromQuantum Espresso (QE).Folding down of the effective Hamiltonian into a set of selected bands.Rotates the numeriacal basis into a representation informed by the user.See alsoList of authors:AUTHORS.mdReferences to cite if we use our code:CITING.mdQuick install intructions and compatibility requirements:INSTALL.md