Saravanan666/pypi-org huggingface dataset:
Package: dsegmenter
A collection of various discourse segmenters (with pre-trained models for German texts).DescriptionThis python module currently comprises three discourse segmenters:edseg,bparseg, andmateseg.edsegis a rule-based system that uses shallow discourse-oriented
parsing to determine the boundaries of elementary discourse units.
The rules are hard-coded in thesubmodule’s fileand are
only applicable to German input.bparsegis an ML-based segmentation module that operates on
syntactic constituency trees (output fromBitPar) and decides
whether a syntactic constituent initiates a discourse segment or not
using a pre-trained linear SVM model.  This model was trained on the
GermanPCCcorpus, but you can also train your own classifer for any
language using your own training data (cf.discourse_segmenter--helpfor further instructions on how to do that).matesegis another ML-based segmentation module that operates on dependency
trees (output fromMateParser) and decides whether a sub-structure
of the dependency graph initiates a discourse segment or not using
a pre-trained linear SVM model.  Again, this model was trained on
the GermanPCCcorpus.InstallationTo install this package from the PyPi index, runpipinstalldsegmenterAlternatively, you can also install it directly from the source
repository by executing:gitclonegit@github.com:discourse-lab/DiscourseSegmenter.gitpipinstall-rDiscourseSegmenter/requirements.txtDiscourseSegmenter/--userUsageAfter installation, you can import the module in your python scripts
(see an examplehere), e.g.:fromdsegmenter.bparsegimportBparSegmentersegmenter=BparSegmenter()or, alternatively, also use the delivered front-end scriptdiscourse_segmenterto process your parsed input data, e.g.:discourse_segmenterbparsegsegmentDiscourseSegmenter/examples/bpar/maz-8727.exb.bparordiscourse_segmentermatesegsegmentDiscourseSegmenter/examples/conll/maz-8727.parsed.conllNote that this script requires two mandatory arguments: the type of
the segmenter to use (bparsegormatesegin the above cases) and the
operation to perform (which meight be specific to each segmenter).EvaluationIntrinsic evaluation scores of the machine learning models on the
predicted vectors will be printed when training and evaluating a
segmentation model.Extrinsic evaluation scores on the predicted segmentation trees can be
calculated with the evaluation script.evaluation{FOLDER:TRUE}{FOLDER:PRED}Note, that the script internally calls theDKpro agreement library,
which requires Java 8.
Package: dse-graph
This package builds on the core DataStax Enterprise Python driver, adding functionality
for interacting with DSE graph features (ie. TinkerPop).This driver is solely for use with DataStax Enterprise. See the License section below.FeaturesTinkerPop GraphTraversal execution on DSEInstallationInstallation through pip is recommended:$ pip install dse-graphDocumentationThe documentation can be found onlinehere.A couple of links for getting up to speed:InstallationGetting started guideReporting ProblemsPlease report any bugs and make any feature requests on theJIRAissue tracker.Getting HelpYour two best options for getting help with this extension are themailing listand the IRC channel.For IRC, use the #datastax-drivers channel on irc.freenode.net.  If you don’t have an IRC client,
you can usefreenode’s web-based client.LicenseCopyright DataStax, Inc.The full license terms are available athttp://www.datastax.com/terms/datastax-dse-driver-license-terms
Package: ds-emer
DS_EMERds_emer es una libreria de prueba.InstalaciónUse the package managerpipto install ds_emer.pipinstallds-emer## Usage```python
fromds_emerimport*# Funcion Eugea=euge()# Funcion Agusb=agus()# Funcion Facuc=facu()ContribucionesNo son necesarias.
Package: dsemu
dsemuDescriptiondsemuis a simple library to help with testing GCP Datastore code written in
Python. The providedEmulatorclass wraps theDatastore
emulatorand
provides basic functionality such as starting/stopping and reseting the
emulator instance from inside the test runner.RequirementsYou must have thegcloudtoolinstalledand available inPATH.Using existing instance of the emulatorIf you're running tests that require datastore access frequently it might be
better to keep an instance of the emulator running at all time instead of
letting the wrapper start and stop it for the duration of the test run. If an
instance of the emulator is running and the required environment variables are
correctly set theEmulatorwrapper will use the running instance instead of
starting a new one and will not tear it down at the end of the test run.Example usage with pytest# conftest.pyimportpytestfromdsemuimportEmulatorfromgoogle.cloudimportdatastore@pytest.fixture(scope="session")defemulator():withEmulator()asemulator:yieldemulator@pytest.fixture(scope="session")defsession_client():client=datastore.Client(project="test")yieldclient@pytest.fixture()defclient(emulator:Emulator,session_client:datastore.Client):emulator.reset()yieldsession_client# datastore_test.pyfromgoogle.cloudimportdatastoredeftest_datastore_put_and_get(client:datastore.Client):kind="Task"name="sampletask1"task_key=client.key(kind,name)task=datastore.Entity(key=task_key)task["description"]="Buy milk"client.put(task)res=client.get(task_key)assertres==task
Package: dsenum
dsenumDerivative structure enumerator for multilatticeGithub:https://github.com/lan496/dsenumPyPI:https://pypi.org/project/dsenum/Installationdsenum works with Python3.8+ and can be installed via PyPI:pipinstalldsenumOr in local:gitclonegit@github.com:lan496/dsenum.gitcddsenum
pipinstall-e.Usageimportnumpyasnpfrompymatgen.coreimportLattice,Structurefrompymatgen.core.periodic_tableimportDummySpeciefromdsenumimportStructureEnumeratorlatt=Lattice(np.array([[0,1,1],[1,0,1],[1,1,0]]))coords=[[0,0,0]]structure=Structure(latt,[DummySpecie('X')]*len(coords),coords)num_type=2index=4dstructs=StructureEnumerator(structure,index,num_type).generate()print(len(dstructs))# -> 12Seedocs/examples/constraints.ipynbfor more complicated use cases.How to cite dsenumIf you usedsenum.ZddStructureEnumeratorin your research, please cite the following articles.@article{doi:10.1063/5.0021663,
    author = {Shinohara,Kohei  and Seko,Atsuto  and Horiyama,Takashi  and Ishihata,Masakazu  and Honda,Junya  and Tanaka,Isao },
    title = {Enumeration of nonequivalent substitutional structures using advanced data structure of binary decision diagram},
    journal = {J. Chem. Phys.},
    volume = {153},
    number = {10},
    pages = {104109},
    year = {2020},
    doi = {10.1063/5.0021663},
    URL = {https://doi.org/10.1063/5.0021663},
}@inproceedings{Horiyama2018,
  memo ={Isomorphism Elimination by Zero-Suppressed Binary Decision Diagrams},
  author={Takashi Horiyama and Masahiro Miyasaka and Riku Sasaki},
  booktitle={the Canadian Conference on Computational Geometry},
  pages={360--366},
  address={Winnipeg, Manitoba, Canada}
  year={2018},
  url={http://www.cs.umanitoba.ca/~cccg2018/papers/session7B-p2.pdf}
}AcknowledgmentsI acknowledge Dr. Takashi Horiyama for sharing his implementation of the frontier method for isomorphism-elimination decision diagram.
I also appreciate his kindness to allow publishing the code.
Package: dserial
DSerial0. IntroductionDSerial is dictionary serializer with type checking methodology by declared type hinting1. Requirementspython 3.7 +2. Quick start2.0. Installationpip3installdserial2.1. Default type{"int_variable":1234,"string_variable":"string_value"}For serialize above dictionary, data class can be below codeclassDefaultData(TypeCheckingDictSerializer):int_variable:intstring_variable:strand usage:>>>dictionary={'int_variable':1234,'string_variable':'string_value'}>>>data=DefaultData(dictionary)>>>data.int_variable1234>>>data.string_variablestring_value2.2. Nested type{"int_variable":1234,"string_variable":"string_value","nested":{"int_variable":4321}}For serialize above dictionary, data class can be below codeclassNestedData(TypeCheckingDictSerializer):int_variable:intclassDefaultData(TypeCheckingDictSerializer):int_variable:intstring_variable:strnested:NestedDataand usage:>>>dictionary={'int_variable':1234,'string_variable':'string_value','nested':{'int_variable':4321}}>>>data=DefaultData(dictionary)>>>data.int_variable1234>>>data.string_variablestring_value>>>data.nested.int_variable43212.3. Repeated type{"int_variable":1234,"string_variable":"string_value","nested":{"int_variable":4321},"repeated":[1,2,3]}For serialize above dictionary, data class can be below codeclassRepeatedData(TypeAnnotatedList):@staticmethoddefget_type():returnintclassNestedData(TypeCheckingDictSerializer):int_variable:intclassDefaultData(TypeCheckingDictSerializer):int_variable:intstring_variable:strnested:NestedDatarepeated:RepeatedDataand usage:>>>dictionary={'int_variable':1234,'string_variable':'string_value','nested':{'int_variable':4321},'repeated':[1,2,3]}>>>data=DefaultData(dictionary)>>>data.int_variable1234>>>data.string_variablestring_value>>>data.nested.int_variable4321>>>len(data.repeated)3>>>data.repeated[0]12.4. More complex{"int_variable":1234,"string_variable":"string_value","repeated_nested":[{"int_variable":3133},{"int_variable":1337}]}For serialize above dictionary, data class can be below codeclassNestedData(TypeCheckingDictSerializer):int_variable:intclassRepeatedData(TypeAnnotatedList):@staticmethoddefget_type():returnNestedDataclassDefaultData(TypeCheckingDictSerializer):int_variable:intstring_variable:strrepeated_nested:RepeatedDataand usage:>>>dictionary={'int_variable':1234,'string_variable':'string_value','repeated_nested':[{'int_variable':3133},{'int_variable':1337}]}>>>data=DefaultData(dictionary)>>>data.int_variable1234>>>data.string_variablestring_value>>>len(data.repeated_nested)3>>>data.repeated[0].int_variable31333. Test>dserial$python3-mtests
Package: d-serialize
d_serialize===========Universal converter of all things Python to adict.  So it can be serializedtoJSON.Installpip install d-serializeUsage:fromd_serializeimportd_serializefromobjectsimportSomeObjectsome_object=SomeObject()json_values=d_serialize(some_object)Usage in Flaskfromd_serializeimportd_serializefromflaskimportjsonifyfromobjectsimportSomeObject@app.route('/a_route')defa_route():some_object=SomeObject()json_values=d_serialize(some_object)returnjsonify(json_values)d_serialize will enumerate all public properties of your object, set, list ordictand convert themto a JSON allowable type.  IE:list, dictionary, string, float, integer or boolean.Any property that is not one of these types will be converted to astring. Enumerables:setandtuplewill be converted tolist.Dictionary, set, tuple and list properties will be followed to ensure all child objects andvalues are also converted.Methods and private properties (starting with_) are not enumerated.Any property or attribute that raises an exception will be excluded orhave aNonevalue.Complex objects that refer to themselves will have their str value as therecursive value instead of the entire complex value.Example:Converting a Class instance.fromd_serializeimportd_serializeclassTestObject:number_value=1float_value=1.1defdont_call_me(self):"""return self.number_value"""print(d_serialize(TestObject()))# dict(number_value=1, float_value=1.1)Release History===============1.0.0 First version of this wondrous package.1.0.1 Check for None when serializing and return None rather than 'None'.  Exceptions on getattr return None for value.1.0.2 Crummy circular reference test.  Convert top level list, set, tuple.1.0.3 Fix deploy workflow1.0.4 Allow for readonly objects1.0.6 Improve recursion check and value
Package: dserve
Tool to serve a dataset over HTTP.Installationpip install dserveUsageThe command below serves the dataset located in/path/to/dataseton port 8080.dserve -d path/to/dataset -p 8080The default port is 5000.The dataset can also be specified using theDSERVE_DATASET_PATHenvironment variable.export DSERVE_DATASET_PATH=path/to/dataset
dserveDataset creationDatasets can be created using the related Python tools:dtoolcoredtool
Package: dset
A Disjoint-Set implementation which makes use of the the “Union by Rank” and “Path Compression” heuristic
optimizations while being as simple and effective as possible.It support the following operations:Find-SetinO(1)amortized running timeUnioninO(1)amortized running timeCheck end notes for additional information about running timesWhy?Sometimes, specially  during algorithm competitions, a Union-Find data structure is needed
to solve a certain problem (e.g. when implementing Kruskal’s MST algorithm) In those cases,
we usually use whatever off-the-shelf implementation we happen to find in PyPI and, many times,
we get the infamous submission failTime Limit Exceed(TLE)because that specific
Disjoint-Set implementation is not fast enough…Worry not! A fast and simple enough (you can copy paste the codehere) disjoint-set implementation
is now available:Installation$pipinstalldsetUsage>>>importdset>>>s1=dset.Set()# To create new sets objects (it runs in O(1))>>>s2=dset.Set()>>>s3=dset.Set()>>>dset.groups()# To check the number of different groups (it runs in O(1))3>>>dset.find(s1)==dset.find(s2)# To check if two sets are in the same groupFalse>>>dset.union(s1,s2)# to group s1 and s2 sets together>>>dset.groups()# Now there are only 2 groups: s1-s2 and s32>>>dset.find(s1)==dset.find(s2)# They are in the same group s1-s2True>>>dset.find(s2)==dset.find(s3)# They are in distinct groups s1-s2 and s3False>>>dset.union(s2,s1)# do nothing (no need to check if s1 and s2 belong to the same group)>>>dset.groups()# Same as before only 2: s1-s2 and s32>>>dset.union(s1,s3)# group s1-s2 and s3 together.>>>dset.groups()# Now there is only one group: s1-s2-s31>>>dset.find(s1)==dset.find(s2)# All the sets are in the same groups s1-s2-s3...dset.find(s2)==dset.find(s3)...dset.find(s1)==dset.find(s3)TrueTrueTrueFuture improvementsMake official Sphinx Documentation. (minor releases)Make test suite. (minor releases)Change to a list-based implementation (major-minor release)Improve running time (constant terms)Improve memory space (constant terms)Make the package C-based. (major-minor release)Improve running time (constant terms)Notes on Complexity Running TimesSeparately, either “Union by Rank” or “Path Compression” improves the running time of
the operations on disjoint-sets. Alone, “Union by Rank” yields a running time
ofO(m log n), wheremis the number of Union and Find-Set operations andnis the number
of sets.The improvement is even greater when we use the two heuristics together; the worst-case running
time isO(m f(n)),wheref(n)is a very slowly growing function, the Inverse Ackermann function
(e.g. fornequal to the number atoms in the universe,f(n)is only 4)
So for any conceivable application, we can consider the running time ofmUnion + Find-Set operations to beO(m)and therefore both Union and Find-Set operations haveO(1)amortized running time in practice.
Package: ds-example-plugin
ds_example_pluginA plugin fordata_slicer'sPIT
that simply serves as an example for how to make such a plugin.It only adds minimal functionality to PIT, namely the functionblurwhich
applies a Gaussian filter to the displayed data and the functionexample_functionwhich just prints out a message to PIT's ipython console.As such, it really is not particularly useful beyond illustrative purposes.Refer to thedocumentation ofdata_slicerfor more information.
Package: dsfaker
DSFaker aims at providing data scientists with functions for generating
and streaming arrays of data such as:Time-seriesDate/DatetimeAutoincrement & random autoincrementTrigonometric (Sin, Sinh, Cos, Cosh, Tan, Tanh)Random distributionsPattern repetitionNoise applicationBasic operators between generators and/or python int/float types (+, -,
/, //, *, …) are also implemented.To generate such data, you can specify the distribution you want amongst
the ones provided by numpy.random. Or you can also implement your own
distribution to generate data.Installationpip install dsfakerDocumentationThe documentation is available here:https://eds-aphp.github.io/dsfaker/.
Package: dsfe
Data Science for EveryonePackage to support the DSFE (Data Science For Everyone) textbook.The most recent version of the textbook at time of writing is theLondon
Interdisciplinary School version.The DSFE package contains various routines to ease some of the more complex
calls to various Numpy and Scipy routines.It is a teaching module as well as a code module.  The idea is to give the
canned routines for common procedures, but treat the canned code as part of
a more advanced tutorial in how the procedures work.
Package: dsff
DataSet File FormatStore a dataset in XSLX-like format.This library contains code for handling the DataSet File Format (DSFF) based on the XSLX format and for converting it toARFF(for use with theWekaframework),CSVor aFilelessDataset structure(from thePacking Box).pipinstall--userdsffUsageCreating a DSFF from a FilelessDataset>>>importdsff>>>withdsff.DSFF()asf:f.write("/path/to/my-dataset")# folder of a FilelessDataset (containing data.csv, features.json and metadata.json)f.to_arff()# creates ./my-dataset.arfff.to_csv()# creates ./my-dataset.csv# while leaving the context, ./my-dataset.dsff is createdCreating a FilelessDataset from a DSFF>>>importdsff>>>withdsff.DSFF("/path/to/my-dataset.dsff")asf:f.to_dataset()# creates ./[dsff-title] with data.csv, features.json and metadata.jsonRelated ProjectsYou may also like these:Awesome Executable Packing: A curated list of awesome resources related to executable packing.Bintropy: Analysis tool for estimating the likelihood that a binary contains compressed or encrypted bytes (inspired fromthis paper).Dataset of packed ELF files: Dataset of ELF samples packed with many different packers.Dataset of packed PE files: Dataset of PE samples packed with many different packers (fork ofthis repository).Docker Packing Box: Docker image gathering packers and tools for making datasets of packed executables.PEiD: Python implementation of the well-known Packed Executable iDentifier (PEiD).PyPackerDetect: Packing detection tool for PE files (fork ofthis repository).REMINDer: Packing detector using a simple heuristic (inspired fromthis paper).
Package: dsfinterp
dsfinterpSummaryCubic Spline Interpolationf of Dynamics Structure Factors S(Q,E,f) as a function of a parameter “f”.DetailsGiven a discrete series of Dynamics Structure Factors S_i(Q,E) that
depends on an extra parameter “f”, S_i(Q,E) = S(Q,E,f_i), construct
and object that returns S(Q,E) for any value of f in the range
[min({f_i} }, max( {f_i} )].Parameter “f” can be an environment variable such as Temperature or an
“internal” parameter, such as a component of a force-field.This interpolator is geared towards determination of S(Q,E,f) using a
series { S_i(Q,E) } computed from molecular dynamics simulations,
although the series could be generated from experiments, too.If the { S_i(Q,E) } do not have associated errors, as it is the case
when computed from simulations, an error will be estimated by a
running regression of the S_i(Q,E) versus f. Thus, the object will
return S(Q,E) and and associated error(Q,E) any value of f in the
allowed range.
Package: dsforge
This package automates the creation of a basic directory structure and files for a data science or machine learning project. It is designed to set up an empty project with standard directories and placeholder files, making it easier to start your project.The package will create the following structure and files:.
├── .github
│   └── workflows
│       └── .gitkeep
├── src
│   └── (projectName)
│       ├── __init__.py
│       ├── modules
│       │   └── __init__.py
│       ├── utilities
│       │   └── __init__.py
│       ├── pipeline
│       │   └── __init__.py
│       ├── constants
│       │   └── __init__.py
├── models
│   └── .gitkeep
├── logs
│   └── .gitkeep
├── data
│   ├── raw
│   │   └── .gitkeep
│   ├── intermediate
│   │   └── .gitkeep
│   └── processed
│       └── .gitkeep
├── exports
│   └── .gitkeep
├── reports
│   └── .gitkeep
├── research
│   └── experiments.ipynb
├── tests
│   └── .gitkeep
├── main.py
├── app.py
├── Dockerfile
├── requirements.txt
├── setup.py
└── README.mdUsageRun the script from the directory where you intend to create the project template.# Replace 'MyProject' according to your project's namefromdsforgeimportcreatorcreator("MyProject")Notes:The script will log each directory and file creation.
Below is a guide for folder usage:src/(projectName):Main source code directory.models:Reserved for storing model files.data:For storing raw/intermediate/processed data files.reports:For storing graphs and reports.exports:For storing outputs and other file exports.research:Jupyter notebook for experiments.tests:Reserved for test scripts.main.py:Main entry point for the project.app.py:Web-app specific code.
Package: dsf-python
Duet Software Framework Python BindingsThis is also availabe as apip package on pypiFind out more aboutDuet Software Framework.Find out more aboutDuet Software Framework Python Bindings.Get in touch with the community atDuet Software Framework Forumfor bug reports, discussion and any kind of exchange.InstallationThis package contains asetup.pyso it can be installed withpython3 setup.py install.UsageSee includedexamples/folder for various use cases.
Package: dsFramework
ds-frameworkdata science framework#how to install
pip install dsframework#how to use
##list of commands:dsf-cli - to get all cli optionsdsf-cli generate project my-new-project - to generate new project named my-new-projectinside generated project folder(cd my-new-folder)dsf-cli g forcer my-new-forcer - creating new forcer (will be automatically injected to pipeline after last exist forcer)dsf-cli g predictable my-new-predictable - creating new predictable (will be automatically injected to pipeline after last exist predictable)dsf-cli run-server - will run local server with port 8080available endpoints depend on projecthttp://localhost:8080/parse,http://localhost:8080/predictwill validate input based on input scheme and execute pipelinehttp://localhost:8080/testwill run mock test using Data science portaldsf-cli create-deploy-files - deploy files generated automatically when creating new project this is an option to create all non exist deploy files,if deploy file was deleted by mistake you can recreate itif dsframework version was update with updated deploy files you can delete deploy files and recreate themdsf-cli create-cloud-eval-files - cloud eval files this is an option to create all non exist cloud eval files
Package: dsFrameworkC
ds-frameworkdata science framework#how to install
pip install dsframework#how to use
##list of commands:dsf-cli - to get all cli optionsdsf-cli generate project my-new-project - to generate new project named my-new-projectinside generated project folder(cd my-new-folder)dsf-cli g forcer my-new-forcer - creating new forcer (will be automatically injected to pipeline after last exist forcer)dsf-cli g predictable my-new-predictable - creating new predictable (will be automatically injected to pipeline after last exist predictable)dsf-cli run-server - will run local server with port 8080available endpoints depend on projecthttp://localhost:8080/parse,http://localhost:8080/predictwill validate input based on input scheme and execute pipelinehttp://localhost:8080/testwill run mock test using Data science portaldsf-cli create-deploy-files - deploy files generated automatically when creating new project this is an option to create all non exist deploy files,if deploy file was deleted by mistake you can recreate itif dsframework version was update with updated deploy files you can delete deploy files and recreate themdsf-cli create-cloud-eval-files - cloud eval files this is an option to create all non exist cloud eval files