Saravanan666/pypi-org huggingface dataset:
Package: dataclassframe
dataclassframeA dataclass container with multi-indexing and bulk operations.
Provides the typed benefits and ergonomics of dataclasses while having the efficiency ofPandas dataframes.The container is based ondata-oriented designby optimising the memory layout of the stored data, providing fast
bulk operations and a smaller memory footprint for large collections.
Bulk operations are enabled using Pandas which has a rich set of vectorised methods for both numerical and string
data types.Multi-indexing provides the ability to use multiple fields as keys to index the records.
This is suitable for bidirectional and inverse dictionary keys.A DataClassFrame provides good ergonomics for production code as columns are immutable
and columns/data types are well defined by the dataclasses.
This makes it easier for users to understand the "shape" of the data in large projects and refactor when necessary.InstallingGet the latest version using pip/PyPipipinstalldataclassframeFeature comparisonContainerPositional indexingKey indexingMulti-key indexingData-oriented designColumn-wise opperationsType hintsUse in prodDataClassFrame✅✅✅✅✅✅✅List✅❌❌❌❌✅✅Dictionary❌✅❌❌❌✅✅MIDict✅✅✅❌❌✅✅Pandas DataFrame✅✅✅✅✅❌❌Show by exampleA container data-type for dataclasses...fromdataclassesimportdataclassfromdataclassframeimportDataClassFrame@dataclassclassExampleDC:field1:strfield2:intrecords=[ExampleDC('a',1),ExampleDC('b',2),ExampleDC('c',3),]dcf=DataClassFrame(record_class=ExampleDC,data=records,index=['field1','field2'])Which acts like a ordered dictionary with multi-indexing...# Obtain record `ExampleDC('b', 2)`row_idx=dcf.iat[1]# Using positional indexrow_f1=dcf.at['b']# Using index of `field1`row_f2=dcf.at[:,2]# Using index of `field2`assertrow_idx==row_f1==row_f2With bulk operations on the columns..assertdcf.cols.field2.sum()==6Works nicely with Python 3 type hints...dcf:DataClassFrame[ExampleDC]dcf.iat[1]:ExampleDCDesignIt's no secret that under the hood DataClassFrames are using Pandas DataFrames to store data.
The data is converted where possible to Pandas Series, which in turn use Numpy arrays. When the user accesses a record the data is then converted back into the dataclass provided at initialisation.Pandas provides many advantages over of using a simple list of dataclasses or similar such as better memory
footprint and fast vectorised operations. However using Pandas DataFrames directly in production code isconsidered by the author and others as an anti-pattern.
Specifically as DataFrames are column-wise mutable and therefore difficult to determine at code-time what columns
the dataframe contains i.e. its shape. It also does not provide any type-hinting benefits.TodoSlicing and dataclassframe views for accessing data and setting dataAppend and insertsData-oriented design for Numpy fieldsChangelogAll notable changes to this project will be documented here.The format is based onKeep a Changelogand this project adheres toSemantic Versioning.[0.1.0] - 2020-10-22AddedInitial release ofdataclassframeLicense© Josh Levy-Kramer 2020. dataclassframe is released under the MIT license.
Package: dataclass-map-and-log
Dataclass Map And LogMap dictionaries to pydantic dataclasses, log any extra attributes.Examplefromdataclass_map_and_log.mapperimportDataclassMapper@dataclassclassChild:name:strsurname:str@dataclassclassSingleChild:name:str@dataclassclassParent:name:strsurname:strchildren:List[Child]single_child:SingleChilddata={"name":"parent_name","surname":"parent_surname","extra":"parent_extra_data","children":[{"name":"child1_name","surname":"child1_surname","extra":"child_extra_data",},{"name":"child2_name","surname":"child2_surname",},],"single_child":{"name":"test"},}definition=DataclassMapper.map(Parent,data)You can then access the dataclass instance like:definition.single_child.nameAnd you would have gotten the following warning log messages:"Unexpected attribute extra on class <class 'tests.test_dataclass_map_and_log.Parent'> with value parent_extra_data",
"Unexpected attribute extra on class <class 'tests.test_dataclass_map_and_log.Child'> with value child_extra_data",
Package: dataclass-mortar
mortarnaturally use dataclasses with AlchemyExperimental concept for easying the use of Dataclasses with Alchemy. Simple models and relationships work, anything more complicated probly won't.UsageDecorate a dataclass with@persistColumn definitions are automatically generated based off the field data, relationships are created for any field that had a type of another dataclass. Column definitions can be passed to mortar via the metadata attribute of the field() method (provided by dataclass). Partial Column data can also be passed any other required fields of the column will be generated based off the field definition.Primary Keysmortar will try to find the proper primary key for a defined class. It will evaluate the columns in this order to find a primary key:Checks for predefined key via A partial column definitionColumn(primary_key=True)Check for a column named <class_name.lower()>_idcheck for an id column namecreate and auto increment column designated as primary key
Package: dataclass-persistence
This program can be used to make dataclasses persistent by adding store and load functionality.
The dataclass is stored in .json format which is by default compressed inside of a .zip file.What makes dataclass-persistence special?Support for numpy arraysSupport for nested dataclassesHuman readable storage format with small file sizeUsageLet your dataclass inherit fromPersistent.
Then the dataclass can be stored on disk using.store()and loaded from disk using.load().In the example below, we create an instance of dataclass, which is stored to and loaded from disk.fromdataclass_persistenceimportPersistentfromdataclassesimportdataclassimportnumpyasnp@dataclassclassSomeData(Persistent):parameter_a:strarray:np.ndarraydata=SomeData('my_string',np.array([0,0]))file='my_file'data.store(file)data_reconstructed=SomeData.load(file)On disk the code above producesmy_file.zipwhich containsmy_file.json:{"parameter_a":"my_string","array":{"data":[0,0],"dtype":"int32"}}
Package: dataclass-plus
dataclass_plusThedataclass_plusis a fastest type validation library for thedataclassInstallpip install dataclass-plusExampleBasic Examplefromdataclass_plusimportdataclass_plusfromtypingimportList,Dict,Tuple@dataclass_plusclassModel:id:intname:strdict_example:Dict[str,str]list_example:List[int]# this field is requiredtuple_example:Tuple[str,float]=None# this field is not required because set default NoneModel(id=1,name='Test Test',dict_example={"test":"test"},list_example=[1,2],tuple_example=("test",1.2))# => Model(id=1, name='Test Test', dict_example={"test": "test"}, list_example=[1,2], tuple_example=("test", 1.2))# Invalid ModelModel(id=1,name='Test Test',dict_example={"test":1},list_example=[1,2],tuple_example=("test",1.2))# => TypeError: {'test': 1} is not typing.Dict[str, str]Nested Examplefromdataclass_plusimportdataclass_plusfromtypingimportList,Dict,Tuple@dataclass_plusclassModel:nested_example:List[List[str]]multi_nested_example:Dict[str,List[Tuple[str,List[int]]]]Model(nested_example=[["test"],["test_2"]],multi_nested_example={"test":[("test",[1,2])]})# => Model(nested_example=[['test'], ['test_2']], multi_nested_example={'test': [('test', [1, 2])]})# Invalid ModelModel(nested_example=[["test"],["test_2"]],multi_nested_example={"test":[("test",[1,"test"])]})# => TypeError: [1, 'test'] is not typing.List[int]Note: Can use every feature of dataclass (Example: frozen=True, init=True ...)
Package: dataclass-property
Add the ability to use properties with dataclasses.
This allows for better input validation on dataclasses.Most of the code base is a direct copy from dataclasses with modifications.
Unfortunately, dataclasses were not written in a way where you can extend the functionality easily.
I changed the code to use a class based system, so the functionality could be extended.
It’s still not fun to work with the dataclasses library.ExamplesfromtypingimportUnionfromdataclass_propertyimportdataclass@dataclassclassTimeDelta:hours:int=0minutes:int=0milliseconds:int=0@propertydefseconds(self)->int:# default_factory is set to `int()` from the annotationreturnself._seconds@seconds.setterdefseconds(self,value:Union[int,float]):ifisinstance(value,float):mill=int((value%1)*1000)ifmill:self.milliseconds=millself._seconds=int(value)td=TimeDelta(minutes=1)td.seconds=0.5asserttd.seconds==0asserttd.milliseconds==500td=TimeDelta(seconds=0.1)asserttd.seconds==0asserttd.milliseconds==100field_propertyexample with custom defaultsfromdataclass_propertyimportdataclass,field_property@dataclassclassPoint:# X@field_property(default=1)defx(self)->int:returnself._x@x.setterdefx(self,value:int):self._x=value# Yy:int=field_property(default=1)@y.getterdefy(self)->int:returnself._y@y.setterdefy(self,value:int):self._y=value# Z@field_propertydefz(self)->int:returnself._zz.default(5)@z.default_factory@classmethoddefz(cls):return5@z.setterdefz(self,value:int):self._z=valuep=Point()assertp.x==1assertp.y==1assertp.z==5p=Point(x=2,y=2,z=6)assertp.x==2assertp.y==2assertp.z==6
Package: dataclass-rest
Dataclass RESTA modern and simple way to create clients for REST like APIsQuickstartStep 1. Installpipinstalldataclass_restrequestsStep 2. Declare models@dataclassclassTodo:id:intuser_id:inttitle:strcompleted:boolStep 3. Create and configure clientfromrequestsimportSessionfromdataclass_rest.http.requestsimportRequestsClientclassRealClient(RequestsClient):def__init__(self):super().__init__("https://example.com/api",Session())Step 4. Declare methods usingget/post/delete/patch/putdecorators.
Type hints are required. Body of method is ignored.Use any method arguments to format URL.bodyargument is sent as request body with json. Other arguments, not used in URL are passed as query parameters.getanddeletedoes not have body.fromtypingimportOptional,ListfromrequestsimportSessionfromdataclass_restimportget,post,deletefromdataclass_rest.http.requestsimportRequestsClientclassRealClient(RequestsClient):def__init__(self):super().__init__("https://example.com/api",Session())@get("todos/{id}")defget_todo(self,id:str)->Todo:pass@get("todos")deflist_todos(self,user_id:Optional[int])->List[Todo]:pass@delete("todos/{id}")defdelete_todo(self,id:int):pass@post("todos")defcreate_todo(self,body:Todo)->Todo:"""Создаем Todo"""AsyncioTo use async client insted of sync:Installaiohttp(instead ofrequests)Changedataclass_rest.http.requests.RequestsClienttodataclass_rest.http.aiohttp.AiohttpClientAddasynckeyword to your methodsConfigurationParsing and serializationAll parsing and serialization is done using instances ofFactoryProtocol.
They are create by client object during its initialization. Default implementation createsRetortfromadaptixThere are 3 of them:Request body factory. Created using_init_request_body_factory. It is used to convert body object to simple python classes before senging to the serverRequest args factory. Created using_init_request_args_factory. It is used to convert other parameters of method. All parameters are passed as a single object.
Type of that object is generated and can be retrieved from your client methods using.methodspec.query_params_typeResponse body factory. Created using_init_response_body_factory. It is used to parse a server response.Error handlingYou can attach error handler to single method using@yourmethod.on_errordecorator in your class.To set same behavior for all methods inherit from BoundMethod class, override_on_error_defaultmethod and set that class asClient.method_classOther paramsYou can use different body argument name if you want. Just passbody_nameto the decorator.
Package: dataclass_structor
dataclass_structorA type aware structor/destructor for python value objects.Installpipinstalldataclass_structorDocumentationThedocs for this project can be found here.Exampleimportdataclassesimporttypingfromdataclass_structorimportstructure,unstructure@dataclasses.dataclassclassInvite:email:strguests:typing.List["Guest"]@dataclasses.dataclassclassGuest:first_name:typing.Optional[str]=Nonevalue_type=Invite(email="testing",guests=[Guest(first_name="John"),Guest(),],)x=unstructure(value_type)assertx=={"email":"","guests":[{"first_name":"John"},{"first_name":None}]}assertstructure(x,Invite)==value_type
Package: dataclass-to-diagram
dataclass_to_diagramСоздание диаграмм из датаклассов pythonТипы диаграммС4Описание -https://c4model.com/Реализация на PlantUML -https://github.com/plantuml-stdlib/C4-PlantUMLСостоит из пакетов:main - основной пакет для запускаmodels - модели на основе dataclass для различных диаграммexporters - экспорт моделей в текстовый форматconverters - конвертирование текстового формата в изображение (svg, png, pdf, ...)exportersdbmlЭкспорт моделей БД (ERD) в форматdbmlconvertersdbml-rendererКонвертирует файл с разметкой dbml в svg-изображение.Установка:npminstall-g@softwaretechnik/dbml-rendererБиблиотека для генерации диаграмм из текстового описания.Диаграммы описываются объектами python. Далее геренируются изображения с помощьюhttps://kroki.io.Как использоватьСоздать две папки:dia_src - папка с исходным описаниемdia_dist - папка со сгенерированными изображениямиВ папке dia_src создаются py-файлы. Названия файлов - произвольные. Можно создавать подкаталоги - структура каталогов будет скопирована в целевую папку dia_dist. Примеры создания можно посмотреть в тестовых диаграммахпакета.В одном файл должна быть только одна диаграмма. Название файла будет названием диаграммы.Для генерации можно создать задачу poetepoet. Прописать в файле pyproject.toml:[tool.poetry.dependencies]konstantin_docs="*"poethepoet="*"[tool.poe.tasks]docs={script="konstantin_docs.main:generate_images('dia_src', 'dia_dist')"}Запустить командой:poetryrunpoedocsДополнительно можно создать задачу в vscode. Для этого в файле .vscode/tasks.json:{"version":"2.0.0","tasks":[{"label":"docs","type":"shell","command":"poetry run poe docs"}]}Запускать командой F1 -> Task: Run task -> docsERDnpminstall-g@softwaretechnik/dbml-rendererРазработкаpoetrybuild&&poetrypublish
Package: dataclass-to-json
Dataclass_to_jsonDataclass_to_jsonis a lightweight and simple library for Converting a dataclass structure to a JSON object. The dataclass_to_json library allows you to retrieve a JSON object representing the classattributesand theirtypes.Installing Dataclass_to_jsondataclass_to_json in available on PyPI:python-mpipinstalldataclass_to_jsonDataclass_to_json officially supports Python 3.7+.Usagefromdataclass_to_jsonimportdataclass_to_jsonfromtypingimportListimportjsonclassEmployee:name:strage:intsalary:floatremarks:List[str]classEntreprise:name:straddress:stremployees:List[Employee]json_structure=dataclass_to_json(Entreprise)print(json.dumps(json_structure,indent=2))The output:{"name":"str","address":"str","employees":[{"name":"str","age":"int","salary":"float","remarks":["str"]}]}
Package: dataclay
dataClay is a distributed data store that enables applications to store and access objects in the same format they have in memory, and executes object methods within the data store. These two main features accelerate both the development of applications and their execution.InstallationTo install dataClay, you can usepip:$pipinstalldataclayAlternatively, you can obtain the latest source code fromGitHub:$gitclonehttps://github.com/bsc-dom/dataclay.git
$cddataclay
$pipinstall.DocumentationdataClay has usage and reference documentation atdataclay.readthedocs.ioContributingWe welcome contributions to dataClay. Please see theContributingfor more details.Maintainers@alexbarcelo(Alex Barcelo)@aqueralt(Anna Queralt)@marcmonfort(Marc Monfort)Other resourcesBSC official website
Package: dataClay-common
dataClay commonprotos: common grpc protocol buffers for dataclayProtosCompile in javaclaymvn protobuf:compile-custom -Pprotogento create just grpc-stubs ormvn clean compile -Pprotogento compile all javaclay including new protosCompile in pyclaypip install grpcio-tools protobufferize
python setup.py protobufferizeNOTE: if protbufferize cannot be installed via pip, please clone
it fromhttps://github.com/bsc-dom/protobufferizeand runpython setup.py installPackaging PythonIt's strongly recommended to use a virtual environment.# Make sure you have the latest build and twine versions installedpython3-mpipinstall--upgradebuildtwine# Remove dist folder if existsrm-rfdist/# Build release distribution with date tagpython3-mbuild-C--global-option=egg_info-C--global-option=--tag-build=$(date+%s)# Publish package to PyPIpython3-mtwineuploaddist/*
Package: datacleanbot
datacleanbotAutomated Data Cleaning Tool.
The main goal is to develop a Python tooldatacleanbotsuch that:
Given a random parsed raw dataset representing a supervised learning problem, the Python tool is capable of automatically identifying the potential issues and reporting the results and recommendations to the end-user in an effective way.Install$pipinstalldatacleanbotQuickStartInstall OpenML (version 0.9.0):OpenML is used to easily import datasets and share models and experiments.$pipinstallopenmlFor Windows, you need to have C++ Compiler installed.Acquire data from OpenML:>>> import openml as oml
>>> data = oml.datasets.get_dataset(id) # id: openml dataset id
>>> X, y, categorical_indicator, features = data.get_data(target=data.default_target_attribute, dataset_format='array')
>>> Xy = np.concatenate((X,y.reshape((y.shape[0],1))), axis=1)Autoclean data with datacleanbot:>>> import datacleanbot.dataclean as dc
>>> Xy = dc.autoclean(Xy, data.name, features)Descriptiondatacleanbotis equipped with the following capabilities:Present an overview report of the given datasetThe most important featuresStatistical information (e.g., mean, max, min)Data types of featuresClean common data problems in the raw datasetDuplicated recordsInconsistent column namesMissing valuesOutliersThe two aspectsdatacleanbotmeaningfully automates are marked in bold.User's GuideThe user's guide can be found atdatacleanbot.
Package: data-cleanup
cleanup_pypi#Reference for creating python packageofficial python page#Reference URLURL helpful for creating python packageThis library has 3 functions:-1) replacing null values
2) standardization the column values 
3) finding the variance_inflation_factor value column wisePoints to keep a noteThe base minimum parameter which it expects is either a numpy array or a DataFrame.

The only things we need to keep in mind is make sure there is no column in date format if so once you have preprocessed it then use these functions.
If any column has values in dataformat then it may give error.
Also in case of a DataFrame kindly ensure the first column is target column or dependent column. Only then start using this function.example of usefrom cleanup.clean import Datacleandf = pd.read_csv('UCI_Credit_Card.csv')dc =Dataclean(df)df1=dc.standardize()print(df1)
Package: datacli
# Data Cli> Productive no-sql data sources exploration with SQL, Pandas and> shortcuts| service               | status        || --------------------- |:-------------:|| travis | [![Build Status](https://travis-ci.org/hackliff/mgocli.svg?branch=master)](https://travis-ci.org/hackliff/mgocli) || quay.io/hackliff/drill | [![Docker Repository on Quay](https://quay.io/repository/hackliff/drill/status "Docker Repository on Quay")](https://quay.io/repository/hackliff/drill) |## Getting started`make container.build` will start and initialize [MongoDB](), [Drill]()and `datacli` with everything installed.It is still possible to register external data sources :```Bash$ cp ./tools/drill/mongo-storage-plugin.json custom-storage.json$ $EDITOR custom-storage.json$ ./tools/drill/manage.sh storage ./custom-storage.json http://$(docker-machine ip default):8047{"result" : "success"}```## Usage### Mgocli```Bash$ ./datacli/__main__.py -H $(docker-machine ip default) appturbo[2016-07-06 06:38:24,564] datacli :: INFO   - connected to Drillbit[ datacli::192.168.99.100::appturbo  ] >>> SHOW DATABASES[2016-07-06 06:38:48,391] datacli :: INFO   - sending query to drill[show databases]SCHEMA_NAME0  INFORMATION_SCHEMA1          cp.default2         dfs.default3            dfs.root4             dfs.tmp```### Drill Repl```sql$ # query data$ docker exec -it $(basename $PWD)_drill_1 /opt/apache-drill-1.7.0/bin/drill-localhost0: jdbc:drill:drillbit=localhost> SHOW DATABASES;+---------------------+|     SCHEMA_NAME     |+---------------------+| INFORMATION_SCHEMA  || cp.default          || dfs.default         || dfs.root            || dfs.tmp             || nosqltest.employee  || nosqltest.local     || nosqltest.test      || sys                 |+---------------------+9 rows selected (0.244 seconds)0: jdbc:drill:drillbit=localhost> SELECT * FROM nosqltest.test.zipsLIMIT 3;+--------+----------+-------------------------+--------+--------+|  _id   |   city   |           loc           |  pop   | state  |+--------+----------+-------------------------+--------+--------+| 01001  | AGAWAM   | [-72.622739,42.070206]  | 15338  | MA     || 01002  | CUSHMAN  | [-72.51565,42.377017]   | 36963  | MA     || 01005  | BARRE    | [-72.108354,42.409698]  | 4546   | MA     |+--------+----------+-------------------------+--------+--------+3 rows selected (0.325 seconds)0: jdbc:drill:drillbit=localhost> !quit```
Package: data-cliff
Data CliffCLI tool to show differences on text data tracked by dvc.Free software: Apache Software License 2.0Documentation:https://data-cliff.readthedocs.io.InstallationYou can install the tool directly frompip:pip install data_cliffQuick start-guidedata_cliffis a tool that helps you visualize differences on text file you’re
tracking on advcrepo. Its CLIcliffaims to be as closed togit diffas possible:cliff [a_rev] [b_rev] [path/to/file]WARNINGYou must be in advcrepository.CreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.0.1 (2023-02-17)First release on PyPI.
Package: datacls
🗂datacls: take the edge offdataclass🗂dataclassesis almost perfect.dataclsis a tiny, thin wrapper arounddataclass.dataclassesmaking it
a bit more self-contained, reflective, and saving a bit of typing.dataclsis exactly likedataclass, except:Adds three new instance methods:asdict(),astuple(),replace(),
and one new class method,fields(), all taken from thedataclassesmodulexmod-ed for less cruft (sodataclsis the same asdatacls.dataclass)The default class isdatacls.immutablewherefrozen=True.Exampleimport datacls

@datacls
class One:
    one: str = 'one'
    two: int = 2
    three: dict = datacls.field(dict)

# `One` has three instance methods: asdict(), astuple(), replace()

o = One()
assert o.asdict() == {'one': 'one', 'two': 2, 'three': {}}

import dataclasses
assert dataclasses.asdict(o) == o.asdict()

assert o.astuple() == ('one', 2, {})

o2 = o.replace(one='seven', three={'nine': 9})
assert o2 == One('seven', 2, {'nine': 9})

# `One` has one new class method: fields()

assert [f.name for f in One.fields()] == ['one', 'two', 'three']

# @datacls is immutable.

try:
    o.one = 'three'
except AttributeError:
    pass
else:
    raise AttributeError('Was mutable!')

# Usec @datacls.mutable or @datacls(frozen=False)
# for mutable classes

@datacls.mutable
class OneMutable:
    one: str = 'one'
    two: int = 2
    three: Dict = datacls.field(dict)

om = OneMutable()
om.one = 'three'
assert str(om) == "OneMutable(one='three', two=2, three={})"

# These four new methods won't break your old dataclass by mistake:
@datacls
class Overloads:
    one: str = 'one'
    asdict: int = 1
    astuple: int = 1
    fields: int = 1
    replace: int = 1

o = Overloads()

assert ov.one == 'one'
assert ov.asdict == 1
assert ov.astuple == 1
assert ov.fields == 1
assert ov.replace == 1

# You can still access the methods as functions on `datacls`:
assert (
    datacls.asdict(ov) ==
    {'asdict': 1, 'astuple': 1, 'fields': 1, 'one': 'one', 'replace': 1}
)API Documentation
Package: dataco
Install$python-mpipinstalldatacoExamples>>>fromdatacoimportS3,GDrive>>>s3=S3('bucket_name')>>>df=s3.read_csv('file_name.csv')>>>df=s3.read_excel('file_name.xlsx',sheet_name='sheet_name')>>>drive=GDrive('credential_file')>>>files=drive.search_files_by_name('cities.csv')>>>df=drive.read_csv('csv_file_id')>>>df=drive.read_excel('excel_file_id','sheet_id')