Saravanan666/pypi-org huggingface dataset:
Package: datatree
SummaryDataTree is a DSL for creating structured documents in python. Inspired byruby builderbut with the goal of reducing the amount of line noise associated
with creating XML documents in python.  As an added bonus the tree can be output
to to any structured format (with XML, JSON and YAML supported in the library).Note:More documentation is coming soon but for now a very basic rough draft can be
found atdatatree.readthedocs.org.InstallationYou can install viaPyPior direct
from thegithubrepo.To install with pip:$ pip install datatreeTo install with easy_install:$ easy_install datatreeExampleA small example:from datatree import Tree, Node

tree = Tree()
with tree.author() as author:
    author.name('Terry Pratchett')
    author.genre('Fantasy/Comedy')
    author // "Only 2 books listed"
    with author.novels(count=2) as novels:
        novels.novel('Small Gods', year=1992)
        novels.novel('The Fifth Elephant', year=1999)
        novels << Node("novel", "Guards! Guards!", year=1989)

print tree(pretty=True)Which produces the XML:<author>
    <name>Terry Pratchett</name>
    <genre>Fantasy/Comedy</genre>
    <!-- Only 2 books listed -->
    <novels count="2">
        <novel year="1992">Small Gods</novel>
        <novel year="1999">The Fifth Elephant</novel>
        <novel year="1989">Guards! Guards!</novel>
    </novels>
</author>Or the JSON:{
    "author": {
        "genre": "Fantasy/Comedy",
        "name": "Terry Pratchett",
        "novels": [
            "Small Gods",
            "The Fifth Elephant",
            "Guards! Guards!"
        ]
    }
}Or the YAML:author:
  genre: Fantasy/Comedy
  name: Terry Pratchett
  novels: [Small Gods, The Fifth Elephant, Guards! Guards!]LicenseThis work is licensed under theApache License, Version 2.0.Souce CodeThe source code can be found ongithub.FeedbackI welcome any and all constructive feedback.  Feel free to contact me (Jason Webb) atwww.bigjason.comor on twitter@bigjasonwebb.ContributingContributions are welcome.  Just fork ongithuband I will try to be as responsive
as possible.
Package: dataTXT
datatxtis a library to query a dataTXT service and to enrich annotations
with queries to DBpedia SPARQL endpoint. It will automatically
convert literals to the corresponding Python types.Visithttp://spaziodati.eu/dataTXTfor documentation and
examples.APIFirst you instantiate a dataTXT class:d = datatxt.Datatxt()Then you ask for annotation:result = d.annotation(text)You will receive a dictionary with annotation as DBpedia resources and for
each resource you will receive all properties found in the DBpedia triplestore.The library will automatically convert typed literals to a coresponding
simple type in Python. Dates are also converted if thedateutillibrary is
available.LicenseThis software is licensed under BSD license, included as LICENSE.txt in the
source distribution.
Package: datatype
For full package documenation, please see:http://datatype.readthedocs.org/Examples>>> from datatype.validation import failures

>>> datatype = {'foo': [{'bar': 'int'}]}
>>> bad_value = {'foo': [{'bar': 'baz'}], 'bif': 'pow!'}

>>> failures(datatype, bad_value)
['unexpected property "bif"', 'foo[0].bar: expected int, got str']Wildcard dictionary keys:>>> datatype = {'_any_': ['int']}
>>> good_value = {'foo': [1, 2, 3], 'bar': [3, 4, 5]}

>>> failures(datatype, good_value)
[]Coercion:>>> from datatype.coercion import coerce_value

>>> coerce_value(['str'], [1, 2, 3])
['1', '2', '3']Copyright and LicenseCopyright 2011-2012 LearningStation, Inc. and Adam WagnerLicensed under the BSD-3 License.  You may obtain a copy of the License in the
LICENSE file.
Package: datatyping
Datatyping is a (pure) Python library with no dependencies that you can use to
verify whether elements in a data structure has the expected types. Great for
incomming json.importdatatypingdatatyping.validate([int],[1,2,3])Check out thedocumentationfor examples,
installation instructions and usage.Develop with me :)Fork the repo first. Use the following lines to setup$gitclonehttps://github.com/your_name/datatyping$cddatatyping$virtualenvvenv$.venv/bin/activate$pythonsetup.pydevelopRun tests$pipinstallpytest$python-mpytestBuild documentation$make-Cdocs/htmlNotesInspired by“How Python Makes Working With Data More Difficult in the Long Run”.Any and all contributions are welcome.Please open an issue if you need help (read: I messed up).Suggest anything you want to see support for!
Package: data-uri-parser
data_uri_parserData URI manipulation made easy.This isn't very robust, and will reject a number of valid data URIs. However, it meets the most useful case: a mimetype, a charset, and the base64 flag.How to installpipinstalldata_uri_parserParsing>>>uri=DataURI('data:text/plain;charset=utf-8;base64,VGhlIHF1aWNrIGJyb3duIGZveCBqdW1wZWQgb3ZlciB0aGUgbGF6eSBkb2cu')>>>uri.mimetype'text/plain'>>>uri.charset'utf-8'>>>uri.is_base64True>>>uri.data'The quick brown fox jumped over the lazy dog.'Note thatDataURI.datawon't decode the data bytestring into a unicode string based on the charset.Creating from a string>>>made=DataURI.make('text/plain',charset='us-ascii',base64=True,data='This is a message.')>>>madeDataURI('data:text/plain;charset=us-ascii;base64,VGhpcyBpcyBhIG1lc3NhZ2Uu')>>>made.data'This is a message.'Creating from a fileThis is really just a convenience method.>>>png_uri=DataURI.from_file('somefile.png')>>>png_uri.mimetype'image/png'>>>png_uri.data'\x89PNG\r\n...'NotesOriginally from:https://gist.github.com/zacharyvoase/5538178
Package: data-url
data_url.pydata_url is a Python 3 library which provides easy methods for creating and working withdata URL's. The full API documentation is availablehere.ExamplesCreating a data URLIf all you need is to create a URL and nothing else there is a shortcut method included so you don't need to directly instantiate the DataURL class.importdata_urlwithopen('image.jpeg','rb')asimage:data=image.read()url=data_url.construct_data_url(mime_type='image/jpeg',base64_encode=True,data=data)If you need the information to persist it is recommended to instantiate a class through one of the factory methods onDataURLimportdata_urlwithopen('image.jpeg','rb')asimage:data=image.read()url=data_url.DataURL.from_data('image/jpeg',True,data)print(str(url))You can access the full data URL by either converting the DataURL object to a string as above or by accessing theurlattribute.Retrieving data from a URLGiven you already have a data URL you can instantiate a DataURL object and retrieve each individual attribute.importdata_urlraw_url="data:application/json;base64,ewogICJ0ZXN0IjogMTIzCn0K"url=data_url.DataURL.from_url(raw_url)print(url.mime_type,url.is_base64_encoded,url.data)
Package: dataurls
dataurlsdataurls isa very simple but fully featured Data URL (also known as data URI) encoder for Python.written in Python Standard Librarydataurls supports toencode data in Data URLsInstallationpipinstalldataurlsQuickstartimportdataurlsLicensedataurls has a BSD-3-Clause license, as found in theLICENSEfile.ContributingChangelog
Package: datautil
Utilities for Data Work=======================The datautil package provides various utilities for working with data:* cache: Url caching and scraping* tabular/*: Processing and transforming tabular data to and from variousformats including csv, json, google spreadsheets, xls* misc, date: Cleaning up and parsing data especially dates.* id: ID generation and shortenening* clitools.py: Command line tools such as creating optparse object and usagefrom a module of object.* deliveranceproxy.py: Deliverance proxy helperCHANGELOG=========v0.4 2011-??-??---------------* Rename swiss to datautilv0.3 2010-08-01---------------* Support for google docs spreadsheets as sources for TabularData* Improve documentation of date module and add FlexiDate.as_datetime()* New clitools module incorporating existing cli tools* deliveranceproxy.py: Deliverance proxy helper for proxying to remotewebsites and retheming with deliverance.* parse/name.py: new (human) name parsing codev0.2 2009-10-23---------------* Extensive refactoring of tabular module/package* Standardized interface with BaseReader and BaseWriter* JsonReader and JsonWriter providing json reading and writing* TxtWriter to support writing to plain text* Improvements to date parsing (support for circa, 'c.', etc)* New id module to do 'compression' of uuids using 32 and 64 bit encodingv0.1 2009-06-03---------------* Bring together existing code (from last 2+ years) into new 'datautil' package* Url caching and scraping* Tabular data handling including csv reader/writer, xls reader, latex writerand associated utilities (such as pivot_table)* Cleaning and parsing data especially dates (misc and date modules)
Package: data-utilities
Data UtilitiesData utilities library focused on machine learning and data analysis.The library relies upon python's scientific/numeric stack to expand their
capabilities. The dependencies are:numpyscipypandasmatplotlibseabornscikit-learnOptional dependencies are:XGBoostdeapHighlights are:matplotlib_utilities: out-of-the-shelf data description withhistogram_of_dataframe.pandas_utilities: easier dataframe preparation withrename_columns_to_lower,categorical_serie_to_binary_dataframe,balance_ndframeandget_numeric_columns.sklearn_utilities: multiprocessing and persistence support for hyper
parameter grid search, both exhaustive and using a genetic algorithmic
approach; convenience functions to the XGBoost module.And much more.Organization and files./data_utilities
├── __init__.py
├── matplotlib_utilities.py
├── pandas_utilities.py
├── python_utilities.py
├── sklearn_utilities
│   ├── evolutionary_grid_search.py
│   ├── grid_search.py
│   └── __init__.py
└── tests
    ├── __init__.py
    ├── test_matplotlib_utilities.py
    ├── test_pandas_utilities.py
    ├── test_python_utilities.py
    ├── test_sklearn_utilities.py
    └── test_support.pyEach of python's significant data modules has its own set of functions.
Optional dependencies functions are interspersed throughout the code.This module does not intend to create its own API or standards. Instead each of
the utilities module should follow the guidelines and APIs provided by the
parent module.Note: This is a primitive project. Expect backwards incompatible changes as I
figure out the best way to to develop the utilities.What's newCheck ourchangelog.Development guidelinesCoding style:PEP 8compliant.Docstrings:google docstrings.Before commiting new versions do a test for different versions of python3:python3.4python3.5python3.6(newer versions)Rationale: even though stability is expected between python versions some
changes occur. See for instance that on commit v1.2.8 (60573d7) there was
as unexpected import error on python34 but not on python36.
Package: datav
DataVThis is an unofficial solution to draw map using Aliyun's DataV.GeoAtlas.这是一个非官方的使用阿里云 DataV.GeoAtlas 绘制地图的解决方案。ExamplesSeeexamplesfor more.importcartopy.crsasccrsimportcartopy.featureascfeatureimportmatplotlib.pyplotaspltimportnumpyasnpfromcartopy.mpl.gridlinerimportLATITUDE_FORMATTER,LONGITUDE_FORMATTERfrommatplotlib.transformsimportoffset_copyfrommatplotlib.font_managerimportFontPropertiesfromdatavimportDataVGeoAtlasFeature# Windows 下字体可以不用这么麻烦，直接 rc 设置 SimSun 即可simsun=FontProperties(fname="/mnt/c/Windows/Fonts/simsun.ttc")proj=ccrs.PlateCarree()fig=plt.figure(figsize=(14,7))main_box=(80.5,82,40.25,41)ax_main=plt.subplot(1,1,1,projection=proj)ax_main.set_extent(main_box,crs=ccrs.PlateCarree())ax_main.add_feature(DataVGeoAtlasFeature(659002,facecolor="darkgreen",alpha=0.2))# 阿拉尔ax_main.add_feature(cfeature.LAKES)ax_main.add_feature(cfeature.RIVERS)site=(81.196,40.624)ax_main.plot(*site,marker="o",color="green",markersize=10,transform=ccrs.Geodetic(),)geodetic_transform=ccrs.Geodetic()._as_mpl_transform(ax_main)text_transform=offset_copy(geodetic_transform,units="dots",x=-25)ax_main.text(site[0],site[1]+0.03,"试验站",transform=text_transform,fontproperties=simsun,)xmain=np.linspace(*main_box[:2],4)ymain=np.linspace(*main_box[2:],4)ax_main.gridlines(xlocs=xmain,ylocs=ymain,linestyle=":")# set custom formatting for the tick labelsax_main.xaxis.set_major_formatter(LONGITUDE_FORMATTER)ax_main.yaxis.set_major_formatter(LATITUDE_FORMATTER)ax_main.yaxis.tick_right()ax_main.set_xticks(xmain,crs=ccrs.PlateCarree())ax_main.set_yticks(ymain,crs=ccrs.PlateCarree())ax_inset=fig.add_axes([0,0.6,0.3,0.35],projection=proj)ax_inset.set_extent((73,136,2,51))ax_inset.add_feature(DataVGeoAtlasFeature(100000,full=True,facecolor="lightgreen",edgecolor="gray"))ax_inset.add_feature(DataVGeoAtlasFeature(650000,facecolor="limegreen"))ax_inset.add_feature(DataVGeoAtlasFeature(652900,facecolor="green",edgecolor="none"))ax_inset.add_feature(DataVGeoAtlasFeature(659002,facecolor="darkgreen",edgecolor="black"))xinset=np.linspace(73,136,5)yinset=np.linspace(2,51,5)ax_inset.gridlines(xlocs=xinset,ylocs=yinset,linestyle=":")ax_inset.xaxis.set_major_formatter(LONGITUDE_FORMATTER)ax_inset.yaxis.set_major_formatter(LATITUDE_FORMATTER)ax_inset.xaxis.tick_top()# set inset tick labelsax_inset.set_xticks(xinset,crs=ccrs.PlateCarree())ax_inset.set_yticks(yinset,crs=ccrs.PlateCarree())plt.savefig("research_site.png",dpi=800)RequirementsCartopy (for ploting)Geopandas (for parsing geojson)NOTEFree For Educational Use. Respect Aliyun's license.仅供学习使用。遵守阿里云的协议。请合理使用地图数据。数据内容和本人无关。数据具体用途也和本人无关。请遵守相关法律规定。TODOrespect the HTTP header last-modified尊重 HTTP 请求头 last-modifiedChinese name alias for adcode使用中文名替代 adcode 查询More features such as line从 GeoJSON 中提取更多可定制的 feature。
Package: datavacuum
datavacuumCLI for data cleaningFree software: Apache Software License 2.0Documentation:https://datavacuum.readthedocs.io.FeaturesTODOCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2019-08-06)First release on PyPI.
Package: datavalues
Simple package for managing data units including conversions and
operationsInstallation# PyPi Installationpipinstalldatavalues# GitHub Installationpipinstallgit+'https://github.com/Scraps23/datavalues.git'Getting StartedThe package is a collection of sub-classed object classes for each data
unit. To import all the object classes, use the import below in your
code:fromdata.unitsimport*This is the import method used for all examples below. ### Object
Classes There are nine 9 object classes, each named for the unit they
represent. - Byte - KiloByte - MegaByte - GigaByte - TeraByte - PetaByte
- ExaByte - YottaByte - ZettaByte ## Usage Each of the object classes
sub-classes the BaseDataModel which allows for arithmetic and comparison
operators, and a sharedconvertmethod. ### Conversion The
conversion method is non-destructive and relies upon the class name of
the data object. Changing the names of the class(es) on import will
break functionality.# Converting the object does not alter it, it returns a new objectas_mb=MegaByte(100)as_gb=as_mb.convert('gb')print(as_mb,as_gb)## RETURNS:100MB0.1GBimportos# Conversion can be done in-line to reduce memory usageas_gb=MegaByte(100).convert('gb')# This is especially useful for bytes-based systems and human-readable input being mergedselected_size=GigaByte(float(input('How many gigabytes? : ')))os.environ['disk_size_var']=selected_size.convert('b')OperatorsData objects can have math applied against them, and be compared to each
other and int/float objects to simplify operations like calculating
total disk usage, RAID viability, and more.Arithmetic OperatorsThe mathematical operators allow objects of different classes to
interact; if another data object is supplied as the other value, both
values are converted to bytes, evaluated, and converted back to the
original unit. Otherwise, if an integer or float is supplied, it is
assumed that value is in the same unit as the original unit.current_disk=GigaByte(1270)end_goal=TeraByte(2)# Will return in Terabytes (0.73 TB)print(end_goal-current_disk)# Will return in Gigabytes (730.0 GB)print((end_goal-current_disk).convert('gb'))Comparison OperatorsThe data objects can also be compared to each other using the comparison
operators (i.e. >, <, >=, etc). In this case, they similarly convert
both values to bytes and compare that float object. Otherwise, if a
float or int is supplied as the comparator, then it is assumed the
number is in the same unit as the object being compared.current_disk=GigaByte(1270)ifcurrent_disk>1000:print(current_disk.convert('tb'))else:print(current_disk)Returning ValuesThe__str__method returns the value in a human-readable format
which allows for clean output in code.The__repr__method returns the creation string for the object.disk1=Byte(150000000000).convert('gb')disk2=MegaByte(328000).convert('gb')# Printing the disks returns the human-readable valueprint(','.join([disk1,disk2]))## RETURNS:150.0GB328.0GB# The object itself returns its creation stringdisk1## RETURNS:GigaByte(150.0)
Package: dataverse-utils
Dataverse utilitiesThis is a generalized set of console utilities and Python 3.6+ package which help with managingDataverserepositories. This means Harvard University's data repository software, not Microsoft's Dataverse product.With these utilities you can:Upload your data sets from a TSV spreadsheetBulk release multiple data setsBulk delete (unpublished) assetsQuickly duplicate recordsReplace licencesand more!All without touching the Dataverse GUI.While the defaults favour a particular Dataverse instance (ie, theAbacus data repositoryat theUniversity of British Columbia Library), they are designed to work with any installation / Dataverse user.Quick installpip install dataverse_utilsDocumentation and API referenceUser-friendly documentation is available athttps://ubc-library-rc.github.io/dataverse_utils. This includes complete descriptions of all console utilities and a complete API reference.Source codeThe primary repository for this software is athttps://github.com/ubc-library-rc/dataverse_utils.
Package: dataview
dataviewis a module that allows you to create views of your sequences or its slicesInstallpip install dataviewTo upgrade a previous installation, use:pip install dataviewUsage && Examples>>> from dataview import DataView
>>> # You have some data, it can be any sequence (str, list, bytes, tuple, etc..)
>>> source_data = list(range(5))
>>> # DataView is just a pointer to your source_data and start/stop/step
>>> DataView(source_data)
[0, 1, 2, 3, 4]
>>> DataView(source_data, 3)
[0, 1, 2]
>>> DataView(source_data, 1, 5)
[1, 2, 3, 4]
>>> DataView(source_data, None, None, -1)
[4, 3, 2, 1, 0]
>>> # You can use slices (completely the same way as list slices)
>>> DataView(source_data)[::-1]
[4, 3, 2, 1, 0]
>>> # Slice return a new DataView object, that points to the previous DataView
>>> DataView(source_data)[::-1][2:4]
[2, 1]
>>> # You can change start/stop/step anytime
>>> view = DataView(source_data, 0, 1)
>>> view.start = 1
>>> view.stop = 2
>>> view
[1]
>>> # View always points to actual data
>>> source_data[1] = 2
>>> view
[2]
>>> # You can change a view source data
>>> view.data = list(range(5))
>>> view
[1]
Package: data-view
DescriptionRequirementspython >= 3.6numpy >= 1.16pandas >= 0.23matplotlib >= 3.02IPython >= 7.2.0Module, provides the functionview, which displays general information on the data:Result of method info()Result of method describe()for numeric / categorical variables - The number of nulls in the data (amount and percentage for each column)grid of histograms for numeric variablesTop-5 of the most frequent categorical variables (for each)Split count statistic by category with less 5 unique valuesmatrix correlationdataframe of higher correlation pair (over 0.7)Parameters (function view):d - table with dataonly_numeric - True / False, default: True. True - information output only by numerical variables, False - information output by numerical and categorical variables.full_stats - True / False, default: False. False - output information on numerical variables without interquartile range, data boundaries without outliers, True - complete output with IQR, min and max by IQR and amount of outliers.histograms - True / False, default: True. True - output with building histograms for numerical variables, False - without building histogramsTop-5 elements of categorical variablesThe postfix (_name / _count) is assigned to the name of the data column to create top-5 elements table:_name- category name_count- number of unique elements in this category. If there are less than 5 unique elements, then the values ​​in the _count field are filled -1Count statistic of numbers variables split by some categorical variablesif number of unique elements in category variable less 5, then show count statisticCorrelation matrixCorrelation matrix with heatmap (pearson corr index)Install$ pip install data_viewUsage$ python3

from data_view import *

d = pd.DataFrame(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), columns=['a', 'b', 'c'])

view(d, only_numeric=True, histograms=False)
Package: dataviper
dataviperExampleWhy?Data SourcesAPIsprofilepivotjoinabilityIssues and TODOsdataviperdataviperis a SQL-based tool to get the basic data preparation done in easy way, with doingCreate "Data Profile" report of a tableOne-hot encode for "Categorical Columns" and create a "one-hot" tableCheck "Joinability" between 2 tables// TODO: and moreExamplepipinstalldataviperYourmain.pywill look like thisfromdataviperimportClientfromdataviper.sourceimportMySQLclient=Client(source=MySQL({'host':'localhost','user':'root','password':'xxxxxx','database':'your_database'}))withclient.connect():profile=client.profile('Your_Table')profile.to_excel()python3main.py# Then you will get 'profile_Your_Table.xlsx' 🤗Why?It's known that "Data Profiling" needs to be done with scanning all the rows in a table. If you try to do this naively bypandasor any libraries which internally usepandas, it's not avoidable to use bunch of memory of your local machine and freeze your work.dataviperis a SQL-based Data Profiling tool, which simply and dynamically generates SQLs and lets the database machine do the annoying calculation.Withdataviper, you don't have to have massive local computer. All you need are a stable network and reachable SQL db.Data SourcesYou can choose your data source fromSQL ServerprofilepivotjoinabilityhistogramMySQLprofilepivotjoinabilityhistogramPostgreSQLCSVprofilepivotjoinabilityhistogramExcelAPIsprofileCreate "Data Profile" excel file of a specified table.When you haveSalestable like thisidregionsales_typepricerep_id1jpphone2401157232usweb901259013jpweb56080034usshop9201822345jpNULL90922316usshop1801004257usshop7052934dowithclient.connect()asconn:table_name='Sales'profile=client.profile(table_name,example_count=3)profile.to_excel()then you will getprofile_Sales.xlsxfile withcolumn_namedata_typenull_countnull_%unique_countunique_%minmaxavgstdexample_top_3example_last_3idbigint007100.00174.02.0[1,2,3][5,6,7]regionvarchar00228.57[jp,us,jp][jp,us,us]sales_typevarchar114.28342.85[phone,web,web][None,shop,shop]priceint00685.7170920307.1428295.379[240,90,560][90,180,70]rep_idint007100.00800318223496778.714251195.79065[115723,125901,8003][92231,100425,52934]pivotSpread categorical columns to N binary columns.When you haveSalestable like above, dowithclient.connect()asconn:table_name='Sales'key='id'categorical_columns=['region','sales_type']profile=client.get_schema(table_name)client.pivot(profile,key,categorical_columns)then you will getSales_PIVOT_YYYYmmddHHMMtable withidregion_jpregion_ussales_type_phonesales_type_websales_type_shop110100201010310010401001510000601001701001joinabilityCount how much 2 tables can be joined.When you haveSalestable like above, andRepstable like thisidnametenure8003Hiromu98972Ochiai652934Taro192231otiai102100425Hanako7125901Chika3182234Mary5199621Jack1dowithclient.connect()asconn:report=client.joinability(on={'Sales':'rep_id','Reps':'id'})report.to_excel()then you will getjoin_Sales_Reps.xlsxfile withtablekeytotalmatchmatch_%dropdrop_%Sales[rep_id]7685.714114.285Reps[id]8675.00225.00Issues and TODOshttps://github.com/otiai10/dataviper/issues