Saravanan666/pypi-org huggingface dataset:
Package: dStorage
dStorage.asl-sl (São Lourenço - MG, Brazil)Academia do software livre.São Lourenço, Minas Gerais - Brazil.the goal of the dStorage meta class is study and implements some other classes.The project structure need a special atention for learn in github usage.the dStorage class give some funcions to create databases, and automaticaly
generate a gui and console interface for the data class and database.Examples of usage are included in examples directory.first versions.My first studies are turned to only build a simple environment.This environment create SQLite database, and make one integration with this database and TkGUIandCLIsimple iterfaces.On the second moment I try made one CLI interface on same database controlled by base class structures.Now I need learn and write the package structure on the right way, following the rules to
devel python applications. at moment this software work, but I think with it is only an
outline.
Older versions creates database file on user home directory, the recent versions have a property to make these paths.
Also on the recent versions I try improve dialogs properties, now is possible change window color for inputGUIdialog. Also you can change de color of outputGUIdialog.This goal, are satisfied.The base class, is based in one multidimensional array data type.That show a simple use of these data types.The integration withCLIandGUIinterface are builded by base class.The base class provide, insert and select funcions.Also are provide one simple find method.One rule for base class use, is at moment, is needed one data member "id" on any derived class.Current.The current version have insert, select, drop, data functions.Also have some good options like optimization of file paths and colors for dialogs onGUI.The library can be installed using pythonpip.python -m pip install --user dStoragethe actual upstream release is on currentgitrepositorybranch.contact/contato:I have some time studying various tecnologic related sciences and aspects.During a long time I teach use, administration and development with some UNIX Like operating systems.Excuse some troubles in my codes, I try be better....Grateful.
Special thanks to python comunity for the oportunity to colaborate.
Excuse for my troubles in package construction, I need more pratice and for me is hard to restart on the right way. Some lectures are not clearly for my compreension, and I only can say forgive me for my limitations.Website for my courses and studies:Here I provide some studies, projects related to tecnology. I teach in my small city during some time and already have some good stories about students. Here I am one of the first to use "UNIX" LikeOsesand try expand and estimulate these culture.On my courses and studieswebsiteI useportuguese,my natural language.Website search engine projects:Here I install one public seaechenginefor studies end learning purposes.The software used, also promotes one metasearch, and small crawler.You are welcome to known and colaborate.
Package: dstr
dstrDeveloper GuideSetup# create conda environment$mambaenvcreate-fenv.yml# update conda environment$mambaenvupdate-ndstr--fileenv.ymlInstallpipinstall-e.# install from pypipipinstalldstrnbdev# activate conda environment$condaactivatedstr# make sure the dstr package is installed in development mode$pipinstall-e.# make changes under nbs/ directory# ...# compile to have changes apply to the dstr package$nbdev_preparePublishing# publish to pypi$nbdev_pypi# publish to conda$nbdev_conda--build_args'-c conda-forge'$nbdev_conda--mambabuild--build_args'-c conda-forge -c dsm-72'UsageInstallationInstall latest from the GitHubrepository:$pipinstallgit+https://github.com/dsm-72/dstr.gitor fromconda$condainstall-cdsm-72dstror frompypi$pipinstalldstrDocumentationDocumentation can be found hosted on GitHubrepositorypages. Additionally you can find
package manager specific guidelines oncondaandpypirespectively.PyTorch Documentation:TorchDataHow to Package PyTorch
Modelstorch.monitor.Eventtorchvisiontorchvision.Datasets.VisionDatasettorchvision.utils.flow_to_imagePyTorch Models to Consider:Diffusion Video
AutoEncodersConvLSTM
AutoEncoderRecurrent All Pairs Field Transforms for Optical
FlowOptical Flow Toolbox
(mmflow)torchvision.models.optical_flow.raft
Package: dstree
A module for visualization of a nested python data structurea nested python objects including list, tuple, dictionary, tensor, ndarray to data-tree, and make a quick peek out of its structuredata = a list of 10 lists of 3 dicts of 2 keys : list of 2 strings

dv.viz(data)

$ list of 10
$ └── list of 3
$     └── dict of 2
$         ├── train : ['data1', 'data2']
$         └── test : ['data3', 'data4']How to useinstall$ pip install dstreeimport and quick_startimportdstreeasdstdata=dst.get_example_data()tree=dst.viz(data)usage in detailimportdstreeasdstdst.quick_start()# Copy & Paste console output to set up the tutorial codesNoteThis package is built based on treelib & graphviz syntax.Supports int, str, float, list, tuple, dict, Tensor, ndarray, Series, DataFrame datatype.And also custom data classes are supported.
Package: dstruct
dstructis designed to map larger data structures onto a smaller ones, which is simple in principle,
but can be complicated in practice - sifting through robust datasets is difficult when the structure
is highly nested, or relevant information is fractured. However, the module’s intuitive api makes pruning
useless data, and parsing its relevant subsets easy.enables complex data parsingstreamlines data trimming heuristicsintuitive api for nested data structuresThe implementation relies on thedescriptorpattern.SeeGitHubfor more info
Package: dst-server-deploy
dst-server-deployA Python package for generating server files for use withDon't Starve Together Docker Images.The image requires docker engine and docker compose to be installed on the host system.This script is indended to be run in the directory in which you want to store the server data to generate most generic servers.
If you need to tune the server after generation, lookherefor reference.How to usePackage Installationpip install dst-server-deployBuild Server Filesdst-server-deployNotesCli arguments may be used to bypass the full questionnaire (See-hflag for more information)The script assumes you are running a single world (forest + caves) on your machine. If you are running multiple servers, you will need to adjust ports accordingly.Deploy Server (requires docker engine and docker-compose)docker-compose up -dReference files.For an example of the files which may be read into the script, lookhere.ContributionDo you want to contribute to the project? Check out thecontribution guide.
Package: ds-tut
ds_tutSome basic infrastructure for aData Science
TutorialInstallpipinstallds_tutInstall Development Versiongitclonehttps://github.com/ephes/ds_tutcdds_tut
python-mpipinstall-e".[dev]"How to useDownload DatasetsfrompathlibimportPatharchive_name="reuters21578.tar.gz"training_data_url="http://www.daviddlewis.com/resources/testcollections/reuters21578/{}".format(archive_name)data_root=Path.cwd()/"data"training_data_path=data_root/archive_namedata_size=download_from_url(training_data_url,training_data_path)print(data_size)8150596
Package: dsu-data-structure
DSU - disjoint-set-unionInfoSee the disjoint set union data structure description at thelink.This implementation is built on sets and not references to parents. Therefore, the complexity of the operation is different from the article. Also added some useful methods.Basic methods and their complexity:same O(m)- check items from the same set;union O(m + n)- union sets into one set based on their items;make O(m)- make a new set for an item or separate this item from its set;remove_items O(m)- remove an item from a structure;remove_sets O(m + n)- remove a set from a structure based on its item.Where:m- number of method parameters (items), usually 1-2;n- number of items in the DSU.Additional methods allow:get sets based on their items;get all items from sets;get a copy DSU;clear the DSU of all items;check the DSU for emptiness;get number of items;get string representation;compare two structures DSU;check whether an item is contained in one of the sets;get iterator over all items.PrerequisitesLanguage version starting from 3.10+$python--version
Python3.10.4Installationpipinstalldsu_data_structureUsageOpen thefolderto view details.>>>fromdsu_data_structureimportDSU>>>dsu=DSU(range(6))>>>print(dsu)[{0},{1},{2},{3},{4},{5}]>>>print(dsu.items()){0,1,2,3,4,5}>>>dsu.union(0,1,2,3)>>>print(dsu)[{0,1,2,3},{4},{5}]>>>dsu.same(0,2)True>>>dsu.same(0,5)False>>>dsu.make(0,10)>>>print(dsu)[{0},{1,2,3},{4},{5},{10}]>>>dsu.remove_items(0,2)>>>print(dsu)[{1,3},{4},{5},{10}]>>>dsu.remove_sets(3)>>>print(dsu)[{4},{5},{10}]AuthorsDmitry VolkovLicenseMIT License - see theLICENSEfile for details
Package: dsutil
python-ds-utilInstallationThe library is available onpip:pip install dsutilExamplesFor full usage examples see notebooks underthe examples directory.PlottingFull examples for plottinghereadd_grid(): reasonable default grid settings, with weak grey lines, light alpha, etc.importnumpyasnpimportmatplotlib.pyplotaspltfromdsutil.plottingimportadd_gridx=np.linspace(0.0,100,10)y=np.random.uniform(low=0,high=10,size=10)plt.bar(x,y)add_grid()add_value_labels()annotates barplots, line plots and scatter plots with values for the coordinatesimportnumpyasnpimportmatplotlib.pyplotaspltfromdsutil.plottingimportadd_value_labelsx=np.linspace(0.0,100,10)y=np.random.uniform(low=0,high=10,size=10)plt.bar(x,y)add_value_labels()format_yaxis_percentage(): turns values between 0 and 1 in y-axis into percentagesimportnumpyasnpimportmatplotlib.pyplotaspltfromdsutil.plottingimportformat_yaxis_percentagex=np.linspace(0.0,100,10)y=np.random.uniform(low=0,high=1,size=10)plt.bar(x,y)plt.yticks(np.arange(0,1.01,0.1))format_yaxis_percentage()format_yaxis_thousands(): uses commas as thousands separator in the y-axis labelsimportnumpyasnpimportmatplotlib.pyplotaspltfromdsutil.plottingimportformat_yaxis_thousandsx=np.linspace(0.0,100,10)y=np.random.uniform(low=10000,high=100000,size=10)plt.bar(x,y)plt.yticks(np.arange(0,100001,10000))format_yaxis_thousands()
Package: dsutp-custom-logger
НазначениеЭто обычный logging, но с более широкими настройкамиПрименениеfrom dsutp_custom_logger import Logging
log = Logging().get_logger()
log.info('Test my logs')Описание параметров класса Logginglog_file_enabled - запись логов в файл. Принимает булево значение True/False. Default = Truelog_console_enabled - запись логов в консоль. Принимает булево значение True/False. Default = Truelog_httpsink_enabled - отправка логов в удаленный http. Принимает булево значение True/False. Default = Falselog_httpsink_url - http ссылка до удаленого лог приемника. Принимает строковое значение ссылки. Default = Nonelog_level - уровень логгирования. Принимает строковое значение DEBUG/INFO/WARNINGS ... .Default = INFOlog_buffer_size_file - размер файла в байтах. Принимает целочисленное значение. Default = 5 * 10**7 (50 Мб)backupCount - число файлов с историей. Принимает целочисленное значение. Default = 1stream - потоки. Принимает список кастомных потоков. Default = []log_location - место файлов для логов. Принимает строкове значение. Default = log.logbase_json_fields - "системные" поля для логов. Принимает словарь. Default = Nonecuston_json_fields - кастомные поля для логов. Принимает словарь. Default = Nonelog_structured_datetimeformat - формат даты. Принимает строковое значение. Default = '%Y-%m-%d %H:%M:%S'Подбробное описаниеlog_httpsink_enabled и log_httpsink_urlЕсли log_httpsink_enabled = True и указана log_httpsink_url ссылка, то в зависимости от выставленного порога log_level будет отправлять тело запроса с полями {'level': '', 'message': ''}.streamВы можете определить свой stream. Для этого нужно:from dsutp_custom_logger import Stream, Logging

class YourCustomStream(Stream):
    def write(self, msg: str) -> None:
        do_something

log = Logging(stream = [YourCustomStream()]).get_logger()base_json_fieldsСловарь с базовыми полям, ключи можно изменять, а ключи статичный.
Данные поля заполняются системными данными.{"level": "levelname", 
"message": "message", 
"loggerName": "name", 
"processName": "processName",
"processID": "process", 
"threadName": "threadName", 
"threadID": "thread",
"timestamp": "asctime",
"filename":"filename",
"module":"module",
"pathname":"pathname"}custon_json_fieldsМожно задать свои кастомные поля с уже заполнеными данными{""RequestID":"123",
"Span":"344",
...}ТестыВ папке tests находятся тесты с результатми
Package: dsvisualizer
dsvisualizerA simple data structure visualizer for Jupyter Notebooks.UsageTo visualize your linked list implementation import the library and add the
corresponding decorators to the linked list node and container.You can see an example notebook inintroduction.InstallationYou can install usingpip:pipinstalldsvisualizerIf using Google ColabYou need to run the following snippet to enable custom widgets.fromgoogle.colabimportoutputoutput.enable_custom_widget_manager()Development InstallationCreate a dev environment:condacreate-ndsvisualizer-dev-cconda-forgenodejsyarnpythonjupyterlab
condaactivatedsvisualizer-devInstall the python. This will also build the TS package.pipinstall-e".[test, examples]"When developing your extensions, you need to manually enable your extensions
with the notebook or lab frontend. For Lab, this is done by the command:jupyter labextension develop --overwrite .
yarn run buildFor classic notebooks, you need to run:jupyter nbextension install --sys-prefix --symlink --overwrite --py dsvisualizer
jupyter nbextension enable --sys-prefix --py dsvisualizerNote that the--symlinkflag doesn't work on Windows, so you will here
have to run theinstallcommand every time that you rebuild your extension.
For certain installations you might also need another flag instead of--sys-prefix, but we won't cover the meaning of those flags here.How to see your changesTypescript:If you use JupyterLab to develop then you can watch the source directory and
run JupyterLab at the same time in different terminals to watch for changes
in the extension's source and automatically rebuild the widget.# Watch the source directory in one terminal, automatically rebuilding when neededyarnrunwatch# Run JupyterLab in another terminaljupyterlabAfter a change wait for the build to finish and then refresh your browser and
the changes should take effect.Python:If you make a change to the python code then you will need to restart the
notebook kernel to have it take effect.
Package: dswe
DSWE-PythonDSWEis a python package, written byProfessor Yu Ding'sresearch team, to supplement the Data Science for Wind Energy (DSWE) book and other state-of-the-art methods used in wind energy applications.Documentation:https://tamu-aml.github.io/DSWE-Python/PyPI:https://pypi.org/project/dswe/
Package: dsws-client
dsws-clientPython wrapper for the Datastream Web Services API (DSWS)To Connect to the Refinitiv Datastream database via Datastream Web Services, you need to have a Datastream subscription and a username/password to use this package.Please note that this is an unofficial client not affiliated with Refinitiv.This package includes all functionalities required to get data from DSWS.Why?There are two client libraries for DSWS that I am aware of, the officialDatastreamDSWSandpydatastream. Both of them return data only as pandas dataframes. I needed something that doesn't depend on pandas, so I decided to make this client.
Package: dsw-sdk
DSW SDKIntroductionThis projects aims at providing unified and easy-to-use Python library for
communicating with the Data Stewardship Wizard API. For more info about
the DSW project itself, seeofficial webpageor theAPI documentation.InstallationYou can install this library via PyPI:pip install dsw-sdkQuickstartThe only mandatory step need in order to get going is to initialize the whole
SDK and tell it, where is the DSW API located and how to connect to it:fromdsw_sdkimportDataStewardshipWizardSDKdsw_sdk=DataStewardshipWizardSDK(api_url='http://localhost:3000',email='albert.einstein@example.com',password='password',)Now you are ready to go.Note that this is onlyillustrative exampleand we
encourage younotto store secrets like passwords in the source code.
There are better mechanisms (env variables) introduced in the docs.Basic usageMost actions should be done via the high-level interfaces provided on an
instance of theDataStewardshipWizardSDKclass. These interfaces operate with
subclasses ofModelclass (e.g.user.User) -- these are the DSW data
entities. Basically they are just data classes with bunch of attributes and
methods for saving the entity (save()) on the server and deleting it
(delete()).importosuser=dsw_sdk.users.create_user(first_name='John',last_name='Doe',email='john.doe@example.com',)user.password=os.getenv('SECRET_PASSWORD')user.save()...user.delete()For more advanced usage, see thedocs.ContributingWant to fix a bug or help with implementing new features? Don't hesitate to contactusand read thecontributing doc.
Package: dsw-tdk
dsw-tdkTemplate Development Kit forData Stewardship WizardInstallationPython packageYou can easily install this tool usingpip(fromPyPI):$pipinstalldsw-tdkOptionally, you can clone any version from this repository and install it directly:$gitclonehttps://github.com/ds-wizard/dsw-tdk.git
$pipinstall-e.We recommend usingvirtual environmentsto avoid clashes in dependencies with other projects.Dockerized toolIf you don't want to use Python directly on your machine, you can easily use Docker image with DSW TDK:$ docker run datastewardshipwizard/dsw-tdk --help
$ docker run datastewardshipwizard/dsw-tdk:develop --helpSeedatastewardshipwizard/dsw-tdk on Docker Hubto check available tags.UsageYou can find out possibilities directly using--helpflag:$dsw-tdk--help
$dsw-tdkput--helpFor further information, visit ourdocumentation.Basic commandsnew= create a new template project locally using interactive wizardlist= list templates available in configured DSW instanceget= download a template from DSW instanceput= upload a template to DSW instance (create or update)verify= check the metadata of local template projectpackage= create a distribution ZIP package that is importable to DSW via web interfaceEnvironment variablesYou can use the following environment variables to avoid repeating CLI options.DSW_API_URL= URL of DSW API you want to use, e.g.,https://api.demo.ds-wizard.org(notice that it isnotthe URL of client, you can find it out by clicking Help > About in DSW)Used when--api-urlnot specifiedDSW_API_KEY= API Key of the user authorized to manage document templatesUsed when--api-keynot specifiedYou can also use them in.envfile which is automatically loaded from current directory or specify it using--dot-envoption:$ls-a
....envmy-other-file
$dsw-tdklist
$dsw-tdk--dot-env/path/to/my/.envlistHow to startPrepare your DSW instance and admin account (optionally, prepare.envfile)Verify the connection by issuingdsw-tdk listCreate a new template projectdsw-tdk newor get existingdsw-tdk get(or re-use some local)Go to the template project and make edits you need to doUpdate template in DSW withdsw-tdk put(or continually withdsw-tdk put --watch)(or) Create a distribution ZIP package that is importable via DSW web interface withdsw-tdk packageVerbosityYou can use--quietand--debugflags to toggle less or more output messages:$dsw-tdk--quietlist
$dsw-tdk--debuglistRequirementsPython 3.8+DSW instance with matching version (e.g. a local one usingDocker)Admin credentials (email+password) to the DSW instanceContributingWe welcome any form of feedback and contribution to this tool:Report bugs or ask in case of uncertainty usingGitHub Issues.Share ideas and feature requests usingDSW Ideas site.Submit enhancements usingPull Requests, just please make sure that you comply with used conventions.For more information readCONTRIBUTING.LicenseThis project is licensed under the Apache 2 License - see theLICENSEfile for more details.
Package: dsx
<< Data Science Utilities (DSX)>>Thedsxpackage contains a collection of wrapper functions to simplify common operations in data analytics tasks.
The core module ds_utils (data science utilities) is designed to work with DataFrame in Pandas
to simplify common tasks.The package can be can be used in the following setup:Jupyter NotebookJupyter LabPyCharm's Python ConsoleiPython ConsolePython ScriptIntallationInstallation using Pip:pip install dsxDocumentationFull Documentation Site:Documentation1. Core Module: "ds_utils"The core module is "ds_utils".
The module contains a list of functions that can accomplish
common data analytics tasks with less codes.
Basically, these functions are wrappers for commonly-used methods in Pandas, particularly
methods of DataFrame object.Some of the key features of the DataFrame utility functions are as following:Generate metadata of columns in a DataFrameNumber & percentage of missing valuesNumber & percentage of unique valuesData TypeGenerate accumulated percentage of values in a columnQuick Rename of a single columnReorder columns of a DataFrameStandardize column names into iPython-friendly namesRetrieve column name(s) by a partial keywordExpand concatenated string in a column into child tableVisualize DataFrame objectDataGrid ViewerPivot Table ViewerQuick Analyzer (Pivot table and visualizations)1.1 UsageBelow is example codes for importing the module:fromdsx.ds_utilsimport*There are 2 categories of methods indsx'sclasses, which are to be called in different ways:Methods:Dynamic functions of the class's instanceInvoke through the extended domain ('ds') of the native DataFrame objectdf=pd.read_excel(os.path.join(os.getcwd(),"data.xlsx"))df.ds.isnull("Column_Name")Static functionsStatic functions from the class's objectInvoke as a static function of pd_utils classdf=pd.read_excel(os.path.join(os.getcwd(),"data.xlsx"))dsx.isnull(df,"Column_Name")2. Data Science Workflow "ds_workflow" (Active Development / Work-In-Progress)The"ml_utils"module contains the methods for simplifying common tasks in
a data science workflow. The methods are built on top of the functions in
the core module"pd_utils".Some of the key features of the module are as the following:Get the column name of the features that are categoricalGet the column name of the features that are numericalCreate or merge the dummy variables created from categorical features
with option to use k-1 dummificationData ExplorationGenerate barplot and accumulated percentage report for all the categorical featuresGenerate distribution plot for all the numerical featuresGenerate heatmap of the the correlation matrixPreprocessingCreate a dataframe with all standardized features merged with other featuresGenerate features listModel AssessmentGenerate Recall-Precision-Threshold CurveGenerate truepositive_falsepositive Curve2.1 UsageThe methods in the module are only callable as the extended domain'ml'in the native Pandas DataFrame object.Calling a method in"ml_workflow":df=pd.read_excel(os.path.join(os.getcwd(),"data.xlsx"))cols_categorical=df.ml.get_features_categorical()