Saravanan666/pypi-org huggingface dataset:
Package: labml-app
Mobile first web app to monitor PyTorch & TensorFlow model trainingRelax while your models are training instead of sitting in front of a computerThis is an open-source library to push updates of your ML/DL model training to mobile.Here's a sample experimentYou can host this on your own.
We also have a smallAWS instance running. and you are welcome to use it. Please consider using your own installation if you are running lots of experiments.Notable FeaturesMobile first design:web version, that gives you a great mobile experience on a mobile browser.Model Gradients, Activations and Parameters:Track and compare these indicators independently. We provide a separate analysis for each of the indicator types.Summary and Detail Views:Summary views would help you to quickly scan and understand your model progress. You can use detail views for more in-depth analysis.Track only what you need:You can pick and save the indicators that you want to track in the detail view. This would give you a customised summary view where you can focus on specific model indicators.Standard ouptut:Check the terminal output from your mobile. No need to SSH.📚 How to track experiments?How to run app locally?Install the PIP packagepipinstalllabml-appStart the serverlabmlapp-serverSet the web api url tohttp://localhost:5005/api/v1/track?when you run experiments.
You can alsoset this on.labml.yaml.fromlabmlimporttracker,experimentwithexperiment.record(name='sample',token='http://localhost:5005/api/v1/track?'):foriinrange(50):loss,accuracy=train()tracker.save(i,{'loss':loss,'accuracy':accuracy})
Package: labml-dashboard
LabML DashboardThis is the web dashboard forLabML.💬 Slack workspace for discussions📗 Documentation📑 Articles & Tutorials👨‍🏫 SamplesHere are some screenshots.Installpip install labml_dashboardRunningNavigate to the path of the project and run the following command to start the server.labml dashboardBuilding from sourcegit clone ...
git submodule init
git submodule update
npm install
make compile
make install
Package: labml-db
LabML DBLabML DB is a minimalistic ORM database that uses JSON, YAML or Pickle files.
It uses Python typehints as much as possible to help with static checking,
and IDE features like autocompletion.You can install this package using PIP.pip install labml_dbExamplefromlabml_dbimportModel,IndexclassProject(Model['Project']):name:strexperiments:int@classmethoddefdefaults(cls):returndict(name='',experiments=0)classUser(Model['User']):name:strprojects:List[Key[Project]]# This is an optional property, will automatically default to Noneoccupation:Optional[str]@classmethoddefdefaults(cls):# Name is not in defaults and not optional.# It will be considered a required propertyreturndict(projects=[])classUsernameIndex(Index['User']):passYou can configure it to use JSON/YAML/Pickle filesModel.set_db_drivers([FileDbDriver(PickleSerializer(),User,Path('./data/user')),FileDbDriver(YamlSerializer(),Project,Path('./data/project'))])Index.set_db_drivers([FileIndexDbDriver(JsonSerializer(),UsernameIndex,Path('./data/UserNameIndex.yaml'))])You can userget_allandKey.loadto retrieve models, andsaveto save models.proj=Project(name='nlp')user=User(name='John')user.projects.append(proj.key)user.occupation='test'user.save()proj.save()keys=User.get_all()print([k.load()forkinkeys])keys=Project.get_all()print([k.load()forkinkeys])And index is a hash-map between strings and keys.user_key=UsernameIndex.get('John')ifnotuser_key:user=User(name='John')user.save()UsernameIndex.set(user.name,user.key)else:print(user_key.load())
Package: labml-python-autocomplete
Python AutocompleteThe full length Python autocompletion Videoand aTwitter thread describing how it worksThis is a learning/demo project to show how deep learning can be used to auto complete Python code.
You can experiment with LSTM and Transformer models.
We also have built a simple VSCode extension to try out the trained models.Training
model:Evaluating trained
model:It gives quite decent results by saving above 30% key strokes in most files, and close to 50% in some. We calculated key
strokes saved by making a single (best)
prediction and selecting it with a single key.The dataset we use is the python code found in repos linked inAwesome-pytorch-list. We download all the repositories as zip
files, extract them, remove non python files and split them randomly to build training and validation datasets.We train a character level model without any tokenization of the source code, since it's the simplest.Try it yourselfClone this repoInstall requirements fromrequirements.txtRunpython_autocomplete/create_dataset.py.It collects repos mentioned inPyTorch awesome listDownloads the zip files of the reposExtract the zipsRemove non python filesCollect all python code todata/train.pyand,data/eval.pyRunpython_autocomplete/train.pyto train the model.Try changing hyper-parameters like model dimensions and number of layers.Runevaluate.pyto evaluate the model.You can also run the training notebook on Google Colab.VSCode extensionClone this repoInstall requirements fromrequirements.txtInstall npm packagesYou need to haveNode.JSinstalledcdvscode_extension
npminstall# This will install the NPM packagesStart the serverpython_autocomplete/serve.pyOpen the extension project (folder) inVSCodecdvscode_extension
code.# This will open vscode_extension in VSCodeIf you don't haveVSCode command line launcherstart VSCode and open the project withFile > OpenRun the extension from VSCodeRun > Start DebuggingThis will open another VSCode editor window, with the extensionCreate or open a python file and start editing!SampleHere's a sample evaluation of a trained transformer model.Colors:yellow: the token predicted is wrong and the user needs to type that character.blue: the token predicted is correct and the user selects it with a special key press,
such as TAB or ENTER.green: autocompleted characters based on the prediction
Package: labMTsimple
TL;DR a simple labMT usage scriptThis script uses the language assessment by Mechanical Turk (labMT) word
list to score the happiness of a corpus. The labMT word list was created
by combining the 5000 words most frequently appearing in four sources:
Twitter, the New York Times, Google Books, and music lyrics, and then
scoring the words for sentiment on Amazon’s Mechanical Turk. The list is
described in detail in the publication Dodds’ et al. 2011, PLOS ONE,
“Temporal Patterns of Happiness and Information in a Global-Scale Social
Network: Hedonometrics and Twitter.”Given two corpora, the script “storylab.py” creates a word-shift graph
illustrating the words most responsible for the difference in happiness
between the two corpora. The corpora should be large (e.g. at least
10,000 words) in order for the difference to be meaningful, as this is a
bag-of-words approach. As an example, a random collection of English
tweets from both Saturday January 18 2014 and Tuesday January 21 2014
are included in the “example” directory. They can be compared by moving
to the test directory, using the commandpythonexample.pyexample-shift.htmland opening the fileexample-shift.htmlin a web browser. For an
explanation of the resulting plot, please visithttp://www.hedonometer.org/shifts.htmlInstallationCloning the github directly is recommended, i.e.gitclonehttps://github.com/andyreagan/labMT-simple.gitand then installing locally usingsudopythonsetup.pyinstallTests can be run by navigating to the test directory, and runningpythontest.pywhich will compare the two days in test/data and print test.html which
shifts them, allowing for a changable lens.This repository can also be installed using pippipinstalllabMTsimplein which case you can download the tests from github and run them, if
desired.
Package: labnode-async
LabNodeThis is the Python3 AsyncIO API library for theLabnodesystem.The library is fully type-hinted.DocumentationThe full documentation can be found on GitHub Pages:https://patrickbaus.github.io/labnode_async/. I use theNumpydocstyle for documentation andSphinxfor compiling it.SetupTo install the library in a virtual environment (always use venvs with every project):python3-mvenvenv# virtual environment, optionalsourceenv/bin/activate
pipinstalllabnode-asyncUsageThis library makes use of asynchronous context managers to hide all connection related stuff and
also handle cleanup. By the way: Context managers are great!Connect to the remote endpoint, and it will automatically enumerate the Labnode and create the correct device.fromlabnode_asyncimportIPConnection# Create a device and start codingasyncwithIPConnection("192.1680.0.2")asdevice:# Add your code here...Seeexamples/for more working examples.VersioningI useSemVerfor versioning. For the versions available, see thetags of this repository.AuthorsPatrick Baus-Initial work-PatrickBausLicenseThis project is licensed under the GPL v3 license - see theLICENSEfile for details.
Package: labo
labo - LaTeX Bootstrapperlabois a CLI-utility for bootstrapping LaTeX-projects based on templates.
It usesJinja2for the templating.ConfigurationBy defaultlabolooks for the configuration file in$HOME/.config/labo/labo.tomland$HOME/.labo.toml, in that order. Currently the only configuration option is the location of the templates. This is specified by thetemplate_dirkey:template_dir=/home/user/.laboTemplatesA template for a LaTeX project forlabois specified by aconfig.tomlfile in a folder in the template directory. Seeexample_template/for a simple example.
The name of the template is determined by the name of this folder. In the config file the template files can be specified using theinstantiatekey.
For the time being it supports rendering LaTeX and Makefile templates using Jinja2.
Files that need to be copied or need to be linked to the new project can also be specified using thecopyandlinkkeys, respectively.
Below is an example of these keysinstantiate=["main.tex","Makefile"]copy=["document.tex"]link=["myclass.cls","mypackage.sty"]VariablesThe templates can contain variables. These variables can either be defined using hooks or by specifying them
in theconfig.tomlunder the[variables]section.laboprompts automatically for the variables during the bootstrapping of the project.
A variable can be defined as followed:[variables]var.prompt="Foo"var.default=1Thepromptkey defines the text used for prompting for the variable and is mandatory.
Thedefaultkey is used to infer the type of the variable and is mandatory.The variable can be used in a LaTeX-template by\VAR{var}and in a Makefile template by{{var}}.HooksBefore the templates are rendered apre_template_hookis ran. This hook can be defined in ahooks.pyfile in the template folder. This hook is passed four arguments: The current working directory, the project path, the options (see next section) and the variables. It is expected to return a dictionary of variables.Since this hook allows for the execution of arbitrary Python code it is important to ensure that the hook does not contain any malicious code.OptionsA template can define extra options with the options key.options=["opt"]HelpUsing thehelpkey a help message can be added to the template.UsageUsinglabo lsall templates can be listed.labo help <template>displays help information about the template<template>. To bootstrap a project uselabo new <template> <project name> <template options>.
Package: labone
PackageMetaCILabOne Python APIThelabone-pythonpackage provides a plain asynchronous Python API forLabOne, the control software of Zurich Instruments.[!CAUTION]This API package is solely being developed to supportLabOne Q, the software framework for quantum computing.For direct access to the instruments without LabOne Q, the standard Python API for all Zurich Instruments' devices is provided through thezhinstpackage and can be obtained fromPyPIbypip install zhinst.[!NOTE]Sincelabone-pythonis not intended for direct usage, we do not offer any support
or external documentation. Please contactZurich Instrumentsif you have any questions.ContributingSeeContributing
Package: laboneq
LabOne QLabOne Qis Zurich
Instrument’s software framework to accelerate progress in quantum computing. Its
Python-based, high-level programming interface enables users to concentrate on
intuitive, efficient experiment design, while automatically accounting for their
instrumentation details and maximizing useful computation time. Tight system
integration between software and hardware ensures a seamless user experience
from setups with a single qubit to those with 100 and more.Requirements⚠️This software requires Python 3.9 or higher.We assume thatpip3andpython3use a corresponding Python version.💡 To ease the maintenance of multiple installations, we recommended to
use Python environments through e.g.venv,pipenvorconda.InstallationThe following commands will make LabOne Q available in your current
environment.$pip3install--upgradelaboneqDocumentationFind the LabOne Q Manual here:https://docs.zhinst.com/labone_q_user_manual/Dive right into using LabOne Q and generate your first pulse sequence:https://docs.zhinst.com/labone_q_user_manual/getting_started/hello_world/The API Documentation is published here:https://docs.zhinst.com/labone_q_user_manual/reference/simple/Architecture
Package: labonneboite-common
_       _                            _           _ _
| | __ _| |__   ___  _ __  _ __   ___| |__   ___ (_) |_ ___
| |/ _` | '_ \ / _ \| '_ \| '_ \ / _ \ '_ \ / _ \| | __/ _ \
| | (_| | |_) | (_) | | | | | | |  __/ |_) | (_) | | ||  __/
|_|\__,_|_.__/ \___/|_| |_|_| |_|\___|_.__/ \___/|_|\__\___|AboutCommon library for labonneboite projects.How tosTestsmake testDocumentationThe documentation is based onmkdocsTo open the docs:make documentationIt will be accessiblehere
Package: laboral-kutxa
laboral-kutxa.pyUnofficial Laboral Kutxa Python libraryInstallpipinstalllaboral-kutxaUsageReading through themisProductoslist:importosfromlaboralkutxa.apiimportlogin,get_my_productsusername=os.environ.get("USERNAME")password=os.environ.get("PASSWORD")login_response=login(username,password)token=login_response["token"]products=get_my_products(token)print([{"alias":product["alias"],"grupo":product["grupo"]}forproductinproducts["misProductos"]])Output:[{'alias':'CUENTA 0,0','grupo':'cuentasCorrientes'},{'alias':'VISA ELECTRÓN','grupo':'tarjetas'},{'alias':'PRESTAMO','grupo':'prestamos'}]Accessing the aggregated amounts per account types:products=get_my_products(token)current_account=products["_Importes"]["_CuentasCorrientes"]financing=products["_Importes"]["_Financiacion"]print({"currentAccount":current_account,"financing":financing})Output:{'currentAccount':{'cantidad':4440.13,'moneda':'EUR'},'financing':{'cantidad':174356.48,'moneda':'EUR'}}CLIThere's also a command line interface available, try with:laboralkutxa--help
Package: lab-orchestrator-lib-auth
Lab Orchestrator Lib AuthThis package contains the lab orchestrator library authentication module.GithubPyPiRead The DocsInstallationpip3 install lab-orchestrator-lib-authDocumentationCheck out the developer documentation atlaborchestratorlib-auth.readthedocs.io.UsageThe library contains one module called auth that contains 3 methods and one dataclass.The first methodgenerate_auth_tokenis used to generate a JWT token. The token contains the user id, and some information about the lab instance for which this token is created. That also contains a list ofvmi_namesthe user should be allowed to connect to.HS256is used as algorithm which is a symmetric algorithm, so you need to use the same secret for both: encryption and decryption of the key.The next methoddecode_auth_tokendecodes the previously encoded JWT token. It returns the information that is contained in the token.The third methodverify_auth_tokencontains the parametervmi_nameand checks if thisvmi_nameis allowed. If it's not allowed the method will return none.See more at:laborchestratorlib-auth.readthedocs.io.ExamplesThere is one example that shows how to create a token with the library. For other examples you need to look into theLabOrchestratorLiborWebsocketProxy.ContributingIssuesFeel free to openissues.Project StructureThesrcfolder contains the source code of the library. Thetestsfolder contains the test cases.examplescontains some example scripts of how to use the library. There is a makefile that contains some shortcuts for example to run the test cases and to make a release. Runmake helpto see all targets. Thedocsfolder contains rst docs that are used inread the docs.Developer DependenciesPython 3.8Makepip install -r requirements.txtpip install -r requirements-dev.txtReleasesYour part:Create branch for your feature (issue/ISSUE_ID-SHORT_DESCRIPTION)CodeMake sure test cases are running and add new ones for your featureCreate MR into masterIncrease version number insrc/lab_orchestrator_lib_auth/__init__.py(semantic versioning)Admin part:Check and accept MRMerge MRRunmake releaseDocsTo generate the docs run:cd docs && make html.
Package: lab-orchestrator-lib-django-adapter
Lab Orchestrator Lib Django AdapterThis package implements the adapters to use thelab orchestrator libraryin django projects. That means that some data of the lab orchestrator library will be saved
in the django database.GithubPyPiRead The DocsSetupInstallationInstall this library with:pip install lab-orchestrator-lib-django-adapter.Project setupAssuming you have a django application, first define the following variables in yoursettings.py:DEVELOPMENT(bool): If this is true the development mode is activated. This means, that no cacert is used and
insecure certs are allowed. If false this assumes you are running this inside a Kubernetes cluster.SECRET_KEY(str): This key is used to create jwt tokens. Create a random key for this and keep this key safe.After that addlab_orchestrator_lib_django_adapterto theINSTALLED_APPSvariable in yoursettings.py.Environment VariablesThe library makes use of some environment variables that you need to set:KUBERNETES_SERVICE_HOST(str): Host of your Kubernetes API (if you runkubectl proxy:localhost)KUBERNETES_SERVICE_PORT(int): Port of your Kubernetes API (if you runkubectl proxy:8001)DEVELOPMENT(bool): If this is true the development mode is activated. This means, that no cacert is used and
insecure certs are allowed. If false this assumes you are running this inside a Kubernetes cluster.UsageYou have three Model classes:DockerImageModelLabModelLabInstanceModelThose three models are used to save data from the library in the django database and those can all be used safely in
read only view sets. TheDockerImageModelandLabModelcan also be used safely in a delete, update and create view
sets. To create and delete aLabInstanceModelyou need to use the lab instance controller. Updating aLabInstanceModelis not allowed.To get extra information from Kubernetes resources you need to use the controllers.A defaultControllerCollectioncan be got with theget_default_ccmethod inlab_orchestrator_lib_django_adapter.controller_collectionmodule. The controllers are part of the lab orchestrator lib
and there you can find documentation about how to use this.An example on how to use this library inViewSetscan be found in theviews.py.
Package: lab-orchestrator-ws-proxy
Websocket ProxyProgram that contains a Proxy for KubeVirts VNC Websockets. It uses JWT Tokens for Authentication and to transfer data.GithubPyPiDocker HubInstallationpip3 install lab-orchestrator-ws-proxyordocker pull biolachs2/lab_orchestrator_ws_proxy:latestDocumentationCheck out the developer documentation atwebsocketproxylib.readthedocs.io.UsageKubernetesTake a look at theLabOrchestrator Kubernetes FilesManualTo start this program you need to set some environment variables first:SECRET_KEY: Secret key that should be used for JWT token decode.LOCAL_DEV_MODE: If False, it's assumed that this is run inside Kubernetes. If True secure connections are disabled. Default: FalseHOST: The host this proxy runs on. Default: 0.0.0.0PORT: The port this proxy runs on. Default: 5001HOST_PATH_PREFIX: Prefix that is removed from the host path.KUBERNETES_SERVICE_HOST: Host where to proxy the websockets. Inside of Kubernetes this has a preset default value.KUBERNETES_SERVICE_PORT: Port where to proxy the websockets. Inside of Kubernetes this has a preset default value.LOGLEVEL: Python Loglevel to use. Default: INFOAfter that run:PYTHONPATH=src python -m lab_orchestrator_ws_proxy.The proxy works as follows:It creates a websocket at the given host and port. Then when you want to access a VM you need to call the websocket with a path that contains the token and the VM-name divided by a slash. Example:localhost:5001/ABCTOKENDEF/ubuntu. The example contains the tokenABCTOKENDEFand tries to access the VM with the nameubuntu. The token contains a list of allowed VM-names and if the given VM-name is part of the token you will be able to access the VM.See more at:websocketproxylib.readthedocs.io.ExamplesFor an example on how to use this program you can take a look at theLabOrchestrator Kubernetes Fileswhich integrates this program into Kubernetes.ContributingIssuesFeel free to openissues.Project StructureThesrcfolder contains the source code of the library. Thetestsfolder contains the test cases. There is a makefile that contains some shortcuts for example to run the test cases and to make a release. Runmake helpto see all targets. The documentation should be in the README.md and inside of the WebsocketProxyLib.Developer DependenciesPython 3.8Makepip install -r requirements.txtpip install -r requirements-dev.txtReleasesYour part:Create branch for your feature (issue/ISSUE_ID-SHORT_DESCRIPTION)CodeMake sure test cases are running and add new ones for your featureCreate MR into masterIncrease version number insrc/lab_orchestrator_ws_proxy/__init__.py(semantic versioning)Admin part:Check and accept MRMerge MRRunmake releaseDocsTo generate the docs run:cd docs && make html.