Saravanan666/pypi-org huggingface dataset:
Package: containerify
Devcontainers Contrib CLIInstallationpip install devcontainer-contrib[generate]Feature Generation:Usage: python -m devcontainer_contrib feature generate [OPTIONS]
                                                        FEATURE_DEFINITION
                                                        OUTPUT_DIR

Arguments:
  FEATURE_DEFINITION  [required]
  OUTPUT_DIR          [required]

Options:
  --help                          Show this message and exit.Input for feature generation is afeature-definition.jsonfile
This is anextendedversion of thedevcontainer-feature.jsonfile withadditional fields:{...regular devcontainer-json fields  like id,name,description etc...# dependencies are list of features and options  (those will be installed as prerequisites to your feature)"dependencies":[{"feature":"ghcr.io/devcontainers-contrib/features/asdf-package:latest","options":{"plugin":"act","version":"latest"}}],# This command will be executed after the dependency feature list has been installed"install_command":"echo'Done'",# this example test scenario checks the default options (empty options dict), each test_command should exit wth code `0` if your feature is installed correctly."test_scenarios":[{"name":"test_defaults","image":"mcr.microsoft.com/devcontainers/base:debian","test_commands":["act--version"],"options":{}}]}redirect input{..."options":{"version":{"type":"string","default":"latest","description":"Selecttheversionofacttoinstall."}},# you can also redirect one of your actual devcontainer-feature.json options values into a dependency input, note the `$options.version` pointer to the asdf-package version option"dependencies":[{"feature":"ghcr.io/devcontainers-contrib/features/asdf-package:latest","options":{"plugin":"act","version":"$options.version"}}],...}Usage examplethis will generate the Elixir feature
It assumes you have the devcontainer-contrib cli already installedgitclonehttps://github.com/devcontainers-contrib/cli--depth1cdcli


devcontainer-contribfeaturegenerate"./test/resources/test_feature_definitions/elixir-asdf/feature-definition.json""./output_dir"
Package: container-inspector
container-inspectoris a suite of analysis utilities and command line tools
for Docker images, containers, root filesystems and virtual machine images.For Docker images, it can process layers and how these relate to each other as
well as Dockerfiles.container-inspectorprovides utilities to:identify Docker images in a file system, its layers and the related metadata.given a Docker image, collect and report its metadata.given a Docker image, extract the layers used to rebuild what how a runtime
rootfs would look.find and parse Dockerfiles.find how Dockerfiles relate to actual images and their layers.given a Docker image, rootfs or Virtual Machime image collect inventories of
packages and files installed in an image or layer or rootfs
(implemented using a provided callable)detect the “distro” of a rootfs of image using os-release files (and an
extensive test suite for these)detect the operating system, architecture andQuick startOnly runs on POSIX OSesGet Python 3.6+Check out a clone or download of container-inspector, then run:./configure –dev.Then runenv/bin/container-inspector -hfor help.Container image formatscontainer-inspector handles the formats of Docker images as created by thedocker savecommand. There are three versions for this Docker image format.
The latest v1.2 is a minor update to v1.1.v1.1 provides improved and richer metadata over v1.0 with a top level manifest.json
file and a Config file for each image with full layer history and ordeing. It also
use checksum for enhanced security and traceability of images and layers.v1.0 uses a simplerepositoriesmeta file and requires infering the ordering of
the layers in an image based on each individual layerjsonmeta file. This
format is no longer support in the latest version of container-inspector.All V1.x formats use the same storage format for layers e.g the layer format V1.0
where each layer is stored in a sub-directories named after the layer id.
Each of this directories contains a “layer.tar” tarball with the layer payload,
a “json” JSON metadata file describing the layer and a “VERSION” file describing
the layer format version. Each tarball represents a slice or diff of the image
root file system using the AUFS conventions.At runtime, in a sequence of layers of an image, each root filesystem slice of a
layer is “layered” on top of each other from the root bottom layer to the latest
layer (or selected tagged layer) using a union file system (e.g. AUFS).
In AUFS, any file or directory prefixed with .wh. are “white outs” files deleting
files in the underlying layers.See the image specifications saved in docs/references/Internal data modelImage: this is a runnable image composed of metadata and a sequence of layers.Layer: this is a slice of an image root filesystem with a payload and metadataResource: this a file or directoryPlansin progress: support OCI image layoutimproved suport for Windows containersRelated toolsFetching Image from remote registry is available in ScanCode.ioExtracting VM Image filesystems as archives is available in ExtractCodeScanning for application and system packages is available in ScanCode Toolkit
Package: containerlog
containerlogA lightweight, optimized, and opinionated structured logging library for Python, intended for containerized applications.containerlogwas born out of a desire to have high-quality structured logging for
containerized applications (e.g. microservices) without having to compromise detailed
logging for application/request latency.structlogis a great general-purpose structured
logging library for Python, but being general-purpose means that there is additional overhead
when logging messages.Whenweupdated a microservice to use structured logging,
we found thatrequest latency went up,
seemingly due to the transition to usestructlog.containerlogis not for everyone. It is highly opinionated, minimally configurable,
and intentionally feature-sparse so that it can achievebetter performancethan
the Python standard loggerNot every application needs optimized logging, but where latency and performance matters,containerlogcould work for you.timestamp='2020-07-23T13:11:28.009804Z' logger='my-logger' level='debug' event='loading configuration' path='./config.yaml'
timestamp='2020-07-23T13:11:28.010137Z' logger='my-logger' level='info' event='starting application'
timestamp='2020-07-23T13:11:28.010158Z' logger='my-logger' level='warn' event='having too much fun' countdown=[3, 2, 1]Installationcontainerlogcan be installed with pip:pip install containerlogIt is only intended to work for Python 3.6+.UsageSee the documentation athttps://containerlog.readthedocs.io/en/latest/OptimizationsThere are numerous sources discussion micro-optimizations in Python. This project probably
does not implement them all, so there is room for improvement. Current optimization work has
leveraged:dis: to disassemble python bytecode for analysistimeit: to measure execution time of code snippetsIf you wish to contribute optimizations and use other libraries, tools, or sources, open a PR to add
them to this list.BenchmarksBenchmarking scripts can be found in thebenchmarksdirectory. To run,$ cd benchmarks
$ ./run.shThis will run benchmarks the Python standard logger and forcontainerlog. The latest results
can be found below.ResultsBenchmarks were measured using Python 3.8.0 on macOS 10.15.1 with a 2.9 GHz 6-Core Intel Core i9
processor and 16 GB 2400 MHz DDR4 memory.Benchmarkstd logger (ns)std proxy (ns)containerlog (ns)baseline0.68 +/- 0.020.69 +/- 0.010.7 +/- 0.02silent108.0 +/- 6.01140.0 +/- 50.051.7 +/- 1.7basic4750.0 +/- 160.01140.0 +/- 60.01070.0 +/- 50.0short-simple5370.0 +/- 160.01280.0 +/- 60.01330.0 +/- 60.0long-simple5280.0 +/- 180.01480.0 +/- 70.02120.0 +/- 60.0short-complex5630.0 +/- 170.01500.0 +/- 150.01480.0 +/- 80.0long-complex6900.0 +/- 190.02870.0 +/- 80.03260.0 +/- 80.0exception10400.0 +/- 300.04440.0 +/- 150.04370.0 +/- 500.0ContributeWhilecontainerlogis intentionally feature-sparse, feature requests are welcome. Additionally,
if you can find any other ways to micro-optimize the codebase, pull requests are very much
appreciated.
Package: container-manager
Container ManagerDNNツール等を用いた実験の環境構築〜モデル学習をサポートするツールです．Requirementnvidia-driverdockerdocker-pynvidia-dockerInstall以下のコマンドでcoctlコマンドがインストールされます．pip install container_managerGetStartexampleディレクトリに最も簡単なサンプルがあります．exampleディレクトリのある場所でcoctlコマンドを使うことでタスクを実行することができます．runrunコマンドでディレクトリを指定しタスクを作成，実行することができます．$ coctl run example上記のコマンドでpytorch1.4がインストールされた環境でタスク名/main.pyが実行されます(例ではexample/main.py)．
指定したディレクトリ名のタスクが作成されます．coctlコマンドは実行したいタスクのディレクトリのある場所で実行してください．$ coctl run xxx/exampleや$ coctl run ../exampleのような指定はできないことに注意してください．
個別に必要なライブラリ等がある場合はexample/Dockerfileに記載されているように適宜必要なライブラリを追加してください．main.pyが実行される環境は起動時に毎回新規に環境が作られるため，必ずDockerfileにpip installなどを記載し，指定するディレクトリの中に配置してください．
runの実行後はmain.pyのログが表示されます．Ctr-Cなどでログの表示を止めることができますが，タスクのプロセス自体は終了しません．
現在走っているタスクを止めるには下記のrmコマンドでタスクを削除するか，同名のタスクをrunすることで削除した後新しいタスクが作成されます．実行時引数実行したいタスクのmain.pyが実行時引数を必要とする場合には-oオプションで引数を渡すことができます．$ coctl run example -o args1:hoge args2:baa arg3とした場合$ main.py --args1=hoge --args2=baa --arg3のように解釈されます．タスクの入出力学習データなどの入力データはtraining_input/タスク名以下に配置してください．
出力データはtraining_output/タスク名に出力されます．
タスクの実行される環境ではtraining_input/タスク名は/inputに，training_output/タスク名は/outputとして認識されるのでタスクのmain.pyでは/input以下からデータを読み込み，/outputへデータを書き込みようにしてください．listlistコマンドで実行中及び完了済みのタスクが表示できます．$ coctl list
task_name       | status
 example        | running
 test   | exited上記の例ではタスク名testが完了済み，exampleが実行中であることを示します．logs実行中のタスクのログを確認するにはlistコマンドを利用します．$ coctl logs example
arg3=default arg
main is run!!!
count 0
count 1
count 2
count 3-fオプションをつけるとログをストリームで監視します．rmrmコマンドでタスクの削除を行うことができます．$ coctl rm example
Task example is deletedcleanタスクの生成を繰り返すと不要なデータが蓄積することがあります．
ディスク容量が圧迫される場合はcleanコマンドで不要なデータを削除してください．$ coctl cleanGCPで学習を行う場合の例以下のコマンドでGPUを持つインスタンスを確保します．gcloud beta compute \
	--accelerator=type=nvidia-tesla-v100,count=1 \
	--image-project=ml-images \
	--boot-disk-size=50GB \
	--zone=us-central1-a \
	--machine-type=n1-standard-1 \
	--subnet=default \
	--network-tier=PREMIUM \
	--maintenance-policy=TERMINATE \
	--service-account=770963190637-compute@developer.gserviceaccount.com \
	--scopes=https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/trace.append,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/devstorage.read_write \
	--image=c3-deeplearning-tf-ent-2-1-cu100-20200131 \
	--boot-disk-type=pd-standard \
	--boot-disk-device-name=gpu-test \
	--reservation-affinity=anygpt-testがインスタンス名なので適宜修正してください．acceleratorをnvidia-tesla-k80にしたり，countを増やしたりすることができます．
また，boot-disk-sizeを変えることでディスク容量が変えられます．
インスタンスの作成が完了し，サーバが起動すると以下のコマンドでログインできます．gcloud compute ssh インスタcoンス名*ログイン時にnvidiaドライバのインストールを行うか聞かれるのでyを選択してください．
ContainerManagerの手順に従い，coctlをインストールします．
上記の手順で作成したインスタンスはContainerManagerのrequirementを満たしているのでそのままpip installが可能です．
Package: container-patcher
container-patcherA tool to automatically fix vulnerable container packages.Free software: MIT licenseDocumentation:https://container-patcher.readthedocs.io.FeaturesTODOCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2020-11-19)First release on PyPI.
Package: container-runtime-interface-api
container-runtime-interface-apiPython library for communication with the KubernetesContainer Runtime Interface API.UsageInstallcontainer-runtime-interface-apiwithpipenvorpip:$pipenvinstallcontainer-runtime-interface-api
Addingcontainer-runtime-interface-apitoPipfile's[packages]…
✔InstallationSucceeded
...This project currently supports Python 3.7+.ConnectionConnection to the CRI API is generally done through a UNIX socket, but any gRPC address supported byinsecure_channelwill work.fromcri_api.channelimportChannelchannel=Channel.from_env()# Loads from RUNTIME_SOCKchannel=Channel("unix:///var/run/dockershim.sock")# Explicit argumentImagesTheImagesclass is a thin wrapper around the existing ImageService API:fromcri_api.imagesimportImageschannel=Channel.from_env()images=Images(channel)images.list_images()images.pull_image("busybox")busybox_images=[i["id"]foriinimages.list_images()ifany("busybox"inrforrini["repoTags"])][images.remove_image(i)foriinbusybox_images]ContainersTheContainersclass is a thin wrapper around the existing RuntimeService API:fromcri_api.imagesimportImagesfromcri_apiimportContainerFilter,ContainerState,ContainerStateValuechannel=Channel.from_env()images=Containers(channel)containers.list_containers()containers.list_containers(ContainerFilter(state=ContainerStateValue(state=ContainerState.CONTAINER_EXITED)))containers.get_container("9d81052cc027a1fb2ec61b898ea0fd6fc88216ce730ad75f4c52b29849cb440f")RawRaw access to the underlying CRI API objects can be done by importing fromcri_api:fromosimportgetenvfromgrpcimportinsecure_channelfromcri_apiimportRuntimeServiceStub,ListContainersRequeststub=RuntimeServiceStub(insecure_channel(getenv("RUNTIME_SOCK")))response=stub.ListContainers(ListContainersRequest())containers=response.containersUpdating Protobuf Python Generated Code$bin/update-proto.shCommit & create a new pull request!DevelopmentInteractive development on MacOS can be done by leveragingminikube:$minikubestart--container-runtime=cri-o
$minikubessh
$socat-d-dTCP4-LISTEN:15432,forkUNIX-CONNECT:/var/run/crio/crio.sock# In another window, you can now connect on $(minikube ip):15432$exportRUNTIME_SOCK=$(minikubeip):15432
...TestingUnit tests are run against all supported Python versions: 3.7, 3.8, and 3.9.A basic set of integration tests is included alongside the unit tests, but are only run if theRUNTIME_SOCKenvironment variable is set correctly.
They are verified in CI against a minikube installation running Docker anddockershim.Run unit tests with:$pytest
Package: container-scheduler
A scheduler for docker containers, made easy. Configure the desired schedule per each container and the library ensures
the container will be running at the correct timing.UsageTo start the scheduler, run the command belowimportcontainer_schedulerschedules=[{"container":"container_name_1","crontab":"*/1 * * * *"},{"container":"container_name_2","crontab":"0 4 * * *"}]container_scheduler.start(schedules)The functionstartis synchronous and will not return until the scheduler is running. To stop it gracefully,
you can send aSIGTERMor aSIGKILLsignal.The library was originally developed to work in a docker container, acting as a scheduler container for other
containers (take a look atdocker-container-scheduler)MetaYusef Maali -contact@yusefmaali.netThe scheduling is executed by theschedule-cronjoblibrary, a fork of the
excellentschedulefrom Daniel Bader.Distributed under the MIT license. SeeLICENSE.txtfor more information.https://github.com/yusefmaali/container-scheduler
Package: container-service-extension
container-service-extensioncontainer-service-extensionis a vCloud Director extension that manages the life cycle of Kubernetes clusters for tenants.This extension is designed to be installed on a vCloud Director instance by the service provider (system administrator).container-service-extensionis distributed as a Python package.DocumentationSee oursite.ContributingThecontainer-service-extensionproject team welcomes contributions from the community. Before you start working withcontainer-service-extension, please read ourDeveloper Certificate of Origin. All contributions to this repository must be signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on as an open-source patch. For more detailed information, refer toCONTRIBUTING.md.LicenseBSD-2
Package: containershare
# Container Share Python
Tools for working with [containershare](https://vsoch.github.io/containershare).<a href=”https://www.github.com/vsoch/containershare” target=”_blank”>
<img src=”https://vsoch.github.io/lessons/assets/img/logo-book.png” width=”100px”>
</a>## Contributors> <@vsoch:Stanford Research Computing Center>## Contributing
We have many ways to contribute, and will briefly provide resources here to get you started.### How to Contribute
If you are a developer interested in working on the Experiment Factory software you should read out [contributing guidelines](.github/CONTRIBUTING.md) If you have any questions, please don’t hesitate to [ask a question](https://www.github.com/expfactory/expfactory/issues).### Code of Conduct
It’s important to treat one another with respect and maintain a fun and respectful environment for the open source community. Toward this aim, we ask that you review our [code of conduct](.github/CODE_OF_CONDUCT.md)## Containershare Library
The containers themselves are now maintained under the [containershare](https://www.github.com/vsoch/containershare), official submissions can be added to the [library](https://www.github.com/vsoch/containershare) (under development) to be tested that they meet minimum requirements.
Package: container-tagbot
TagbotTagbot retags OCI Container Images without needing a full Docker Pull / Docker Push workflow by working directly with the registry API.UsageLocal Usagetagbot\--usernameexample\--passwordpassword\--sourceexample.azurecr.io/debian:latest\--tagsv1.0.0,1.0.0This would add an additional tag ofv1.0.0toexample.azurecr.io/debian:latest. The container image can then be pulled with eitherexample.azurecr.io/debian:latest,example.azurecr.io/debian:v1.0.0, orexample.azurecr.io/debian:1.0.0GitHub Actions Usagename:releaseon:push:tags:["v[0-9]+.[0-9]+.[0-9]+"]jobs:release:uses:binkhq/tagbot/.github/workflows/retag.yaml@masterwith:username:examplesource:example.azurecr.io/${{ github.event.repository.name }}:${{ github.ref_name }}tags:${{ matrix.environment }}-v1.0.0,${{ matrix.environment }}envirnment:${{ matrix.environment }}secrets:password:${{ secrets.ACR_BINKCORE_PASSWORD }}strategy:matrix:environment:[staging,production]FAQWhats going on withghcr.io/binkhq/tagbot?This still exists, but is formally deprecated and will no longer recieve updates or supportAre Registries other than Azure Container Registry supported?Azure Container Registry is the only officially supported Registry, but other simple registries like Docker Hub are expected to work just fine.
Package: containerutil
containerutil.Pathprovides API similar topathlib.Pathfor inspectingdocker containerfile system:importdockerimportcontainerutilclient=docker.from_env(version='auto')ctnr=client.containers.get('my-container')p=containerutil.Path(ctnr,'/foo/bar/path')p.exists()# True if file exists (regardless of type)p.is_file()# True if file exists and is a regular filep.is_symlink()# True if file exists and is a symbolic linkp.is_fifo()# True if file is a named pipe (FIFO)InstallationFromPyPIusingpippackage manager:pipinstall--upgradecontainerutilOr install the latest sources from GitHub:pipinstallhttps://github.com/pavdmyt/containerutil/archive/master.zipDevelopmentClone the repository:gitclonehttps://github.com/pavdmyt/containerutil.gitInstall dependencies:makeinstallLint code:makelintRun tests:maketestContributingFork it!Create your feature branch:git checkout-bmy-new-featureCommit your changes:git commit-m'Add some feature'Push to the branch:git push originmy-new-featureSubmit a pull requestMake sure tests are passingLicenseMIT - Pavlo Dmytrenko
Package: container-utils
Container utilitiesThe container-utils Python package contains utility functionality for working with Docker containers via theDocker SDK for Python. The core functionality was originally part of theue4-ci-helperspackage, and was subsequently factored out into a standalone package that can be used with minimal dependencies.To install the package, simply run:pip3 install container-utilsLegalCopyright © 2020, Adam Rehn. Licensed under the MIT License, see the fileLICENSEfor details.
Package: container-workflow-tool
Container Workflow ToolA python3 tool to make rebuilding container images easier by automating several steps of the process.MotivationThe actual rebuild of container images consists of several steps that have been so far done manually. Some of these steps are:Rebase against upstream repositoryCheck the changes made by the rebasePush the changes into dist-git and run the buildAll of the steps are currently automated or semi-automated bycwt(but still need to be manually started). This should help a bit with the image rebuild workflow.Requirementspython3python3-GitPythonpython3-requests-kerberosfedpkgOptionsusage: cwt [options] command
    Command:
        koji            - List builds, base images, hash ids
        build           - Command for building images
        git             - Work with upstream/downstream git repositories
        utils           - Other actions tied to the rebuild (communication, repository preparation etc.)

    Options:
        -v, --verbosity      - Verbosity level, 1 (Critical only) - 5 (Debug messages), default 4 (Info)
        --base               - Specific base image release, required for some actions
        --clear-cache        - Clears tmp dir before running the command
        --latest-release     - Work with latest brew builds by release value
        --config             - Overrides default configuration file, expects the name of file a inside the config folder, optionally takes image_set argument
                               example usage: --config default.yaml:fedora27
        --do-image           - Use a custom set of images instead of all from the config (use dist-git names)
        --exclude-image      - Exclude an image from the list of images defined by config (use dist-git names)
        --do-set             - Use a specific set of images instead of all from the config (use dist-git names)
        --tmp                - Overrides default temporary working directory
        --disable-klist      - Disables getting kerberos token by klistTo get the usage of a specific command, you can run:cwt command --helpcontainer-workflow-tool in the quay.io registrycontainer-workflow-toolis automatically built and pushed inquay.io/rhscl/cwt-generatoras soon as changes
are merged intomasterbranch.TestThis repository also contains test suites for python'sunittestframework that check the basic functionality of cwt.
These test can be run directly from the repository's root via Makefile:make testIf you do not need to run all of the test cases provided you can run a module-specific subset like this:make test_distgitIf you want to run all the test cases in container, run it like this:maketest-in-container