Saravanan666/pypi-org huggingface dataset:
Package: job-farmer
Job FarmerDescriptionJob Farmer is a python API designed to prepare jobs for HPC platforms supporting SLURM.
It collects the job configurations of the associated tasks and prepares the task list and slurm configuration files
based on the system constraints and the required resources.InstallationFrom PyPIpipinstalljob-farmerFrom sourcesgitclonegit@gitlab.com:riffard/job_farmer.gitcdjob_farmer
pipinstall.UsageJob configurationIn the filetest/simple_job.py, we are showing a simple job submission.First, you import the job-farmer:importjob_farmerThen you create theSystemobject (NERSC/CORI-Haswell in this example) and theFarmerobjectsystem=job_farmer.CoriHaswell()farmer=job_farmer.Farmer(system)The nex step consists on creation theJobConfigurationand add it to theFarmer:job_configuration=job_farmer.JobConfiguration(name='test_job',account='your_account',queue='queue',duration='xx:xx:xx')farmer.add_global_job_configuration(job_configuration)You add theTasksto theFarmerand you link them to the job_configurationtask=job_farmer.Task(commands=[f'export MY_MESSAGE="Hello World!"','echo "${MY_MESSAGE}"'],n_threads=1)farmer.add_task(job_configuration.name,task)And it builds the task list and the slurm configuration file:farmer.build()Finally, you can submit the jobs to the batch system using:farmer.submit(dry_run=False)(by default,dry_run = True).Note:By default, the job-farmer logs are not displayed.
To display them, you need to import theloggingpackage and set the log level tologging.INFOsuch as:**logging.basicConfig(level=logging.INFO)Running jobsJob runnerBatch submissionSupportIf you are encountering any issue, please open a git issue:hereContributingAny contribution is more than welcome. To add any new feature, please follow this procedure:Create a git issuehereCreate a new branch with the following format:[initials]_[feature title]_issue_[issue number].Once the development is nearly completed, push the branch to the repository and create a merge request with the following guideline:The MR title must contain a short feature/bugfix  descriptionThe MR description must contain a detailed description of the feature/bug fix. Add here any relevant resources.After reviewing the merge request, it will be merged to themainbranch.Adding a new data center and a new systemTo add a new data center, create a new python file injob_farmer/systemwith the name of the data center in the filename (use lower case).To add a new system, add a newSystemclass that inherits fromSystemin the correct data center file.
To see an example, take a look atjob_farmer/system/nersc.pyLicenseThis software is licenced under:GNU Affero General Public License v3.0
Package: jobflow-remote
jobflow-remoteFull DocumentationJobflow Remote is a Python package to runjobflowworkflows on remote resources.Need help?Ask questions about Jobflow Remote on theJobflow Remote support forum.
If you've found an issue with jobflow-remote, please submit a bug report onGitHub Issues.What’s new?Track changes to jobflow-remote through thechangelog.ContributingWe greatly appreciate any contributions in the form of a pull request.
Additional information on contributing to jobflow-remote can be foundhere.
We maintain a list of all contributorshere.Code of conductHelp us keep jobflow-remote open and inclusive.
Please read and follow ourCode of Conduct.Licensejobflow-remote is released under a modified BSD license; the full text can be foundhere.CitationIf you use Jobflow in your work, please cite it as follows:"Jobflow: Computational Workflows Made Simple", A.S. Rosen, M. Gallant, J. George, J. Riebesell, H. Sahasrabuddhe, J.X. Shen, M. Wen, M.L. Evans, G. Petretto, D. Waroquiers, G.‑M. Rignanese, K.A. Persson, A. Jain, A.M. Ganose,Journal of Open Source Software, 9(93), 5995 (2024) DOI:10.21105/joss.05995Acknowledgementsjobflow-remote is developed and maintained by Matgenix SRL.A full list of all contributors can be foundhere.
Package: job-generator
Resume and Cover Letter GeneratorThis project is a software development project that utilizes web scraping, machine learning APIs, and HTML templates to generate a resume and cover letter based on a job posting. The purpose of this project is to automate the process of job application by generating a personalized resume and cover letter in a matter of seconds.Table of ContentsResume and Cover Letter GeneratorTable of ContentsInstallationUsageContributingInstallationClone the repository to your local machine.Install the required packages by runningpip install -r requirements.txt.Create a.env.localfile and add the following variables:API_KEY=<your_api_key>Note: You will need to obtain an API key from the machine learning API provider.Runpython main.pyto start the application.UsageThis project is currently limited to a command line tool. The program is controlled by a number of flags.-hor--help: Displays the help menu.-oor--output: Determines if files will be output.-vor--verbose: Determines if the program will print the output.-ror--resume: Determines if a resume will be generated.-lor--letter: Determines if a cover letter will be generated.-por--posting: Determines if a job posting will be generated.ContributingContributions are always welcome! If you have any ideas or suggestions, please create a pull request.
Package: job-generator-timhunterdev
Resume and Cover Letter GeneratorThis project is a software development project that utilizes web scraping, machine learning APIs, and HTML templates to generate a resume and cover letter based on a job posting. The purpose of this project is to automate the process of job application by generating a personalized resume and cover letter in a matter of seconds.Table of ContentsResume and Cover Letter GeneratorTable of ContentsInstallationUsageContributingInstallationClone the repository to your local machine.Install the required packages by runningpip install -r requirements.txt.Create a.env.localfile and add the following variables:API_KEY=<your_api_key>Note: You will need to obtain an API key from the machine learning API provider.Runpython main.pyto start the application.UsageThis project is currently limited to a command line tool. The program is controlled by a number of flags.-hor--help: Displays the help menu.-oor--output: Determines if files will be output.-vor--verbose: Determines if the program will print the output.-ror--resume: Determines if a resume will be generated.-lor--letter: Determines if a cover letter will be generated.-por--posting: Determines if a job posting will be generated.ContributingContributions are always welcome! If you have any ideas or suggestions, please create a pull request.
Package: jobio
jobio is used for executing processes on a Unix-like environment.
The execution runtime at the moment is just a simple shell execution of the job command and the optional arguments.In addition, the package supports the staging of S3 buckets on the executing node before the process scheduled.
Furthermore, upon a successful execution, thejobiopackage supports the automatic export of persistent results
to the same S3 bucket.To distinquish between what is the staging inputs in the bucket and the subsequent results,
jobio sets defaultbucket_input_prefix=’input’andbucket_output_prefix=’output’to divide these two sets in the bucket.Very much in anAlphastage.
Not ready for anything!InstallationInstallation from pypi:pip install jobioInstallation from local git repository:cd jobio
pip install .
Package: jobject
jobjectjobject: A dictionary replacement that gives additional access to data using C struct notation, just like JavaScript ObjectsInstallationpipinstalljobjectImportfromjobjectimportjobjectmy_dict=jobject({'one':1,'two':2,'three':3})print(my_dict.three)# prints '3'InheritanceBecause jobject extends dict it can be dropped into any code that requires
dict notation or iteration. Because of this, jobject makes sure any dictionary
instances that are passed to it are also converted into jobjectsfromjobjectimportjobjectmy_dict=jobject({'one':{'two':{'three':123}}})print(my_dict.one.two.three)# prints '123'It will even follow lists to make sure everything under it is converted to a
jobjectfromjobjectimportjobjectmy_dict=jobject({'array':[{'one':1},{'two':2},{'three':3}]})print(my_dict[2].three)# prints '3'This even includes data set after the factfromjobjectimportjobjectmy_dict=jobject()my_dict.test={'one':[{'two':2}]}print(my_dict.test.one[0].two)# prints '2'print(my_dict['test']['one'][0]['two'])# prints '2'
Package: joblib
The homepage of joblib with user documentation is located on:https://joblib.readthedocs.ioGetting the latest codeTo get the latest code using git, simply type:git clone git://github.com/joblib/joblib.gitIf you don’t have git installed, you can download a zip or tarball
of the latest code:http://github.com/joblib/joblib/archives/masterInstallingYou can usepipto install joblib:pip install joblibfrom any directory or:python setup.py installfrom the source directory.DependenciesJoblib has no mandatory dependencies besides Python (supported versions are
3.7+).Joblib has an optional dependency on Numpy (at least version 1.6.1) for array
manipulation.Joblib includes its own vendored copy oflokyfor process management.Joblib can efficiently dump and load numpy arrays but does not require numpy
to be installed.Joblib has an optional dependency onpython-lz4as a faster alternative to
zlib and gzip for compressed serialization.Joblib has an optional dependency on psutil to mitigate memory leaks in
parallel worker processes.Some examples require external dependencies such as pandas. See the
instructions in theBuilding the docssection for details.Workflow to contributeTo contribute to joblib, first create an account ongithub. Once this is done, fork thejoblib repositoryto have your own repository,
clone it using ‘git clone’ on the computers where you want to work. Make
your changes in your clone, push them to your github account, test them
on several computers, and when you are happy with them, send a pull
request to the main repository.Running the test suiteTo run the test suite, you need the pytest (version >= 3) and coverage modules.
Run the test suite using:pytest joblibfrom the root of the project.Building the docsTo build the docs you need to have sphinx (>=1.4) and some dependencies
installed:pip install -U -r .readthedocs-requirements.txtThe docs can then be built with the following command:make docThe html docs are located in thedoc/_build/htmldirectory.Making a source tarballTo create a source tarball, eg for packaging or distributing, run the
following command:python setup.py sdistThe tarball will be created in thedistdirectory. This command will
compile the docs, and the resulting tarball can be installed with
no extra dependencies than the Python standard library. You will need
setuptool and sphinx.Making a release and uploading it to PyPIThis command is only run by project manager, to make a release, and
upload in to PyPI:python setup.py sdist bdist_wheel
twine upload dist/*Note that the documentation should automatically get updated at each git
push. If that is not the case, try building th doc locally and resolve
any doc build error (in particular when running the examples).Updating the changelogChanges are listed in the CHANGES.rst file. They must be manually updated
but, the following git command may be used to generate the lines:git log --abbrev-commit --date=short --no-merges --sparse
Package: joblibgcs
joblibgcsA google cloud storage memory backend for joblibinspired by and closely followingjoblib-s3relies heavily ongcsfsimportnumpyasnpfromjoblibimportMemory,register_store_backendfromjoblibgcs.gcs_backendimportGCSFSStoreBackendregister_store_backend("gcs",GCSFSStoreBackend)mem=Memory("cache-location",backend="gcs",verbose=100,backend_options={"project":"project-name"},)multiply=mem.cache(np.multiply)array1=np.arange(10000)array2=np.arange(10000)result=multiply(array1,array2)print(result)result2=multiply(array1,array2)print(result2)
Package: joblib-progress
joblib-progressA contextmanager to track progress ofjoblibexecution usingrich.progress.WhyThe vanillamultiprocessingdoes not work when an object to multiprocess is notpickle-able. Thejoblibsolves this, but then its progress is not tracked nicely. This library solves that tracking issue withjoblib.Install>pipinstalljoblib-progressUsageIf you know the number of itemsimporttimefromjoblibimportParallel,delayedfromjoblib_progressimportjoblib_progressdefslow_square(i):time.sleep(i/2)returni**2withjoblib_progress("Calculating square...",total=10):Parallel(n_jobs=4)(delayed(slow_square)(number)fornumberinrange(10))If you don't know the number of itemswithjoblib_progress("Calculating square..."):Parallel(n_jobs=4)(delayed(slow_square)(number)fornumberinrange(10))AcknowledgmentsThe idea of usingjoblib.parallel.BatchCompletionCallBackis referenced fromhttps://stackoverflow.com/a/58936697/5133167
Package: joblibs3
JoblibS3Python Package used to handle saving objects and Models on S3 BucketsFree software: MIT licenseDocumentation:https://joblibs3.readthedocs.io.FeaturesTODOCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2018-04-05)First release on PyPI.
Package: joblib-zstd
joblib-zstd is a plugin, which enables Zstandard (.zst) compression and decompression through joblib.dump and joblib.load.PrerequisitesYou needjoblib.Installpip install joblib-zstdIf you failed to install, update version of pip and setuptools:pip install -U pip setuptoolsUsageimportjoblibimportjoblib_zstdjoblib_zstd.register()joblib.dump({'a':1,'b':2},'obj.zst',compress=5)# implicit compressionjoblib.dump({'a':1,'b':2},'obj.bin',compress=('zstd',5))# explicit compressionjoblib.load('obj.zst')# implicit decompressionLISENCEMIT
Package: jobman
Jobman automates the process of running and monitoring jobs on the command line. Jobman supportsrunning commands in the background immune to hangupslogging command outputretrying commandsaborting commands after timeoutdelaying command execution for a specified time or eventsending notifications on command success or failureDocumentationVisit the :book:jobman documentation site:book: for complete information on using jobman.RequirementsJobman runs on UNIX-like operating systems including Linux and MacOS. Windows is not supported.Jobman requires Python3.9+.InstallationInstall or upgrade from PyPI with$ pip install jobmanExampleThe example below uses jobman to run a Python scripttrain.pyin the background and immune to hangups (e.g., a SIGHUP from an SHH timeout).Jobman will ensure 60 seconds have passedandthat the filedata.csvexists before starting the program. If those conditions haven't been met by 5:00PM on March 5, 2032, jobman will abort the job.Jobman will retry the program up to five times until there's one successful run, defined as an exit code of0or42, waiting ten seconds between retries.If the job succeeds, jobman will send a notification email. If the job fails, jobman will send an SMS message.$jobman\--wait-duration60s--wait-for-filedata.csv\--abort-time"2032-03-05T17:00:00"\--retry-attempts5--retry-delay10s-c0-c42\--notify-on-job-successmy-email--notify-on-job-failuremy-cell\train.py
12e4b604After submitting thetrain.pyjob above, usejobman showto display details on job progress:jobmanshow12e4b604To view a running log of the consolidated stdout and stderr streams of the latest run of the job, usejobman logs:jobmanlogs12e4b604--followAlternativesJobman aims to be reliable and fully-featured. It operateswithoutrequiring a system service/daemon for orchestration.Alternative tools for similar use cases include:cron: for scheduling repeated executions of a jobAirflow,Prefect, andDagster: for managing dependencies between multiple jobssupervisord: for daemon-based job managementDevelopingJobman uses pyenv for Python version management and poetry for builds. Before working on Jobman, ensure you havepyenvandpoetryinstalled.TheMakefiledefines targets for common operations during development, including the following:make setup: set up and install the packagemake fmt: run the autoformattersmake test: run the type tests and unit test suitemake build: build the jobman wheelTo release a new version of the package, use thebumpver.shscript. For example, to update to version 1.2.3, run./bumpver.sh 1.2.3.ContributingFeature requests, bug reports, and pull requests are welcome! SeeCONTRIBUTING.mdfor details on how to contribute to jobman.
Package: jobmanager
jobmanagerEasy distributed computing based on the python class SyncManager for remote communication and python module multiprocessing for local parallelism.:Note:
Although the documentation is not complete, the project is currently being
used successfully (which is the only reason why we have not reached a first
major release with a leading '1' as version number)Documentation (out of date)The documentation is available athttp://cimatosa.github.io/jobmanager/Developer's note (out of date)After cloning into jobmanager, create a virtual environmentvirtualenv --system-site-packages ve_jm
source ve_jm/bin/activateInstall all dependenciespython setup.py developRunning an examplepython examples/simple_example.pyRunning testspython setup.py test
Package: jobmanager-common
AboutThis library attempts to provide a Job Manager common library to use with client or apiProject url :https://github.com/ronhanson/python-jobmanager-commonUsage*Lib Usage*<BLAH>CompatibilityThis client can be used on Linux, OSX systems, or Windows.This libraries are compatibles with Python 2.X and Python 3.X.Mainly tested on 2.7 and 3.4.Author & LicenceCopyright (c) 2007-2015 Ronan DelacroixThis program is released under MIT Licence. Feel free to use it or part of it anywhere you want.
Package: jobmonitor
AFlaskweb application to monitorrqjobs, with a preference for
monitoring histograms usingd3.plotable.This base package allows you to quickly get a robust submit-poll loop up
on running on the client side, with the server handling job submission
and results retrieval.Seeexample-monitoring-appfor an example application that usesjobmonitorto displayhistogramsfrom aROOTfile, including
example deployment scripts in aVagrantvirtual machine.InstallingPipis the recommend way of installing thejobmonitormodule. As
the latest release is available onPyPI, installing it is simple.$pipinstalljobmonitorThe latest development version, thehead of the master branch, can be
installed instead, if desired.$pipinstall"git+https://github.com/alexpearce/jobmonitor.git#egg=jobmonitor"Either option will automatically install the monitor’s dependencies.RunningThe job monitor doesn’t do much user-facing stuff by itself, instead it
is expected that you will want tocreate a ‘child’ applicationthat
usesjobmonitor. You can run the application if you like though,
once it isinstalled, by creating and running a script to start
Flask’s development server.importjobmonitorapp=jobmonitor.create_app()app.run(debug=True)Therq workerscan be started with a separate script. Anexampleis included. ARedis databaseis
expected to be running when the workers start.TestingToxis recommended to run the test suite for thejobmonitormodule.$gitclonehttps://github.com/alexpearce/jobmonitor.git$cdjobmonitor$pipinstalltox$toxThis will run the test suite under the Python environments defined in
thetox.inifile.
Package: jobpilot
jobpilotjobpilotis a straightforward job scraping library.Current limitationsCurrently,jobpilotonly supports LinkedIn. However, integration with other platforms, such as Indeed, is in the pipeline.InstallationYou can installjobpilotfrom PyPI using popular Python package managers likepoetry,pip,pipx, and others.UsageA basic usage script is available in theexamplesdirectory.DevelopmentIf you're interested in contributing tojobpilot, start by cloning the repository:gitclonehttps://github.com/fabifont/jobpilot.gitcdjobpilotOnce cloned, install the development dependencies with:poetryinstallThis project maintains code quality and consistency using tools likepre-commit,ruff,black, andpyright. After cloning and installing the necessary dependencies, set up the pre-commit hooks:pre-commitinstallThe pre-commit configuration checks for various common issues. These hooks run automatically with every commit. If a hook detects an issue it cannot fix automatically, it will abort the commit, offering an explanation of the problem.To manually run pre-commit without committing, for instance, to validate your code before committing, use:pre-commitrun--all-filesAll contributors must adhere to these checks. Submitted code in pull requests must pass all Continuous Integration (CI) workflows. GitHub workflows will verify this automatically. Pull requests with failing checks will not be merged.jobpilotfollows theSemantic Versioning 2.0.0standard.ContributingContributions tojobpilotare very welcome! Whether you're reporting bugs, enhancing documentation, or improving the code, every contribution helps.Branching and commit naming conventionsTo ensure consistency and clarity across the codebase,jobpilotfollows a simplified naming convention for branches and commits. The naming conventions are based on the concepts shared in thisarticle.To assist with crafting conformant commit messages,jobpilotincludescommitizenin its development dependencies. After staging your changes, you can utilize:czcommitcommitizenwill guide you in generating a structured and consistent commit message, adhering to our project's conventions.Licensejobpilotis licensed under the GPL-3.0. For full details, see theLICENSEfile.
Package: job-pool
job-poolEnhanced Job Pool for Python MultiprocessingUsagefromjob_poolimportJobPooldefadd_one(i):returni+1defmultiprocessed_add_one():pool=JobPool(4)foriinrange(20):pool.applyAsync(add_one,[i])results=pool.checkPool(printProgressEvery=5)assertresults==list(range(1,21))Installationjob-pool is available on PyPI and can be installed withpip:pipinstalljob-poolAlternatively, you can install job-pool after cloning from this repository:gitclonehttps://github.com/matthewthe/job-pool.git
pipinstall.
Package: jobprogress
When doing complex processing that has to report progress indication to the user, things can get complex quick. Often, we don’t know beforehand how many work unit our processing will have, because knowing it depends on another work unit for which the progress should also be reported to the user. One example of such situation is processing files after having collected them. When we start the process, we don’t know how many files we’ll collect, so it’s hard to set a maximum value on our progress bar.jobprogresshandles that.Also, most of the time, we want to run our task in a separate thread so the GUI stays smooth.jobprogresstakes care of synchronizing the threaded task and the GUI.For now, only PyQt is supported, but the toolkit specific layer is pretty thin, so it should be easy to add new toolkits.The documentation has to be built with Sphinx. You can build the documentation with:cd docs
make htmlThe documentation is also available online athttp://www.hardcoded.net/docs/jobprogressChangesVersion 1.0.4 – 2012/03/20MadeThreadedJobPerformer.reraise_if_error()public and stopped calling it automatically.Version 1.0.3 – 2011/12/07Fixed weird meta-crash on ThreadedJobPerformer._reraise_if_error()Version 1.0.2 – 2011/07/11Added Job.check_if_cancelled().Version 1.0.1 – 2011/05/29Added debug information for a weird crash in ThreadedJobPerformer._reraise_if_error()Version 1.0.0 – 2010/11/20Initial Release