Saravanan666/pypi-org huggingface dataset:
Package: archive-cdx
A library to interact with the archive.org Wayback Machine CDX server.How toarchivecdx.Listingis the basic list class.PassCDX URL
argumentsas keyword args. (For arguments that are Python keywords, append an
underscore and the library will automatically remove it for you.)Special args:url- theurlparamdelay- the delay between automatic page count detection and
grabbing the first pageimportarchivecdxlisting=archivecdx.Listing("archive.org",fl=["original","timestamp","digest"])print(listing[0])# "Row(original='http://www.archive.org:80/',timestamp='19970126045828',digest='Q4YULN754FHV2U6Q5JUT6Q2P57WEWNNY')"archivecdx.PaginatedListinguses thePagination
APIto return less values at once.It supports thepagekeyword argument and adds anext_pageattribute, which will automatically pull the next page from the API.importarchivecdxlisting=archivecdx.PaginatedListing("archive.org",page=1,fl=["original","timestamp","digest"])print(len(listing.listing))# 15000
Package: archive-file-urls
archive-file-urlsarchive-file-urlsscans a file for URLs
and submits them to theInternet Archive Wayback Machinein order for them to be archived.
The program optionally supports exporting the resulting Internet Archive URLs.This is useful in all sorts of situations.
Take personal blog articles in any format (Markdown, HTML, ...) for example: You can use this program to save the linked web pages.The same goes for when you create a LaTex document and want to save the websites cited inside aBibTeXfile.And of course, there are many more ways to utilize this program.Installationpip install archive-file-urlsUsagearchive-file-urls --help
Package: archive-images
archive-imagesDescriptionSmall python package to create an image archive from a bunch of unordered images. The images are archived on the basis of the timestamp in the exif data.InstallationYou can install the simpleicp package fromPyPI:pip install archive-imagesThen call the help screen of executable witharchive-images --help.Usage notesThe default target path of an image in the archive isyear/month/%Y_%m_%d_%H_%M_%S.%f.ext, e.g.image_archive\2018\04\2018_04_03_17_21_20.219.jpg. Note that the extension of an image is automatically converted to lowercase, e.g..JPGto.jpg.Help screenusage: archive-images [-h] -i IMAGEFOLDER -a IMAGEARCHIVE [-e IMAGEEXTENSIONS] [-m {copy,move}]
                      [-fs FILTERBYIMAGESIZE FILTERBYIMAGESIZE] [-d] [-n] [-c]

required arguments:
  -i IMAGEFOLDER, --imageFolder IMAGEFOLDER
                        Path to input folder with images to archive. (default: None)
  -a IMAGEARCHIVE, --imageArchive IMAGEARCHIVE
                        Path to output folder with image archive. (default: None)

optional arguments:
  -e IMAGEEXTENSIONS, --imageExtensions IMAGEEXTENSIONS
                        Extensions of images to archive separated by commas.
                        (default: ['jpg', 'jpeg'])
  -m {copy,move}, --mode {copy,move}
                        Move or copy image files to archive? (default: copy)
  -fs FILTERBYIMAGESIZE FILTERBYIMAGESIZE, --filterByImageSize FILTERBYIMAGESIZE FILTERBYIMAGESIZE
                        Archive only images with a specific image size. The image size must be
                        specified as two integer numbers separated by a single space,
                        i.e. num1 num2. The orientation of the images (portrait or landscape) is
                        not considered by this filter, i.e. images with size num1-by-num2 or
                        num2-by-num1 are archived. (default: [0, 0])
  -d, --addDuplicates   Add duplicates to a subfolder "duplicates" in image archive?
                        (default: False)
  -n, --addNoExif       Add images with no exif information to a subfolder "no_exif" in image
                        archive? (default: False)
  -c, --confirm         Confirm each operation before execution? (default: False)
Package: archiveIO
Here are some decorators for reading and writing to compressed archives.Installationeasy_install -U archiveIOUsageimport archiveIO
import os
from six import BytesIO

# Define a function that generates archive contents
@archiveIO.save
def save(targetPath):
    open(targetPath, 'wt').write('xxx')
# Define a function that processes archive contents
@archiveIO.load
def load(sourcePath):
    return open(sourcePath, 'rt').read()

# Save archives
save('sample.txt')
save('sample.txt.zip')
save('sample.txt.tar.gz')
save('sample.txt.tar.bz2')
save('sample.txt.tar')
# Load archives
assert 'xxx' == load('sample.txt')
assert 'xxx' == load('sample.txt.zip')
assert 'xxx' == load('sample.txt.tar.gz')
assert 'xxx' == load('sample.txt.tar.bz2')
assert 'xxx' == load('sample.txt.tar')

# Create an archive containing two files
@archiveIO.save
def save(targetPath):
    open(targetPath + '.txt', 'wt')
    open(targetPath + '.csv', 'wt')
targetPath = 'sample.zip'
save(targetPath)
# Target CSV files before TXT files
@archiveIO.load(extensions=['.csv', '.txt'])
def load(sourcePath):
    return os.path.basename(sourcePath)
assert 'sample.csv' == load(targetPath)

# Use MyException instead of IOError
class MyException(Exception):
    pass
@archiveIO.load(CustomException=MyException)
def load(sourcePath):
    return sourcePath
try:
    load('xxx.tar.gz')
except MyException, error:
    print error

# Compress directly into a string buffer
archive = archiveIO.Archive(BytesIO(), '.tar.gz')
archive.save([
    'sample.txt',
    'sample.txt.zip',
])
# Uncompress into a temporary folder
with archiveIO.TemporaryFolder() as temporaryFolder:
    for filePath in archive.load(temporaryFolder):
        print filePath0.5.0Fixed Python 3 support0.4.4Added option to target file extensions for uncompressed files0.4.3Added CustomException coverage if opening archive fails0.4.2Added CustomException option to load() decorator0.4.1Excluded extraneous ‘.’ when saving zip archives0.4.0Added support for file-like objects when saving or loading archivesAdded support for folderPaths when saving archivesIncreased test coverage to 100%0.3.0Added Archive class0.2.0Ported code from zip_storeAdded support for .tar.gz .tar.bz2 .tar
Package: archiveis
archiveisA simple Python wrapper for thearchive.iscapturing service.InstallationpipenvinstallarchiveisPython UsageImport it.>>>importarchiveisCapture a URL.>>>archive_url=archiveis.capture("http://www.example.com/")See where it's stored.>>>printarchive_urlhttp://archive.md/WxlRKIf a URL has been recently cached, archive.is may return the URL to that page rather
than conduct a new capture.Command-line usageThe Python library is also installed as a command-line interface. You can run it from your terminal like so:archiveishttp://www.example.com/The command has the same options as the Python API, which you can learn about from its help output.$archiveis--help
Usage:archiveis[OPTIONS]URLArchivestheprovidedURLusingthearchive.iscapturingservice.

Options:-ua,--user-agentTEXTUser-Agentheaderforthewebrequest--helpShowthismessageandexit.ContributingInstall dependencies for development.pipenvinstall--devRun tests.maketestShip new version to PyPImakeshipDeveloping the CLIThe command-line interface is implemented using Click and setuptools. To install it locally for development inside your virtual environment, run the following installation command, asprescribed by the Click documentation.pipinstall--editable.
Package: archive-manager
Archive ManagerManage files in directories in a backup root directory. It will ensure the total size of all the files in the directories are less than a maximum size, but will always keep the minimum number of files. Once the total size is ensured, it will then delete files until the maximum number of files exist.The assumption is that some other process, such as a backup script, is generating "archives" in a directory or directories.archive-managerwill help to manage those files to ensure that there is always room for the most recent backup file.RequirementsThis will probably only run on LinuxInstallationFuture releases will include installation via pip and RPM.virtualenvarchive-manager-venv
.archive-manager-venv/bin/activate
pipinstallarchive-managerNow that it's installed, thearchive-managercommand should be available.$whicharchive-manager
<venv>/bin/archive-managerUsageBy defaultarchive-managerlooks for its config file in/etc/archive-manager/config.yml. It does not require any options to run, and can simply be executed byarchive-manager.Options:$archive-manager--help
Usage:archive-manager[OPTIONS]COMMAND[ARGS]...Managesizeandnumberoffilesinasetofdirectories

Options:-c,--configTEXTConfigfilelocation-v,--verboseSettoverbosemode--helpShowthismessageandexit.Configuration FileThere is anexample configurationfile in this repository. All of the options in the example file are required.mkdir/etc/archive-manager
cpconfig.yaml.example/etc/archive-manager/config.yml
vi/etc/archive-manager/config.yml#edit file to suit requirementsTestUnit Testsmake testwill run the available unit tests.
Package: archivematica-omekas
archivematica-omekasInstallpipinstallarchivematica_omekas設定Archivematicaの設定dip_zip_file_path="data/sample/ex02-1-TIFF-007c28cc-4671-4d5d-be2a-facace4c7b84.zip"mapping_json_file_path="data/sample/mapping.json"Omekaの設定API_URL="https://omekas.aws.ldas.jp/sandbox/api"KEY_IDENTITY="2st7sDKL2UAL8OKtfah8ffZS5UjefA7g"KEY_CREDENTIAL="M4s6xyqkzbxgJZieQfsMysVUqzdC6ZPl"実行output_dir=ArchiveMaticaOmeka.convert(dip_zip_file_path,mapping_json_file_path,task_id="test")metadata_csv_path=f"{output_dir}/metadata.csv"media_csv_path=f"{output_dir}/media.csv"ArchiveMaticaOmeka.upload(metadata_csv_path,media_csv_path,API_URL,KEY_IDENTITY,KEY_CREDENTIAL)アイテムの登録

100%|██████████| 4/4 [00:26<00:00,  6.60s/it]


IDの取得

100%|██████████| 4/4 [00:01<00:00,  2.35it/s]


メディアの登録

100%|██████████| 4/4 [00:33<00:00,  8.49s/it]以下のURLから、アイテムが登録されていることをご確認ください。url=f"{API_URL.replace('/api','')}"url'https://omekas.aws.ldas.jp/sandbox'
Package: archivematica-tools
Archivematica ToolsInstallpipinstallmatica_toolsHow to useSee the documentationfor full details of the Archivematica
API Client.fromdotenvimportload_dotenvimportosload_dotenv(override=True)dashboard_url=os.environ.get("DASHBOARD_URL")dashboard_username=os.environ.get("DASHBOARD_USERNAME")dashboard_api_key=os.environ.get("DASHBOARD_API_KEY")storage_service_url=os.environ.get("STORAGE_SERVICE_URL")storage_service_username=os.environ.get("STORAGE_SERVICE_USERNAME")storage_service_password=os.environ.get("STORAGE_SERVICE_PASSWORD")client=ArchivematicaAPIClient(dashboard_url,dashboard_username,dashboard_api_key,storage_service_url,storage_service_username,storage_service_password)transfer_UUID=client.v2beta_package("standard","","bfe37714-61c2-432e-b375-af9253e114da","/home/archivematica/archivematica-sampledata/SampleTransfers/DemoTransferCSV","transfer v2 demo","automated")transfer_UUID'2d68be44-48f4-4b15-8db8-d8d1250f50bf'Seethe documentationfor more examples.Created bySatoru
Nakamura.
Package: archive-path
archive-pathA package to provide pathlib like access to zip & tar archives.Installation$pipinstallarchive-pathUsageFor reading zip (ZipPath) or tar (TarPath) files:fromarchive_pathimportTarPath,ZipPathpath=TarPath("path/to/file.tar.gz",mode="r:gz")sub_path=path/"folder"/"file.txt"assertsub_path.filepath=="path/to/file.tar.gz"assertsub_path.at=="folder/file.txt"assertsub_path.exists()andsub_path.is_file()assertsub_path.parent.is_dir()content=sub_path.read_text()forsub_pathinpath.iterdir():print(sub_path)For writing files, you should use within a context manager, or directly call theclosemethod:withTarPath("path/to/file.tar.gz",mode="w:gz")aspath:(path/"new_file.txt").write_text("hallo world")# there are also some features equivalent to shutil(path/"other_file.txt").putfile("path/to/external_file.txt")(path/"other_folder").puttree("path/to/external_folder",pattern="**/*")Note that archive formats do not allow to overwrite existing files (they will raise aFileExistsError).For performant access to single files:fromarchive_pathimportread_file_in_tar,read_file_in_zipcontent=read_file_in_tar("path/to/file.tar.gz","file.txt",encoding="utf8")These methods allow for faster access to files (using less RAM) in archives containing 1000's of files.
This is because, the archive's file index is only read until the path is found (discarding non-matches),
rather than the standardtarfile/zipfileapproach that is to read the entire index into memory first.Windows compatibilityPaths within the archives arealwaysread and written as being/delimited.
This means that the package works on Windows,
but will not be compatible with archives written outside this package with\\path delimiters.DevelopmentSeeCONTRIBUTING.mdfor details on how to contribute to this package.
Package: archive-project
archive-projectCLI tool to archive a Python project under the current working directory
ignoring things like.idea/,.mypy_cache,.venv, etc. and send the
resultzipto the desktopInstallationpip install -U --user archive-projectThe package will be installed in your user home directory. Seepipdocumentation aboutuser installs. You need the installation directory
to be present inPATHto runarchive-projectfrom the terminal.Usage$archive-project--helpUsage: archive-project [OPTIONS]Zip current Python projectOptions:--version  Show the version and exit.--help     Show this message and exit.$archive-project/Users/user/Desktop/archive-project.zip was successfully createdRequirementsPython 3.8+macOS
Package: archive-reddit-user
archive-reddit-userThis program allows a user to export their own comments (and others) and preserve the context surrounding the comment. The comments and the context surrounding the comment (if applicable) are saved in JSON format.You can take it a step further and and create a hostable page to easily read and search through the comments.Installation and UsageInstall using pip:pip install archive-reddit-user.Follow the setup wizard to create aconfig.inifile.Run the command:archive-reddit-userin your Terminal.Arguments--user: specify a specific user to archive their comments.--limit: returns your rate usage (does uses one call).Coming SoonTesting--nuke: edit the comment and delete them.--publish: Create a page you can upload anywhere that provides a nice front-end to navigate
through the JSON-ified comments, with search!Is this safe?When you first runarchive-reddit-user, you're prompted to enter your client id, client secret (also known as an API key in other services), and credentials to your reddit account. These credentials are stored locally so you don't have to enter them everytime you run the program. You may wonder why it is in plaintext, I wrote something in thewikifor more context.
Package: archive-repo
archive-repoCaptures and stores data about a GitHub repo for archival purposesArchive commandusage: python3 -m archive-repo archive [-h] [--reference [REFFILE]] [--issues-only] [--quiet] repo githubToken [outFile]Archive repo issues and PRs.positional arguments:repo:  The GitHub repository to fetch, e.g. "quicwg/base-drafts"githubToken:  A token for a GitHub account with permission to read the
selected repository; seehttps://github.com/settings/tokensoutfile:  Where the output should be written; if omitted, stdout is used.optional arguments:-h: Displays help text--reference: Optionally, previous output of this script.  Makes the new run
faster, since only new/changed issues and PRs need to be fetched.--issues-only:  Only fetch issues, not PRs--quiet:  Only write errors to stderr; otherwise, status is written to
stderrNote that providing an output file can make processing faster if the respository
has not changed since the reference file was generated.Generate PR Branchesusage: python3 -m archive-repo pr_branches [-h] [--include-merged] [--dry-run] input repo git_dirIdentify the source repos and forks of PRs from an archive, creating local branches as neededpositional arguments:input:  Archive file generated bypython3 -m archive-repo archiverepo:  Target GitHub repogit_dir:  Local directory with git repo (default: local directory)optional arguments:-h: Display help message--include-merged: include merged PRs in analysis--dry-run: Show what would be done, without making any changes
Package: archive-rotator
Flexible utility for rotating backup files.Free software: MIT licenseDocumentation:https://archive-rotator.readthedocs.org.Code:https://github.com/maxharp3r/archive-rotatorThis utility rotates files - typically backup archives. It offers three rotation algorithms - FIFO, Tower of Hanoi, and
tiered (a generalization of grandfather-father-son). It requires no configuration file, just command-line parameters.
The script is stand-alone (python required) and tracks its state by applying a naming convention to rotated files.Learn about the concept of archive rotation:http://en.wikipedia.org/wiki/Backup_rotation_schemeExample UseWe assume that you have an archive, say/path/to/foo/mydump.tgzthat is the result of an external process (e.g.,tar
-czf, ormysqldump) that recurs (e.g., usingcron). You will use this script to add this file into a rotation set.
If the size of the set is greater than the maximum number of files to keep (configurable), then the rotation set will
be trimmed according to a rotation algorithm, which is configurable.Example of running:archive-rotator -v -n 5 /path/to/foo/mydump.tgzThis will rename mydump.tgz to something like this:/path/to/foo/mydump.tgz.2012-12-20-133640.backup-0Given this configuration, the rotation script automatically keep at most 5 files in the rotation. When the script is run
six times, the set of archives would be too big, so the oldest will be deleted. This is an example of the simple (FIFO)
rotation algorithm.History0.2.1 (2015-12-03)Improve and document programmatic access0.2.0 (2015-12-03)Much improved test coverage and refactored codebaseAdded –destination-path parameter0.1.0 (2015-11-23)First release on PyPI
Package: archives
archives: a new way to do python code documentationarchivesis a new style of python code documentation, as well as a
linter for the documentation itself. It can help you ensure that your
docstrings in your classes and functions stay up to date, and that they
adequately explain their purpose, arguments, and return value.Featureslinter for docstrings (work in progress, but usable!)(coming soon) documentation generatorUsageInstallationpipinstallarchivesRun the Linter# run archives (on itself!)archivesarchives/# archives.py:846:0: F104 function 'path_empty' missing @ret tag## Impossible! Perhaps your archives are incomplete?# 1 issues found.# list tags!archives--list-tag# @arg    describe an argument of a function# @author denote the author of a module/class/function# @cc     denote the complexity of a function# @desc   describe a module/class/function# @link   add a link to the generated documentation# @nodoc  disable this module/class/function in the documentation# @nolint disable archives linting in this module/class/function# @note   add a note to a module/class/function# @ret    describe the return value of a function# @todo   tag something as a todo# @warn   add a warning to a module/class/function# list rules!archives--list-rules# disable rules!archives--disableM100.# different formats for output! defaults to flake8archives--formatpylintarchives.pyTestingTests can be run with tox!# run teststox# only one version of pythontox-epy36Todomore rulesbetter system for multi-check rulesmore output formatspotentially spell-checking inside desc?documentation generatortests
Package: archives_org_latin_toolkit
What ?This piece of software is intended to be used with the 11K Latin Texts produced by David Bamman (http://www.cs.cmu.edu/~dbamman/latin.html). It supports only the plain text formats and the metadata github repo CSV file. This has been tested withPython3only. I welcome any new functions or backward compatibility support.How to install ?With development version:Clone the repository :git clone https://github.com/ponteineptique/archives_org_latin_toolkit.gitGo to the directory :cd archives_org_latin_toolkitInstall the source with develop option :python setup.py installWith pip:Install from pip :pip install archives_org_latin_toolkitExampleThe following example should run with the data in tests/test_data. The example can be run withpython example.py# We import the main classes from the modulefromarchives_org_latin_toolkitimportRepo,Metadatafrompprintimportpprint# We initiate a Metadata object and a Repo objectmetadata=Metadata("./test/test_data/latin_metadata.csv")# We want the text to be set in lowercaserepo=Repo("./test/test_data/archive_org_latin/",metadata=metadata,lowercase=True)# We define a list of token we want to search fortokens=["ecclesiastico","ecclesia","ecclesiis","&quot;"]# We instantiate a result storageresults=[]# We iter over text having those tokens :# Note that we need to "unzip" the listfortext_matchinginrepo.find(*tokens):# For each text, we iter over embeddings found in the text# We want 3 words left, 3 words right,# and we want to keep the original token (Default behaviour)forembeddingintext_matching.find_embedding(*tokens,window=3,ignore_center=False):# We add it to the resultsresults.append(embedding)# We print the result (list of list of strings)pprint(results)
Package: archivesspace-jsonmodel-converter
ArchivesSpace JSONModel ConverterInstallationNavigate to the top level of this repository, and run:pipinstall-e.After installation, the tool should be available asajc, runningajcwill emit a help page describing any extant subcommands.ConfigurationAJC is configured via a YAML file. Some values are hardcoded as defaults and can be omitted - currently, this is limited to anasnake_configkey containing values for the default admin account of a local ASpace instance.By default, AJC looks for a.archivesspace_jsonmodel_converter.ymlin the user's home directory.  A different file location can be specified via the AJC_CONFIG_FILE variable, or by providing a path to theajccommend via the--config-fileoption.Conversion of Access DB to postgresql DBFor various reasons, it's more convenient to move the data from access to another data format before trying to convert it to ASpace JSONModel objects.  For this example, we will convert the data to a postgresql-compatible SQL file.  Additionally, we will deal with some issues specific to the Fort Lewis DB but that will commonly occur when converting data from MDB format.ToolsThe tools we'll be using are:mdbtools- a set of command line tools for accessing and working with Access DBsa Linux shell with mdbtools installedpsqla text editor of the user's choiceProcessStep 1First, we need to create an SQL script that represents the access DB's schema.  We do this with the following commend:mdb-schemamy_access_file.mdbpostgres>my_postgres_schema.sqlStep 2Next, we need to add insert statements for all the tables in the schema.  This can be done via a shell loop, thusly:fornamein`mdb-tablesmy_access_file.mdb`;domdb-export-Ipostgresmy_access_file.mdb$name>>my_postgres_schema.sql;doneAt this point, we have a schema with all tables, constraints, and data from the Access db - if we'revery lucky, this might just import cleanly into postgres at this point.Step 3Assuming postgres is already set up and the user account you're logged in as can create databases:createdbmy_access_db
psqlmy_access_db<my_postgres_schema.sql&>psql_import_output.txtAfter this completes, check the input - there's a good chance you'll see some errors - Postgres (and MySQL, and most things for that matter) are more strict about foreign keys and indexes than Access, and don't let you do things like embed a whole word document as a column value. If there are errors, open the SQL file in a text editor that supports large files, correct the errors or drop the overly strict indexes, and then drop and recreate the table and run Step 3 again. When it runs without errors, you've successfully migrated the data.