neulab/tldr huggingface dataset:
client_config_backend: settings client_config: client_id: <Client ID from developers' console> client_secret: <Client secret from developers' console> save_credentials: True save_credentials_backend: file save_credentials_file: <filename to cache credentials> get_refresh_token: True
In this scenario, the username and host parts of the URL play no role; only the path matters. During the first run, you will be prompted to visit an URL in your browser to grant access to your drive. Once granted, you will receive a verification code to paste back into Duplicity. The credentials are then cached in the file references above for future use.
GDrive: is a rewritten PyDrive: backend with less dependencies, and a simpler setup - it uses the JSON keys downloaded directly from Google Cloud Console.
Note Google has 2 drive methods, `Shared(previously Team) Drives` and `My Drive`, both can be shared but require different addressing
For a Google Shared Drives folder
Share Drive ID specified as a query parameter, driveID,  in the backend URL.  Example: gdrive://developer.gserviceaccount.com/target- folder/?driveID=<SHARED DRIVE ID>
For a Google My Drive based shared folder
MyDrive folder ID specified as a query parameter, myDriveFolderID, in the backend URL Example export GOOGLE_SERVICE_ACCOUNT_URL=<serviceaccount- name>@<serviceaccount-name>.iam.gserviceaccount.com gdrive://${GOOGLE_SERVICE_ACCOUNT_URL}/<target-folder-name-in- myDriveFolder>?myDriveFolderID=<google-myDrive-folder-id>
There are also two ways to authenticate to use GDrive: with a regular account or with a "service account". With a service account, a separate account is created, that is only accessible with Google APIs and not a web login.  With a regular account, you can store backups in your normal Google Drive.
To use a service account, go to the Google developers console at https://console.developers.google.com. Create a project, and make sure Drive API is enabled for the project. In the "Credentials" section, click "Create credentials", then select Service Account with JSON key.
The GOOGLE_SERVICE_JSON_FILE environment variable needs to contain the path to the JSON file on duplicity invocation.
export GOOGLE_SERVICE_JSON_FILE=<path-to-serviceaccount-credentials.json>
The alternative is to use a regular account. To do this, start as above, but when creating a new Client ID, select "Create OAuth client ID", with application type of "Desktop app". Download the client_secret.json file for the new client, and set the GOOGLE_CLIENT_SECRET_JSON_FILE environment variable to the path to this file, and GOOGLE_CREDENTIALS_FILE to a path to a file where duplicity will keep the authentication token - this location must be writable.
During the first run, you will be prompted to visit an URL in your browser to grant access to your drive. Once granted, you will receive a verification code to paste back into Duplicity. The credentials are then cached in the file references above for future use.
As a sanity check, GDrive checks the host and username from the URL against the JSON key, and refuses to proceed if the addresses do not match. Either the email (for the service accounts) or Client ID (for regular OAuth accounts) must be present in the URL. See URL FORMAT above.
Rclone is a powerful command line program to sync files and directories to and from various cloud storage providers.
Once you have configured an rclone remote via
and successfully set up a remote (e.g. gdrive for Google Drive), assuming you can list your remote files with
rclone ls gdrive:mydocuments
you can start your backup with
duplicity /mydocuments rclone://gdrive:/mydocuments
Please note the slash after the second colon. Some storage provider will work with or without slash after colon, but some other will not. Since duplicity will complain about malformed URL if a slash is not present, always put it after the colon, and the backend will handle it for you.
The ssh backends support sftp and scp/ssh transport protocols.  This is a known user-confusing issue as these are fundamentally different.  If you plan to access your backend via one of those please inform yourself about the requirements for a server to support sftp or scp/ssh access.  To make it even more confusing the user can choose between several ssh backends via a scheme prefix: paramiko+ (default), pexpect+, lftp+... . paramiko & pexpect support --use-scp, --ssh-askpass and --ssh-options. Only the pexpect backend allows to define --scp-command and --sftp- command.
SSH paramiko backend (default) is a complete reimplementation of ssh protocols natively in python. Advantages are speed and maintainability. Minor disadvantage is that extra packages are needed as listed in REQUIREMENTS.  In sftp (default) mode all operations are done via the according sftp commands. In scp mode ( --use-scp ) though scp access is used for put/get operations but listing is done via ssh remote shell.
SSH pexpect backend is the legacy ssh backend using the command line ssh binaries via pexpect.  Older versions used scp for get and put operations and sftp for list and delete operations.  The current version uses sftp for all four supported operations, unless the --use-scp option is used to revert to old behavior.
SSH lftp backend is simply there because lftp can interact with the ssh cmd line binaries.  It is meant as a last resort in case the above options fail for some reason.
Why use sftp instead of scp?  The change to sftp was made in order to allow the remote system to chroot the backup, thus providing better security and because it does not suffer from shell quoting issues like scp.  Scp also does not support any kind of file listing, so sftp or ssh access will always be needed in addition for this backend mode to work properly. Sftp does not have these limitations but needs an sftp service running on the backend server, which is sometimes not an option.
Certificate verification as implemented right now [02.2016] only in the webdav and lftp backends. older pythons 2.7.8- and older lftp binaries need a file based database of certification authority certificates (cacert file). Newer python 2.7.9+ and recent lftp versions however support the system default certificates (usually in /etc/ssl/certs) and also giving an alternative ca cert folder via --ssl-cacert-path.
The cacert file has to be a PEM formatted text file as currently provided by the CURL project. See
http://curl.haxx.se/docs/caextract.html
After creating/retrieving a valid cacert file you should copy it to either
~/.duplicity/cacert.pem ~/duplicity_cacert.pem /etc/duplicity/cacert.pem
Duplicity searches it there in the same order and will fail if it can't find it.  You can however specify the option --ssl-cacert-file <file> to point duplicity to a copy in a different location.
Finally there is the --ssl-no-check-certificate option to disable certificate verification alltogether, in case some ssl library is missing or verification is not wanted. Use it with care, as even with self signed servers manually providing the private ca certificate is definitely the safer option.
A NOTE ON SWIFT (OPENSTACK OBJECT STORAGE) ACCESS Swift is the OpenStack Object Storage service. The backend requires python-switclient to be installed on the system. python-keystoneclient is also needed to use OpenStack's Keystone Identity service.  See REQUIREMENTS.
It uses following environment variables for authentification: SWIFT_USERNAME (required), SWIFT_PASSWORD (required), SWIFT_AUTHURL (required), SWIFT_USERID (required, only for IBM Bluemix ObjectStorage), SWIFT_TENANTID (required, only for IBM Bluemix ObjectStorage), SWIFT_REGIONNAME (required, only for IBM Bluemix ObjectStorage), SWIFT_TENANTNAME (optional, the tenant can be included in the username)
If the user was previously authenticated, the following environment variables can be used instead: SWIFT_PREAUTHURL (required), SWIFT_PREAUTHTOKEN (required)
If SWIFT_AUTHVERSION is unspecified, it will default to version 1.
PCA is a long-term data archival solution by OVH. It runs a slightly modified version of Openstack Swift introducing latency in the data retrieval process.  It is a good pick for a multi backend configuration where receiving volumes while an other backend is used to store manifests and signatures.
The backend requires python-switclient to be installed on the system. python-keystoneclient is also needed to interact with OpenStack's Keystone Identity service.  See REQUIREMENTS.
It uses following environment variables for authentification: PCA_USERNAME (required), PCA_PASSWORD (required), PCA_AUTHURL (required), PCA_USERID (optional), PCA_TENANTID (optional, but either the tenant name or tenant id must be supplied) PCA_REGIONNAME (optional), PCA_TENANTNAME (optional, but either the tenant name or tenant id must be supplied)
If the user was previously authenticated, the following environment variables can be used instead: PCA_PREAUTHURL (required), PCA_PREAUTHTOKEN (required)
If PCA_AUTHVERSION is unspecified, it will default to version 2.
This backend requires mediafire python library to be installed on the system. See REQUIREMENTS.
Use URL escaping for username (and password, if provided via command line):
mf://duplicity%40example.com@mediafire.com/some_folder
The destination folder will be created for you if it does not exist.
Signing and symmetrically encrypt at the same time with the gpg binary on the command line, as used within duplicity, is a specifically challenging issue.  Tests showed that the following combinations proved working.
1. Setup gpg-agent properly. Use the option --use-agent and enter both passphrases (symmetric and sign key) in the gpg-agent's dialog.
2. Use a PASSPHRASE for symmetric encryption of your choice but the signing key has an empty passphrase.
3. The used PASSPHRASE for symmetric encryption and the passphrase of the signing key are identical.
KNOWN ISSUES / BUGS Hard links currently unsupported (they will be treated as non-linked regular files).
Bad signatures will be treated as empty instead of logging appropriate error message.
This section describes duplicity's basic operation and the format of its data files.  It should not necessary to read this section to use duplicity.
The files used by duplicity to store backup data are tarfiles in GNU tar format.  They can be produced independently by rdiffdir(1).  For incremental backups, new files are saved normally in the tarfile.  But when a file changes, instead of storing a complete copy of the file, only a diff is stored, as generated by rdiff(1).  If a file is deleted, a 0 length file is stored in the tar.  It is possible to restore a duplicity archive "manually" by using tar and then cp, rdiff, and rm as necessary. These duplicity archives have the extension difftar.
Both full and incremental backup sets have the same format.  In effect, a full backup set is an incremental one generated from an empty signature (see below).  The files in full backup sets will start with duplicity- full while the incremental sets start with duplicity-inc.  When restoring, duplicity applies patches in order, so deleting, for instance, a full backup set may make related incremental backup sets unusable.
In order to determine which files have been deleted, and to calculate diffs for changed files, duplicity needs to process information about previous sessions.  It stores this information in the form of tarfiles where each entry's data contains the signature (as produced by rdiff) of the file instead of the file's contents.  These signature sets have the extension sigtar.
Signature files are not required to restore a backup set, but without an up-to-date signature, duplicity cannot append an incremental backup to an existing archive.
To save bandwidth, duplicity generates full signature sets and incremental signature sets.  A full signature set is generated for each full backup, and an incremental one for each incremental backup.  These start with duplicity-full-signatures and duplicity-new-signatures respectively. These signatures will be stored both locally and remotely. The remote signatures will be encrypted if encryption is enabled.  The local signatures will not be encrypted and stored in the archive dir (see --archive-dir ).
Duplicity requires a POSIX-like operating system with a python interpreter version 2.6+ installed.  It is best used under GNU/Linux.
Some backends also require additional components (probably available as packages for your specific platform):
Amazon Drive backend python-requests - http://python-requests.org python-requests-oauthlib - https://github.com/requests/requests- oauthlib
azure backend (Azure Storage Blob Service) Microsoft Azure Storage Blobs client library for Python - https://pypi.org/project/azure-storage-blob/
boto backend (S3 Amazon Web Services, Google Cloud Storage) boto version 2.0+ - http://github.com/boto/boto
box backend (box.com) boxsdk - https://github.com/box/box-python-sdk
cfpyrax backend (Rackspace Cloud) and hubic backend (hubic.com) Rackspace CloudFiles Pyrax API - http://docs.rackspace.com/sdks/guide/content/python.html
dpbx backend (Dropbox) Dropbox Python SDK - https://www.dropbox.com/developers/reference/sdk
gdocs gdata backend (legacy Google Docs backend) Google Data APIs Python Client Library - http://code.google.com/p/gdata-python-client/
gdocs pydrive backend(default) see pydrive backend
gio backend (Gnome VFS API) PyGObject - http://live.gnome.org/PyGObject D-Bus (dbus)- http://www.freedesktop.org/wiki/Software/dbus
lftp backend (needed for ftp, ftps, fish [over ssh] - also supports sftp, webdav[s]) LFTP Client - http://lftp.yar.ru/
MEGA backend (only works for accounts created prior to November 2018) (mega.nz) megatools client - https://github.com/megous/megatools
MEGA v2 and v3 backend (works for all MEGA accounts) (mega.nz) MEGAcmd client - https://mega.nz/cmd
multi backend Multi -- store to more than one backend (also see A NOTE ON MULTI BACKEND ) below.
ncftp backend (ftp, select via ncftp+ftp://) NcFTP - http://www.ncftp.com/
OneDrive backend (Microsoft OneDrive) python-requests-oauthlib - https://github.com/requests/requests- oauthlib
Par2 Wrapper Backend par2cmdline - http://parchive.sourceforge.net/
pydrive backend PyDrive -- a wrapper library of google-api-python-client - https://pypi.python.org/pypi/PyDrive (also see A NOTE ON PYDRIVE BACKEND ) below.
rclone backend rclone - https://rclone.org/
rsync backend rsync client binary - http://rsync.samba.org/
ssh paramiko backend (default) paramiko (SSH2 for python) - http://pypi.python.org/pypi/paramiko (downloads); http://github.com/paramiko/paramiko (project page) pycrypto (Python Cryptography Toolkit) - http://www.dlitz.net/software/pycrypto/
ssh pexpect backend sftp/scp client binaries OpenSSH - http://www.openssh.com/ Python pexpect module - http://pexpect.sourceforge.net/pexpect.html
swift backend (OpenStack Object Storage) Python swiftclient module - https://github.com/openstack/python- swiftclient/ Python keystoneclient module - https://github.com/openstack/python-keystoneclient/
webdav backend certificate authority database file for ssl certificate verification of HTTPS connections - http://curl.haxx.se/docs/caextract.html (also see A NOTE ON SSL CERTIFICATE VERIFICATION). Python kerberos module for kerberos authentication - https://github.com/02strich/pykerberos
MediaFire backend MediaFire Python Open SDK - https://pypi.python.org/pypi/mediafire/
Original Author - Ben Escoto <bescoto@stanford.edu>
Current Maintainer - Kenneth Loafman <kenneth@loafman.com>
Continuous Contributors Edgar Soldin, Mike Terry
Most backends were contributed individually.  Information about their authorship may be found in the according file's header.
Also we'd like to thank everybody posting issues to the mailing list or on launchpad, sending in patches or contributing otherwise. Duplicity wouldn't be as stable and useful if it weren't for you.
A special thanks goes to rsync.net, a Cloud Storage provider with explicit support for duplicity, for several monetary donations and for providing a special "duplicity friends" rate for their offsite backup service.  Email info@rsync.net for details.
rdiffdir(1), python(1), rdiff(1), rdiff-backup(1).
jarsigner(1)                     Security Tools                     jarsigner(1)
jarsigner - Signs and verifies Java Archive (JAR) files.
jarsigner [ options ] jar-file alias
jarsigner -verify [ options ] jar-file [alias ...]
options The command-line options. See Options.
-verify The -verify option can take zero or more keystore alias names after the JAR file name. When the -verify option is specified, the jarsigner command checks that the certificate used to verify each signed entry in the JAR file matches one of the keystore aliases. The aliases are defined in the keystore specified by -keystore or the default keystore.
If you also specified the -strict option, and the jarsigner command detected severe warnings, the message, "jar verified, with signer errors" is displayed.
jar-file The JAR file to be signed.
If you also specified the -strict option, and the jarsigner command detected severe warnings, the message, "jar signed, with signer errors" is displayed.
alias  The aliases are defined in the keystore specified by -keystore or the default keystore.
The jarsigner tool has two purposes:
• To sign Java Archive (JAR) files.
• To verify the signatures and integrity of signed JAR files.
The JAR feature enables the packaging of class files, images, sounds, and other digital data in a single file for faster and easier distribution. A tool named jar enables developers to produce JAR files. (Technically, any zip file can also be considered a JAR file, although when created by the jar command or processed by the jarsigner command, JAR files also contain a META-INF/MANIFEST.MF file.)
A digital signature is a string of bits that is computed from some data (the data being signed) and the private key of an entity (a person, company, and so on). Similar to a handwritten signature, a digital signature has many useful characteristics:
• Its authenticity can be verified by a computation that uses the public key corresponding to the private key used to generate the signature.
• It cannot be forged, assuming the private key is kept secret.
• It is a function of the data signed and thus cannot be claimed to be the signature for other data as well.
• The signed data cannot be changed. If the data is changed, then the signature cannot be verified as authentic.
To generate an entity's signature for a file, the entity must first have a public/private key pair associated with it and one or more certificates that authenticate its public key. A certificate is a digitally signed statement from one entity that says that the public key of another entity has a particular value.
The jarsigner command uses key and certificate information from a keystore to generate digital signatures for JAR files. A keystore is a database of private keys and their associated X.509 certificate chains that authenticate the corresponding public keys. The keytool command is used to create and administer keystores.
The jarsigner command uses an entity's private key to generate a signature. The signed JAR file contains, among other things, a copy of the certificate from the keystore for the public key corresponding to the private key used to sign the file. The jarsigner command can verify the digital signature of the signed JAR file using the certificate inside it (in its signature block file).
The jarsigner command can generate signatures that include a time stamp that lets a systems or deployer (including Java Plug-in) to check whether the JAR file was signed while the signing certificate was still valid. In addition, APIs allow applications to obtain the timestamp information.
At this time, the jarsigner command can only sign JAR files created by the jar command or zip files. JAR files are the same as zip files, except they also have a META-INF/MANIFEST.MF file. A META-INF/MANIFEST.MF file is created when the jarsigner command signs a zip file.
The default jarsigner command behavior is to sign a JAR or zip file. Use the -verify option to verify a signed JAR file.
The jarsigner command also attempts to validate the signer's certificate after signing or verifying. If there is a validation error or any other problem, the command generates warning messages. If you specify the -strict option, then the command treats severe warnings as errors. See Errors and Warnings.
KEYSTORE ALIASES All keystore entities are accessed with unique aliases.
When you use the jarsigner command to sign a JAR file, you must specify the alias for the keystore entry that contains the private key needed to generate the signature. For example, the following command signs the JAR file named MyJARFile.jar with the private key associated with the alias duke in the keystore named mystore in the working directory. Because no output file is specified, it overwrites MyJARFile.jar with the signed JAR file.
jarsigner -keystore /working/mystore -storepass <keystore password> -keypass <private key password> MyJARFile.jar duke
Keystores are protected with a password, so the store password must be specified. You are prompted for it when you do not specify it on the command line. Similarly, private keys are protected in a keystore with a password, so the private key's password must be specified, and you are prompted for the password when you do not specify it on the command line and it is not the same as the store password.
KEYSTORE LOCATION The jarsigner command has a -keystore option for specifying the URL of the keystore to be used. The keystore is by default stored in a file named .keystore in the user's home directory, as determined by the user.home system property.
On Oracle Solaris systems, user.home defaults to the user's home directory.
The input stream from the -keystore option is passed to the KeyStore.load method. If NONE is specified as the URL, then a null stream is passed to the KeyStore.load method. NONE should be specified when the KeyStore class is not file based, for example, when it resides on a hardware token device.
KEYSTORE IMPLEMENTATION The KeyStore class provided in the java.security package supplies a number of well-defined interfaces to access and modify the information in a keystore. You can have multiple different concrete implementations, where each implementation is for a particular type of keystore.
Currently, there are two command-line tools that use keystore implementations (keytool and jarsigner), and a GUI-based tool named Policy Tool. Because the KeyStore class is publicly available, JDK users can write additional security applications that use it.
There is a built-in default implementation provided by Oracle that implements the keystore as a file, that uses a proprietary keystore type (format) named JKS. The built-in implementation protects each private key with its individual password and protects the integrity of the entire keystore with a (possibly different) password.
Keystore implementations are provider-based, which means the application interfaces supplied by the KeyStore class are implemented in terms of a Service Provider Interface (SPI). There is a corresponding abstract KeystoreSpi class, also in the java.security package, that defines the Service Provider Interface methods that providers must implement. The term provider refers to a package or a set of packages that supply a concrete implementation of a subset of services that can be accessed by the Java Security API. To provide a keystore implementation, clients must implement a provider and supply a KeystoreSpi subclass implementation, as described in How to Implement a Provider in the Java Cryptography Architecture at http://docs.oracle.com/javase/7/docs/technotes/guides/security/crypto/HowToImplAProvider.html
Applications can choose different types of keystore implementations from different providers, with the getInstance factory method in the KeyStore class. A keystore type defines the storage and data format of the keystore information and the algorithms used to protect private keys in the keystore and the integrity of the keystore itself. Keystore implementations of different types are not compatible.
The jarsigner and policytool commands can read file-based keystores from any location that can be specified using a URL. In addition, these commands can read non-file-based keystores such as those provided by MSCAPI on Windows and PKCS11 on all platforms.
For the jarsigner and keytool commands, you can specify a keystore type at the command line with the -storetype option. For Policy Tool, you can specify a keystore type with the Edit command in the KeyStore menu.
If you do not explicitly specify a keystore type, then the tools choose a keystore implementation based on the value of the keystore.type property specified in the security properties file. The security properties file is called java.security, and it resides in the JDK security properties directory, java.home/lib/security, where java.home is the runtime environment's directory. The jre directory in the JDK or the top-level directory of the Java Runtime Environment (JRE).
Each tool gets the keystore.type value and then examines all the installed providers until it finds one that implements keystores of that type. It then uses the keystore implementation from that provider.
The KeyStore class defines a static method named getDefaultType that lets applications and applets retrieve the value of the keystore.type property. The following line of code creates an instance of the default keystore type as specified in the keystore.type property:
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
The default keystore type is jks (the proprietary type of the keystore implementation provided by Oracle). This is specified by the following line in the security properties file:
keystore.type=jks
Case does not matter in keystore type designations. For example, JKS is the same as jks.
To have the tools use a keystore implementation other than the default, change that line to specify a different keystore type. For example, if you have a provider package that supplies a keystore implementation for a keystore type called pkcs12, then change the line to the following:
keystore.type=pkcs12
Note: If you use the PKCS 11 provider package, then see "KeyTool" and "JarSigner" in Java PKCS #11 Reference Guide at http://docs.oracle.com/javase/7/docs/technotes/guides/security/p11guide.html
SUPPORTED ALGORITHMS By default, the jarsigner command signs a JAR file using one of the following algorithms:
• Digital Signature Algorithm (DSA) with the SHA1 digest algorithm
• RSA algorithm with the SHA256 digest algorithm
• Elliptic Curve (EC) cryptography algorithm with the SHA256 with Elliptic Curve Digital Signature Algorithm (ECDSA).
If the signer's public and private keys are DSA keys, then jarsigner signs the JAR file with the SHA1withDSA algorithm. If the signer's keys are RSA keys, then jarsigner attempts to sign the JAR file with the SHA256withRSA algorithm. If the signer's keys are EC keys, then jarsigner signs the JAR file with the SHA256withECDSA algorithm.
These default signature algorithms can be overridden using the -sigalg option.
THE SIGNED JAR FILE When the jarsigner command is used to sign a JAR file, the output signed JAR file is exactly the same as the input JAR file, except that it has two additional files placed in the META-INF directory:
• A signature file with an .SF extension
• A signature block file with a .DSA, .RSA, or .EC extension
The base file names for these two files come from the value of the -sigFile option. For example, when the option is -sigFile MKSIGN, the files are named MKSIGN.SF and MKSIGN.DSA
If no -sigfile option appears on the command line, then the base file name for the .SF and .DSA files is the first 8 characters of the alias name specified on the command line, all converted to uppercase. If the alias name has fewer than 8 characters, then the full alias name is used. If the alias name contains any characters that are not allowed in a signature file name, then each such character is converted to an underscore (_) character in forming the file name. Valid characters include letters, digits, underscores, and hyphens.
Signature File
A signature file (.SF file) looks similar to the manifest file that is always included in a JAR file when the jarsigner command is used to sign the file. For each source file included in the JAR file, the .SF file has three lines, such as in the manifest file, that list the following:
• File name
• Name of the digest algorithm (SHA)
• SHA digest value
In the manifest file, the SHA digest value for each source file is the digest (hash) of the binary data in the source file. In the .SF file, the digest value for a specified source file is the hash of the three lines in the manifest file for the source file.
The signature file, by default, includes a header with a hash of the whole manifest file. The header also contains a hash of the manifest header. The presence of the header enables verification optimization. See JAR File Verification.
Signature Block File
The .SF file is signed and the signature is placed in the signature block file. This file also contains, encoded inside it, the certificate or certificate chain from the keystore that authenticates the public key corresponding to the private key used for signing. The file has the extension .DSA, .RSA, or .EC, depending on the digest algorithm used.
SIGNATURE TIME STAMP The jarsigner command can generate and store a signature time stamp when signing a JAR file. In addition, jarsigner supports alternative signing mechanisms. This behavior is optional and is controlled by the user at the time of signing through these options. See Options.
-tsa url -tsacert alias -altsigner class -altsignerpath classpathlist -tsapolicyid policyid
JAR FILE VERIFICATION A successful JAR file verification occurs when the signatures are valid, and none of the files that were in the JAR file when the signatures were generated have changed since then. JAR file verification involves the following steps:
1.  Verify the signature of the .SF file.
The verification ensures that the signature stored in each signature block (.DSA) file was generated using the private key corresponding to the public key whose certificate (or certificate chain) also appears in the .DSA file. It also ensures that the signature is a valid signature of the corresponding signature (.SF) file, and thus the .SF file was not tampered with.
2.  Verify the digest listed in each entry in the .SF file with each corresponding section in the manifest.
The .SF file by default includes a header that contains a hash of the entire manifest file. When the header is present, the verification can check to see whether or not the hash in the header matches the hash of the manifest file. If there is a match, then verification proceeds to the next step.
If there is no match, then a less optimized verification is required to ensure that the hash in each source file information section in the .SF file equals the hash of its corresponding section in the manifest file. See Signature File.
One reason the hash of the manifest file that is stored in the .SF file header might not equal the hash of the current manifest file is that one or more files were added to the JAR file (with the jar tool) after the signature and .SF file were generated. When the jar tool is used to add files, the manifest file is changed by adding sections to it for the new files, but the .SF file is not changed. A verification is still considered successful when none of the files that were in the JAR file when the signature was generated have been changed since then. This happens when the hashes in the non-header sections of the .SF file equal the hashes of the corresponding sections in the manifest file.
3.  Read each file in the JAR file that has an entry in the .SF file. While reading, compute the file's digest and compare the result with the digest for this file in the manifest section. The digests should be the same or verification fails.
If any serious verification failures occur during the verification process, then the process is stopped and a security exception is thrown. The jarsigner command catches and displays the exception.
Note: You should read any addition warnings (or errors if you specified the -strict option), as well as the content of the certificate (by specifying the -verbose and -certs options) to determine if the signature can be trusted.
MULTIPLE SIGNATURES FOR A JAR FILE A JAR file can be signed by multiple people by running the jarsigner command on the file multiple times and specifying the alias for a different person each time, as follows:
jarsigner myBundle.jar susan jarsigner myBundle.jar kevin
When a JAR file is signed multiple times, there are multiple .SF and .DSA files in the resulting JAR file, one pair for each signature. In the previous example, the output JAR file includes files with the following names:
SUSAN.SF SUSAN.DSA KEVIN.SF KEVIN.DSA
Note: It is also possible for a JAR file to have mixed signatures, some generated by the JDK 1.1 by the javakey command and others by jarsigner. The jarsigner command can be used to sign JAR files that are already signed with the javakey command.
The following sections describe the various jarsigner options. Be aware of the following standards:
• All option names are preceded by a minus sign (-).
• The options can be provided in any order.
• Items that are in italics or underlined (option values) represent the actual values that must be supplied.
• The -storepass, -keypass, -sigfile, -sigalg, -digestalg, -signedjar, and TSA-related options are only relevant when signing a JAR file; they are not relevant when verifying a signed JAR file. The -keystore option is relevant for signing and verifying a JAR file. In addition, aliases are specified when signing and verifying a JAR file.
-keystore url Specifies the URL that tells the keystore location. This defaults to the file .keystore in the user's home directory, as determined by the user.home system property.
A keystore is required when signing. You must explicitly specify a keystore when the default keystore does not exist or if you want to use one other than the default.
A keystore is not required when verifying, but if one is specified or the default exists and the -verbose option was also specified, then additional information is output regarding whether or not any of the certificates used to verify the JAR file are contained in that keystore.
The -keystore argument can be a file name and path specification rather than a URL, in which case it is treated the same as a file: URL, for example, the following are equivalent:
-keystore filePathAndName -keystore file:filePathAndName
If the Sun PKCS #11 provider was configured in the java.security security properties file (located in the JRE's $JAVA_HOME/lib/security directory), then the keytool and jarsigner tools can operate on the PKCS #11 token by specifying these options:
-keystore NONE -storetype PKCS11
For example, the following command lists the contents of the configured PKCS#11 token:
keytool -keystore NONE -storetype PKCS11 -list
-storetype storetype Specifies the type of keystore to be instantiated. The default keystore type is the one that is specified as the value of the keystore.type property in the security properties file, which is returned by the static getDefaultType method in java.security.KeyStore.
The PIN for a PCKS #11 token can also be specified with the -storepass option. If none is specified, then the keytool and jarsigner commands prompt for the token PIN. If the token has a protected authentication path (such as a dedicated PIN-pad or a biometric reader), then the -protected option must be specified and no password options can be specified.
-storepass[:env | :file] argument Specifies the password that is required to access the keystore. This is only needed when signing (not verifying) a JAR file. In that case, if a -storepass option is not provided at the command line, then the user is prompted for the password.
If the modifier env or file is not specified, then the password has the value argument. Otherwise, the password is retrieved as follows:
• env: Retrieve the password from the environment variable named argument.
• file: Retrieve the password from the file named argument.
Note: The password should not be specified on the command line or in a script unless it is for testing purposes, or you are on a secure system.
-keypass [:env | :file] argument Specifies the password used to protect the private key of the keystore entry addressed by the alias specified on the command line. The password is required when using jarsigner to sign a JAR file. If no password is provided on the command line, and the required password is different from the store password, then the user is prompted for it.
-sigfile file Specifies the base file name to be used for the generated .SF and .DSA files. For example, if file is DUKESIGN, then the generated .SF and .DSA files are named DUKESIGN.SF and DUKESIGN.DSA, and placed in the META-INF directory of the signed JAR file.
The characters in the file must come from the set a-zA-Z0-9_-. Only letters, numbers, underscore, and hyphen characters are allowed. All lowercase characters are converted to uppercase for the .SF and .DSA file names.
If no -sigfile option appears on the command line, then the base file name for the .SF and .DSA files is the first 8 characters of the alias name specified on the command line, all converted to upper case. If the alias name has fewer than 8 characters, then the full alias name is used. If the alias name contains any characters that are not valid in a signature file name, then each such character is converted to an underscore (_) character to form the file name.
-sigalg algorithm Specifies the name of the signature algorithm to use to sign the JAR file.
For a list of standard signature algorithm names, see "Appendix A: Standard Names" in the Java Cryptography Architecture (JCA) Reference Guide at http://docs.oracle.com/javase/7/docs/technotes/guides/security/crypto/CryptoSpec.html#AppA
This algorithm must be compatible with the private key used to sign the JAR file. If this option is not specified, then SHA1withDSA, SHA256withRSA, or SHA256withECDSA are used depending on the type of private key. There must either be a statically installed provider supplying an implementation of the specified algorithm or the user must specify one with the -providerClass option; otherwise, the command will not succeed.
-digestalg algorithm Specifies the name of the message digest algorithm to use when digesting the entries of a JAR file.
For a list of standard message digest algorithm names, see "Appendix A: Standard Names" in the Java Cryptography Architecture (JCA) Reference Guide at http://docs.oracle.com/javase/7/docs/technotes/guides/security/crypto/CryptoSpec.html#AppA
If this option is not specified, then SHA256 is used. There must either be a statically installed provider supplying an implementation of the specified algorithm or the user must specify one with the -providerClass option; otherwise, the command will not succeed.
-certs If the -certs option appears on the command line with the -verify and -verbose options, then the output includes certificate information for each signer of the JAR file. This information includes the name of the type of certificate (stored in the .DSA file) that certifies the signer's public key, and if the certificate is an X.509 certificate (an instance of the java.security.cert.X509Certificate), then the distinguished name of the signer.
The keystore is also examined. If no keystore value is specified on the command line, then the default keystore file (if any) is checked. If the public key certificate for a signer matches an entry in the keystore, then the alias name for the keystore entry for that signer is displayed in parentheses. If the signer comes from a JDK 1.1 identity database instead of from a keystore, then the alias name displays in brackets instead of parentheses.
-certchain file Specifies the certificate chain to be used when the certificate chain associated with the private key of the keystore entry that is addressed by the alias specified on the command line is not complete. This can happen when the keystore is located on a hardware token where there is not enough capacity to hold a complete certificate chain. The file can be a sequence of concatenated X.509 certificates, or a single PKCS#7 formatted data block, either in binary encoding format or in printable encoding format (also known as Base64 encoding) as defined by the Internet RFC 1421 standard. See the section Internet RFC 1421 Certificate Encoding Standard in keytool and http://tools.ietf.org/html/rfc1421.
-verbose When the -verbose option appears on the command line, it indicates verbose mode, which causes jarsigner to output extra information about the progress of the JAR signing or verification.
-internalsf In the past, the .DSA (signature block) file generated when a JAR file was signed included a complete encoded copy of the .SF file (signature file) also generated. This behavior has been changed. To reduce the overall size of the output JAR file, the .DSA file by default does not contain a copy of the .SF file anymore. If -internalsf appears on the command line, then the old behavior is utilized. This option is useful for testing. In practice, do not use the -internalsf option because it incurs higher overhead.
-sectionsonly If the -sectionsonly option appears on the command line, then the .SF file (signature file) generated when a JAR file is signed does not include a header that contains a hash of the whole manifest file. It contains only the information and hashes related to each individual source file included in the JAR file. See Signature File.
By default, this header is added, as an optimization. When the header is present, whenever the JAR file is verified, the verification can first check to see whether the hash in the header matches the hash of the whole manifest file. When there is a match, verification proceeds to the next step. When there is no match, it is necessary to do a less optimized verification that the hash in each source file information section in the .SF file equals the hash of its corresponding section in the manifest file. See JAR File Verification.
The -sectionsonly option is primarily used for testing. It should not be used other than for testing because using it incurs higher overhead.
-protected Values can be either true or false. Specify true when a password must be specified through a protected authentication path such as a dedicated PIN reader.
-providerClass provider-class-name Used to specify the name of cryptographic service provider's master class file when the service provider is not listed in the java.security security properties file.
Used with the -providerArg ConfigFilePath option, the keytool and jarsigner tools install the provider dynamically and use ConfigFilePath for the path to the token configuration file. The following example shows a command to list a PKCS #11 keystore when the Oracle PKCS #11 provider was not configured in the security properties file.
jarsigner -keystore NONE -storetype PKCS11 \ -providerClass sun.security.pkcs11.SunPKCS11 \ -providerArg /mydir1/mydir2/token.config \ -list
-providerName providerName If more than one provider was configured in the java.security security properties file, then you can use the -providerName option to target a specific provider instance. The argument to this option is the name of the provider.
For the Oracle PKCS #11 provider, providerName is of the form SunPKCS11-TokenName, where TokenName is the name suffix that the provider instance has been configured with, as detailed in the configuration attributes table. For example, the following command lists the contents of the PKCS #11 keystore provider instance with name suffix SmartCard:
jarsigner -keystore NONE -storetype PKCS11 \ -providerName SunPKCS11-SmartCard \ -list
-Jjavaoption Passes through the specified javaoption string directly to the Java interpreter. The jarsigner command is a wrapper around the interpreter. This option should not contain any spaces. It is useful for adjusting the execution environment or memory usage. For a list of possible interpreter options, type java -h or java -X at the command line.
-tsa url If -tsa http://example.tsa.url appears on the command line when signing a JAR file then a time stamp is generated for the signature. The URL, http://example.tsa.url, identifies the location of the Time Stamping Authority (TSA) and overrides any URL found with the -tsacert option. The -tsa option does not require the TSA public key certificate to be present in the keystore.
To generate the time stamp, jarsigner communicates with the TSA with the Time-Stamp Protocol (TSP) defined in RFC 3161. When successful, the time stamp token returned by the TSA is stored with the signature in the signature block file.
-tsacert alias When -tsacert alias appears on the command line when signing a JAR file, a time stamp is generated for the signature. The alias identifies the TSA public key certificate in the keystore that is in effect. The entry's certificate is examined for a Subject Information Access extension that contains a URL identifying the location of the TSA.
The TSA public key certificate must be present in the keystore when using the -tsacert option.
-tsapolicyid policyid Specifies the object identifier (OID) that identifies the policy ID to be sent to the TSA server. If this option is not specified, no policy ID is sent and the TSA server will choose a default policy ID.
Object identifiers are defined by X.696, which is an ITU Telecommunication Standardization Sector (ITU-T) standard. These identifiers are typically period-separated sets of non-negative digits like 1.2.3.4, for example.
-altsigner class This option specifies an alternative signing mechanism. The fully qualified class name identifies a class file that extends the com.sun.jarsigner.ContentSigner abstract class. The path to this class file is defined by the -altsignerpath option. If the -altsigner option is used, then the jarsigner command uses the signing mechanism provided by the specified class. Otherwise, the jarsigner command uses its default signing mechanism.
For example, to use the signing mechanism provided by a class named com.sun.sun.jarsigner.AuthSigner, use the jarsigner option -altsigner com.sun.jarsigner.AuthSigner.
-altsignerpath classpathlist Specifies the path to the class file and any JAR file it depends on. The class file name is specified with the -altsigner option. If the class file is in a JAR file, then this option specifies the path to that JAR file.
An absolute path or a path relative to the current directory can be specified. If classpathlist contains multiple paths or JAR files, then they should be separated with a colon (:) on Oracle Solaris and a semicolon (;) on Windows. This option is not necessary when the class is already in the search path.
The following example shows how to specify the path to a JAR file that contains the class file. The JAR file name is included.
-altsignerpath /home/user/lib/authsigner.jar
The following example shows how to specify the path to the JAR file that contains the class file. The JAR file name is omitted.
-altsignerpath /home/user/classes/com/sun/tools/jarsigner/
-strict During the signing or verifying process, the command may issue warning messages. If you specify this option, the exit code of the tool reflects the severe warning messages that this command found. See Errors and Warnings.
-verbose suboptions For the verifying process, the -verbose option takes suboptions to determine how much information is shown. If the -certs option is also specified, then the default mode (or suboption all) displays each entry as it is being processed, and after that, the certificate information for each signer of the JAR file. If the -certs and the -verbose:grouped suboptions are specified, then entries with the same signer info are grouped and displayed together with their certificate information. If -certs and the -verbose:summary suboptions are specified, then entries with the same signer information are grouped and displayed together with their certificate information. Details about each entry are summarized and displayed as one entry (and more). See Examples.
During the signing or verifying process, the jarsigner command may issue various errors or warnings.