Package: chkbit
chkbitchkbit is a lightweight tool to check the data integrity of your files. It allows you to verifythat the data has not changedsince you put it there and that it is still the same when you move it somewhere else.Use itOn your DiskOn your BackupFor Data in the CloudInstallationUsageRepairIgnore filesFAQShould I runchkbiton my whole drive?Why is chkbit placing the index in.chkbitfiles (vs a database)?How does chkbit work?I wish to use a stronger hash algorithmHow can I delete the index files?Can I test if chkbit is working correctly?DevelopmentUse itOn your Diskchkbit starts with your primary disk. It creates checksums for each folder that will follow your data onto your backups.Here it alerts you todamage on the diskdamage caused by filesystem errorsdamage caused by malware (when it encrypts your files)The built in checksums from your filesystems only cover some of these cases.On your BackupNo matter what storage media or filesystem you use, chkbit stores its indexes in hidden files that are backed up together with your data.When you run chkbit on your backup media you can verify that every byte was correctly transferred.If your backup media fails or experiencesbitrot/data degradation, chkbit allows you to discover what files were damaged and need to be replaced by other backups. You should always keep multiple backups :)For Data in the CloudSome cloud providers re-encode your videos or compress your images to save space. chkbit will alert you of any changes.InstallationThe easiest way to install python CLI tools is withpipx.pipx install chkbitYou can also use pip:pip install --user chkbitNOTEversion 3 now uses the blake3 hash algorithm by default as it is not only better but also faster than md5.UsageRunchkbit -u PATHto create/update the chkbit index.chkbit willcreate a.chkbitindex in every subdirectory of the path it was given.update the index with blake3 (see --algo) hashes for every file.report damage for files that failed the integrity check since the last run (check the exit status).Runchkbit PATHto verify only.usage: chkbit [-h] [-u] [--show-ignored-only] [--algo ALGO] [-f] [-s] [-l FILE] [--log-verbose] [--index-name NAME] [--ignore-name NAME] [-w N] [--plain] [-q] [-v] [PATH ...]

Checks the data integrity of your files. See https://github.com/laktak/chkbit-py

positional arguments:
  PATH                  directories to check

options:
  -h, --help            show this help message and exit
  -u, --update          update indices (without this chkbit will verify files in readonly mode)
  --show-ignored-only   only show ignored files
  --algo ALGO           hash algorithm: md5, sha512, blake3 (default: blake3)
  -f, --force           force update of damaged items
  -s, --skip-symlinks   do not follow symlinks
  -l FILE, --log-file FILE
                        write to a logfile if specified
  --log-verbose         verbose logging
  --index-name NAME     filename where chkbit stores its hashes, needs to start with '.' (default: .chkbit)
  --ignore-name NAME    filename that chkbit reads its ignore list from, needs to start with '.' (default: .chkbitignore)
  -w N, --workers N     number of workers to use (default: 5)
  --plain               show plain status instead of being fancy
  -q, --quiet           quiet, don't show progress/information
  -v, --verbose         verbose output

.chkbitignore rules:
  each line should contain exactly one name
  you may use Unix shell-style wildcards (see README)
  lines starting with `#` are skipped
  lines starting with `/` are only applied to the current directory

Status codes:
  DMG: error, data damage detected
  EIX: error, index damaged
  old: warning, file replaced by an older version
  new: new file
  upd: file updated
  ok : check ok
  ign: ignored (see .chkbitignore)
  EXC: internal exceptionchkbit is set to use only 5 workers by default so it will not slow your system to a crawl. You can specify a higher number to make it a lot faster if the IO throughput can also keep up.Repairchkbit cannot repair damage, its job is simply to detect it.You shouldbackup regularly.run chkbitbeforeeach backup.check for damage on the backup media.in case of damagerestorefrom a checked backup.Ignore filesAdd a.chkbitignorefile containing the names of the files/directories you wish to ignoreeach line should contain exactly one nameyou may useUnix shell-style wildcards*matches everything?matches any single character[seq]matches any character in seq[!seq]matches any character not in seqlines starting with#are skippedlines starting with/are only applied to the current directoryyou can usepath/sub/nameto ignore a file/directory in a sub pathhidden files (starting with a.) are ignored by defaultFAQShould I runchkbiton my whole drive?You would typically run it only oncontentthat you keep for a long time (e.g. your pictures, music, videos).Why is chkbit placing the index in.chkbitfiles (vs a database)?The advantage of the .chkbit files is thatwhen you move a directory the index moves with itwhen you make a backup the index is also backed upThe disadvantage is obviously that you get hidden.chkbitfiles in your content folders.How does chkbit work?chkbit operates on files.When run for the first time it records a hash of the file contents as well as the file modification time.When you run it again it first checks the modification time,if the time changed (because you made an edit) it records a new hash.otherwise it will compare the current hash to the recorded value and report an error if they do not match.I wish to use a stronger hash algorithmchkbit now uses blake3 by default. You can also specify--algo sha512or--algo md5.Note that existing index files will use the hash that they were created with. If you wish to update all hashes you need to delete your existing indexes first. A conversion mode may be added later (PR welcome).How can I delete the index files?List them withfind . -name .chkbitand add-deleteto delete.Can I test if chkbit is working correctly?On Linux/OS X you can try:Create test and set the modified time:$ echo foo1 > test; touch -t 201501010000 test
$ chkbit -u .
new ./test

Processed 1 file.
- 0:00:00 elapsed
- 192.31 files/second
- 0.00 MB/second
- 1 directory was updated
- 1 file hash was added
- 0 file hashes were updatednewindicates a new file was added.Now update test with a new modified:$ echo foo2 > test; touch -t 201501010001 test # update test & modified
$ chkbit -u .
upd ./test

Processed 1 file.
- 0:00:00 elapsed
- 191.61 files/second
- 0.00 MB/second
- 1 directory was updated
- 0 file hashes were added
- 1 file hash was updatedupdindicates the file was updated.Now update test with the same modified to simulate damage:$ echo foo3 > test; touch -t 201501010001 test
$ chkbit -u .
DMG ./test

Processed 1 file.
- 0:00:00 elapsed
- 173.93 files/second
- 0.00 MB/second
chkbit detected damage in these files:
./test
error: detected 1 file with damage!DMGindicates damage.DevelopmentWith pipenv (install withpipx install pipenv):# setup
pipenv install

# run chkbit
pipenv run python3 -m chkbit_cli.mainTo build a source distribution package from pyproject.tomlpipx run buildYou can then install your own package withpipx install dist/chkbit-*.tar.gz