Package: git-doc-history
git_doc_historyCopy and track files ingit, and a library to traverse the historyReally, thedocisn't super accurate, but 'file/index/history' is such an overloaded term when it comes togitI use this to track mytodo.txtfiles, changes to configuration files, any shell histories which don't support timestamps (see all of my config fileshere)This copies the files to a different directory, so it doesn't interfere with the application/configurationBy copying those files to a separate directory, I can always roll back to previous file, or see what the file was like a couple days/months ago.For shell histories/files which are unique lines of text (e.g., mytodo.txtfile) this also lets me estimate timestamps for when new lines were added to the history/text files, using theiter_diffsbelow, which emits added/removed events for individual events with estimated timesInstallationRequirespython3.8+To install with pip, run:pip install git_doc_historyUsageThe main script to backup data is [bin/git_doc_history], which gets installed into your~/.local/bin/directory.If uses a config file (parsed withpython-dotenv) like:SOURCE_DIR=~/.todo  # copy from
BACKUP_DIR=~/data/todo_git_history # copy to
# multiple lines means multiple files
COPY_FILES="todo.txt
done.txt"You can either provide the full path to that config file, or place the file in ~/.config/git_doc_historyFor example, after placing it at ~/.config/git_doc_history/todo -- to copy/commit any changes, run:$git_doc_historytodoGenerated configuration:
SOURCE_DIR: /home/sean/data/todo
BACKUP_DIR: /home/sean/data/todo_git_history
COPY_FILES: todo.txt
done.txt
'/home/sean/data/todo/todo.txt' -> '/home/sean/data/todo_git_history/todo.txt'
'/home/sean/data/todo/done.txt' -> '/home/sean/data/todo_git_history/done.txt'
'/home/sean/data/todo/.gitignore' -> '/home/sean/data/todo_git_history/.gitignore'
[master f927490] update
 1 file changed, 1 insertion(+)
 create mode 100644 .gitignoreThat usespython3 -m git_doc_history shell todoto parse the configuration file, like:eval"$(python3-mgit_doc_historyshelltodo)"The python library comes with a small CLI interface to extract a file from some time ago:$ python3 -m git_doc_history extract-file-at --at 2020-09-20 -c todo todo.txt -
setup command of completionTheBACKUP_DIRis of course just a regular git directory -- you canreset --hardto some point in the past to get rid of recent commits,rebase/squashto merge commits or do whatever you pleaseLibrary UsageMost things will be done withgit_doc_history.DocHistoryThis doesn't assume the filetype is readable text (you may be storing images/binary doc files in the git repository), so the default is to return the data asbytes-- you can.decode("utf-8")to convert that to readable textTo traverse the entire history:fromgit_doc_historyimportDocHistoryfromgit_doc_history.configimportparse_config,resolve_config# parse the config from the env filedoc=DocHistory.from_dict(parse_config(resolve_config("todo")))# iterate through the history for the todo.txt fileforsnapshotindoc.iter_commit_snapshots("todo.txt"):print(str(snapshot.commit_sha))print(str(snapshot.dt))print(snapshot.data.decode("utf-8"))Parsing DiffsIterates through the git history in chronological order, keeping track
of when data was added or removed. By default, this parses thefilegiven by splitting it into lines. If lines are added/removed, this returns an
event which specifies when in the history, and what was added/removedAlternatively, can pass aparse_func, which is a function which
accepts theDocHistorySnapshot, and retuns a list of hashable items
to store as stateFor an example of parsing diffs, seeexamples/todotxt_diff.py:Example output looks something like:added 2022-03-08 12:14:45 (C) 2022-03-08 create shebang script +programming
removed 2022-03-08 13:14:58 (C) 2022-03-08 create shebang script +programming
added 2022-03-08 22:23:39 save formhistory.sqlite in browserexport
removed 2022-03-08 23:23:45 save formhistory.sqlite in browserexport
added 2022-03-09 02:49:58 (C) create a python fzf wrapper because apparently I cant find a good one
added 2022-03-10 16:24:24 (B) 2022-03-10 create plaintext playlist parser module +music
removed 2022-03-11 01:30:49 (B) 2022-03-10 create plaintext playlist parser module +music
added 2022-03-11 10:37:06 (C) 2022-03-11 sync tmux from home directory +programming
added 2022-03-12 03:44:24 install undotree +vim +programming
removed 2022-03-12 04:44:51 (C) 2022-03-11 sync tmux from home directory +programming
removed 2022-03-12 10:51:20 install undotree +vim +programmingIn this case, 'removed' would mean I either changed the text on the line, or (more likely) I completed itTestsgitclone'https://github.com/seanbreckenridge/git_doc_history'cd./git_doc_history
pipinstall'.[testing]'pytest
flake8./git_doc_history
mypy./git_doc_history