Crib-Notes: Manifest Deltas
Tags: deltas, diff, git, log
Published: 2019-07-17 13:38:21

---
title: Crib-Notes: Manifest Deltas
published: true
tags: deltas,diff,git,log
canonical_url: https://thjones2.blogspot.com/2019/07/crib-notes-manifest-deltas.html
---

Each month, the group I work for publishes new CentOS and Red Hat [AMIs](https://github.com/plus3it/spel#current-published-images) (and Azure templates and Vagrant boxes). When we complete the publication-event, we post a news announcement to our user-portal so that subscribers can receive an alert of the new publication. Included in that news announcement is a "what's changed" section.  
  
In prior months, figuring out "what changed" was left as a manual step for the team-member charged with running the automation for a given month's publication event. This month, no one generated that news article _and_ there were several updated and new RPMs included in the new image. So, I set about figuring out "how to extract this information programmatically so as to more-easily suss-out what to include in the announcement posting." The following does so (though, presumably, in a not-particularly-optimized) fashion:

> ```
> git diff $(
> git log --pretty='%H' --follow -- <PATH_TO_MANIFEST_FILE> | \
> head -2 | \
> tac | \
> sed 'N;s/\n/../'
> ) -- <PATH_TO_MANIFEST_FILE> | \
> grep -E '(amazon|aws|ec2)-' | \
> sed 's/^./& /' | \
> sort -k 2
> 
> ```

To explain:  

1. Use `git log` to output the commit-hashes for all the commits for the target file (in this case, the project's manifest-file)
2. Use `head -2` to grab only the two most-recent commit hashes from the output-stream
3. Use the `tac` command to invert the order of the two lines returned from the `head` command
4. Use the `sed` command to join the two lines, replacing the first line's line-ending newline character with "`..`"
5. Use `git diff` against the output created in steps 1-4, and constrain the diff-activity to just the manifest-file.
6. Pipe that output through `grep` to suppress all information other than the bits containing the `amazon-`, `aws-` and `ec2-` substrings.
7. Pipe that through `sed` so that the +/- that `git diff` uses to show new and removed files, respectively, becomes an easily-tokenized substring.
8. Sort the remaining output-stream (with `sort`) so that the lines are groups by manifest-element (the second key/token in the sorted output) 

Taking that output and converting to a news article is still manual, but it at least makes it a lot easier to do than either hand-diffing two files or having to "just know" what's changed.

  

**Notes**  
  
Because Red Hat has placed EL6 is in its final stage of de-support, we've stopped publishing images for CentOS6 and RHEL6. We did this to discourage our subscribers from doing new deployments on EL6 (since the underlying OS will go into final de-support come November of this year).  
  
Similarly, due to current lack of CentOS offering for EL8, lack of security-related build- or hardening-guidance for EL8 and associated lack of subscriber-demand for an EL8 build, we don't yet include builds for CentOS8 or RHEL8 in our process. Thus, for the time being, we only need to provide a "whats changed" for EL7 builds. Given this, (combined with the 99% overlap we implement in our CentOS vice RHEL builds) we currently only need to do change-queries against the "`manifests/spel-minimal-centos-7-hvm.manifest.txt`" file.