Update notes for submitted changes

* Store list of collected configs in cachedir