Refactor `find_bindings` to be recursive

Allows the nested data structure to be built much more directly. Code
simplification FTW.
Use different method of removal in `match` so bindings are not refound