(info "(elisp) Searching Keymaps")

(or (cond
     (overriding-terminal-local-map
      (FIND-IN overriding-terminal-local-map))
     (overriding-local-map
      (FIND-IN overriding-local-map))
     ((or (FIND-IN (get-char-property (point) 'keymap))
          (FIND-IN TEMP-MAP)
          (FIND-IN-ANY emulation-mode-map-alists)
          (FIND-IN-ANY minor-mode-overriding-map-alist)
          (FIND-IN-ANY minor-mode-map-alist)
          (if (get-text-property (point) 'local-map)
              (FIND-IN (get-char-property (point) 'local-map))
            (FIND-IN (current-local-map))))))
    (FIND-IN (current-global-map)))

Instead of playing with `minor-mode-map-alist', it's simpler and cleaner to add `view-mode-map' to `minor-mode-overriding-map-alist'.

It's original purpose is to let major mode overriding minor mode keymaps, but it can also be used by a minor mode to override the others.
install list-utils (for list-utils-safe-equal)
enable paredit in IELM