If the command that invokes the recursive edit selects a different buffer first, that is the buffer you will edit recursively. You could probably do all the rest of your editing inside the recursive edit, visiting files and all. So remember to exit or abort the recursive edit when you no longer need it. In general, we try to minimize the use of recursive editing levels in GNU Emacs. When possible, we present different activities in separate buffers.
Some commands switch to a new major mode but provide a way to switch back.
These approaches give you more flexibility to go back to unfinished tasks in the order you choose.