Remove an unnecessary try/finally in a context manager.

Context managers always call __exit__, so there's no need to attempt to further fortify them with `finally`.

Unless I'm missing something....