That is the case and I started putting the logic in to handle that, but I spoke to Adam about it and he said it would be better to have the link available all the time so that people know that the feature is available.

Pulling up the restore page with no deleted items will just produce a nice empty list.