The All Iron hostel is a nice simple place to stay in Bilbao. Very clean, kind people, the best you can find at that budget.
You can walk to the Guggenheim and have a look at the city in the meantime (around 40 Minutes walk) and come back to the hostel by metro (as it is in a steep hill). I would recommend it.