It’s not a good idea to put bread back in the fridge after setting it on the counter for a few hours. All the back and forth with the temperature will harden the bread much more quickly.  Take out what you need and keep the rest in the fridge.