Yes, you can leave the airport and explore Reykjavik for part of the day. Your layover would need to be 7-9 hours for you to have enough time. Please know once you leave you will have to go back through security so plan extra time for that.