It is okay to look in a bird box, though you should avoid doing it more than necessary. If you live in the UK, the British Trust for Ornithology has methods for looking into your birdbox and making observations. Whether your birdbox is hosting Blue Tits or other birds, it is helpful for researchers to have as much data as possible, so always enter your information into the form if you can.