Question: is it worth visiting oxford?
Answer: I would say it is definitely worth visiting Oxford even if you only have one day. The three places you mention are some of my favourite places in Oxford. The Botanical Gardens and a walk up the High Street taking in the Radcliffe Camera and which ever colleges you pass.