That depends on your playstyle. The Oghma Infinium serves no purpose beyond the skill points it gives you, but reading it causes the book to disappear. If you prefer to show off your unique items and Daedric artifacts as trophies in your home, you should not read it.