If you're only visiting Universal to experience the Wizarding World of Harry Potter, we do not believe that you need the Universal Express Pass. We believe the most important thing to consider is making sure that you are taking advantage of early park admission. Specifically, try to visit Universal during a time when they do have early admission at the Islands of Adventure theme park in which you can use your time to go on Hagrid's Magical Creatures Motorbike Adventure and then Harry Potter and the Forbidden Journey right after.
The main reason is that most guests who take advantage of early park admission can get onto these two rides within the first hour and a half. This will allow for plenty of time to explore the Wizarding World of Harry Potter and get on the remaining 4 Harry Potter rides. and even grab a Harry Potter Butterbeer. So if visiting the Wizarding World of Harry Potter is your only objective we would say that the Universal Express Pass is not worth it.