Spherical panoramas are incredibly processor intensive.
You need to recalculate every single pixel each time the viewer looks in a different direction. It's perfectly possible in Java to do this, but it can be slow. Smaller images of say 150 pixels by 150 will run quite well, but you will see the frame rate drop dramatically as the image size increases. Not only are spherical panoramas tricky to view, they can also be tricky to create. Cameras work by projecting images onto a flat piece of film.