Spherical panoramas are incredibly processor intensive. You need to recalculate every single pixel each time the viewer looks in a different direction. It's perfectly possible in Java to do this, but it can be slow. Smaller images of say 150 pixels by 150 will run quite well, but you will see the frame rate drop dramatically as the image size increases. Not only are spherical panoramas tricky to view, they can also be tricky to create.
Cameras work by projecting images onto a flat piece of film.
Panoramic cameras that work by rotating on a tripod will only ever create cylindrical panoramas, and movie cameras will need special software to warp individual frames so that they can be overlaid to create a single image. The software to do this needs to be incredibly clever, and many packages will have trouble recognising spherical panoramas. The result is that most tend to be very wide and not very high, so the scope for the viewer to look up or look down is extremely limited anyway. For the most part, spherical mapping is not worth it.