The performance difference is significant - particularly when the data locality is reasonably good.
The first preference would be for hyper local (or in process locality as you called it), next for node locality - both of which qualify for host locality (along with no prefs tasks).
Next comes rack local tasks - after which any node would do.

Obviously between host local (both kinds) and rack local/any, we would definitely prefer host local; but if choice is between hyper local and any container on that node, we would prefer hyper local to completely bypass the IO need entirely.
If we do not support hyper local scheduling, for each rdd block, we can keep ending up transferring data between containers on the same node - the cost for single block might not be very high, but cummulatively adds up to being pretty expensive : not to mention completely thrashing the memory and disk caches on that node.

Unfortunately, there are a bunch of other factors which interact with this in non trivial ways to always end up with qualitative figures - 
* whether data is still in memory or has been pushed to disk, 
* how many other slaves are pulling data from this worker, 
* how many other slaves are scheduled on the same node, 
etc

To give an example, time to complete for some of our machine algorithms went down from 18+ minutes to 3.x - 4.x minutes after all the scheduling changes were done on a small 40 node cluster.
Ofcourse it was not just this change though - there are a few other flags which control the full set : 
TASK_REVIVAL_INTERVAL and TASK_SCHEDULING_AGGRESSION in ClusterScheduler have significant impact on the runtime when the cluster size goes up. The default value has been set to preserve existing behavior, but in our clusters we have TASK_SCHEDULING_AGGRESSION set to RACK_LOCAL and TASK_REVIVAL_INTERVAL set to 200 or so.