Optimization: Only raycast when there are mouse listeners