Some of those who argue they would not throw the switch say they could not morally do so, since they would be initiating force against the 50 people who would die. The individual throwing the switch is simply deflecting the force to a less destructive outcome. A good example is the current debate between the income tax vs a national sales tax. Assume for the moment that the amount of taxes to be collected has been driven down as far as you can immediately drive it, and you're faced with how to collect that amount. Assume a revenue neutral shift that eliminates the income tax and replaces it with a national sales tax, and increases economic activity as most who study the issue agree that it would.
Some people are better off, some people are worse off, and the system, as a whole, is better off.
The majority of options for improving the world are of this kind. We should never stop fighting to drive taxes down to zero, and while we're doing that, we should not pass up opportunities to make the world a better place without the initiation of force. The distinction between the initiation and the deflection of force is critical to our ability to be politically effective, and morally sound at the same time. The cause of freedom is best served by making the choices that we actually have today, while working to have the choices we really want for tomorrow.