Your first responsibility is to yourself. We all must think about the possibility, but there's no need to be tormented by it. Crimes of violence occur least often. In Minnesota, only a minority of the crimes reported involve a confrontation that could lead to personal injury. In other words, you're much more likely to have your property stolen than your life threatened.
Most crimes are crimes of opportunity.
A dangling handbag invites a purse snatcher. An unlocked window invites an intruder. If you eliminate the opportunity, you could avoid the crime. Carry a flashlight. Stand near the control panel while you ride.