Poor facilities at work, lack of storage space for a cycle at work, the lack of the culture of cycling that exists in other countries - these all play a part.
The number one concern is fear for safety, whether that be from proximity to traffic or fear of assault or robbery on poorly lit paths.