Because of the high pay, investment bankers are often able to retire earlier than other workers. Many bankers have the financial security to call it quits by the age of 40. However, many choose to continue, either as investment bankers or in different fields.