If you work at your employer's place of business, it is implied that your employer has control. This is especially true if the type of work you do could be done elsewhere. Working at the employer's place of business places you under your employer's direction and supervision. Even if you do not work on your employer's premises, it does not necessarily mean that you are not an employee.
Some occupations require services to be performed off premises.
For example, employees of construction contractors must work off premises to perform their job duties.