It depends on how specific you want to be. If you want to indicate that you are working inside the building, use "in the office." If you want to give a general indication of your location, use "at the office."