It depends on the cafe’s policies and norms. Some cafes allow it, especially if not busy, but it’s considerate to make a purchase or ask the staff.