The best places to buy stamps on a Sunday would be an ATM or a retailer that is open on Sundays. Post offices and banks typically aren’t open on Sundays, so retailers and ATMs are likely your best bet. Check your local Post office to see if they have 24-hour kiosk in their lobby as well.