Yes, you can refuse consent to a search if the officer doesn’t have a warrant or probable cause. However, remain polite and non-confrontational when doing so.