I've seen young men take off their cap leaving a bad hair day for all to see rather than insult the lady at the table or hear their mama scream.
Women were not expected to take off their hats when indoors or dining which continues today.