Leo: Given the sentence "A man in an orange shirt is holding an object and speaking to a girl in a blue shirt who is looking back at him." is it true that "The man is asking the girl what she wants for lunch."?
Mei: OK, So, let's think first...
Me:
If you're holding an object and speaking to a girl you can't be certain he asks about lunch.
Thus, the answer is it is not possible to tell.