The comedian Alvy Singer (Woody Allen) is trying to understand why his relationship with Annie Hall (Diane Keaton) ended a year ago. Growing up in New York, he vexed his mother with impossible questions about the emptiness of existence, but he was precocious about his innocent sexual curiosity.
Annie and Alvy, in a line for The Sorrow and the Pity, overhear another man deriding the work of Federico Fellini and Marshall McLuhan; McLuhan himself steps in at Alvy's invitation to criticize the man's comprehension. That night, Annie shows no interest in sex with Alvy. Instead, they discuss his first wife (Carol Kane), whose ardor gave him no pleasure. His second marriage was to a New York writer who didn't like sports and was unable to reach orgasm.
With Annie, it is different. The two of them have fun making a meal of boiled lobster together. He teases her about the unusual men in her past. He met her playing tennis doubles with friends. Following the game, awkward small talk led her to offer him first a ride up town and then a glass of wine on her balcony. There, what seemed a mild exchange of trivial personal data is revealed in "mental subtitles" as an escalating flirtation. Their first date follows Annie's singing audition for a night club ("It Had to be You"). He suggests they kiss first, to get it out of the way. After their lovemaking that night, Alvy is "a wreck", while she relaxes with a joint.
Soon Annie admits she loves him, while he buys her books on death and says that his feelings for her are more than just love. When she moves in with him, things become very tense. Eventually, he finds her arm in arm with one of her college professors and the two begin to argue whether this is the "flexibility" they had discussed. They eventually break up, and he searches for the truth of relationships, asking strangers on the street about the nature of love, questioning his formative years, until he casts himself in Snow White opposite Annie's Evil Queen.
Alvy returns to dating, but the effort is marred by neurosis, bad sex, and finally an interruption from Annie, who insists he come over immediately. It turns out she needs him to kill a spider. A reconciliation follows, coupled with a vow to stay together come what may. However, their separate discussions with their therapists make it evident there is an unspoken divide. When Alvy accepts an offer to present an award on television, they fly out to Los Angeles, with Alvy's friend, Rob (Tony Roberts). However, on the return trip, they agree that their relationship is not working. After losing her to her record producer, Tony Lacey (Paul Simon), he unsuccessfully tries rekindling the flame with a marriage proposal. Back in New York, he stages a play of their relationship but changes the ending: now she accepts.
The last meeting for them is a wistful coda on New York's Upper West Side, when they have both moved on to someone new. Alvy's voice returns with a summation: love is essential, especially if it is neurotic. Annie sings "Seems Like Old Times" and the credits roll.

Question: Why were Alvy upset with Annie?
Answer: He saw Annie walking with another man arm in arm.
Alternative answer: He caught her with another man