In 1978, con artists Irving Rosenfeld (Christian Bale) and Sydney Prosser (Amy Adams) have started a relationship and are working together. Sydney has improved Rosenfeld's scams, posing as English aristocrat "Lady Edith Greensly." Irving loves Sydney, though is hesitant to leave his unstable and histrionic wife Rosalyn (Jennifer Lawrence), fearing he will lose contact with her son Danny, whom Irving has adopted. Rosalyn has also threatened to report Irving to the police if he leaves her. FBI agent Richie DiMaso (Bradley Cooper) catches Irving and Sydney in a loan scam, but offers to release them if Irving can line up four additional arrests. Richie believes Sydney is English but has proof that her claim of aristocracy is fraudulent. Sydney tells Irving she will manipulate Richie, distancing herself from Irving.
Irving has a friend pretending to be a wealthy Arab sheikh looking for potential investments in America. An associate of Irving's suggests the sheikh do business with Mayor Carmine Polito (Jeremy Renner) of Camden, New Jersey, who is campaigning to revitalize gambling in Atlantic City, New Jersey, but has struggled in fundraising. Carmine seems to have a genuine desire to help the area's economy and his constituents. Richie devises a plan to make mayor Polito the target of a sting operation, despite the objections of Irving and of Richie's boss, Stoddard Thorsen (Louis C.K.). Sydney helps Richie manipulate an FBI secretary into making an unauthorized wire transfer of $2,000,000. When Stoddard's boss, Anthony Amado (Alessandro Nivola), hears of the operation, he praises Richie's initiative, pressuring Stoddard to continue.
Carmine leaves their meeting when Richie presses him to accept a cash bribe. Irving convinces Carmine the sheikh is legitimate, expressing his dislike of Richie, and the two become friends. Richie arranges for Carmine to meet the sheikh, and without consulting the others, has Mexican-American FBI agent Paco Hernandez (Michael Pe単a) play the sheikh, which displeases Irving. Carmine brings the sheikh to a casino party, explaining mobsters are there and it is a necessary part of doing business. Irving is surprised to hear that Mafia overlord Victor Tellegio (Robert De Niro), right-hand man to Meyer Lansky, is present, and that he wants to meet the sheikh. Mafia man Tellegio explains that the business needs the sheikh to become an American citizen and that Carmine will need to expedite the process. Tellegio also requires a $10,000,000 wire transfer to prove the sheikh's legitimacy.
Richie confesses his strong attraction to Sydney but becomes confused and aggressive when she drops her English accent and admits to being from Albuquerque, New Mexico. Rosalyn starts an affair with a mobster Pete Musane (Jack Huston), whom she met at the party. She mentions her belief that Irving is working with the Internal Revenue Service, causing Pete to threaten Irving, who promises to prove the sheikh's investment is real. Irving later confronts Rosalyn, who admits she told Pete and agrees to keep quiet but wants a divorce. With Carmine's help, Richie and Irving videotape members of Congress receiving bribes. Richie assaults Stoddard in a fight over the money and later convinces Amado that he needs the US$10,000,000 to get Tellegio, but gets only US$2,000,000. A meeting is arranged at the offices of Tellegio's lawyer, Alfonse Simone (Paul Herman), but Tellegio does not appear.
Irving visits Carmine and admits to the scam, but says he has a plan to help him. Carmine throws Irving out and the loss of their friendship hits Irving hard. The feds inform Irving that their US$2,000,000 is missing, and that they have received an anonymous offer to return the money in exchange for Irving and Sydney's immunity and a reduced sentence for Carmine. Amado accepts the deal and Stoddard removes Richie from the case, which effectively ends his career, dropping him back into obscurity. Irving and Sydney move in together and open a legitimate art gallery, while Rosalyn lives with Pete and shares custody of Danny with Irving.

Question: Why is Irving afraid to leave Sydney?
Answer: He doesn't want to lose contact with her son
Alternative answer: Fears loosing contact with his son.