On 27 August 1965, Harrison was appointed as Ambassador to the Soviet Union. In 1968, he engaged in a brief affair with a Russian chambermaid who was working at the British Embassy. Harrison recalled not asking or knowing if she worked for the KGB, but he said that it was assumed that every Soviet employee at the embassy worked or was an agent for the Soviet secret service. When security concerns arose over the Soviet occupation of Czechoslovakia, and he had been sent incriminating photographs taken by the KGB, Harrison informed the Foreign Office of his indiscretion, which immediately terminated his appointment and recalled him to Britain. Harrison revealed the affair to "The Sunday Times" newspaper in 1981.