Chhang was born on 22 January 1961 in Tuol Kouk District, Phnom Penh, Cambodia. He was the youngest of nine children. His father was a gem merchant. On 17 April 1975, the day of the fall of Phnom Penh, Chhang was alone at home, and was separated from his family in the subsequent Khmer Rouge-ordered evacuation of the city. He walked to Takeo Province, knowing that his mother's home village was somewhere in the vicinity of Phnom Chisor. Villagers there sheltered him until his mother's arrival, without alerting the Khmer Rouge village chief to his presence, but after his mother's arrival, he and his family members were sent to a work camp in the country's northwest, in what is today Banteay Meanchey Province. After the Vietnamese invasion of Cambodia, he returned to Phnom Penh, but then, at his mother's urging, fled Cambodia and snuck into neighbouring Thailand. He lived at the Khao-I-Dang refugee camp for several years until 1985 when he was chosen for resettlement in the United States, and after some time living at an orientation and training camp in the Philippines, he arrived in the United States in 1986.