Forss was born in South Bronx on May 4, 1941. His father, Hank, was deported from the United States to Finland after Forss' birth; his mother, Norma, practised vernacular photography. He suffered from polio as a child. He was removed from his mother's custody and consequently spent five years in an orphanage. After moving out of the orphanage system, Forss went back to his mother – who had osteoporosis and rheumatoid arthritis – and became her caregiver. They grew close over their shared interest in photography. He became a self-taught photographer after being introduced to photography at the 1964 New York World's Fair.