Baberowski was born in Radolfzell am Bodensee, West Germany. He grew up in a Catholic working class and social democratic family. His grandfather was Polish; the Baberowski family thus belonged to the Ruhr Polish minority, Polish immigrants to the rapidly industrializing areas of the Ruhr Valley. Baberowski's wife is Iranian-born and fled to Germany following the Iranian Revolution.