Goldman was married to Betty Rosenburg, a former model. Her family was from Saint Petersburg, Russia; and while they were of German-Jewish ancestry, they practiced the Russian Orthodox faith, as was required for Jews residing in Saint Petersburg. Goldman suffered from Crohn's disease. His wife suffered from Chronic brain syndrome. He was affectionately known as the "Coach" by his students.