After retiring from professional football his first training appointment came from Jupp Derwall while they were both at Galatasaray. Terim's coaching career began when he was appointed the coach of Ankaragücü. He coached the club for 18 months before moving on to coach Göztepe in İzmir for a year. He had no significant success with either team. He was appointed as assistant to Turkey's national coach, Sepp Piontek, in 1990. He also coached the Turkey under-21 team. After serving as assistant coach for three years, he was appointed coach in 1993. Under his management, Turkey qualified for the final tournament of European Football Championship in 1996, for the first time in its history. Although they did not perform well in the tournament, losing all their matches and not scoring any goals, qualification was still considered a great achievement for Turkish football.