Goumri improved one place to 14th in the long race at the 2004 IAAF World Cross Country Championships, but the Moroccans were edged into fourth by the Eritrean team. Track became his focus that year and he made his Olympic debut in August, reaching the 5000 m final at the 2004 Athens Olympics and ending the race in 13th place. He was ninth at the 2004 IAAF World Athletics Final, but gained a medal on the regional stage as he took the silver behind Khoudir Aggoune at the Pan Arab Games. Eighteenth in the short race at the 2005 IAAF World Cross Country Championships, he and his teammates Adil Kaouch, Mohamed Moustaoui and Hicham Bellani were again the fourth best (beaten this time by Qatar).