Many of the Asians that left Uganda came back after Amin was thrown out .
Amin 's ally Muammar Gaddafi told Amin to expel Asians from Uganda .