You will find the memory fragments during the summoning. When you get a character you already own during the summoning, then it will be converted to memory fragments.