removing distinct, adding tests, fixing bugs