The Kagrumez Gauntlet is actually a miscellaneous quest. So, it doesn't appear as its own individual entry in your quest log. Both following this guide to collect the gems or heading straight to the Kagrumez ruins will both cause the misc quest to trigger.