Revert "Figure out the gem path instead of guess"

This reverts commit ef4a67a8648509dddf1088b9b9a322a3c41d71f4.