ENH: Release GIL in the gram variant