Add q to toggle the pager.
Just close the pager with q, not toggle.