Infinite Scroll: Do not enqueue all scripts on single pages

Fixes #634
Previous commit 19fd934d9d29684bd0535851fd5b2215c40af66c didn't completely fix things
Merge pull request #963 from jeherve/fix/enqueues-infinite-scroll

Infinite Scroll: Do not enqueue all scripts on single pages