Calculate where to scroll to based off the position in the div

Also, only scroll if not currently visible.

Corrects broken behaviour added in bb86373