ItemGrid: fix glib criticals.

We let a timeout lapse, but didn't update our state to reflect that.
Later we would then re-remove the source triggering glib criticals.