fix some issues with inserting URLs, improve logic a bit, and move allowed checks into here (makes more sense)