The Client/Server timer variable was only increasing at it's tick times, this lead to their timers being very innaccurate. This is probably part of all the ClientSide prediction issues I've been having.