Found the issue. The pool was sending work before sending the Extranonce1. Getting rid of the changes to change the difficluty on the first block becasue they were probably unnecessary and will only hurt hashrate.