Remove a bogus sign check

Found with Coverity (CID 1163351).