You cannot increase patch vetsion if you break backward compatibility. You must increase major version in such case. see http://semver.org/
It is important to follow it as it indicate to users of library if they need modify their own code.