This assumes that the released button is also the primary button, no? Which it does not need to be, with people/cats doing crazy mouse button combination pressings (actually expected with cats to do crazy mouse handling).

Not sure what proper handling is here, might need more rewrites to care for crazy input. Perhaps just keep it for now as is. There are not that many cats among testers and users, and I cannot see data loss happening by that code.