Be more specific when disconnecting the signal

Fixes a crash, amends be48cb646c99e350e08605ebcc08e84fa4ec1cb0