Track addition/deletion of all widgets, not only the top most ones

Fixes #191