Fix a bug where some dummy edges were leaked to the layout graph