Probabl the right way to implement this would be to add a skip_connections interface to the MLP itself. The Layers don't need to be aware that they're getting input from multiple other Layers. They can just get input that's formatted as a CompositeSpace. They don't need to know where it came from.