Fix #1226: Store minimal info in IR to construct hierarchy

Hum but if that happens you might be missing a parent. I think it's best to crash in that event.