Correctly remove non-links during flattening