[#2026] Fix cleanup for test_segments cleanup as link.


Problem description
===================

The code cleanup up test_segments was dump and it was not handling links.

Why we got into this (5 whys)
=============================

No test for that.


Changes description
===================

Fix code and added tests.


How to try and test the changes
===============================

reviewers @alibotean 

Check that changes are ok.

Thanks!