Use Set ADT to keep track of colors which should be preserved for image filtering