Filter out known Swift catalog sources.