remove clients that are no longer fetching frames