Use microsecond precision time for insert/remove