Fix deletion of factors

Reviewer: Tiffany Go

Issue: None