Fix the overlay closing for realz

Summary:

Reviewers: alex, eric, grant, tyrell, song