We don't have to add annotations to the view if they're replies.

Fixes #540 

@tilgovi: Do you see any pitfall about this solution?