Firing callback on replies only

Fixes #2252 

@johnsimons please review