Horav Chaim Shmuelevitz, zl, suggests that the Talmud implies the answer when it refers to Chushim's deafness. The brothers had slowly been drawn into a discussion with Eisav. While the debate continued back and forth they thought they were besting Eisav. Although this might have been true, during this time period Yaakov's body lay in shame. They did not realize what they were doing.
Human nature causes one to adapt slowly to a given situation.
The brothers' dispute with Eisav dulled their sensitivity to their father's shame. Chushim, who was not a part of the debate because of his impairment, was consumed with anger when he was told what was occurring. His senses were not dulled. He reacted in an expectable manner. We adapt to the conditions in which we are placed.