Regardless of what some think, a successful plan A can include talking about things like hurt. Don't be discouraged about your H being unable to talk to you about this. On both sides of this equation, answers are needed and that is part of the healing process. By asking the question you did, you gave him notice that you want to know what hurts and how to fix it. This is better than trying to ignore what happened and acting as if nothing has occurred.
Some people just take longer to open up.
Keep up the Plan A and only once in a while bring up the sensitive issues. If your H wants to re-build then at some point he will open up. It's very important that when he does for you to react in a caring, interested fashion and not let yourself drop into anger or disgust no matter what he tells you. This worked for my W as far as communication went. As in my case, even if you get this far, it doesn't mean the marriage can or will be saved, but you will feel better knowing the truth and so will you H. If it's any comfort, I have been exactly where you are at now as it pertains to your frustration.