If he gets asylum you will not be eligible for it with him since you were not married at the time of his filing.  Secondly, if he submitted his application after he was already out of status, he will have to appear before an immigration judge who will determine his eligibility.  If he does not get asylum granted, he will need to leave the country unless he chooses to appeal somehow.  It does not mean that you have to leave the US with him just because you are married.  You can stay in the US as long as your visa and your authorized period of stay is still valid.