No, you only apply the Asian Handicap to the competitors that you backed. For example, if you back a home team to win at -0.5 and the final score is 1-1, then the handicap score is 0.5-1, not 0.5-1.5.