Its even odds, but I'll bet on the swordsman. He knows how to use his weapon. Most people explore. My point is if the swordsman is facing someone with even a passing familiarity with his tool of choice, the swordsman is in some deep shit. Size DOES matter.
Lets admit it and move on.
I must say that technique does indeed matter more. Now, hold up there hulk, before you show me how you fist, let me explain. He'll get stabbed when the now mightily pissed off swordsman stabs him through the heart. Packing a big bad gun will do you no good if your aim stinks. I once knew a guy who practically tripped over himself when he walked, yet had the prowess of a dung beetle.