Not really. When comparing the best fighters in A Song of Ice and Fire, Barristan Selmy outranks Robert Baratheon. Though Robert is a sheer mass of brute force, Barristan is the most acclaimed fighter in all of Westeros. He is beyond proficient and would defeat Robert.