He only has 100 - 20 = <<100-20=80>>80 gold bars after losing 20 of them.
He then gives each of his friends 80 ÷ 4 = <<80/4=20>>20 gold bars.
#### 20