Riding on gravel is harder because it uses more muscles. This includes your upper body, which usually goes along for the ride when riding on pavement.