You misread in_forward code.
This listen_address is not bind address: https://github.com/fluent/fluentd/blob/master/lib/fluent/plugin/in_forward.rb#L57