  This paper presents a multi-agent reinforcement learning algorithm to
represent strategic bidding behavior in freight transport markets. Using this
algorithm, we investigate whether feasible market equilibriums arise without
any central control or communication between agents. Studying behavior in such
environments may serve as a stepping stone towards self-organizing logistics
systems like the Physical Internet. We model an agent-based environment in
which a shipper and a carrier actively learn bidding strategies using policy
gradient methods, posing bid- and ask prices at the individual container level.
Both agents aim to learn the best response given the expected behavior of the
opposing agent. A neutral broker allocates jobs based on bid-ask spreads.
  Our game-theoretical analysis and numerical experiments focus on behavioral
insights. To evaluate system performance, we measure adherence to Nash
equilibria, fairness of reward division and utilization of transport capacity.
We observe good performance both in predictable, deterministic settings (~95%
adherence to Nash equilibria) and highly stochastic environments (~85%
adherence). Risk-seeking behavior may increase an agent's reward share, as long
as the strategies are not overly aggressive. The results suggest a potential
for full automation and decentralization of freight transport markets.
