Entropy based Independent Learning in Anonymous MultiAgent Settings. Efficient sequential matching of supply and demand is a problem of interest in many online to offline services. For instance, Uber, Lyft, Grab for matching taxis to customers; Ubereats, Deliveroo, FoodPanda etc for matching restaurants to customers. In these online to offline service problems, individuals who are responsible for supply (e.g., taxi drivers, delivery bikes or delivery van drivers) earn more by being at the right place at the right time. We are interested in developing approaches that learn to guide individuals to be in the right place at the right time (to maximize revenue) in the presence of other similar learning individuals and only local aggregated observation of other agents states (e.g., only number of other taxis in same zone as current agent).  A key characteristic of the domains of interest is that the interactions between individuals are anonymous, i.e., the outcome of an interaction (competing for demand) is dependent only on the number and not on the identity of the agents. We model these problems using the Anonymous MARL (AyMARL) model. The key contribution of this paper is in employing principle of maximum entropy to provide a general framework of independent learning that is both empirically effective (even with only local aggregated information of agent population distribution) and theoretically justified.  Finally, our approaches provide a significant improvement with respect to joint and individual revenue on a generic simulator for online to offline services and a real world taxi problem over existing approaches. More importantly, this is achieved while having the least variance in revenues earned by the learning individuals, an indicator of fairness.

Unveiling Taxi Drivers Strategies via cGAIL: Conditional Generative Adversarial Imitation Learning Smart passengerseeking strategies employed by taxi drivers contribute not only to driversu0027 incomes, but also higher quality of service passengers received. Therefore, understanding taxi driversu0027 behaviors and learning the good passengerseeking strategies are crucial to boost taxi driversu0027 wellbeing and public transportation quality of service. However, we observe that driversu0027 preferences of choosing which area to find the next passenger are diverse and dynamic across locations and drivers. It is hard to learn the locationdependent preferences given the partial data (i.e., an individual driveru0027s trajectory may not cover all locations). In this paper, we make the first attempt to develop conditional generative adversarial imitation learning (cGAIL) model, as a unifying collective inverse reinforcement learning framework that learns the driveru0027s decisionmaking preferences and policies by transferring knowledge across taxi driver agents and across locations. Our evaluation results on three months of taxi GPS trajectory data in Shenzhen, China, demonstrate that the driveru0027s preferences and policies learned from cGAIL are on average 34.7% more accurate than those learned from other stateoftheart baseline approaches.