Intro to Agent Based Modeling. An example of how agent based modeling... | by Bassel Karami | Towards Data Science

Table of contents:
1- Intro: why agent based modeling?
2- Our supermarket queueing problem
3- Model design
4- Model execution
5- Conclusion
Agent based modeling (ABM) is a bottom-up simulation technique where we analyze a system by its individual agents that interact with each other.
Suppose we want to predict the number of COVID cases in a particular region, we can create agents representing the people in that region.
We simulate the behavior of several agents (people) where some may be wearing masks while others aren’t, some are vaccinated, some are very social while others aren’t etc. We then simulate their interactions with each other and the associated probabilities of COVID being transmitted...
After running the simulation, we are able to better understand the overall system... we see the number of infections in the population, the number of people recovered, transmission rates, whether we hit herd immunity etc. We can also analyze how the system would look like (infections, recoveries etc.) if we were to impose measures such as forced vaccinations.
The cornerstone of ABM is the ability to understand macro phenomena emerging from micro-scale behavior.
In the above example, the alternative to using ABM is to directly look at macro-level stats such as the number of infections and recoveries and to directly analyze the trends of those stats. Instead, ABM emphasizes the behavior of individual agents which is a more flexible and granular bottom-up approach that can capture aspects a macro-scale model can’t.
ABM has uses in biology, social sciences, traffic management, and supply chain management amongst many other disiplines.
The behavior of those agents should represent reality as closely as possible (we can validate that), so for example, if 75% of people in that region wear masks, then we create agents such that 75% of them are modeled as mask-wearing agents.
Suppose that we are managing staffing costs at a supermarket.
Problem: For a particular hour, we expect 500 customers to show up and we want to determine how many counters should we have open during that hour.
The average customer is worth $10 of gross margin while the average cost to operate a single counter for one hour is $300.
In contrast to standard queueing theory models (M/M/1, M/G/1, M/M/c etc.), agent based modeling can more easily capture behavioral aspects in this multi-waiting-line, multi-server environment.
Common behaviors including queue balking (customer doesn’t join the queue because it is too long), reneging (customer joins the queue but leaves after waiting for too long), and jockeying (customers moving from a longer queue to a shorter one).
We shall attempt to solve this problem using ABM.
We’ll build a simple ABM model with the following assumptions:
1- We are optimizing net profit for this single hour.
2- Exactly 500 customers will be arriving to the Supermarket during this hour.
3- Every customer is worth $10.
4- Every counter costs $300 to operate.
5- Customers arrive at the supermarket queues at different times during the day, based on a beta distribution, such that they are more likely to arrive at the middle of the hour.
6- The average time to process a customer order is 45 seconds, but the processing time differs between customers. It follows a Poisson distribution (for simplicity and to ensure we get discrete non-negative numbers) with an average of 45.
7- Every customer has a balking tolerance which is the number of people in the queue that a customer is willing to join. If there are more people in the queue, the customer won’t join the queue and we lose the sale. The customers’ balking tolerance follows a poisson distribution. The average tolerance is 6 i.e. most customers will balk if there are 6 or more other customers in the queue.
8- Customers choose the counter with the shortest queue.
Note that in real-world applications, we validate the assumptions (or pass them as parameters to be optimized) against available data. Also, in this introductory example, balking is the only behavior we consider.
We run the ABM simulation for 3600 steps (ticks) representing 3600 seconds in the hour. Below is the initial configuration code.
# Configticks = 3600 # 3600 ticks = 3600 seconds = 1 hourno_customers = 500avg_service_time = 45 # ticks/seconds per customergross_margin_per_customer = 10 # dollarscost_per_counter = 300 # dollars
Now, we use the Mesa library in python to code the rest of our model. Let’s start with defining our Customer class and the required attributes. The Customer class is one of two type of agents we will be dealing with, the other is Counter.
from mesa import Agentclass Customer(Agent):    def __init__(self, unique_id, model):        super().__init__(unique_id, model)        # Time required to process the customer's transaction        self.service_time = ss.poisson(45).rvs()        # Time of arrival at queue        self.entry_time = np.int(ss.beta(3, 3).rvs() * ticks) + 1        self.balk_tolerance = ss.poisson(5).rvs() + 1        # Whether or not the customer has arrived at the queue        self._arrived = False        self._chosen_counter = None        self._q_entry = None        self._q_exit = None        # Start time when customer is being served        self._service_entry = None        # End time        self._service_exit = None
We also need to create a “step” method for our customers which describes what do they do at each step (tick/second) of the simulation.
class Customer(Agent):    def __init__(self, unique_id, model):        super().__init__(unique_id, model)        # Time required to process the customer's transaction        self.service_time = ss.poisson(45).rvs()        # Time of arrival at queue        self.entry_time = np.int(ss.beta(3, 3).rvs() * ticks) + 1        self.balk_tolerance = ss.poisson(5).rvs() + 1        # Whether or not the customer has arrived at the queue        self._arrived = False        self._chosen_counter = None        self._q_entry = None        self._q_exit = None        # Start time when customer is being served        self._service_entry = None        # End time        self._service_exit = None    def select_counter(self):        self._arrived = True        # Queue at shortest counter        self._chosen_counter_idx = np.argmin([            len(counter.queue) for counter in self.model.counters])        self._chosen_counter = self.model.counters[            self._chosen_counter_idx]        # Balk if there are too many people at the counter        if len(self._chosen_counter.queue) < self.balk_tolerance:            self._chosen_counter.queue.append(self)            self._q_entry = self.model._current_tick    def pay_n_leave(self):        self._service_exit = self.model._current_tick        self._chosen_counter.active_customer = None    def step(self):        if (self._arrived == False) & \        (self.model._current_tick >= self.entry_time):            self.select_counter()        elif isinstance(self._service_entry, int):            if self.model._current_tick - self._service_entry \            == self.service_time:                self.pay_n_leave()
In the above code, at every simulation step, if we have arrived at the simulation step/time which matches the customer’s entry time (time at which he arrives at the supermarket queue), then we run the select_counter method where he chooses the counter with the shortest queue or he balks if that queue is too long.
If the customer is already being served at the counter, we check if we have already run enough steps to arrive at his service time (time required to process his transaction) then we run the pay_n_leave method.
Similarly, the below code is used for the 2nd type of agents, Counter.
class Counter(Agent):    def __init__(self, unique_id, model):        super().__init__(unique_id, model)        self.queue = []        self.active_customer = None    def dequeue(self):        try:            self.active_customer = self.queue.pop(0)            self.active_customer._service_entry = self.model._current_tick            self.active_customer._q_exit = self.model._current_tick        except:            pass    def step(self):        if self.active_customer is None:            self.dequeue()
Finally, we create a QueueModel class which is used to manage the overall simulation/system that the agents belong to.
from mesa import Modelfrom mesa.time import RandomActivationclass QueueModel(Model):    """Queueing model with customers and counters as two    types of agents that interact with each other    """    def __init__(self, no_customers, no_counters, ticks):        self.ticks = ticks        self._current_tick = 1        self.no_customers = no_customers        self.no_counters = no_counters        self.schedule = RandomActivation(self)        # Create agents        self.customers = []        self.counters = []        for i in range(self.no_customers):            customer = Customer(i, self)            self.schedule.add(customer)            self.customers.append(customer)        for i in range(self.no_counters):            counter = Counter(i + no_customers, self)            self.schedule.add(counter)            self.counters.append(counter)    def step(self):        self.schedule.step()        self._current_tick += 1
The QueueModel class can be initiated with the number of customers (in our example 500), the number of counters (this is what we are trying to determine), and the ticks (number of simulation steps, 3600 representing 3600 seconds in an hour).
We have a “schedule” attribute where we describe which agents will be activated and when. We use RandomActivation which activates all agents in random order. You can read up the Mesa documentation for different settings.
Upon initialization, there are 2 for loops where we create the Customer and Counter agents. We also add them to the model’s schedule.
There is also a “step” method that orchestrates running one step of the simulation and it also triggers the “step” methods of the agents that are part of the schedule.
We can now run the simulation by running the below code...
model = QueueModel(ticks=ticks,    no_customers=no_customers, no_counters=no_counters)for i in range(ticks):    model.step()
However, we can also add a “data collector” to our QueueModel class that can help us track metrics of interest. We can track those metrics at every step of our simulation.
Let’s track 8 metrics:
1- Number of customers arrived. At the end of the 3600 simulation steps (representing 1 hour), we should have 500 customers.
2- Number of customers already served.
3- Number of customers who arrived at the queue but balked. We lost those customers.
4- The average queue size across all counters.
5- The average waiting time before a customer is served.
6- Gross margin = number of customers served x $10 per customer
7- Operating costs = number of counters x $300
8- Total profit = Gross margin - Operating costs
In order to add the metrics to our ABM model, we first create them as functions that take our QueueModel as an argument...
def get_customers_arrived(model):    customers_arrived = [        customer._arrived for customer in model.customers]    no_customers_arrived = np.sum(customers_arrived)    return no_customers_arriveddef get_customers_served(model):    customers_served = [not(customer._service_exit is None)                        for customer in model.customers]    no_customers_served = np.sum(customers_served)    return no_customers_serveddef get_customers_balked(model):    customers_arrived = [        customer._arrived for customer in model.customers]    # Customers who never joined a queue    customers_no_q = np.array([        customer._q_entry is None for customer in model.customers])    no_customers_balked = np.sum(customers_arrived * customers_no_q)    return no_customers_balkeddef get_avg_queue_size(model):    queue_size = [len(counter.queue) for counter in model.counters]    avg_queue_size = np.mean(queue_size)    return avg_queue_sizedef get_avg_waiting_time(model):    customers_wait = [np.nan if customer._q_exit is None else                      customer._q_exit - customer._q_entry for customer in model.customers]    avg_customer_wait = np.nanmean(customers_wait)    return avg_customer_waitdef get_gross_margin(model):    return gross_margin_per_customer * get_customers_served(model)def get_operating_costs(model):    return cost_per_counter * no_countersdef get_total_profit(model):    return get_gross_margin(model) - get_operating_costs(model)
Finally, we update our QueueModel class as follows...
from mesa.datacollection import DataCollectorclass QueueModel(Model):    """Queueing model with customers and counters as two    types of agents that interact with each other    """    def __init__(self, no_customers, no_counters, ticks):        self.ticks = ticks        self._current_tick = 1        self.no_customers = no_customers        self.no_counters = no_counters        self.schedule = RandomActivation(self)        # Create agents        self.customers = []        self.counters = []        for i in range(self.no_customers):            customer = Customer(i, self)            self.schedule.add(customer)            self.customers.append(customer)        for i in range(self.no_counters):            counter = Counter(i + no_customers, self)            self.schedule.add(counter)            self.counters.append(counter)        self.datacollector = DataCollector(            model_reporters={    'Customers Arrived': get_customers_arrived,                          'Customers Served': get_customers_served,    'Customers Balked': get_customers_balked,    'Average Waiting Time': get_avg_waiting_time,    'Average Queue Size': get_avg_queue_size,    'Gross Margin': get_gross_margin,    'Operating Costs': get_operating_costs,    'Total Profit': get_total_profit})    def step(self):        self.datacollector.collect(self)        self.schedule.step()        self._current_tick += 1
We are now ready to run our model...
Let’s try to run our model with 5 counters...
no_counters = 5model = QueueModel(ticks=ticks,    no_customers=no_customers, no_counters=no_counters)for i in range(ticks):    model.step()
Below are the results and the code used to produce them...
run_stats = model.datacollector.get_model_vars_dataframe()fig, (ax1, ax2, ax3, ax4) = plt.subplots(4, 1)fig.figure.set_figwidth(12)fig.figure.set_figheight(16)fig.suptitle(f'Simulations stats using {no_counters} counters', fontsize=20)ax1.plot(run_stats[['Customers Arrived',                'Customers Served',                'Customers Balked',               ]])ax1.legend(['Customers Arrived',            'Customers Served',            'Customers Balked',            ])ax1.set_ylabel('Customers')ax1.set_xlim(0)ax1.set_ylim(0)ax2.plot(run_stats['Average Queue Size'], color='red')ax2.legend(['Average Queue Size'])ax2.set_ylabel('Customers')ax2.set_xlim(0)ax2.set_ylim(0)ax3.plot(run_stats['Average Waiting Time'], color='grey')ax3.legend(['Average Waiting Time (across full hour)'])ax3.set_ylabel('Seconds')ax3.set_xlim(0)ax3.set_ylim(0)ax4.plot(run_stats[['Gross Margin',                'Operating Costs',                'Total Profit'               ]])ax4.legend(['Gross Margin',            'Operating Costs',            'Total Profit'            ])ax4.set_ylabel('Dollars')ax4.set_xlim(0)fig.show()
The x-axis represents the 3600 ticks/seconds in the 1 hour that we are simulating.
The first chart shows us how after ~1200 seconds (~20 minutes), customers start to balk as too many customer are arriving.
In the 2nd chart, we can see how the queue size starts to increase after roughly ~10 minutes and it peaks at approximately 6 customers per counter at the middle of the hour.
The 3rd chart shows us how average waiting time (cumulative) increases from the beginning of the hour towards the end of the hour as many customers arrive at the middle of the hour and they wait in long queues pushing the average waiting time up.
Finally, the 4th chart shows us our financials, where we accumulate just over $1500 towards the end of the hour.
Based on the above, 5 counters seems to be too low as we can see that just under 200 customers (40%) have balked and they are lost.
Now, let’s try to run the simulations with 5–15 counters to determine what is the ideal number of counters...
full_stats = {}for no_counters in range (5, 16):    model = QueueModel(ticks=ticks,    no_customers=no_customers, no_counters=no_counters)    for i in range(ticks):        model.step()       run_stats = model.datacollector.get_model_vars_dataframe()    full_stats[no_counters] = run_stats.iloc[-1]    pd.DataFrame(full_stats).transpose().astype(int)
In the above code, we run simulations with 5–15 counters and we look at the stats at the end of each simulation (not every step of the simulation). The dataframe output shows the results with different numbers of counters...
We can see that total profit is maximized at $2030 with 9 counters.
We can optimize based on any metric of interest... immediate profit is not necessarily the best metric.
Average waiting time per customer using 9 counters is 94 seconds... by just adding one additional counter, we cut the waiting time by 33 seconds to 61 and we only lose $90 (total profit is $1940). Reduced waiting time can likely create a better customer experience leading to more profit in the long-run.
The goal of the above example is to introduce the topic of agent based modeling and its unique benefits, including the ability to easily capture different agent behaviors. Often, it is argued that we take decisions based on bounded rationality.
Furthermore, another major benefit is the ability to see how micro-scale behavior impacts the overall system. We see, for example, how balking tolerance of individuals can dictate different average waiting time in queues and different accumulated profit.
In subsequent articles, we can look at how to select the optimal model assumptions and parameters for a model after validating against available data and stylized facts.