A Socially Aware Bayesian Model for Competitive Foraging

A Socially Aware Bayesian Model for Competitive Foraging Sheeraz Ahmad (sahmad@cs.ucsd.edu) Department of Computer Science and Engineering University of California, San Diego Angela J. Yu (ajyu@ucsd.edu) Department of Cognitive Science University of California, San Diego Abstract can do better by moving to another patch. Competitive for- aging, therefore, presents an ecologically convenient case study for how individual-based, decentralized learning and decision-making, can lead to coherent and desirable behav- ioral patterns at the group level. Surprisingly, despite the rich body of empirical literature on the subject, the cognitive and neural mechanisms leading to IFD are not well understood. Most of the models (see (Pyke, 1984) for a detailed review) are descriptive in nature, and make unrealistic assumptions about foraging agents’ knowledge about the reward environ- ment and their behavioral constraints. In particular, agents are assumed to have precise and unbiased knowledge about the reward distribution (the ideal assumption), and that they can freely travel from one patch to another without any lim- itation or cost (the free assumption). In more natural scenar- ios, humans and animals often have to forage without (initial) knowledge about the relative reward value of the different op- tions, and there may be costs associated with switching be- tween options (e.g. time, energy, opportunity cost, money). A more complete understanding of competitive foraging be- havior needs to incorporate these factors. Effectively solving the problem of when and where to forage is critical for survival in many animal species. The task is further complicated when there are other agents, potentially competing for the same limited resources. Previous models of foraging consider agents either in isolation or in groups but without competition. Here, we present a novel Bayesian model for competitive foraging, Socially Aware Bayesian Agent (SABA), that takes into explicit account the presence of other agents for both learning and decision-making. For comparison, we also implement a simple Naive Agent model that completely ignores the presence of other agents. We find that although all models perform well in a stationary environ- ment, converging quickly to the optimal population-level solu- tion, only SABA with the stochastic foraging policy can read- ily adapt when the environment is non-stationary. These results represent a first step toward cognitively sophisticated represen- tations for learning and decision-making in competitive forag- ing. Keywords: competitive foraging; Bayesian learning; decision-making; public information; Ideal Free Distribution (IFD). Introduction Foraging is the general problem of deciding how to choose among different reward options (e.g. food patches) adap- tively, based on sequentially observed, noisy data, and thus in the presence of uncertainty associated with reward values of the different options. In competitive foraging, there is both benefit, in terms of information, and cost, in terms of com- petition, due to the presence of other foraging agents. Daily life is rife with such competitive foraging examples, such as picking a restaurant, asking someone out on a date, bidding on a house, picking a vacation location, etc. In the experimen- tal literature, this has most often been studied in the context of animals foraging in an environment containing patches of food (see (Pyke et al., 1977; Perry & Pianka, 1997) for re- view). The optimal solution for the population is to have the agents distributed proportional to the underlying reward rates at the different food patches, as that ensures maximal efficiency of food intake in the population. This distribu- tion is also known as the Ideal Free Distribution (IFD), and has been observed in humans (Sokolowski et al., 1999; Kraft & Baum, 2001; Madden et al., 2002; Goldstone & Ashpole, 2004; Goldstone et al., 2005) and other species such as ducks (Harper, 1982), guppies (Dill, 1987), sticklebacks (Milinski, 1979), Bumblebees (Dreisig, 1995), etc. Interestingly, IFD is also the individually optimal Nash equilibrium solution (Nash et al., 1950), in which no agent Recently, there has been a shift toward models that re- lax these strong assumptions, such as EPICURE (Roberts & Goldstone, 2006), Polyworld (Griffith & Yaeger, 2011), Avidian agents (Walker, 2011), Omega agents (Seth, 2011), and Dynamic system (Krivan et al., 2008). While these mod- els have demonstrated that IFD can be achieved even when the ideal assumption is relaxed, they still make the free as- sumption. More importantly, these models are overly sim- plistic in their assumptions about each agent’s representation of the environmental reward distribution and other compet- ing agents. Most of these assume that learning only proceeds based on private information, i.e. an agent’s own history of choices and reward outcomes (Griffith & Yaeger, 2011; Walker, 2011; Seth, 2011). EPICURE (Roberts & Goldstone, 2006) does consider the effect of competition but it just fits an ad-hoc, descriptive parameter corresponding to agent den- sity, so that the competition has a unidirectional (negative for their data) effect on agent choices, failing to account for the fact that depending on agent’s knowledge, at times high agent density might make an option more lucrative. A large body of empirical ecology research shows that animals readily use public information, i.e. other agents’ choices and possibly also their outcomes, in foraging (e.g. (Krebs, 1973; Boyd, 1988; Templeton & Giraldeau, 1995)), as well as in other bio-