"SECTION I.Introduction
Over the past several years the game industry has undergone a series of major changes. The increasing prominence of persistent online games, digital distribution platforms and portals, mobile and social games, and the emergence of new business models have all changed fundamental aspects of making and playing games. Developers increasingly focus on the live operation of a game, rather than creating a boxed and finalized product. Players are more diverse, have access to games in more places and at more times, and produce more data and content for developers to leverage than ever before. Across these changes four forces have come to the fore:

Games and cross-game play is increasingly persistent and presenting longer-term experiences

Game developers are starting to see their game titles as an ecosystem wherein players may move from game to game

Player communities and social gameplay have gained prominence

Developers and players have a growing interest in coupling the real world and virtual world(s)

Some of these forces have been around for a number of years, while others are just beginning to emerge. The consequence of these forces is a profound opportunity for Artificial Intelligence (AI), Computational Intelligence (CI), and Machine Learning (ML) in games (collectively referred to as Game AI) to play an even greater role in the development of computer games and the delivery of engaging real-time experiences to players.
Through these forces we see an opportunity for Game AI to address new research questions that have the potential to dramatically impact game development practices. The most significant consequence of the shifting landscape of computer game development is the generation of massive amounts of data. Researchers and game developers can leverage this data in a new paradigm of data-driven Game AI focused on how to use these sources of data to improve game development practices and to deliver more engaging experiences. However, we are not merely advocating that researchers and game developers adopt data-driven analogues to existing Game AI practices (although that would be a worthy endeavor). Instead, we are proposing that the market forces enumerated above are forcing the industry to change how they think about game development processes. The modern development environment presents significant challenges to scalability-of both the practice of creating games and also the size and scope of games themselves-that are readily addressed by artificial intelligent, computational intelligence, and machine learning research.

Game AI was initially about supporting the interaction between player and the game itself. Recently there has been a push for procedural content generation as a means to support and augment the game developer on a game by game basis. In this paper, we present our desiderata on the future of Game AI in which intelligent systems support the entire game production pipeline from game creation to live operation, and across a number of games and game genres. This perspective of Game AI for production does not supplant or replace prior perspectives on Game AI, but presents a new lens through which to see an expanded role for Game AI in computer game production. We will enumerate 12 novel research questions that arise when considering the role of AI as Producer and discuss steps toward how these challenges may be addressed.

SECTION II.The Shifting Landscape of Game Development
In this section, we enumerate some of the prominent new forces that are shaping the way that computer game development companies create games. Many of these forces have arisen as a confluence of advances in computing technology and market forces that result from a maturation of the computer game industry.

Persistent Games: The rise of Massively Multiplayer Online Games (MMOGs), social games, online game portals and long-term or recurring game experiences is creating longterm data on players within games. Many games have players join, play over long periods of time, and potentially rejoin at later times. Developers are increasingly pressed to develop content that provides long-term engagement with a game, rather than a closed experience with clear beginning and end.

Ecosystem of Games: Developers have a wider variety of tools to build games and lowered barriers to distributing games to new users through digital platforms and online portals. Together this puts greater emphasis on keeping players engaged within an ecosystem of games from a single developer, rather than focusing on experiences only within one game. Developers are driven to provide new content of multiple kinds (including meta-game objectives) that engage players across games. The growing diversity of game players has led to additional emphasis on ensuring an ecosystem of games provides a diverse range of experiences.

Player Communities: Player communities have emerged as a major driving force for the success (and failure) of games. Players generate a mass of content from reviews and walkthoughs through add-ons and full game modifications. Continually engaging communities, supporting player socializing within the community, managing how players impact ones another's experiences (positively and negatively), and leveraging user-generated content are growing concerns.

Coupling of Real and Virtual Worlds: Games are more widely adopting ways to connect to the real world. Sensing systems from the Kinect and Wiimote to mobile phone GPS provide more data on players' real-world environment. Output modalities from second screen experiences and mobile phone augmented reality to virtual reality are emerging as new ways to view game content. At the same time these technologies have introduced a host of challenges around how games can interface with and use this real-world context and respond to more unstructured types of information.

Each of these forces presents new opportunities to solve problems of real-world applicability to game developers. A side effect of each of these forces is the massive amounts of data being generated about game players. While it is not necessarily the case that new research problems will require data-driven AI, CI, and ML techniques, we see this data as a tool for tackling real-world game development problems.

SECTION III.Background: Three Roles of Game AI
In 2001, Laird and van Lent [1] put forth their seminal argument for AI in computer games as an academic pursuit. They specifically argued that the pursuit of “human-level” AI systems could use computer games as testbeds for research because games were intermediate environments between the toy domains researchers had been using and the full complexity of the real world. While this opened Game AI as an academic endeavor, the automation of various aspects of computer games has been a part of the industrial practice of creating commercial computer games since the beginning.

Game AI has come to refer to the set of tools-algorithms and representations-developed specifically to aid the creation and management of interactive, real-time, digital entertainment experiences. While games are played by humans, there are a number of aspects of the game playing experience that must be automated: roles that would be best performed by humans but are not practical to do so:

Opponents and enemies that are meant to survive for only a short time before losing.

Non-player characters in roles that are not “fun” to play such as shopkeepers, farmers, or victims.

Companions in single-player experiences and non-player characters in support roles.

Drama management at scale.

Game designer for personalized experiences at scale.

As we go down this list, Game AI is charged with taking progressively more responsibility for the quality of the human player's experience in the game.
We group Game AI approaches into three broad roles that AI, CI, or ML takes in the creation and delivery of engaging entertainment experiences. Each role targets a different stakeholder: players, designers, and producers. The first role is AI as Actor, in which the Game AI system mediates between the player and the game to create a compelling real time experience. The most common manifestation of AI under this paradigm is controlling or managing bots and non-player characters. The second role is AI as Designer, in which the AI mediates between a game designer and a single player-game system. Under this paradigm, AI systems might procedurally generate game content or adapt the game to particular players to scale the game development process. Finally, we propose a third role, AI as Producer, in which the AI mediates between game producers and a number of systems of players, designers, and games. Figure 1 shows how the different roles relate to each other and the three primary human stakeholders.

These three roles are not distinct phases in the pursuit of Game AI, but overlapping sets of concerns and driving problems, all of which need to be pursued individually or in unison. We see AI Producers as a superset of AI Designers, encompassing a broader set of research questions. Equivalently, we see this as a shift from Game AI for game design to Game AI for game production. In industry the differences between designers and producers have blurred as technical barriers to content production have lowered and gameplay and business (e.g. monetization and marketing) are more tightly coupled.

A. Artificial Intelligence as Actor
Historically, the earliest uses of artificial intelligence in computer games was to mediate between users and the game. AI served the role of an artificial human opponent or playmate, enabling play without requiring other people or filling roles humans would be loathe to fill in a game. Compared to nonGame AI, the intelligence built into games places a greater emphasis on creating engaging and entertaining experiences for users, rather than maximizing a utility function such as score or win/loss rates.

For the AI as Actor role, research has focused on non-player character (NPC) path planning and decision making. Game agent decision making emphasizes the believability of characters to support the suspension of disbelief that the player is interacting with software instead of a monster, human opponent, or human companion. These problems are still being pursued by industry developers and academic researchers.

Drama management is another way for AI to mediate between users and the NPCs and other aspects of a game or virtual world [2], [3], [4]. A drama manager is an omniscient agent responsible for delivering an enjoyable and coherent narrative experience to players, much as a “Dungeon Master” does the same for tabletop role playing games.

Fig. 1. - Three roles of Game AI and their human stakeholders. AI as Actor describes how AI mediates between the player and the game itself. AI as Designer describes how AI mediates between a single game designer and the human-computer system of player and game. AI as Producer is a new role describing how AI mediates between producers and many systems of designers, users, and games.
Fig. 1. Three roles of Game AI and their human stakeholders. AI as Actor describes how AI mediates between the player and the game itself. AI as Designer describes how AI mediates between a single game designer and the human-computer system of player and game. AI as Producer is a new role describing how AI mediates between producers and many systems of designers, users, and games.
Show All

B. Artificial Intelligence as Designer
The second role of Game AI is to mediate between the human designer (or developer) and the human-computer system comprised of game and player. In our metaphor, game designers are responsible for building and defining a game, analyzing how players interact with the game, and iteratively refining a game to achieve a design vision. This paradigm for artificial intelligence is often referred to as Procedural Content Generation (PCG)—algorithms and representations for generating any and all components of games [5], [6], [7]. Offline content generation emphasizes producing content designers may curate or refine as a means of increasing the efficiency of the game development process. Online, or just-in-time, generation focuses on providing larger amounts of variation than human designers can achieve alone and/or tailoring content to a given user. A primary concern for PCG researchers has been (a) ways to appropriately represent game content to suit generation algorithms, while (b) providing means for users to interact with generation systems to author desired content and outcomes.

Game adaptation combines content generation and player modeling to enable AI designers to tailor games to individual players, emphasizing a closed loop of modeling player actions and automated adjustment of game content based on design goals for player behavior [8], player skill acquisition [9], or maximizing player enjoyment [10], [11], [12]. Game adaptation taken to its extreme introduces the problem of game generation-the automatic creation of entire games driven by (real or simulated) player feedback.

The availability of large data sets is having an impact on those who see AI as designer. In particular, game analytics involves capturing, aggregating, understanding, and visualizing player behavior to support designer understanding [13]. Player modeling research examines methods to describe and predict player behavior, potentially to be used by designers or automated AI systems in response [14]. While not a traditional domain of Game AI, player modeling has becoming increasingly prominent as AI agents shift to learning to adapt to players. Machine learning (e.g. [15], [16]) and evolutionary computing [17] are the primary areas currently being employed to perform these modeling tasks.

C. Artificial Intelligence as Producer
The third role of Game AI uses a metaphor of AI as game producer. In our metaphor, producers concern themselves with the entire set of games and game content being made by a company, along with related aspects of managing player communities. AI Producers mandate a shift from single player experiences within a closed game to long-term player experiences within an open game, understanding a player across multiple games in an ecosystem, and understanding how multiple players interact as an in-game and out-of-game community. AI Producers extend many methods of AI designers, driving a shift to model and adapt games that distinguishes characters (in-game avatars or personas) from players (agents manipulating those characters).

AI Producers also leverage a broader sense of context for enhancing game experiences including the community of players and the real-world context of their activities. AI for game production puts a premium on broadening the scope of Game AI to better integrate the increasingly pervasive nature of games into how games entertain and engage users. Games no longer are sharply bound by a single delivered product and Game AI ought to respond by incorporating these newly opened borders for new research domains. Effectively responding to these challenges will require new methods to leverage the masses of data being produced by players to improve interactive experiences.

SECTION IV.Game AI Producer: Research Questions
AI producers will use the wealth of data being generated to meet the needs imposed by the four forces of (1) persistent games, (2) game ecosystems, (3) player communities, and (4) real-virtual world coupling. In each case these forces provide unique opportunities for data-driven approaches to Game AI that enhance player experience through richer sources of information and emerging modes of game-related interaction. Rather than replace earlier roles and stakeholders of Game AI, we assert that the new role of AI as Producer must emerge to address the novel concerns raised by the four forces. This new role for Game AI enhances and extends the AI techniques defined from earlier roles, leading to new research questions and techniques that potentially overlap multiple roles. In the next sections, we revisit the forces and the research questions that accompany them. Due to the overlapping nature of roles, in many cases research questions may address more than one role simultaneously.

A. Persistent Games
Persistent games shift from closed games comprising time investments typically on the order of days to ongoing and extended game experiences spanning months or years generating long-term data on player history and activities. AI Producers address the full lifetime of a player, spanning the standard business concerns of acquiring new players, retaining players over a long period of time, and reacquiring lapsed players. Key research questions around using long-term data to improve player engagement focus on: lifelong agents, gameplay support, and engagement-oriented content generation. Unifying these is a view of Game AI providing for long-term player engagement across an increasingly diverse group of players.

1) How can an AI agent interact with players over very long periods of time?
An agent that persists for months, years, or a over a user's lifetime is referred to as a lifelong agent. In the context of computer games, lifelong agents are NPCs that learn about you as a player over time. Lifelong agents serve as long term companions (or adversaries) that recognize and adapt to changes in players over time and use historical interactions with players to shift their behavior. That is, a lifelong agents get to know players, and become familiar with the player, and familiar to the player.

Lifelong agents are relatively unexplored in the domain of games. In the domain of virtual agents for healthcare, Bickmore and colleagues have been designing and developing agents that interface with humans longitudinally [18]. In the context of games, lifelong agents may foster player empathy for companions-and enmity for rivals-or engage users in social interactions with game world NPCs. In addition, adapting agent behaviors to foster long-term engagement stands as a key to the problem of many online games face in creating vibrant and stable communities of players.

A central challenge for lifelong agents will be adapting to games with continually evolving content in the form of patches, expansions, downloadable content, and other incremental updates to the game the agent inhabits. Research in lifelong machine learning investigates how agents can transfer knowledge between particular tasks, continually learn and refine knowledge, uncover representations for complex information, and incorporate guidance or feedback from humans [19]. Addressing these challenges for game agents can provide enormous benefits to developers of live and ongoing games.

2) How can a Game AI system support deep gameplay?
A key to persistent games is player retention. Many contemporary games have high ceilings for player skills to build up to-through complex underlying systems or ever-evolving multiplayer competition. However, complex games often lose players early in the game, especially as players increasingly have more diverse backgrounds and skills. Gameplay support agents act as mentors to players to help them overcome challenges that might otherwise cause them to quit playing a game. Gameplay support AI observes players, learns their gameplay strengths and weaknesses, and intervenes to provide players with appropriate hints, training materials, or content adjustments as needed. For example, if a player shows an inability to counter a particular strategy in an RTS the AI would identify the missing player skills and could provide instruction about the appropriate response, training video demonstrations, or set up game scenarios to practice the requisite skill. At the extreme a gameplay support agent could coach players to climb their way to the top of multiplayer competition.

Intelligent Tutoring Systems [20] present one vision for gameplay support. Applying interactive tutors to support longterm player engagement will need advances in representing player skills, modeling players based on their game activities, devising interventions to improve those skills, and adjusting those interventions in response to tutoring success or failure. Ultimately, gameplay support systems should aspire to automatically generate tutorials-for introducing new content through coaching players along difficult missions-based on game features and gameplay data. Extracting examples appropriate to demonstrate skills from gameplay data, recognizing player skills, and choosing appropriate timing and presentation style for tutorial information are all key challenges to apply gameplay data toward automated game tutoring.

Another technique for gameplay support might lie in Dynamic Difficulty Adjustment (DDA). Dynamic difficulty adjustment systems make real-time adjustments to game parameters, item placement, enemy behavior, and other content to suit player abilities. Techniques for DDA have involved classical cybernetic systems [21], production systems [22], optimization of generator output from neuro-evolutionary [12] or machine learning [23] systems, or logic programming on possible player behavior [8]. Adapting these techniques for gameplay support requires capturing the long-term effects of interventions on player engagement [24], richer models of player skills related to various gameplay domains, and techniques to intelligently reuse high quality human-authored content where possible.

3) How can a Game AI system generate motivational game content?
Content generation for long-term engagement models player values, preferences, and motivations to generate content that continues player engagement over long periods of time. The goal is to improve player retention or reacquire players who have lost interest in a game. Whereas gameplay support addresses potential “pain points” of gameplay to prevent player early dropout and improve player acquisition, long-term motivational content generation focuses on how to best retain players once they have committed to a game and encourage players who have lapsed from a game to return.

Compared to previous work on content generation, motivational content generation should incentivize players to take advantage of aspects of a game they already enjoy or to explore new elements of a game they might not have encountered. For example, generation of personalized achievements can encourage players to try alternative ways to complete a mission or level. Likewise, a system might generate mini-games within the context of a larger, persistent games based on the behaviors that a player already favors. Other forms of motivational content may exist. Regardless, a system must use longitudinal player data to determine what content to create, when to create it, and what value the content will provide to different players.

Drama managers are disembodied virtual agents that monitor virtual worlds and intervene to drive a narrative forward based on models of player experience quality [4]. Drama managers implemented in Left 4 Dead and Darkspore have demonstrated the ability to increase game replay value by varying game content and modulating player intensity levels. Drama managers that procedurally generate narratives have been demonstrated in the context of short-term experiences. Extending these systems to motivate persistent game players through narratives requires further work to personalize narratives to players through data [10], create a potentially infinite variety of narratives or quests [25], and chain narratives to create a persistent, coherent sense of progression in an evergrowing game story.

B. Ecosystem of Games
Digital distribution and online game portals have led to increasingly diverse games and a growing long-tail of smaller games appealing to niche interests. Ecosystems of games push entertainment goals to players' experiences across a number of potentially unrelated games, rather than within a single game. Connecting characters and gameplay from a single player across multiple games opens new opportunities in cross-game agents, cross-game content generation, and automated online game designs experiments. Previous player modeling and content generation research can play new roles when players interact with many distinct games, requiring advances in representing, collecting, and reasoning on game design knowledge.

4) How can AI agents interact with players across games?
Starting from the premise that lifelong virtual agents learn how to interact with individual players, we speculate that it may be advantageous for characters to interact with their players across many games in a company's ecosystem. To some degree, the solution involves designing for cross-game characters that manifest as recurring characters or playmates who join players across many games. However, as a lifelong agent adapts to an individual player, the question becomes one of how an agent with a constantly adapting personality, memory, and history of interactions with a player can manifest its individualized nature in the context of new games.

Cross-game agents can draw from research on competitive cross-game AI and work on socially present agents. The general game playing competition has spawned research on representing and reasoning on generic game state and rule systems to enable AI agents to play games of generic specification [26]. Cross-game agents must also consider how the personality and memory of the agent translates into new game contexts. To that end, research into socially present agents has explored how to control in-game behavior in reference to out-of-game context. Techniques explored include referring to historical interactions with players, simulating social roles common in a game, and explicitly signaling social and affective states not immediately relevant to in-game behavior [27]. Linking cross-game data on player-agent interactions to how social actions affect players will be crucial.

Cross-game agents may also serve as “game universe guides,” acting as a curator or tour guide to ensure players get the most out of a space of possible games. Important challenges involve understanding and eliciting player feedback on games, guiding players to new games as their interests shift, and sequencing the order of games played to optimize player experience. Cross-game data will be the linchpin to recommending different games and modeling how playing games in different orders (and ways) affects player experience.

5) How can a Game AI system generate cross-game content?
To date procedural content generation systems have been developed on a game-by-game basis. Cross-game content generation and adaptation explores how data acquired about a player in one game can improve content generation in another game in the ecosystem. Cross-game data enables mining game designs for general design knowledge-a model mapping game mechanics to player behavior. Taken to its extreme, such design knowledge can lead to generating new games that span genres, moving beyond the current genre-focused efforts. Understanding cross-game player behavior can also allow recommending content from other games and ultimately lead to pre-adapting game experiences to players based on their behavior in other games.

6) How can a Game AI system add to the game ecosystem?
If AI as Producer focuses on an ecosystem of computer games, we might ask whether intelligent systems can automatically create new games that add to the ecosystem in meaningful ways. Computer game generation is a nascent area of Game AI research [28], [29], [30], [31]. Work to date has focused on a single genre at a time. Cross-game play puts a premium on new research to represent more generic game structures in a way that spans multiple genres. Many of the existing formalisms may be able to support such extensions, but how to best bridge genres remains an open question. More ambitious work will ultimately aspire to AI Producers that generate sets of games that complement one another, creating game ecosystems using a wealth of accumulated design knowledge.

Despite substantial efforts to reason on design knowledge, relatively little work has explored ways to acquire or refine general design knowledge. Current game industry efforts employ A/B testing for online game design-exemplified by Zynga's practices. Analogous experimental methods have been used to understand how game designs impact player engagement and learning [32] or negative behavior [33]. Leveraging the unique potential of online distribution methods for automated rapid iteration and experimentation can open the way to addressing the challenges of extracting cross-game design knowledge. However, a number of research questions must first be addressed. First, an automated system must choose which designs to test, balancing benefits of testing against the high costs (in terms of player time, money, or negative reactions) of automated crowdsourced testing. Second, an automated system must be able to interpret the results of testing, including attributing credit/blame to aspects of design choices and appropriately changing designs in response. Third, an automated system must ultimately devise new design goals to explore when feedback indicates a given goal is unfeasible or unvaluable. Additionally, if user-generated content is available, a system might mine design principles from the content as a means of bootstrapping learning PCG systems.

C. Community in Games
Game communities demand greater attention to how players impact one another's experiences within a game, beyond the dynamics of cooperation and competition limited to a single session or round of play. Player communities extend in-game activities to a broader out-of-game social content. Careful matchmaking can group players for entertainment and engagement while group-oriented content generation can provide experiences tailored to sets of players. Further, communities themselves yield a wealth of user-generated content, posing opportunities to enhance interactions with PCG systems for better user- and system-generated content. However, with community comes a dark side: fraud, security violations, cheating, and abusive behavior. Game AI has an unique opportunity to enhance negative behavior detection and automate responses beyond merely banning players.

7) How can Game AI systems improve matchmaking and group content?
Matchmaking has traditionally focused on pairing players for competition to ensure even win rates. Online and social games, however, require grouping players for cooperative or synergistic goals. Beyond balancing win rates, players can be grouped to have complementary abilities or for purposes such as mentoring. Developing techniques to model player value as a social partner and using that information to create groups stand as key research challenges. Future developments will likely require adopting more sophisticated techniques from the social matching literature [34].

Group-oriented content generation extends PCG to the setting of engaging multiple players-with potentially conflicting interests-at once. Algorithms that balance the diverse needs of players in a given group, live game constraints (e.g. in terms of available players), and meet design goals are relevant research vectors. Both matchmaking and grouporiented content generation will require advances in modeling how players relate to one another socially and how these social and personal attributes interact with game content.

8) How can a Game AI system reduce negative behavior?
Negative player behaviors in online games include fraud, security violations, cheating, and abusive behaviors. Fraudulent behavior commonly involves counterfeit game items sold for real-world money. Security violations compromise games through stealing players' accounts or financial data. Cheating spans hacking game systems to change their functionality to exploiting game bugs for personal gain or harm to others. Abusive behaviors include insulting other players or intentionally attempting to ruin their game experiences. Across these issues are a broad set of Game AI research challenges associated with responding appropriately to these behaviors. Note that many of these behaviors implicitly require cross-game agents-many forms of negative behavior manifest at the level of human players who act both within specific games and on game forums or other out-of-game socialization venues.

There is a wealth of research on fraud detection, security violations, and related challenges [35]. Little work, however, has addressed how Game AI agents should respond to these activities once detected. Game companies have only recently begun to systematically investigate ways to reduce negative behavior beyond banning players (e.g. [33]). We hypothesize that negative behavior can be reduced or mitigated by automatically adapting game content, rules, and mechanics to incentivize players toward pro-social behaviors. Game AI agents may minimize a player's negative impact by redirecting their actions away from others players. Detection may be improved by intentionally eliciting fraudulent or cheating behavior through an AI double-agent. Beyond negative reinforcement or punishment, Game AI agents can reward positive behavior or channel players toward more constructive pursuits.

9) How can a Game AI system induce user-generated content?
User-generated content has become a major force for players' continuing interest in aging games. Minecraft, Spore, Second Life, and a host of other games have demonstrated the power of end-users to continually extend and enhance a live game. Despite the growing ubiquity of user-generated content little research has developed methods to elicit needed content, interact with end-users to improve content, or mine design knowledge from this content.

Open questions for future user-generated content systems relate to how to best incorporate user content and feedback to improve the content created. Existing research has examined providing preference information (e.g. [36], [37]) or directly authoring the space of content (e.g. [29], [38], [39]). Enhancing user-generated content will require enabling users to teach PCG systems in new ways. Interactive machine learning has been developing techniques that optimize human abilities to train learning algorithms [40]. Integrating interactive machine learning approaches with user content creation interfaces can enable a new wave of learning PCG systems that improve through interaction with a player community. Key research problems include devising means to gather new modes of feedback, incorporating that feedback into PCG systems, and aggregating feedback from a diverse community of players.

D. Coupling the Real and Virtual Worlds
Games are increasingly coupled to the real world through input and output modalities that put a greater premium on a player's context. New input devices provide novel sensor information including GPS location (mobile devices), room layouts (Microsoft Kinect), motion data (Nintendo Wiimote, Playstation Move), sound (Kinect, webcams), and brain activity (NeuroSky, Emotiv). Output modalities have similarly become richer, including 3D displays (3D TV, Nintendo 3DS), second-screen experiences (Nintendo WiiU), virtual reality (Oculus Rift), augmented reality (mobile devices of all sorts), and potentially projection technologies. These technologies introduce nuances of player physical context including location, bodily motion data, and various physiological indicators. Beyond this information, games must also account for other aspects of player context including social circumstances or economic situation. Together this additional contextual information and output opportunities open research avenues related to incorporating real-world context into games, using out-of-game social network data in games and using games to proactively sense real-world information.

10 ) How can a Game AI system utilize real-world context within a game?
Real-world data opens many avenues for game experiences that overlay on real-world settings or leverage real-world semantic information for new forms of gameplay. Human-computer interaction research has a rich literature on addressing the challenges and nuances of context, but has seen little adoption in the context of Game AI [41]. Understanding how AI agents can (or should) use context is an open problem for future Game AI agents.

Real-world settings enable new game types including augmented reality games and alternate reality games. Augmented reality games visually project virtual game content onto the real-world situation. Alternate reality games overlay fictional contexts on real-world settings without necessarily requiring a visual overlay-Google's Ingress is a prominent example. Both kinds of games, however, are currently circumscribed by challenges in authoring new content and fitting it to new, unanticipated real world contexts-a promising avenue for future PCG research. Preliminary work on merging the virtual and real has explored dynamically adapting alternate reality game quests to new physical locations [42], [43]. The next stages of this work may use add other real world context in the game, such as social or economic context.

An alternative perspective on coupling real and virtual worlds involves using the semantic information present in real-world data. Data Games engage players in understanding semantic information in open data sets (e.g. US census data) through automatically mapping data into game content [44]. Future work should explore how in-game agents can leverage out-of-game data for richer interaction with players and tighter coupling of game worlds and real-world contexts.

11) How can a Game AI system leverage out-of-game social networks?
Modeling in-game social interactions, social networks, and player communities can enable Game AI agents to interact with players as part of a social world that (partially) overlaps the in-game world. Current research on online game social networks has explored detecting social information including group identities [45], shared housing networks [46], and real-money trade [47]. Outside of games a wealth of prior work on modeling techniques for social and economic networks exists [48]. Here we ask: how can Game AI systems use social networks from outside a game to improve in-game experiences?

Overall, relating in-game interactions to out-of-game social interaction remains underexplored. Open research problems include uncovering how network structures can be used to draw users into a company's game ecosystem or how to encourage users to move among games. Out-of-game social networks such as Twitter and Facebook additionally contain a wealth of user-generated material that reveals sentiments, preferences, attitudes, and non-game product usage. We speculate this data could be used to influence motivational content generation and integrate ads into game content in non-trivial ways.

12) Can an intelligent system use games to “proactively sense” the real world?
In the age of big data, companies routinely collect data about users to improving products or otherwise monetizing user behavior. Yet, many aspects of player motivations or real-world context remain hidden from these efforts. Proactive sensing is the use of game content to elicit data about the real-world that might not otherwise be collected by passive observation of users. For example, consider attempting to learn whether a new coffee shop has good coffee. Game content-possibly a quest in an alternate reality game-can be generated that requires players within proximity of the coffee shop to visit and rate the shop, thus generating new data that might not otherwise have been generated. Additional applications of the proactive sensing paradigm can target other hidden real-world information such as player preferences, skills, and attitudes or semantic information about real-world objects and locations.

Proactive sensing research will require modeling the quality of information known about the world player abilities to provide that information, and challenges in generating game-relevant content and agent behaviors to elicit this information. Human computation research has modeled human abilities to perform tasks that provide computers with information about the real world [49]. Extending these approaches to games will involves incorporating player motivation to perform or complete game tasks. Key research problems will include balancing between game design goals and information needs and generating game tasks and agent behaviors appropriate to a wide variety of information needs.

SECTION V.Conclusions
The digital game industry is experiencing a shift to persistent games, business models emphasizing game ecosystems, more support for game communities, and new considerations for incorporating real world context into game worlds. We see these advances as positive signs of growth and maturity in the game industry. We also see these advances pushing the bounds on the scalability of game development practices.

Artificial intelligence, computational intelligence, and machine learning have always excelled at addressing problems of scalability by automating tasks and dynamically adapting system behavior. Game AI has always been an integral part of computer game development. AI Actors have enhanced player experiences by supporting players' suspension of disbelief and dynamically managing dramatic contexts. AI Designers have supported and augmented the development of individual games through procedural content generation. We envision AI Producers taking on a new role of augmenting and scaling the game production pipeline, supporting the entire span of live operations in games, enhancing cross-game interoperations, nurturing strong player communities, and coupling real and virtual contexts. This vision is bolstered by the massive amounts of data being generated and collected by the game development industry.

The twelve new Game AI research questions here require game developers to see Game AI as part of live game production. AI for game production does not supplant previous challenges in Game AI-it extends the scope of the field of Game AI as a whole. Addressing these problems with credible AI solutions will result in immediate relevance to game developers and may present a new vector for industry game development and academic Game AI research collaboration."
Did the above paper explain why AI and machine learning are critical to AR gaming ?