Deceptive Opponent Modeling with Proactive Network Interdiction for Stochastic Goal Recognition Control
Authors: Junren Luo
Wei Gao
Zhiyong Liao
Weilin Yuan
Wanpeng Zhang
Shaofei Chen
Venue:  (2020)

Abstract: Goal recognition based on the observations of the behaviors collected online has been used to model some potential applications. Newly formulated problem of goal recognition design aims at facilitating the online goal recognition process by performing offline redesign of the underlying environment with hard action removal.
In this paper, we propose the stochastic goal recognition control (S-GRC) problem with two main stages: (1) deceptive opponent modeling based on maximum entropy regularized Markov decision processes (MDPs) and (2) goal recognition control under proactively static interdiction.
For the purpose of evaluation, we propose to use the worst case distinctiveness (wcd) as a measure of the non-distinctive path without revealing the true goals, the task of S-GRC is to interdict a set of actions that improve or reduce the wcd.
We empirically demonstrate that our proposed approach control the goal recognition process based on opponent's deceptive behavior.

Under review as a conference paper at ICLR 2020
DECEPTIVE OPPONENT MODELING WITH PROACTIVE NETWORK INTERDICTION FOR STOCHASTIC GOAL RECOGNITION CONTROL
Anonymous authors Paper under double-bile blind review
ABSTRACT
Goal recognition based the observations of the behaviors collected online has been used to model some potential applications. Newly formulated problem of goal recognition design aims at facilitating the online goal recognition process by performing offline redesign of the underlying environment with hard action re- moval. In this paper, we propose the stochastic goal recognition control (S (S-GRC) problem with two main stages: (1) deceptive opponent modeling based on maxi- mum entropy regularized Markov decision processes (MDPs) and (2) goal recog- nition control under proactively static interdiction. For the purpose of evaluation, we propose to use the worst case distinctiveness (wed) as a measure of the non- distinctive path without revealing the true goals, the task of S-GRC is to interdict a set of actions that improve or reduce the wed. We empirically demonstrate that our proposed approach control the goal recognition process based opponent's deceptive behavior.
INTRODUCTION
Goal recognition (GR), also called intention recognition, is the task of inferring the goals of an agent according to the observed actions or states collected online (Sadri) (011I) which enables humans or agents to make proactive response plans. Goal recognition design (GR)) is to deliberately redesign the environment for improved online goal recognition ability (Keren et al. 2014), which includes two key models, the first measures how efficiently and effectively the online goal recognition system performs in a given setting, the second optimizes the goal recognition setting via redesign (Keren eet al. 2019). Goal recognition control (GRC) aims at soft action interdiction under bounded re- source by adding cost compared to the hard action removal of GRD (Luo et al., 2019). in which the interdiction be proactively static inhibition online dynamic block by allocating security resources to protect goals against the attacker.
In fact, under complex environment (both adversarial and cooperative), the information of op- ponent's goals are asymmetric and can not be obtained through communication (Le Guillarme) 2016). Most recent advancements in GR utilize planning recognition as planning (PRAP) (Ramírez & Geffner 2009), generative game-theoreti, frameworks (Ang et al. 2017). and goal recognition as planning (GRAP) (Pereira et al. 2019). There three key assumptions widely used: (1) agents performs optimal plans to real the goals; (2) the environment is fully observable, that is the states and actions of the agents are observable; and (3) the agent's actions are deterministic. However, these existing frameworks seldom address the deceptive behaviors of actively misleading the goal recognition process. We will relax the first and third assumptions to handle non-optimal agents and stochastic actions. Such as came-theoretic approach provided with unified treatment of both threat assessment and response planning in (Guillame et al., 2017), after evaluating the threat, the defender will allocate road barrier patrolling force to protect critical infrastructure, illustrated in Figure T
In order to identify the goals and improve security. employ opponent modeling deal with the non-stationary strategies stemming from deception, in which maximum entropy regularized Markov decision process (MDP) is utilized to shape the multi-modal adversarial strategies (Shen & How

---

Under review as a conference paper at ICLR 2020
G2
Opportunity
Intention
Capability
Figure 1: Illustration of critical infrastructure protection (CIP) (Guillarme al. 2017): the defender can perform proactive road network interdiction to accelerate the opponent's 's goal recognition even with possible deceptive behaviors.
2019). After finding the maximal non-distinctive agent path, we are seeking some optimal modifi- cations (e.g., interdiction block or inhibit action) to accelerate the online goal recognition.
In this work, we propose the stochastic goal recognition design (S-GRC) problem with deceptive opponent modeling and proactive network interdiction: the opponent's deceptive policy modeled as one soft multi-criteria decision policy with one tunable parameter to balance goal achievement and deception preference; and the primary objective is offline redesigning the environment by soft action interdiction to control online GR.
To validate the model, we evaluate our approach in two different environment representations: ran- dom generated connected graph and real road network. Our experiments demonstrate that (1) de- ceptive opponent modeling with entropy regularization make robust to multi-modal stochastic be- haviors; (2) soft decision policy based stochastic goal recognition design model for action removal bridge the gap between observation and decision making.
RELATED WORK
2.1 GOAL RECOGNITION AND DESIGN
Goal Recognition (GR): The ability of inferring the goals of others can assist us to reason about what they are doing, why they doing and what they will do next (Sukthankar al. 2014). The recognition setting can be generally be divided into keyhole, where the agent is unaware of being observed and recognized as if the agent is looking through keyhole; adversarial, where the agent actively hostile to the observations of the actions and the inference the goals; and intended, where the agent wants to convey the goal be understood and helps the recognizer detect the objective (Keren et al. 2019). In this work, we focus on the adversarial setting, where the agent would hide the goal and attempt to thwart the recognition process by deception. GR is strongly related to the topic of
and explainable planning (Keren et al. 2016 Chakraborti
al. 2019 Kulkarni et a..).
Goal Recognition Design (GRD): The GRD problems contains two models: one goal recogni- tion setting analyzed environments, acting agent (attacker), and recognition system (recognizer, ob- server); one design model of possible ways to change the environment (Keren et al.] 2019). The environment will induce a set of possible behaviors for the attacker, GRD can divided into deterministic GRD (erren al. 2019), stochastic GRD (S-GRD) (Wayllace et al. 2018 Wayl- lace 2019), GRD for Agents with partial knowledge (GRD-APK) (Sarah et al., 2019). and game- theoretic GRD (Ang al. 2017 Masters & Sardinal 2017). Also, many design measures have been employed, such as action removal (AR), action sensor refinement (SAR), and action conditioning
(AC) (Keren al. 2018
Wayllace al. 2018
2

---

Under review as a conference paper at ICLR 2020
2.2 OPPONENT MODELING AND NETWORK INTERDICTION
Deceptive Opponent Modeling: Opponent modeling has been employed to reason the
stationary strategies of autonomous agents (Albrecht & Stone, 2018 Papoudakis et al., 2019)) De- ception involves combination of hiding the truth and showing the false (Masters Sardina 2017). which articulated three distinct strategies in each case: dissimulation (masking, repackaging, and dazzling), simulation (mimicking, inventing, and decoying) (Ettinger & Jehiel, 2010). As for de-
ceptive path planning, game-theoretic method (Root, 2005) and model based (Masters & Sardina 2017) method have been investigated.
Network Interdiction: In recent researches, interdiction has been used to counter the attacker's 's be- havior, MDP interdiction (Panda & Vorobeychik] (017). plan interdiction /Vorobeychik & Pritchard, 2018), and network interdiction (Xu et al.) show one promising research thread of Observe-Orient Decide-Act (OODA).
2.3 REGULARIZED MDPS
Most sequential decision making problems can be described as MDP based reinforcement learning problems or stochastic game problems. Regularization for MDPs can be divided into temporal regularization problems in temporal space (Thodoroff et al. 2018), spatial regularization in feature or state space (Farahmand) 2011; Harrigan), and entropy or information regularization in policy space (Neu et al.] [017 Geist et al. 2019 Belousov & Peters 2019). Regularized reinforcement learning focuses on the smoothness and sparsity of the value function (Farahmand) 2011). acquiring of diverse robot skills (Haarnoja 2018), and sparsity and multi multi-modality of the optimal policy (Lee)
jet al. 2019). Regularized stochastic game employees various entropy constraints to balance or
control behavior (Grau-Moya al. 2018 Savas et al. 2019 Ahmadi et al. 2018 Tian al. 2019).
STOCHASTIC GOAL RECOGNITION CONTROL
The goal recognition control (GRC) problems try to balance the intended and adversarial goal recog- nition process under network interdiction (Luo et al., 2019). which extends to assume that the agent choose to reveal obfuscate (Kulkarni et al. 2019b ab. share or hide (Strouse al. (018) the goals. In this paper, we focus on stochastic GRC (S-GRC) under adversarial, cooperative, and non-stationary environment. The framework of our proposed approach is illustrated in Figure
We embed deceptive opponent modeling and proactive network interdiction into the OODA cycle as two modules, which could bridge the gap between observation and pecision-making The deceptive opponent modeling module will shape the multi-modal opponent, the proactive network interdiction module will provide bounded resource allocation for defense. So, GRC will identify the goals with topological soft q learning and provide proactive interdiction resource allocation.
3.1 SSP-MDP FOR S-GRC
Different from the GRD problem, we employ more applicable and soft measure of interdiction with bounded resource to control the goal recognition process. With this motivation in mind, we use the Stochastic Shortest-Path MDP (SSP-MDP) (Kolobov & Kolobov 2012) model, which is widely used for uncertainty given an initial state, a set of goal states, actions with probabilistic outcomes and an action cost function. SSP-MDP is represented set of states S, initial state 8p. set of actions A, a transition function T S x A x S 00, of state transition from 8 to s' after executing the action a, and cost function C
tuple (S, su. A, T, C, G). which consists a
A x S
that gives the cost of executing action a, and set of terminal goals G C S. With cost function to replace the reward function, the objective is to find a optimal policy π so as to minimize the smallest expected cost or travel time of the policy, which forms the map from states to actions.
So, the optimal policy of the SSP-MDP can be find by using the Value Iteration (VI) algorithm, which use value function V to represent expected costs. The expected cost of an optimal policy T for the starting state sg E S is the expected cost ((n)), and the expected cost for all states s can be calculated by the Bellman equation (Kolobov & Kolobov 2012
3

---

Under review as a conference paper at ICLR 2020
Goal Recognition Control
Deceptive Opponent Modeling
Proactive Network Interdiction)
Orient
Decide
Sensors
Actuator
Observe
Enviromment
Advencrtial
Cooperative
stationary
Figure 2: The OODA cycle embedded with stochastic goal recognition control (S-GRC): one de- ceptive opponent modeling module and one goal recognition control module.
V(s) min Σ T( a
S'ES
A previous work on stochastic shortest path network interdiction modeled the problem as one se- quential bi-level attacker-defender game, where the defender moves first by deploying sensors to the ares in the network maximizing the expected shortest traveling time of the adversary (Zhang al. (0188). Here we formulate the stochastic goal recognition control (S-GRC) problem as an ex- tension of the GRD problem to allows non-optimal agents and stochastic actions, which means the agent's action are not deterministic and the possible successor states are with probability, the mea- sures we can take are interdictions with cost. The bounded interdiction resources allocations can better model the real world applications. A S-GRC problem is represented as a tuple / = (D,G). be represented as SSP-MDPs. Similar to
the domain information D (S Su. A, T, C, G) the S-GRD WWyylaae et al. 2018), we employ a metric to assess the largest number of actions an agent can take the largest P = (D, G), the worst case distinctiveness (wed) of the problem can be defined as:
agent will ineur before revealing the goal. Given problem
wed (P) max Va (So)
#ell
So the objective of S-GRC under network interdiction can be reformulated one optimization problem of finding a subset of actions AA" x C A:
AA argminwed(() P))
AACA
s.t. VA (sp) = VaS (sp)
Ag E G
(3)
(1AA k
where P (D,G) the problem with domain ò ss so. A.T,C, À is the redesigned action set with AA* interdicted, I is the corresponding cost to the interdiction resource, the total interdiction resource allocation is bounded by At
3.2 AUGMENTED SSP-MDP WITH SCCS
With the metric wed, we can measure the maximal number of actions an agent can take before its goal is revealed. However, the formal wed related metrics for S-GRDs are inconsistent with the intuitive definition, the wed computation is not Markovian owing to the set possible goals depend the observed trajectory to state. To deal with the uncertain inconsistence, wedaag and
4

---

Under review as a conference paper at ICLR 2020
ECD together with topologiaal value iteration are employed to model the problem, in which the augmented MDPs with strongly connected components do not have loops (Wayllace et al.).
The state space of an MDP can be represented shown in Figure 3(a) where the states are denoted by nodes, the actions are denoted by edges, the transitions related to the arrow. The wed may be infinite when the state space contains some cycle loop, here we can use the Tarjan algorithm [Tajaan 1972 Hou et al. 2014) to generate strongly connected components (SCCs), which form directed acyclic graph (DAG) as shown in Figure (bb)
directed connectivity graph. One example MDP
14
Bq
%0
KJ1621
(3)
R1
1/22
(a)
(b)
Figure 3: Augmented MDP with strong connected components (SCCs).
Augmented SSP-MDP for S-GRC: Given a SSP-MDP, an augmented
SSP-MDP
($, So. T,T.C,G augments each component of the tuple:
Each state å E S is represented by (s, G').
The augmented initial state is 5g = (so: G).
Each augmented action à E Å is tuple (a, G'), where aA and G' is the set of all goals for which that action is an optimal action.
The new transition function T $ À $ -> [0,1] gives the probability (8, a, a)),, where z = (s, G'), â = (a, G'), and G' n G > 1 and equals 0 otherwise.
(s', G' G""",,", (5) = T (s, a, 8'),
The cost function Č x + R+ gives the cost Ò (3, ā, 8¹) of executing action à in augmented state ă and arriving in st1. This cost equals the cost Ö' (5, 8¹) C(s, a, 8') under the same condition as above.
x
The augmented goal states G C $ are those augmented states (s, G') for which any exe- cution of an augmented action will transition an augmented state (s/, G with one goal or no goals.
3.3 DECEPTIVE OPPONENT MODELING
As the opponent may change the goals midway mislead the goal recognition process. We use binary variable À E (0, }} to characterize whether the opponent is a optimal or deceptive with decoy goals.
Depending on X, the opponent is expected to exhibit different behaviors. which is fully described by an opponent policy π" (" st). This model is restrictive since the action probability only depend on the current state. Nonetheless, we use this model only for policy learning, and use general history dependent opponent policy for the evaluation of the learned autonomous agent policy. Another implicit assumption of this model is that the opponent has full observability over the states. This assumption could be released by modeling the opponent as a POMDP agent.
Neutral Opponent: If the opponent is civilian, i.e. x cill (aflst) is available to model the opponent:
O, we assume simple reactive policy
n (ag St, À 0) m (a. (st)
(4)
5

---

Under review as a conference paper at ICLR 2020
Table 1:
Sample table title
PART
DESCRIPTION
Dendrite
Input terminal
Axon
Output terminal
Soma
Cell body (contains cell nucleus)
Deceptive Opponent: We use the following equation to model adversarial agent's policy n":
T"(a]|sı,>= 1; 3) =argmin ECA [ am x (KL( TT (+|s; 0)))
(5)
7ta
MDP
(a À 1) e (())))))) (s(),,
(6)
where KL(-I-) denotes the Kullback-Leibler divergence between two distributions, The goal- achieving policy Ta MDP is associated with the optimal Q function Q(s,, ai), of a goal-achieving adversary MDP defined later. The temperature parameter in (6 represents the level of rational- ity of the adversary. The other parameter 3 indicates the level of deception. ((s;) the partition function that normalizes r.MMPP
Soft Q Learning: We use soft-Q Q learning (Haarnoja 2018) to learn a stochastic belief space policy. The soft Q learning objective is maximize the expected reward regularized by the entropy of the policy,
Σ Ba [r(b, 8t, az) a (T((be,
The parameter a controls the 'softness' the policy. The nice interpretation of this objective func- tion is maximizing accumulative reward while behaving as uncertain as possible, which is desired property against an adversary. This maximum entropy problem is solved using soft Q iteration. For discrete action space, the fixed point iteration:
Qbo St, St, ae) Ti +
P. ?
Von (brn. 8t-1]]
(8)
(be, st) +- a log Σ exp(
Quan (be, St, a)),
8EA
converges to the optimal soft value functions Q and can be obtained from:
HHarnoja 2018), and the optimal policy
soft
Maxtan (at (b,, 9x) expl
( (Doon(be 8t, ac) Vaon (be, 8c))).
(10)
3.4
PROACTIVE NETWORK INTERDICTION
Proactive network interdiction
EXPERIMENTS
4.1
GRID WORLD
4.2
NETWORK GRAPH
4.3
RESULTS
CONCLUSION AND FUTURE WORK
Lisk-Sensitive MDPs (Hou al. 2014) illustrate applicable scenario where the cost function is constrained with one threshold or deadline.
See Table

---

Under review as a conference paper at ICLR 2020
ACKNOWLEDGMENTS
This work partially sponsored by the National Natural Science Foundation of China under Grants No. 61702528, No. 61806212.
REFERENCES
Mohamadreza Ahmadi, Suda Bharadwaj, Takashi Tanaka, and Ufuk Topeu. Stochastic games with sensing costs. In 2018 56th Annual Allerton Conference on Communication, Control, and Com- puting (Allerton), pp. 275-288 IEEE, 2018.
Stefano V Albrecht and Peter Stone. Autonomous agents modelling other agents: A comprehensive survey and open problems. Artificial Intelligence, 258:66 95, 2018.
Samuel Ang, Hau Chan, Albert Xin Jiang, and William Yeoh. Game-theoretic goal recognition models with applications to security domains. In International Conference on Decision and Game Theory for Security, pp. 256-272. Springer, 2017.
Boris Belousov and Jan Peters. Entropic regularization of markov decision processes. Entropy, (7):674, 2019.
Tathagata Chakraborti, Anagha Kulkarni, Sarath Sreedharan, David E Smith, and Subbarao Kamb- hampati. Explicability? legibility? predictability? transparency? privacy? security? the emerging landscape of interpretable agent behavior. In Proceedingy of the International Conference on Automated Planning and Scheduling, volume 29, pp. 86-96, 2019.
David Ettinger and Philippe Jehiel. A theory of deception. American Economic Journal: Microeco- nomics, 2(1):1-20, 2010.
Amir-massoud Farahmand. Regularization in Reinforcement Learning. PhD thesis, University of Alberta, 2011.
Matthieu Geist, Bruno Scherrer, and Olivier Pietquin. A theory of regularized markov decision
processes. In International Conference on Machine Learning, pp. 2160-2169, 2019.
Jordi Grau-Moya, Felix Leibfried, and Haitham Bou-Ammar. Balancing two-player stochastic games with soft q-learning. arXiv preprint /rXiv::802..8:: 2018.
Nicolas Le Guillarme, Abdel Illah Mouaddib, Sylvain Gatepaille, and Amandine Bellenger. Ad- versarial intention recognition as inverse game-theone theoretic planning for threat assessment. In JEEE International Conference Tools with Artificial Intelligence. 2017.
Tuomas Haarnoja. Acquiring Diverse Robot Skills via Maximum Entropy Deep Reinforcement Learning. PhD thesis, UC Berkeley, 2018
Cosmo Harrigan. Deep reinforcement learning with regularized convolutional neural fitted q itera- tion. differences, 14:1.
Ping Hou, William Yeoh, and Pradeep Varakantham. Revisiting risk-sensitive mdps: New al- gorithms and results. In Twenty-Fourth International Conference on Automated Planning and Scheduling, 2014.
Sarah Keren, Avigdor Gal, and Erez Karpas. Goal recognition design. In Twenty-Fourts Interna- lional Conference on Automated Planning and Scheduling, 2014.
Sarah Keren, Avigdor Gal, and Erez Karpas. Privacy preserving plans in partially observable envi- ronments. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelli- gence, pp. 3170-3176 AAAI Press, 2016.
Sarah Keren, Avigdor Gal, and Erez Karpas. Strong stubborn sets for efficient goal recognition design. In (wenty-Eighth International Conference on Automated Planning and Scheduling, 2018.
Sarah Keren, Avigdor Gal, and Erez Karpas. Goal recognition design in deterministic environments. Journal of Artificial Intelligence Research, 65:209-269, 2019.

---

Under review as a conference paper at ICLR 2020
Mausam Kolobov and Andrey Kolobov. Planning with Markov decision processes. An Al perspec- tive. Morgan & Claypool Publishers, 2012.
Anagha Kulkarni, Sarath Sreedharan, Sarah Keren, Tathagata Chakraborti, David E Smith, and Sub- barao Kambhampati. Design for interpretability.
Anagha Kulkarni, Siddharth Srivastava, and Subbarao Kambhampati. Balancing goal obfusca- tion and goal legibility in settings with cooperative and adversarial observers. arXiv preprint arXiv:1905. 10672, 2019a.
Anagha Kulkarni, Siddharth Srivastava, and Subbarao Kambhampati. A unified framework for planning in adversarial and cooperative environments. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 2479-2487, 2019b.
Nicolas Le Guillarme. A Game-Theorete Planning Framework for Intentional Threat Assessment. PhD thesis, Thèse de doctorat, Université de Caen, 2016.
Kyungjae Lee, Sungyub Kim, Sungbin Lim, Sungjoon Choi, and Songhwai Oh. Tsallis reinforce- ment learning: A unified framework for maximum entropy reinforcement learning. arXiv preprint arXiv: 2019.
Junren Luo, Xiang Ji, Wei Gao, Wanpeng Zhang, and Shaofei Chen. Goal recognition control under network interdiction using a privacy information metric. Symmetry, 11(8):1059, 2019.
Peta Masters and Sebastian Sardina. Deceptive path path-planning. In IJCAI, pp. 4368 4375, 2017.
Gergely Neu, Anders Jonsson, and Vicenç Gómez. A unified view of rntropy-regulariize markov decision processes. arXiv preprint arXiv:1705.07730 2017.
Swetasudha Panda and Yevgeniy Vorobeychik. Near-optimal interdiction of factored mdps. In Conference on Uncertainty in Artificial Intelligence, 2017.
Georgios Papoudakis, Filippos Christianos, Arrasy Rahman, and Stefano Albrecht. Dealing with non-stationarity in multi-agent deep reinforcement learning. arXiv preprint arXiv:1906.04737,
2019.
Ramon Fraga Pereira, Nir Oren, and Felipe Meneguzzi. Landmark-based approaches for goal recog- nition as planning, arXiv preprint arXiv:1904.17739 2019.
Miquel Ramírez and Hector Geffner. Plan recognition as planning. In Twenty-First International Joint Conference on Artificial Intelligence, 2009.
PhilipJ Root. Collaborative UAV path planning with deceptive strategies. PhD thesis, Massachusetts Institute of Technology, 2005.
Fariba Sadri. Logic-based approaches to intention recognition. In Handbook of Research on Ambient Intelligence and Smart Environments: Trends and Perspectives, pp. 346-375. IGI Global, 2011.
Keren Sarah, Xu Haifeng, Kwapong Kofi, Parkes David, and Grosz Barbara. Information shaping for enhanced goal recognition of partially-informed agents. In Heuristics and Search for Domain- Independent Planning Workshop of ICAPS, 2019.
Yagiz Savas, Mohamadreza Ahmadi, Takashi Tanaka, and Ufuk Topcu. rntropy-reguulrizzed stochas- tic games. arXiv preprint 19Xiv:/907.1133 2019.
Macheng Shen and Jonathan How. Active perception in adversarial scenarios using maximum entropy deep reinforcement learning. arXiv preprint arXiv:1902.05:: 2019.
DJ Strouse, Max Kleiman- Weiner,, Josh Tenenbaum, Matt Botvinick, and David Schwab. Learning to share and hide intentions using information regularization. In Advances in Neural Information Processing Systems, pp. (0249-10259, 2018.
Gita Sukthankar, Christopher Geib, Hung Hai Bui, David Pynadath, and Robert Goldman. Plan, activity, and intent recognition: Theory and practice. Newnes, 2014.
8

---

Under review as a conference paper at ICLR 2020
Robert Tarjan. Depth-firt first search and linear graph algorithms. SIAM journal on computing, 1(2): 146-160, 1972.
Pierre Thodoroff, Audrey Durand, Joelle Pineau, and Doina Precup. Temporal regularization for markov decision process. In Advances in Neural Information Processing Systems, pp. 1779-1789, 2018.
Zheng Tian, Ying Wen, Zhichen Gong, Faiz Punakkath, Shihao Zou, and Jun Wang. A regularized opponent model with maximum entropy objective. arXiv preprint arXiv:1905, 08087, 2019.
Yovgeniy Vorobeychik and Michael Pritchard. arXiv:1811.06162, 2018.
Plan interdiction games.
arXiv preprint
Christabel Wayllace. Stochastic goal recognition design. In Proceedings the AAAI Conference on Artificial Intelligence, volume 33, pp. 9904-9905, 2019.
Christabel Wayllace, Ping Hou, and William Yeoh. New metrics and algorithms for stochastic goal recognition design problems.
Christabel Wayllace, Sarah Keren, William Yeoh, Avigdor Gal, and Erez Karpas. Accounting for partial observability in stochastic goal recognition design: Messing with the marauders map. In Heuristics and Search for Domain- Innnain--ndeendentt Planning Workshop of ICAPS, 2018.
Kai Xu, Kaiming Xiao, Quanjun Yin. Yabing Zha, and Cheng Zhu. Bridging the gap between observation and decision making: Goal recognition and flexible resource allocation in dynamic network interdiction.
Jing Zhang, Jun Zhuang, and Brandon Behlendorf. Stochastic shortest path network interdiction with a case study arizona-mexico border. Reliability Engineering & System Safety, 179:62-73, 2018.


Reviews:
[{"review_id": "HJeoRzyD9r", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper aims to provide a goal recognition framework. The paper reviews previous literature and outlines a model, but it seems to be at the draft stage as it stands as the experiment section is missing (section 4). Also, in some parts the sentences are broken and hard to follow, for example, “Here we formulate the stochastic goal recognition control (S-GRC) problem as an extension of the GRD problem to allows non-optimal agents and stochastic actions, which means the agent’s action are not deterministic and the possible successor states are with probability, the measures we can take are interdictions with cost.”\nBased on the above comments I think the paper is not ready for publication.\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "N/A", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "BklzD_02Yr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to use reinforcement learning to model an agent that is reaching goal that it is intended to reach. The authors consider the case where the agent is (1) indifferent to being observed, (2) trying to help an observer reach its goal, and (3) trying to fool the observer into not reach its goal. The paper propose to use a value function to quantify how easy it is to predict where the agent is going (\"worst case distinctiveness\"). The authors propose to then train an agent to modify the action space to make it difficult for an agent to fool the observer.\n\nWhile this is an interesting idea, the paper is clearly incomplete. The experiment section is missing, and much of the method section is partly written.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "N/A", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJxVeg3kKS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies goal recognition control given a deceptive opponent, who selects actions to intentionally mislead or confusing the learner to learn the true goal. The problem has been studied in the security game, resource allocation game and Stackelberg game, where the defender is to play a resource allocation game given the best response of the attacker. in this paper, the authors use stochastic-shortest-path MDP to model the attacker's planning problem. The defender's objective is stated in eq (2) and (3) but not explained clearly.\n\nThe work considers a deceptive attacker whose objective is not to maximize the total reward/minimize the total cost, but rather to balance two objectives: minimizing the total cost while maximizing the KL divergence between its current policy and the optimal goal-achieving policy. This opponent model seems to be simplistic, given security game has studied the opponent using more rigorous equilibrium analysis and with dynamic Bayesian game formulation. \n\nsee Manshaei, Mohammad Hossein, et al. \"Game theory meets network security and privacy.\" ACM Computing Surveys (CSUR) 45.3 (2013): 25.\n\nThe practical application is well motivated.  The paper itself is incomplete and contains many typos. The paper is unfinished.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "N/A", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper has been withdrawn by the authors.
Decision: Reject