TASK DEFINITION: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
PROBLEM: Distributed constraint optimization (DCOP) problems are a popular way of formulating and solving agent-coordination problems. A DCOP problem is a problem where several agents coordinate their values such that the sum of the resulting constraint costs is minimal. It is often desirable to solve DCOP problems with memory-bounded and asynchronous algorithms. We introduce Branch-and-Bound ADOPT (BnB-ADOPT), a memory-bounded asynchronous DCOP search algorithm that uses the message-passing and communication framework of ADOPT (Modi, Shen, Tambe, & Yokoo, 2005), a well known memory-bounded asynchronous DCOP search algorithm, but changes the search strategy of ADOPT from best-first search to depth-first branch-and-bound search. Our experimental results show that BnB-ADOPT finds cost-minimal solutions up to one order of magnitude faster than ADOPT for a variety of large DCOP problems and is as fast as NCBB, a memory-bounded synchronous DCOP search algorithm, for most of these DCOP problems. Additionally, it is often desirable to find bounded-error solutions for DCOP problems within a reasonable amount of time since finding cost-minimal solutions is NP-hard. The existing bounded-error approximation mechanism allows users only to specify an absolute error bound on the solution cost but a relative error bound is often more intuitive. Thus, we present two new bounded-error approximation mechanisms that allow for relative error bounds and implement them on top of BnB-ADOPT.

SOLUTION: An Asynchronous Branch-and-Bound DCOP Algorithm

PROBLEM: The subpath planning problem is a branch of the path planning problem, which has widespread applications in automated manufacturing process as well as vehicle and robot navigation. This problem is to find the shortest path or tour subject for travelling a set of given subpaths. The current approaches for dealing with the subpath planning problem are all based on meta-heuristic approaches. It is well-known that meta-heuristic based approaches have several deficiencies. To address them, we propose a novel approximation algorithm in the O(n3) time complexity class, which guarantees to solve any subpath planning problem instance with the fixed ratio bound of 2. Beside the formal proofs of the claims, our empirical evaluation shows that our approximation method acts much better than a state-of-the-art method, both in result and execution time. Note to Practitioners—In some real world applications such as robot and vehicle navigation in structured and industrial environments as well as some of the manufacturing processes such as electronic printing and polishing, it is required for the agent to travel a set of predefined paths. Automating this process includes three steps: 1) capturing the environment of the actual problem and formulating it as a subpath planning problem; 2) solving subpath planning problem to find the near optimal path or tour; 3) command the robot to follow the output. The most challenging phase is the second one that this paper tries to tackle it. To design an effective automation for the aforementioned applications, it is essential to make use of methods with low computational cost but near optimal outputs in the second phase. According to the fact that the length of the final output has a direct effect on the cost of performing the task, it is desirable to incorporate methods with low complexity that can guarantee a bound for the difference between length of the optimal path and 1 ar X iv :1 60 3. 06 21 7v 1 [ cs .R O ] 2 0 M ar 2 01 6 the output. Current approaches for solving subpath planning problem are all meta-heuristic based. These methods do not provide such a bound. And plus, they are usually very time consuming. They may find promising results for some instances of problems, but there is no guarantee that they always exhibit such a good behaviour. In this paper, in order to avoid the issues of metaheuristics methods, we present an approximation algorithm, which provides an appropriate bound for the optimality of its solution. To gauge the performance of proposed methods, we conducted a set of experiments the results of which show that our proposed method finds shorter paths in less time in comparison with a state-of-the-art method.

SOLUTION: An Approximation Approach for Solving the Subpath Planning Problem

PROBLEM: In the wake of the vast population of smart device users worldwide, mobile health (mHealth) technologies are hopeful to generate positive and wide influence on people’s health. They are able to provide flexible, affordable and portable health guides to device users. Current online decision-making methods for mHealth assume that the users are completely heterogeneous. They share no information among users and learn a separate policy for each user. However, data for each user is very limited in size to support the separate online learning, leading to unstable policies that contain lots of variances. Besides, we find the truth that a user may be similar with some, but not all, users, and connected users tend to have similar behaviors. In this paper, we propose a network cohesion constrained (actor-critic) Reinforcement Learning (RL) method for mHealth. The goal is to explore how to share information among similar users to better convert the limited user information into sharper learned policies. To the best of our knowledge, this is the first online actor-critic RL for mHealth and first network cohesion constrained (actor-critic) RL method in all applications. The network cohesion is important to derive effective policies. We come up with a novel method to learn the network by using the warm start trajectory, which directly reflects the users’ property. The optimization of our model is difficult and very different from the general supervised learning due to the indirect observation of values. As a contribution, we propose two algorithms for the proposed online RLs. Apart from mHealth, the proposed methods can be easily applied or adapted to other health-related tasks. Extensive experiment results on the HeartSteps dataset demonstrates that in a variety of parameter settings, the proposed two methods obtain obvious improvements over the state-of-the-art methods.

SOLUTION:
Cohesion-based Online Actor-Critic Reinforcement Learning for mHealth Intervention