bluuebunny/crossref_metadata_2025 huggingface dataset:
Hypothalamic γ-melanocyte stimulating hormone gene delivery reduces fat mass in male mice
Authors: Eerola K , Virtanen S , Vähätalo L , Ailanen L , Cai M , Hruby V , Savontaus M , Savontaus E
Published: July 2018

γ-Melanocyte stimulating hormone (γ-MSH) is an endogenous agonist of the melanocortin 3-receptor (MC3R). Genetic disruption of MC3Rs increases adiposity and blunts responses to fasting, suggesting that increased MC3R signaling could be physiologically beneficial in the long term. Interestingly, several studies have concluded that activation of MC3Rs is orexigenic in the short term. Therefore, we aimed to examine the short- and long-term effects of γ-MSH in the hypothalamic arcuate nucleus (ARC) on energy homeostasis and hypothesized that the effect of MC3R agonism is dependent on the state of energy balance and nutrition. Lentiviral gene delivery was used to induce a continuous expression of γ-Msh only in the ARC of male C57Bl/6N mice. Parameters of body energy homeostasis were monitored as food was changed from chow (6 weeks) to Western diet (13 weeks) and back to chow (7 weeks). The γ-MSH treatment decreased the fat mass to lean mass ratio on chow, but the effect was attenuated on Western diet. After the switch back to chow, an enhanced loss in weight (−15% vs −6%) and fat mass (−37% vs −12%) and reduced cumulative food intake were observed in γ-MSH-treated animals. Fasting-induced feeding was increased on chow diet only; however, voluntary running wheel activity on Western diet was increased. The γ-MSH treatment also modulated the expression of key neuropeptides in the ARC favoring weight loss. We have shown that a chronic treatment intended to target ARC MC3Rs modulates energy balance in nutritional state-dependent manner. Enhancement of diet-induced weight loss could be beneficial in treatment of obesity.
IRF5 is increased in labouring myometrium and regulates pro-labour mediators
Authors: Lim Ratana , Barker Gillian , Lappas Martha
Published: July 2018

Preterm birth continues to be the leading cause of neonatal mortality and morbidities that can extend into adult life. Few treatment options stem from our incomplete understanding of the mechanisms of human labour and delivery. Activation of the inflammatory response in gestational tissues by inflammation and/or infection leads to the production of pro-inflammatory and pro-labour mediators, thus preterm birth. Interferon regulatory factor 5 (IRF5) has recently emerged as an important pro-inflammatory transcription factor involved in acute and chronic inflammation. The aims of this study were to determine the expression of IRF5 in human myometrium from labouring and non-labouring women, and whether IRF5 is involved in the genesis of pro-inflammatory and pro-labour mediators induced by pro-inflammatory cytokines or toll-like receptor (TLR) ligands. IRF5 mRNA and protein expression was significantly higher in human myometrium after spontaneous term labour, compared to non-labouring tissues. IRF5 mRNA expression was also significantly higher in primary myometrial cells treated with the pro-inflammatory cytokines IL1B or TNF. In primary myometrial cells, IRF5 knockdown by siRNA (siIRF5) was associated with significantly decreased expression and or secretion of pro-inflammatory cytokines (IL1A, IL6), chemokines (CXCL8, CCL2), adhesion molecules (ICAM1, VCAM1) and contraction-associated proteins PTGS2, PGF2α and PTGFR when in the presence of IL1B, TNF, fsl-1 (TLR2/6 ligand) or flagellin (TLR5 ligand). siIRF5-transfected cells also displayed decreased NF-κB RELA transcriptional activity in the presence of these preterm birth mediators. Our study suggests a novel role for IRF5 in the regulation of the inflammatory response in human myometrium.
Impact of Aortic Valve Replacement on Glycemic Control in Diabetes Mellitus
Authors: Rodgers-Fischl Peter Michael , Davenport Daniel L. , Saha Sibu P. , Guglin Maya E.
Published: July 2018

Introduction: The Framingham Studies revealed that diabetes mellitus (DM) predisposed subjects to a two- to eight-fold increase in the risk of developing heart failure (HF). However, there is much less information available about the reverse issue; namely, whether there is an increased risk of developing DM in patients with HF. We sought to determine if reversal or partial reversal of HF through aortic valve replacement (AVR) would improve glycemic control in patients with DM at our institution.
Methods: The electronic medical records of 57 consecutive diabetic patients were retrospectively analyzed. These patients had undergone AVR at a medium-sized academic medical center from May 2005 through May 2015, and had glycated hemoglobin (HbA1C) measured before and after the procedure. The variables of interest included HbA1C, and echocardiographic parameters such as left ventricular ejection fraction (LVEF), tricuspid regurgitation velocity (TRV), and right ventricular systolic pressure (RVSP) before and after valve replacement.
Results: HbA1C decreased significantly during the first year after replacement, from 7.1% (range 4.4 - 13.0%) before surgery to 6.5% in the first year (P < .05). In addition, the calculated RVSP decreased from 44 mmHg (20 - 79 mmHg) to 37 mmHg (P < .05 from the preoperative value). LVEF and TRV did not change significantly. Reductions in HbA1C and RVSP during the first year were greater in patients who experienced an increase of 5% or more in EF at their first postoperative measurement. Patients with higher baseline HbA1C values had a greater decline in glycated Hb during the first year (P < .01).
Conclusion: AVR was associated with a reduction of HbA1C and a decrease in pulmonary artery systolic pressure within one year of the procedure.
Nurses and Internet Health-Related Information: Review on Access and Utility
Authors: Ahmad Muayyad M , Musallam Ruba , Habeeb Allah Abla
Published: July 2018

Purpose. To review literature on the nurses’ use and access to Internet Health Related Information (HRI).There is relatively little evidence in published literature on barriers, attitudes and how nurses utilize online health-related information.Methods. Literature search was carried-out on Cumulative Indexes to Nursing and Allied Health Literature (CINAHL), MEDLINE, Google Scholar, Springer and Sage publications. The search timeframe has focused on the outburst of Internet usage between the years 2000 and 2014.Results. Quality of available websites and databases containing health-related information vary widely in their accuracy, validity and reliability that require nurses to continuously evaluate their relevance.Conclusion: Emphasis on the necessity for training in the use of information technology is important to the nurses’ continuous professional development. Literature showed a strong evidence of the increased access to the Internet by nurses to retrieve information related to clinical practice, which in turns enhance the quality of care and communication among nurses.
Teoria das Fundações Morais: o Nativismo Moral em Jonathan Haidt
Authors: Rex Roger Valério de Vargas
Published: July 2018

Jonathan Haidt propôs uma das versões mais conhecidas de nativismo moral, na qual ele defende a existência de certas fundações morais. No presente artigo, examino as principais críticas feitas a essa teoria, em especial (a) a de que Haidt estaria comprometido com uma tese bastante questionável acerca da mente humana; e (b) a de que a descrição de Haidt seria incompleta, em razão de não fornecer uma explicação para as etapas mentais prévias ao desencadeamento das intuições. Quanto à primeira crítica, ao aceitar a tese da modularidade massiva, Haidt de fato compromete-se com uma visão a respeito da mente humana que não é consiliente com os resultados de pesquisas neurocientíficas. No entanto, em razão de ter ciência dessa objeção, ele propõe para os críticos da modularidade massiva uma versão mais amena de sua teoria. Com relação à segunda crítica, a teoria de Haidt é realmente incompleta, pois ignora que só é possível que uma ação produza determinada intuição após aquele que a percebe tenha formado algum tipo de representação. A partir da análise dessas críticas, concluo que a teoria das fundações morais, em sua versão amena, oferece uma explicação plausível para os julgamentos morais, mas falha ao não oferecer uma explicação a respeito do papel que inferências sobre os estados mentais do autor da ação desempenham no desencadeamento das intuições morais.
Planning with Durative Actions in Stochastic Domains
Authors: Mausam  , Weld D. S.
Published: July 2018

Probabilistic planning problems are typically modeled as a Markov Decision Process (MDP). MDPs, while an otherwise expressive model, allow only for sequential, non-durative actions. This poses severe restrictions in modeling and solving a real world planning problem. We extend the MDP model to incorporate 1) simultaneous action execution, 2) durative actions, and 3) stochastic durations. We develop several algorithms to combat the computational explosion introduced by these features. The key theoretical ideas used in building these algorithms are -- modeling a complex problem as an MDP in extended state/action space, pruning of irrelevant actions, sampling of relevant actions, using informed heuristics to guide the search, hybridizing different planners to achieve benefits of both, approximating the problem and replanning. Our empirical evaluation illuminates the different merits in using various algorithms, viz., optimality, empirical closeness to optimality, theoretical error bounds, and speed.
CUI Networks: A Graphical Representation for Conditional Utility Independence
Authors: Engel Y. , Wellman M. P.
Published: July 2018

We introduce CUI networks, a compact graphical representation of utility functions over multiple attributes. CUI networks model multiattribute utility functions using the well-studied and widely applicable utility independence concept. We show how conditional utility independence leads to an effective functional decomposition that can be exhibited graphically, and how local, compact data at the graph nodes can be used to calculate joint utility. We discuss aspects of elicitation, network construction, and optimization, and contrast our new representation with previous graphical preference modeling.
CTL Model Update for System Modifications
Authors: Zhang Y. , Ding Y.
Published: July 2018

Model checking is a promising technology, which has been applied for verification of many hardware and software systems. In this paper, we introduce the concept of model update towards the development of an automatic system modification tool that extends model checking functions. We define primitive update operations on the models of Computation Tree Logic (CTL) and formalize the principle of minimal change for CTL model update. These primitive update operations, together with the underlying minimal change principle, serve as the foundation for CTL model update. Essential semantic and computational characterizations are provided for our CTL model update approach. We then describe a formal algorithm that implements this approach. We also illustrate two case studies of CTL model updates for the well-known microwave oven example and the Andrew File System 1, from which we further propose a method to optimize the update results in complex system modifications.
Extended RDF as a Semantic Foundation of Rule Markup Languages
Authors: Analyti A. , Antoniou G. , Damasio C. V. , Wagner G.
Published: July 2018

Ontologies and automated reasoning are the building blocks of the Semantic Web initiative. Derivation rules can be included in an ontology to define derived concepts, based on base concepts. For example, rules allow to define the extension of a class or property, based on a complex relation between the extensions of the same or other classes and properties. On the other hand, the inclusion of negative information both in the form of negation-as-failure and explicit negative information is also needed to enable various forms of reasoning. In this paper, we extend RDF graphs with weak and strong negation, as well as derivation rules. The ERDF stable model semantics of the extended framework (Extended RDF) is defined, extending RDF(S) semantics. A distinctive feature of our theory, which is based on Partial Logic, is that both truth and falsity extensions of properties and classes are considered, allowing for truth value gaps. Our framework supports both closed-world and open-world reasoning through the explicit representation of the particular closed-world assumptions and the ERDF ontological categories of total properties and total classes.
The Complexity of Planning Problems With Simple Causal Graphs
Authors: Gimenez O. , Jonsson A.
Published: July 2018

We present three new complexity results for classes of planning problems with simple causal graphs. First, we describe a polynomial-time algorithm that uses macros to generate plans for the class 3S of planning problems with binary state variables and acyclic causal graphs. This implies that plan generation may be tractable even when a planning problem has an exponentially long minimal solution. We also prove that the problem of plan existence for planning problems with multi-valued variables and chain causal graphs is NP-hard. Finally, we show that plan existence for planning problems with binary state variables and polytree causal graphs is NP-complete.
Loosely Coupled Formulations for Automated Planning: An Integer Programming Perspective
Authors: Van den Briel M. H.L. , Vossen T. , Kambhampati S.
Published: July 2018

We represent planning as a set of loosely coupled network flow problems, where each network corresponds to one of the state variables in the planning domain. The network nodes correspond to the state variable values and the network arcs correspond to the value transitions. The planning problem is to find a path (a sequence of actions) in each network such that, when merged, they constitute a feasible plan. In this paper we present a number of integer programming formulations that model these loosely coupled networks with varying degrees of flexibility. Since merging may introduce exponentially many ordering constraints we implement a so-called branch-and-cut algorithm, in which these constraints are dynamically generated and added to the formulation when needed. Our results are very promising, they improve upon previous planning as integer programming approaches and lay the foundation for integer programming approaches for cost optimal planning.
Optimal and Approximate Q-value Functions for Decentralized POMDPs
Authors: Oliehoek F. A. , Spaan M. T. J. , Vlassis N.
Published: July 2018

Decision-theoretic planning is a popular approach to sequential decision making problems, because it treats uncertainty in sensing and acting in a principled way. In single-agent frameworks like MDPs and POMDPs, planning can be carried out by resorting to Q-value functions: an optimal Q-value function Q* is computed in a recursive manner by dynamic programming, and then an optimal policy is extracted from Q*. In this paper we study whether similar Q-value functions can be defined for decentralized POMDP models (Dec-POMDPs), and how policies can be extracted from such value functions. We define two forms of the optimal Q-value function for Dec-POMDPs: one that gives a normative description as the Q-value function of an optimal pure joint policy and another one that is sequentially rational and thus gives a recipe for computation. This computation, however, is infeasible for all but the smallest problems. Therefore, we analyze various approximate Q-value functions that allow for efficient computation. We describe how they relate, and we prove that they all provide an upper bound to the optimal Q-value function Q*. Finally, unifying some previous approaches for solving Dec-POMDPs, we describe a family of algorithms for extracting policies from such Q-value functions, and perform an experimental evaluation on existing test problems, including a new firefighting benchmark problem.
Spectrum of Variable-Random Trees
Authors: Liu F. T. , Ting K. M. , Yu Y. , Zhou Z. H.
Published: July 2018

In this paper, we show that a continuous spectrum of randomisation exists, in which most existing tree randomisations are only operating around the two ends of the spectrum. That leaves a huge part of the spectrum largely unexplored. We propose a base learner VR-Tree which generates trees with variable-randomness. VR-Trees are able to span from the conventional deterministic trees to the complete-random trees using a probabilistic parameter. Using VR-Trees as the base models, we explore the entire spectrum of randomised ensembles, together with Bagging and Random Subspace. We discover that the two halves of the spectrum have their distinct characteristics; and the understanding of which allows us to propose a new approach in building better decision tree ensembles. We name this approach Coalescence, which coalesces a number of points in the random-half of the spectrum. Coalescence acts as a committee of ``experts'' to cater for unforeseeable conditions presented in training data. Coalescence is found to perform better than any single operating point in the spectrum, without the need to tune to a specific level of randomness. In our empirical study, Coalescence ranks top among the benchmarking ensemble methods including Random Forests, Random Subspace and C5 Boosting; and only Coalescence is significantly better than Bagging and Max-Diverse Ensemble among all the methods in the comparison. Although Coalescence is not significantly better than Random Forests, we have identified conditions under which one will perform better than the other.
A General Theory of Additive State Space Abstractions
Authors: Yang F. , Culberson J. , Holte R. , Zahavi U. , Felner A.
Published: July 2018

Informally, a set of abstractions of a state space S is additive if the distance between any two states in S is always greater than or equal to the sum of the corresponding distances in the abstract spaces. The first known additive abstractions, called disjoint pattern databases, were experimentally demonstrated to produce state of the art performance on certain state spaces. However, previous applications were restricted to state spaces with special properties, which precludes disjoint pattern databases from being defined for several commonly used testbeds, such as Rubik's Cube, TopSpin and the Pancake puzzle. In this paper we give a general definition of additive abstractions that can be applied to any state space and prove that heuristics based on additive abstractions are consistent as well as admissible. We use this new definition to create additive abstractions for these testbeds and show experimentally that well chosen additive abstractions can reduce search time substantially for the (18,4)-TopSpin puzzle and by three orders of magnitude over state of the art methods for the 17-Pancake puzzle. We also derive a way of testing if the heuristic value returned by additive abstractions is provably too low and show that the use of this test can reduce search time for the 15-puzzle and TopSpin by roughly a factor of two.
Dynamic Control in Real-Time Heuristic Search
Authors: Bulitko V. , Lustrek M. , Schaeffer J. , Bjornsson Y. , Sigmundarson S.
Published: July 2018

Real-time heuristic search is a challenging type of agent-centered search because the agent's planning time per action is bounded by a constant independent of problem size. A common problem that imposes such restrictions is pathfinding in modern computer games where a large number of units must plan their paths simultaneously over large maps. Common search algorithms (e.g., A*, IDA*, D*, ARA*, AD*) are inherently not real-time and may lose completeness when a constant bound is imposed on per-action planning time. Real-time search algorithms retain completeness but frequently produce unacceptably suboptimal solutions. In this paper, we extend classic and modern real-time search algorithms with an automated mechanism for dynamic depth and subgoal selection. The new algorithms remain real-time and complete. On large computer game maps, they find paths within 7% of optimal while on average expanding roughly a single state per action. This is nearly a three-fold improvement in suboptimality over the existing state-of-the-art algorithms and, at the same time, a 15-fold improvement in the amount of planning per action.
M-DPOP: Faithful Distributed Implementation of Efficient Social Choice Problems
Authors: Petcu A. , Faltings B. , Parkes D. C.
Published: July 2018

In the efficient social choice problem, the goal is to assign values, subject to side constraints, to a set of variables to maximize the total utility across a population of agents, where each agent has private information about its utility function. In this paper we model the social choice problem as a distributed constraint optimization problem (DCOP), in which each agent can communicate with other agents that share an interest in one or more variables. Whereas existing DCOP algorithms can be easily manipulated by an agent, either by misreporting private information or deviating from the algorithm, we introduce M-DPOP, the first DCOP algorithm that provides a faithful distributed implementation for efficient social choice. This provides a concrete example of how the methods of mechanism design can be unified with those of distributed optimization. Faithfulness ensures that no agent can benefit by unilaterally deviating from any aspect of the protocol, neither information-revelation, computation, nor communication, and whatever the private information of other agents. We allow for payments by agents to a central bank, which is the only central authoritythat we require. To achieve faithfulness, we carefully integrate the Vickrey-Clarke-Groves (VCG) mechanism with the DPOP algorithm, such that each agent is only asked to perform computation, report information, and send messages that is in its own best interest. Determining agent i's payment requires solving the social choice problem without agent i. Here, we present a method to reuse computation performed in solving the main problem in a way that is robust against manipulation by the excluded agent. Experimental results on structured problems show that as much as 87% of the computation required for solving the marginal problems can be avoided by re-use, providing very good scalability in the number of agents.
On unstructured problems, we observe a sensitivity of M-DPOP to the density of the problem, and we show that reusability decreases from almost 100% for very sparse problems to around 20% for highly connected problems. We close with a discussion of the features of DCOP that enable faithful implementations in this problem, the challenge of reusing computation from the main problem to marginal problems in other algorithms such as ADOPT and OptAPO, and the prospect of methods to avoid the welfare loss that can occur because of the transfer of payments to the bank.
A Multiagent Approach to Autonomous Intersection Management
Authors: Dresner K. , Stone P.
Published: July 2018

Artificial intelligence research is ushering in a new era of sophisticated, mass-market transportation technology. While computers can already fly a passenger jet better than a trained human pilot, people are still faced with the dangerous yet tedious task of driving automobiles. Intelligent Transportation Systems (ITS) is the field that focuses on integrating information technology with vehicles and transportation infrastructure to make transportation safer, cheaper, and more efficient. Recent advances in ITS point to a future in which vehicles themselves handle the vast majority of the driving task. Once autonomous vehicles become popular, autonomous interactions amongst multiple vehicles will be possible. Current methods of vehicle coordination, which are all designed to work with human drivers, will be outdated. The bottleneck for roadway efficiency will no longer be the drivers, but rather the mechanism by which those drivers' actions are coordinated. While open-road driving is a well-studied and more-or-less-solved problem, urban traffic scenarios, especially intersections, are much more challenging.

We believe current methods for controlling traffic, specifically at intersections, will not be able to take advantage of the increased sensitivity and precision of autonomous vehicles as compared to human drivers. In this article, we suggest an alternative mechanism for coordinating the movement of autonomous vehicles through intersections. Drivers and intersections in this mechanism are treated as autonomous agents in a multiagent system. In this multiagent system, intersections use a new reservation-based approach built around a detailed communication protocol, which we also present. We demonstrate in simulation that our new mechanism has the potential to significantly outperform current intersection control technology -- traffic lights and stop signs. Because our mechanism can emulate a traffic light or stop sign, it subsumes the most popular current methods of intersection control. This article also presents two extensions to the mechanism. The first extension allows the system to control human-driven vehicles in addition to autonomous vehicles. The second gives priority to emergency vehicles without significant cost to civilian vehicles. The mechanism, including both extensions, is implemented and tested in simulation, and we present experimental results that strongly attest to the efficacy of this approach.
A Heuristic Search Approach to Planning with Continuous Resources in Stochastic Domains
Authors: Meuleau N. , Benazera E. , Brafman R. I. , Hansen E. A. , Mausam
Published: July 2018

We consider the problem of optimal planning in stochastic domains with resource constraints, where the resources are continuous and the choice of action at each step depends on resource availability. We introduce the HAO* algorithm, a generalization of the AO* algorithm that performs search in a hybrid state space that is modeled using both discrete and continuous state variables, where the continuous variables represent monotonic resources. Like other heuristic search algorithms, HAO* leverages knowledge of the start state and an admissible heuristic to focus computational effort on those parts of the state space that could be reached from the start state by following an optimal policy. We show that this approach is especially effective when resource constraints limit how much of the state space is reachable. Experimental results demonstrate its effectiveness in the domain that motivates our research: automated planning for planetary exploration rovers.
Complexity of Strategic Behavior in Multi-Winner Elections
Authors: Meir R. , Procaccia A. D. , Rosenschein J. S. , Zohar Aviv
Published: July 2018

Although recent years have seen a surge of interest in the computational aspects of social choice, no specific attention has previously been devoted to elections with multiple winners, e.g., elections of an assembly or committee. In this paper, we characterize the worst-case complexity of manipulation and control in the context of four prominent multi-winner voting systems, under different formulations of the strategic agentâs goal.
Online Planning Algorithms for POMDPs
Authors: Ross S. , Pineau J. , Paquet S. , Chaib-draa B.
Published: July 2018

Partially Observable Markov Decision Processes (POMDPs) provide a rich framework for sequential decision-making under uncertainty in stochastic domains. However, solving a POMDP is often intractable except for small problems due to their complexity. Here, we focus on online approaches that alleviate the computational complexity by computing good local policies at each decision step during the execution. Online algorithms generally consist of a lookahead search to find the best action to execute at each time step in an environment. Our objectives here are to survey the various existing online POMDP methods, analyze their properties and discuss their advantages and disadvantages; and to thoroughly evaluate these online approaches in different environments under various metrics (return, error bound reduction, lower bound improvement). Our experimental results indicate that state-of-the-art online heuristic search methods can handle large POMDP domains efficiently.
Learning Partially Observable Deterministic Action Models
Authors: Amir E. , Chang A.
Published: July 2018

We present exact algorithms for identifying deterministic-actions' effects and preconditions in dynamic partially observable domains. They apply when one does not know the action model(the way actions affect the world) of a domain and must learn it from partial observations over time. Such scenarios are common in real world applications. They are challenging for AI tasks because traditional domain structures that underly tractability (e.g., conditional independence) fail there (e.g., world features become correlated). Our work departs from traditional assumptions about partial observations and action models. In particular, it focuses on problems in which actions are deterministic of simple logical structure and observation models have all features observed with some frequency. We yield tractable algorithms for the modified problem for such domains.

Our algorithms take sequences of partial observations over time as input, and output deterministic action models that could have lead to those observations. The algorithms output all or one of those models (depending on our choice), and are exact in that no model is misclassified given the observations. Our algorithms take polynomial time in the number of time steps and state features for some traditional action classes examined in the AI-planning literature, e.g., STRIPS actions. In contrast, traditional approaches for HMMs and Reinforcement Learning are inexact and exponentially intractable for such domains. Our experiments verify the theoretical tractability guarantees, and show that we identify action models exactly. Several applications in planning, autonomous exploration, and adventure-game playing already use these results. They are also promising for probabilistic settings, partially observable reinforcement learning, and diagnosis.
Identification of Pleonastic It Using the Web
Authors: Li Y. , Musilek P. , Reformat M. , Wyard-Scott L.
Published: July 2018

In a significant minority of cases, certain pronouns, especially the pronoun it, can be used without referring to any specific entity. This phenomenon of pleonastic pronoun usage poses serious problems for systems aiming at even a shallow understanding of natural language texts. In this paper, a novel approach is proposed to identify such uses of it: the extrapositional cases are identified using a series of queries against the web, and the cleft cases are identified using a simple set of syntactic rules. The system is evaluated with four sets of news articles containing 679 extrapositional cases as well as 78 cleft constructs. The identification results are comparable to those obtained by human efforts.
A Multiagent Reinforcement Learning Algorithm with Non-linear Dynamics
Authors: Abdallah S. , Lesser V.
Published: July 2018

Several multiagent reinforcement learning (MARL) algorithms have been proposed to optimize agents' decisions. Due to the complexity of the problem, the majority of the previously developed MARL algorithms assumed agents either had some knowledge of the underlying game (such as Nash equilibria) and/or observed other agents actions and the rewards they received.

We introduce a new MARL algorithm called the Weighted Policy Learner (WPL), which allows agents to reach a Nash Equilibrium (NE) in benchmark 2-player-2-action games with minimum knowledge. Using WPL, the only feedback an agent needs is its own local reward (the agent does not observe other agents actions or rewards). Furthermore, WPL does not assume that agents know the underlying game or the corresponding Nash Equilibrium a priori. We experimentally show that our algorithm converges in benchmark two-player-two-action games. We also show that our algorithm converges in the challenging Shapley's game where previous MARL algorithms failed to converge without knowing the underlying game or the NE. Furthermore, we show that WPL outperforms the state-of-the-art algorithms in a more realistic setting of 100 agents interacting and learning concurrently.

An important aspect of understanding the behavior of a MARL algorithm is analyzing the dynamics of the algorithm: how the policies of multiple learning agents evolve over time as agents interact with one another. Such an analysis not only verifies whether agents using a given MARL algorithm will eventually converge, but also reveals the behavior of the MARL algorithm prior to convergence. We analyze our algorithm in two-player-two-action games and show that symbolically proving WPL's convergence is difficult, because of the non-linear nature of WPL's dynamics, unlike previous MARL algorithms that had either linear or piece-wise-linear dynamics. Instead, we numerically solve WPL's dynamics differential equations and compare the solution to the dynamics of previous MARL algorithms.
Learning Document-Level Semantic Properties from Free-Text Annotations
Authors: Branavan S. R. K. , Chen H. , Eisenstein J. , Barzilay R.
Published: July 2018

This paper presents a new method for inferring the semantic properties of documents by leveraging free-text keyphrase annotations. Such annotations are becoming increasingly abundant due to the recent dramatic growth in semi-structured, user-generated online content. One especially relevant domain is product reviews, which are often annotated by their authors with pros/cons keyphrases such as ``a real bargain'' or ``good value.'' These annotations are representative of the underlying semantic properties; however, unlike expert annotations, they are noisy: lay authors may use different labels to denote the same property, and some labels may be missing. To learn using such noisy annotations, we find a hidden paraphrase structure which clusters the keyphrases. The paraphrase structure is linked with a latent topic model of the review texts, enabling the system to predict the properties of unannotated documents and to effectively aggregate the semantic properties of multiple reviews. Our approach is implemented as a hierarchical Bayesian model with joint inference. We find that joint inference increases the robustness of the keyphrase clustering and encourages the latent topics to correlate with semantically meaningful properties. Multiple evaluations demonstrate that our model substantially outperforms alternative approaches for summarizing single and multiple documents into a set of semantically salient keyphrases.
Solving #SAT and Bayesian Inference with Backtracking Search
Authors: Bacchus F. , Dalmao S. , Pitassi T.
Published: July 2018

Inference in Bayes Nets (BAYES) is an important problem with numerous applications in probabilistic reasoning. Counting the number of satisfying assignments of a propositional formula (#SAT) is a closely related problem of fundamental theoretical importance. Both these problems, and others, are members of the class of sum-of-products (SUMPROD) problems. In this paper we show that standard backtracking search when augmented with a simple memoization scheme (caching) can solve any sum-of-products problem with time complexity that is at least as good any other state-of-the-art exact algorithm, and that it can also achieve the best known time-space tradeoff. Furthermore, backtrackings ability to utilize more flexible variable orderings allows us to prove that it can achieve an exponential speedup over other standard algorithms for SUMPROD on some instances.

The ideas presented here have been utilized in a number of solvers that have been applied to various types of sum-of-product problems. These systems have exploited the fact that backtracking can naturally exploit more of the problems structure to achieve improved performance on a range of probleminstances. Empirical evidence of this performance gain has appeared in published works describing these solvers, and we provide references to these works.
Sentence Compression as Tree Transduction
Authors: Cohn T. A. , Lapata M.
Published: July 2018

This paper presents a tree-to-tree transduction method for sentence compression. Our model is based on synchronous tree substitution grammar, a formalism that allows local distortion of the tree topology and can thus naturally capture structural mismatches. We describe an algorithm for decoding in this framework and show how the model can be trained discriminatively within a large margin framework. Experimental results on sentence compression bring significant improvements over a state-of-the-art model.
Policy Iteration for Decentralized Control of Markov Decision Processes
Authors: Bernstein D. S. , Amato C. , Hansen E. A. , Zilberstein S.
Published: July 2018

Coordination of distributed agents is required for problems arising in many areas, including multi-robot systems, networking and e-commerce. As a formal framework for such problems, we use the decentralized partially observable Markov decision process (DEC-POMDP). Though much work has been done on optimal dynamic programming algorithms for the single-agent version of the problem, optimal algorithms for the multiagent case have been elusive. The main contribution of this paper is an optimal policy iteration algorithm for solving DEC-POMDPs. The algorithm uses stochastic finite-state controllers to represent policies. The solution can include a correlation device, which allows agents to correlate their actions without communicating. This approach alternates between expanding the controller and performing value-preserving transformations, which modify the controller without sacrificing value. We present two efficient value-preserving transformations: one can reduce the size of the controller and the other can improve its value while keeping the size fixed. Empirical results demonstrate the usefulness of value-preserving transformations in increasing value while keeping controller size to a minimum. To broaden the applicability of the approach, we also present a heuristic version of the policy iteration algorithm, which sacrifices convergence to optimality. This algorithm further reduces the size of the controllers at each step by assuming that probability distributions over the other agents' actions are known. While this assumption may not hold in general, it helps produce higher quality solutions in our test problems.
Wikipedia-based Semantic Interpretation for Natural Language Processing
Authors: Gabrilovich E. , Markovitch S.
Published: July 2018

Adequate representation of natural language semantics requires access to vast amounts of common sense and domain-specific world knowledge. Prior work in the field was based on purely statistical techniques that did not make use of background knowledge, on limited lexicographic knowledge bases such as WordNet, or on huge manual efforts such as the CYC project. Here we propose a novel method, called Explicit Semantic Analysis (ESA), for fine-grained semantic interpretation of unrestricted natural language texts. Our method represents meaning in a high-dimensional space of concepts derived from Wikipedia, the largest encyclopedia in existence. We explicitly represent the meaning of any text in terms of Wikipedia-based concepts. We evaluate the effectiveness of our method on text categorization and on computing the degree of semantic relatedness between fragments of natural language text. Using ESA results in significant improvements over the previous state of the art in both tasks. Importantly, due to the use of natural concepts, the ESA model is easy to explain to human users.
Exploiting Single-Cycle Symmetries in Continuous Constraint Problems
Authors: Ruiz de Angulo V. , Torras C.
Published: July 2018

Symmetries in discrete constraint satisfaction problems have been explored and exploited in the last years, but symmetries in continuous constraint problems have not received the same attention. Here we focus on permutations of the variables consisting of one single cycle. We propose a procedure that takes advantage of these symmetries by interacting with a continuous constraint solver without interfering with it. A key concept in this procedure are the classes of symmetric boxes formed by bisecting a n-dimensional cube at the same point in all dimensions at the same time. We analyze these classes and quantify them as a function of the cube dimensionality. Moreover, we propose a simple algorithm to generate the representatives of all these classes for any number of variables at very high rates. A problem example from the chemical ﬁeld and the cyclic n-roots problem are used to show the performance of the approach in practice.
Research on electrical conductivity of graphene/cement composites
Authors: Bai Shuya , Jiang Linhua , Jiang Yu , Jin Ming , Jiang Shaobo , Tao Debiao
Published: July 2018

Graphene can be used in cement-based materials to enhance their electrical properties. In this work, electrical conductivity measurements on graphene/cement nanocomposites were performed using the four-probe method. The influences of graphene content, water/cement ratio, curing age and water content on the electrical conductivity of the system were studied. The micro-morphology of the composites was observed by scanning electron microscopy (SEM). The results showed that electrical conductivity increases with increasing graphene content, which can be explained by percolation theory. A logistic function is introduced to describe the relationship between the electrical conductivity of the system and graphene content. SEM observation revealed that the conductivity of system depends on the distribution and connection of graphene in the cement matrix. When the graphene content exceeded a threshold concentration, curing age and water content had little influence on the electrical conductivity of system. The findings provide an important guide for research on the electrical properties of graphene/cement composites.
An analytical model for contaminant transport in cut-off wall and aquifer system
Authors: Xie Haijian , Wang Shaoyi , Chen Yun , Jiang Jianqun , Qiu Zhanhong
Published: July 2018

The vertical cut-off wall is a practical means of site contamination control. It is of great importance to predict the service time of a vertical cut-off wall. Previous analytical models on simulation of a cut-off wall neglected the existence of the aquifer, which can lead to simulation errors. An analytical model was developed for contaminant transport in a vertical cut-off wall and an aquifer system. Three cases were considered – that is, advection and diffusion of contaminant towards the same direction, advection and diffusion towards the opposite direction and the pure diffusion case. The analytical solution is obtained by using Laplace transform and binomial theorem. Increasing the scale of the aquifer can extend the breakthrough time. With a fixed hydraulic gradient of 0·5, as the dispersivity of the aquifer increases from 1 to 10 m, the breakthrough time for chloride (Cl−) increased by a factor of 1·7. The breakthrough behaviour of chloride is more sensitive to the scale of the aquifer than the breakthrough behaviour of lead is. The breakthrough time of lead obtained by the single-layer model is always longer than that of the proposed model due to overestimation of retardation.
New observational method framework for embedded walls
Authors: Hardy Stuart , Nicholson Duncan , Ingram Peter , Gaba Asim , Chen Ying , Biscontin Giovanna
Published: July 2018

This paper introduces the new holistic observational method (OM) framework described in the Construction Industry Research and Information Association’s guide C760 – the new revision to the Embedded Retaining Walls design guide C580. This new OM framework is described by four approaches, and these are explained with reference to case histories. The maximum benefit is achieved by back-analysing case histories in similar ground conditions to derive the most probable soil parameters. These parameters can then be used in an ab initio design to maximise saving in the wall thickness, toe embedment and propping requirements. A contingency plan is based on characteristic soil parameters, and this may involve additional propping. The term ‘ipso tempore’ is introduced for the circumstance where wall redesign occurs after construction starts. This may be to achieve saving in propping where movements are much smaller than predicted. Alternatively, additional propping or ground treatment may be required, and this is consistent with Peck’s ‘best-way-out’ approach.
Optimisation of flange and web slenderness for pre-engineered built-up steel sections
Authors: Saleem Muhammad Umair , Khurram Nauman , Qureshi Hisham Jahangir , Kazmi Zaheer Abbas , Siddiqui Zahid Ahmad
Published: July 2018

A design-based optimisation for pre-engineered steel frames comprising built-up sections is presented in this paper and the effect of unbraced length ratios on flange and web slenderness ratios is elaborated. A wide range of built-up sections was selected with different unbraced length ratios, web slenderness ratios and flange slenderness ratios. Web slenderness and flange slenderness were varied from a fairly compact range to highly slender values. The capacity of the design sections was determined using different design codes under the combined effect of compression and bending forces. The optimised values for flange and web slenderness were determined for each unbraced length ratio and a relationship between unbraced length and optimum flange slenderness values was established. Analysis of the results revealed that the use of appropriate values of flange and web slenderness in accordance with the member unbraced length can significantly reduce the section weight and improves its efficiency.
Modelling bond between concrete and bonded and anchored carbon-fibre polymer strips
Authors: Sakin Shaimaa , Anil Özgür , Ghoroubi Rahim , Mercimek Ömer
Published: July 2018

An analytical model was developed using finite-element software to simulate the load-transfer mechanisms between concrete and strengthening strips of carbon-fibre-reinforced polymer. The strips were bonded to the concrete surface and additionally fixed with fan-type carbon-fibre-reinforced polymer anchors. The model was developed using the results of 14 pull tests on concrete blocks strengthened with variously sized and anchored strips. The anchors were represented by increasing the bonding strength in the anchor contact area. The numerical results were compared with experimental results and showed good correlation. The model was then used to investigate the influence of strip width and length, number of anchors and concrete strength on load-transfer behaviour. Finally, a mathematical model was proposed for calculating the bond strength between concrete and a bonded and anchored carbon-fibre-reinforced polymer strip.
Magnetically inspired deformation of the liquid/vapor interface drives soap bubbles
Authors: Legchenkova Irina , Chaniel Gilad , Frenkel Mark , Bormashenko Yelena , Shoval Shraga , Bormashenko Edward
Published: July 2018

The possibility of driving millimetrically scaled, soap and glycerol bubbles floating on a liquid, using a steady magnetic field of c. 0·5 T, is demonstrated. The bubbles are repelled by the magnet, starting from a threshold initial lateral distance of L0cr= 4 mm from the vertical axis of the magnet, and are attracted to the magnet when the initial lateral distance L0< L0cr. The displacement of bubbles is due to the deformation of the liquid/vapor interface by the magnetic field, known as the Moses effect. This deformation gives rise to the change in apparent contact angles driving the bubble. Switching of repulsion to attraction is explained in terms of a model of an interaction between two capillary charges, one of which represents the bubble and the second represents the well, produced by the magnet on the liquid/vapor interface. The experiments with bubbles demonstrate that the role of gravity in the displacement of floating diamagnetic objects driven by steady magnetic fields may be negligible. The mathematical model of the process is suggested. The experimental data are well explained by the proposed model.