Query: Inverse Reinforcement Learning

Relevant passages:
Maximum margin planning <=SEP=> Imitation learning of sequential, goal-directed behavior by standard supervised techniques is often difficult. We frame learning such behaviors as a maximum margin structured prediction problem over a space of policies. In this approach, we learn mappings from features to cost so an optimal policy in an MDP with these cost mimics the expert's behavior. Further, we demonstrate a simple, provably efficient approach to structured maximum margin learning, based on the subgradient method, that leverages existing fast algorithms for inference. Although the technique is general, it is particularly relevant in problems where A* and dynamic programming approaches make learning policies tractable in problems beyond the limitations of a QP formulation. We demonstrate our approach applied to route planning for outdoor mobile robots, where the behavior a designer wishes a planner to execute is often clear, while specifying cost functions that engender this behavior is a much more difficult task.

Maximum Entropy Inverse Reinforcement Learning <=SEP=> Recent research has shown the benefit of framing problems of imitation learning as solutions to Markov Decision Problems. This approach reduces learning to the problem of recovering a utility function that makes the behavior induced by a near-optimal policy closely mimic demonstrated behavior. In this work, we develop a probabilistic approach based on the principle of maximum entropy. Our approach provides a well-defined, globally normalized distribution over decision sequences, while providing the same performance guarantees as existing methods. We develop our technique in the context of modeling realworld navigation and driving behaviors where collected data is inherently noisy and imperfect. Our probabilistic approach enables modeling of route preferences as well as a powerful new approach to inferring destinations and routes based on partial trajectories.

Learning Agents for Uncertain Environments (Extended Abstract) <=SEP=> This talk proposes a very simple “baseline architecture” for a learning agent that can handle stochastic, partially observable environments. The architecture uses reinforcement learning together with a method for representing temporal processes as graphical models. I will discuss methods for leaming the parameters and structure of such representations from sensory inputs, and for computing posterior probabilities. Some open problems remain before we can try out the complete agent; more arise when we consider scaling up. A second theme of the talk will be whether reinforcement learning can provide a good model of animal and human learning. To answer this question, we must do inverse reinforcement learning: given the observed behaviour, what reward signal, if any, is being optimized? This seems to be a very interesting problem for the COLT, UAI, and ML communities, and has been addressed in econometrics under the heading of structural estimation of Markov decision processes. 1 Learning in uncertain environments AI is about the construction of intelligent agents, i.e., systems that perceive and act effectively (according to some performance measure) in an environment. I have argued elsewhere Russell and Norvig (1995) that most AI research has focused on environments that are static, deterministic, discrete, and fully observable. What is to be done when, as in the real world, the environment is dynamic, stochastic, continuous, and partially observable? ‘This paper draws on a variety of research efforts supported by NSF @I-9634215), ONR (N00014-97-l-0941), and AR0 (DAAH04-96-1-0341). Permission to make digital or hard copies of all or p.art of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prolit or commercial adwantage and that copies bear this notice and the full citation on the first page. To copy otherwise. to republish, to post on servers or to redistribute to lists, requires prior specific pemlission and/or a fee. COLT 98 Madison WI IJSA Copyright ACM 1998 1-5X1 13-057--0/9X/ 7...$5.00 In recent years, reinforcement learning (also called neurodynamic programming) has made rapid progress as an approachfor building agents automatically (Sutton, 1988; Kaelbling et al., 1996; Bertsekas & Tsitsiklis, 1996). The basic idea is that the performance measure is made available to the agent in the form of a rewardfunction specifying the reward for each state that the agent passes through. The performance measure is then the sum of the rewards obtained. For example, when a bumble bee forages, the reward function at each time step might be some combination of the distance flown (weighted negatively) and the nectar ingested. Reinforcement learning (RL) methods are essentially online algorithmd for solving Markovdecisionprocesses (MDPs). An MDP is defined by the reward function and a model, that is, the state transition probabilities conditioned on each possible action. RL algorithms can be model-based, where the agent learns a model, or model-free-e.g., Q-learning citeWatkins: 1989, which learns just a function Q(s, a) specifying the long-term value of taking action a in state s and acting optimally thereafter. Despite their successes, RL methods have been restricted largely tofully observable MDPs, in which the sensory input at each state is sufficient to identify the state. Obviously, in the real world, we must often deal with partially observable MDPs (POMDPs). Astrom (1965) proved that optimal decisions in POMDPs depend on the belief state b at each point in time, i.e., the posterior probability distribution over all possible actual states, given all evidence to date. The functions V and Q then become functions of b instead of s. Parr and Russell (1995) describes a very simple POMDP RL algorithm using an explicit representation of b as a vector of probabilities, and McCallum (1993) shows a way to approximate the belief state using recent percept sequences. Neither approach is likely to scale up to situations with large numbers of state variables and long-term temporal dependencies. What is needed is a way of representing the model compactly and updating the belief state efficiently given the model and each new observation. Dynamic Bayesian networks (Dean & Kanazawa, 1989) seem to have some of the required properties; in particular, they have significant advantages over other approaches such as Kalman filters and hidden Markov models. Our baseline architecture, shown in Figure 1, uses DBNs to represent and update the belief state as new sensor information arrives. Given a representation for b, the reward signal is used to learn a Q-function represented by some “black-box” function approximator such as a neural network. Provided we can handle hybrid (dis-

Algorithms for Inverse Reinforcement Learning <=SEP=> 

Learning to search: Functional gradient techniques for imitation learning <=SEP=> Programming robot behavior remains a challenging task. While it is often easy to abstractly define or even demonstrate a desired behavior, designing a controller that embodies the same behavior is difficult, time consuming, and ultimately expensive. The machine learning paradigm offers the promise of enabling “programming by demonstration” for developing high-performance robotic systems. Unfortunately, many “behavioral cloning” (Bain & Sammut, 1995; Pomerleau, 1989; LeCun et al., 2006) approaches that utilize classical tools of supervised learning (e.g. decision trees, neural networks, or support vector machines) do not fit the needs of modern robotic systems. These systems are often built atop sophisticated planning algorithms that efficiently reason far into the future; consequently, ignoring these planning algorithms in lieu of a supervised learning approach often leads to myopic and poor-quality robot performance. While planning algorithms have shown success in many real-world applications ranging from legged locomotion (Chestnutt et al., 2003) to outdoor unstructured navigation (Kelly et al., 2004; Stentz, 2009), such algorithms rely on fully specified cost functions that map sensor readings and environment models to quantifiable costs. Such cost functions are usually manually designed and programmed. Recently, a set of techniques has been developed that explore learning these functions from expert human demonstration. These algorithms apply an inverse optimal control approach to find a cost function for which planned behavior mimics an expert’s demonstration. The work we present extends the Maximum Margin Planning (MMP) (Ratliff et al., 2006a) framework to admit learning of more powerful, non-linear cost functions. These algorithms, known collectively as LEARCH (LEArning to seaRCH ), are simpler to implement than most existing methods, more efficient than previous attempts at non-linearization (Ratliff et al., 2006b), more naturally satisfy common constraints on the cost function, and better represent our prior beliefs about the function’s form. We derive and discuss the framework both mathematically and intuitively, and demonstrate practical realworld performance with three applied case-studies including legged locomotion, grasp planning, and autonomous outdoor unstructured navigation. The latter study includes hundreds of kilometers of autonomous traversal through complex natural environments. These case-studies address key challenges in applying the algorithm in practical settings that utilize state-of-the-art planners, and which may be constrained by efficiency requirements and imperfect expert demonstration.

The benefits of hybridization <=SEP=> This article presents the impact of the performance of an FC and control strategies on the benefits of hybridization of fuel cell/supercapacitor hybrid sources for vehicle applications. Then, the storage device can complement the main source to produce the compatibility and performance characteristics needed in a load. The studies of two hybrid power systems for vehicle applications, FC/battery and FC/supercapacitor hybrid power sources, are explained. Experimental results with small-scale devices (a PEMFC of 500 W, 40 A, and 13 V; a lead-xicid battery module of 33 Ah and 48 V; and a supercapacitor module of 292 F, 500 A, and 30 V) in laboratory will illustrate the performance of the system during motor-drive cycles.

RF-MEMS Switches for Reconfigurable Integrated Circuits <=SEP=> This paper deals with a relatively new area of radio-frequency (RF) technology based on microelectromechanical systems (MEMS). RF MEMS provides a class of new devices and components which display superior high-frequency performance relative to conventional (usually semiconductor) devices, and which enable new system capabilities. In addition, MEMS devices are designed and fabricated by techniques similar to those of very large-scale integration, and can be manufactured by traditional batch-processing methods. In this paper, the only device addressed is the electrostatic microswitch—perhaps the paradigm RF-MEMS device. Through its superior performance characteristics, the microswitch is being developed in a number of existing circuits and systems, including radio front-ends, capacitor banks, and time-delay networks. The superior performance combined with ultra-low-power dissipation and large-scale integration should enable new system functionality as well. Two possibilities addressed here are quasi-optical beam steering and electrically reconfigurable antennas.

Modeling Complex Air Traffic Management Systems <=SEP=> In this work, we propose the use of multi-agent system (MAS) models as the basis for predictive reasoning about various safety conditions and the performance of Air Traffic Management (ATM) Systems. To this end, we describe the engineering of a domain-specific MAS model that provides constructs for creating scenarios related to ATM systems and procedures; we then instantiate the constructs in the ATM model for different scenarios. As a case study we generate a model for a concept that provides the ability to maximize departure throughput at La Guardia airport (LGA) without impacting the flow of the arrival traffic; the model consists of approximately 1.5 hours real time flight data. During this time, between 130 and 150 airplanes are managed by four en-route controllers, three TRACON controllers, and one tower controller at LGA who is responsible for departures and arrivals. The planes are landing at approximately 36 to 40 planes an hour. A key contribution of this work is that the model can be extended to various air-traffic management scenarios and can serve as a template for engineering large-scale models in other domains.

Union and Difference of Models, 10 years later <=SEP=> This paper contains a summary of the talk given by the author on the occasion of the MODELS 2013 most influential paper award. The talk discussed the original paper as published in 2003, the research work done by others afterwards and the author’s personal reflection on the award. 1 Version Control of Software and System Models There are two main usage scenarios for design models in software and system development: models as sketches, that serve as a communication aid in informal discussions, and models as formal artifacts, to be analyzed, transformed into other artifacts, maintained and evolved during the whole software and system development process. In this second scenario, models are valuable assets that should be kept in a trusted repository. In a complex development project, these models will be updated often and concurrently by different developers. Therefore, there is a need for a version control system for models with optimistic locking. This is a system to compare, merge and store all versions of all models created within a development project. We can illustrate the use of a version control system for models as follows. Let us assume that the original model shown at the top of Figure 1 is edited simultaneously by two developers. One developer has decided that the subclass B is no longer necessary in the model. Simultaneously, the other developer has decided that class C should have a subclass D. The problem is to combine the contributions of both developers into a single model. This is the model shown at the bottom of Fig. 1. We presented the basic algorithms to solve this problem in the original paper published in the proceedings of the UML 2003 conference [1]. The proposed solution is based on calculating the final model as the merge of the differences between the original and the edited models. Figure 2 shows an example of the difference of two models, in this case the difference between the models edited by the developers and the original model. The result of the difference is not always a model, in a similar way that the difference between two natural numbers is not a natural number but a negative one. An example of this is shown in the bottom B C A Original Model

Stakeholder Identification in the Requirements Engineering Process <=SEP=> Adequate, timely and effective consultation of relevant stakeholders is of paramount importance in the requirements engineering process. However, the thorny issue of making sure that all relevant stakeholders are consulted has received less attention than other areas which depend on it, such as scenario-based requirements, involving users in development, negotiating between different viewpoints and so on. The literature suggests examples of stakeholders, and categories of stakeholder, but does not provide help in identifying stakeholders for a specific system. In this paper, we discuss current work in stakeholder identification, propose an approach to identifying relevant stakeholders for a specific system, and propose future directions for the work. 1. What is a ÔstakeholderÕ? There is a large body of literature in the strategic management area which discusses organisations in terms of a stakeholder model. Stakeholder analysis, it is claimed, can be used to analyse an organisationÕs performance and determine its future strategic direction. An oft-quoted definition of ÔstakeholderÕ, taken from a key reference in this literature is: ÔA stakeholder in an organisation is (by definition) any group or individual who can affect or is affected by the achievement of the organisationÕs objectives.Õ [10] A much broader definition, which has also been attributed to Freeman, is that a stakeholder is Ôanything influencing or influenced byÕ the firm, but it has been claimed that this definition is problematic because it leads to the identification of a very broad set of stakeholders. It is important to distinguish between influencers and stakeholders because while some potential stakeholders may indeed be both stakeholders and influencers, some who have a real stake in an enterprise may have no influence, e.g. a job applicant, while some influencers may have no stake, e.g. the media [8]. Information systems (IS) researchers have also taken up the idea of stakeholders: ÔWe define stakeholders as these participants <in the development process> together with any other individuals, groups or organisations whose actions can influence or be influenced by the development and use of the system whether directly or indirectly.Õ [22] In software engineering, stakeholders have been defined as: ÔThe people and organisations affected by the applicationÕ [3] ÔSystem stakeholders are people or organisations who will be affected by the system and who have a direct or indirect influence on the system requirementsÕ [16] ÔStakeholders are people who have a stake or interest in the projectÕ [4] A more explicit refinement of this definition is: ÔÉ anyone whose jobs will be altered, who supplies or gains information from it, or whose power or influence within the organisation will increase or decrease.Õ [7] They go on to say that ÔIt will frequently be the case that the formal ÔclientÕ who orders the system falls very low on the list of those affected. Be very wary of changes which take power, influence or control from some stakeholders without returning something tangible in its place.Õ [7] When faced with the practical problem of how to identify the set of stakeholders relevant to a specific project, these definitions are not particularly helpful. The main concern is that, although such definitions are usually accompanied by example groups of stakeholders, they are vague and may lead to consideration of inappropriate or incomplete groups of stakeholders. Categories of stakeholder include end-users, managers and others involved in the organisational processes influenced by the system, engineers responsible for system development and maintenance, customers of the Sharp, Finkelstein & Galal 11/12/99 2 organisation who will use the system to provide a service, external bodies such as regulators, domain experts, and so on. They will each have different goals, and will try to satisfy their own without recourse to others [16]. Cotterell and Hughes [4] suggest that stakeholders might be in one of three categories: internal to the project team; external to the project team, but internal to the organisation; and external to both the project team and the organisation. Newman and Lamming [20] suggest a different division: into those who will use the system directly or indirectly, and those who will be involved in developing the system. This distinction is also taken up with respect to the development of knowledge-based systems [22]. The set of stakeholders in the knowledge acquisition process and the set of stakeholders in the use of a knowledge-based system, are not necessarily identical; they are likely to vary in membership, and for those members in common, the type and level of stake they have is likely to vary. The IS literature suggests a different division again, into ÔhubsÕ or ÔsponsorsÕ and ÔspokesÕ or ÔadaptorsÕ, where the former are those initiating and sustaining the system, while the latter are those participating [21]. Macaulay [19] identifies four categories of stakeholder in any computer system: 1. Those responsible for design and development; 2. Those with a financial interest, responsible for its sale or purchase; 3. Those responsible for introduction and maintenance; 4. Those who have an interest in its use. But again, she offers no guidelines for identifying specific stakeholders for a given system. So far, we have not distinguished between individuals or groups and roles. As with other situations, the mapping between stakeholders and individuals or groups is not oneto-one. It is therefore more appropriate to think of stakeholders as roles rather than as specific people. In subsequent discussion, we shall use ÔstakeholderÕ to mean Ôstakeholder roleÕ. 2. Identifying stakeholders There has been little focus on the participants of the requirements engineering (RE) process, for example in terms of how to trace participants in the RE process and how to identify stakeholders [13]. All of the references cited above emphasise the importance of identifying stakeholders, and although they provide examples, or broad guidance for identifying them, none describes a model or a concrete approach for identifying stakeholders for a specific project. This deficiency has been noted in the management literature, and in the IS literature [21], where the approaches have been criticised for either assuming that stakeholders are ÔobviousÕ, or for providing broad categories which are too generic to be of practical use. Expert identification for knowledge-based systems development has similarities with stakeholder identification in RE, although here too there has been an assumption that experts are readily identifiable [22]. Pouloudi and Whitley [21] suggest four principles of stakeholder identification, and describe an approach which is based on these principles, and which they used to identify stakeholders in the drug use management domain. Lyytinen and Hirschheim [17] also provides some guidance for stakeholder identification for IS, while acknowledging that Ôthe identification of the set of all stakeholders is far from trivialÕ. Methods for requirements engineering, e.g. KAOS [5], do not directly support stakeholder identification; it seems to be assumed that it is straightforward. The task of identifying actors for use case development [15] has similarities with stakeholder identification, but is targeted only at a fraction of system stakeholders. Stakeholders are related to each other and interact with each other [22,11,17]. Interactions between them include: exchanging information, products, or instructions, or providing supporting tasks. Information about the stakeholders and the nature of their relationships and interactions needs to be identified and recorded. Dimensions of importance are: relationships between stakeholders, the relationship of each stakeholder to the system, and the priority to be given to each stakeholderÕs view. This information is needed to manage, interpret, balance and process stakeholder input. Reconciling different stakeholder views is beyond this paper, but is considered in [12]. 3. An approach to identifying stakeholders for requirements engineering Here, we propose an approach to discovering all relevant stakeholders of a specific system that we believe is domain-independent, effective and pragmatic. We draw on the literature cited above, and on aspects of ethical decision-making in software systems [18]. Our starting point is a set of stakeholders that we refer to as ÔbaselineÕ stakeholders. From these, we can recognise ÔsupplierÕ stakeholders and ÔclientÕ stakeholders: the former provides information or supporting tasks to the baseline, and the latter processes or inspects the products of the baseline. Other stakeholders that we call ÔsatelliteÕs interact with the baseline in a variety of ways. ÔInteractionÕ may involve communicating, reading a set of rules or guidelines, searching for information and so on. Our approach focuses on interactions between stakeholders rather than relationships between the system and the stakeholder, Sharp, Finkelstein & Galal 11/12/99 3 because they are easier to follow. Figure 1 illustrates the main elements of stakeholder identification. Figure 1. The main elements of stakeholder identification addressed by our approach 3.1 Baseline stakeholders We have identified four groups of baseline stakeholder: users, developers, legislators and decision-makers. We have dubbed these ÔbaselineÕ because the web of stakeholders and their relationships can be identified from them. The nature of each group is explored below. 3.1.1 Users. The term ÔuserÕ has many interpretations. For example, Holtzblatt and Jones [14] include in their definition of ÔusersÕ those who manage direct users, those who receive products from the system, those who test the system, those who make the purchasing decision, and those who use competitive products. Eason [9] identifies three categories of user: primary, secondary and te

A Smart Cloud Robotic System Based on Cloud Computing Services <=SEP=> In this paper, we present a smart service robotic system based on cloud computing services. The design and implementation of infrastructure, computation components and communication components are introduced. The proposed system can alleviate the complex computation and storage load of robots to cloud and provide various services to the robots. The computation components can dynamically allocate resources to the robots. The communication components allow easy access of the robots and provide flexible resource management. Furthermore, we modeled the task-scheduling problem and proposed a max-heaps algorithm. The simulation results demonstrate that the proposed algorithm minimized the overall task costs.

The spread of behavior in an online social network experiment. <=SEP=> How do social networks affect the spread of behavior? A popular hypothesis states that networks with many clustered ties and a high degree of separation will be less effective for behavioral diffusion than networks in which locally redundant ties are rewired to provide shortcuts across the social space. A competing hypothesis argues that when behaviors require social reinforcement, a network with more clustering may be more advantageous, even if the network as a whole has a larger diameter. I investigated the effects of network structure on diffusion by studying the spread of health behavior through artificially structured online communities. Individual adoption was much more likely when participants received social reinforcement from multiple neighbors in the social network. The behavior spread farther and faster across clustered-lattice networks than across corresponding random networks.

Voltage buffer compensation using Flipped Voltage Follower in a two-stage CMOS op-amp <=SEP=> In Miller and current buffer compensation techniques, the compensation capacitor often loads the output node. If a voltage buffer is used in feedback, the compensation capacitor obviates the loading on the output node. In this paper, we introduce an implementation of a voltage buffer compensation using a Flipped Voltage Follower (FVF) for stabilizing a two-stage CMOS op-amp. The op-amps are implemented in a 180-nm CMOS process with a power supply of 1.8V while operating with a quiescent current of 110μA. Results indicate that the proposed voltage buffer compensation using FVF improves the Unity Gain Frequency from 5.5MHz to 12.2MHz compared to Miller compensation. Also, the proposed technique enhances the transient response while lowering the compensation capacitance by 47% and 17.7% compared to Miller and common-drain compensation topologies. Utilization of FVF or its variants as a voltage buffer in a feedback compensation network has wide potential applications in the analog design space.

Dynamic question generation system for web-based testing using particle swarm optimization <=SEP=> One aim of testing is to identify weaknesses in students’ knowledge. Computerized tests are now one of the most important ways to judge learning, and, selecting tailored questions for each learner is a significant part of such tests. Therefore, one current trend is that computerized adaptive tests (CATs) not only assist teachers in estimating the learning performance of students, but also facilitate understanding of problems in their learning process. These tests, must effectively and efficiently select questions from a large-scale item bank, and to cope with this problem we propose a dynamic question generation system for web-based tests using the novel approach of particle swarm optimization (PSO). The dynamic question generation system is built to select tailored questions for each learner from the item bank to satisfy multiple assessment requirements. Furthermore, the proposed approach is able to efficiently generate near-optimal questions that satisfy multiple assessment criteria. With a series of experiments, we compare the efficiency and efficacy of the PSO approach with other approaches. The experimental results show that the PSO approach is suitable for the selection of near-optimal questions from large-scale item banks. 2007 Elsevier Ltd. All rights reserved.

Deep Embedding for Spatial Role Labeling <=SEP=> This paper introduces the visually informed embedding of word (VIEW), a continuous vector representation for a word extracted from a deep neural model trained using the Microsoft COCO data set to forecast the spatial arrangements between visual objects, given a textual description. The model is composed of a deep multilayer perceptron (MLP) stacked on the top of a Long Short Term Memory (LSTM) network, the latter being preceded by an embedding layer. The VIEW is applied to transferring multimodal background knowledge to Spatial Role Labeling (SpRL) algorithms, which recognize spatial relations between objects mentioned in the text. This work also contributes with a new method to select complementary features and a fine-tuning method for MLP that improves the F1 measure in classifying the words into spatial roles. The VIEW is evaluated with the Task 3 of SemEval-2013 benchmark data set, SpaceEval.

To recognize shapes, first learn to generate images. <=SEP=> The uniformity of the cortical architecture and the ability of functions to move to different areas of cortex following early damage strongly suggest that there is a single basic learning algorithm for extracting underlying structure from richly structured, high-dimensional sensory data. There have been many attempts to design such an algorithm, but until recently they all suffered from serious computational weaknesses. This chapter describes several of the proposed algorithms and shows how they can be combined to produce hybrid methods that work efficiently in networks with many layers and millions of adaptive connections.

Design and optimization of thermo-mechanical reliability in wafer level packaging <=SEP=> Article history: Received 4 July 2009 Received in revised form 16 November 2009 Available online 29 January 2010 0026-2714/$ see front matter 2009 Elsevier Ltd. A doi:10.1016/j.microrel.2009.11.010 * Corresponding author. Address: Department of M University, P.O. Box 10028, Beaumont, TX 77710, USA +1 409 880 8121. E-mail address: xuejun.fan@lamar.edu (X.J. Fan). In this paper, a variety of wafer level packaging (WLP) structures, including both fan-in and fan-out WLPs, are investigated for solder joint thermo-mechanical reliability performance, from a structural design point of view. The effects of redistribution layer (RDL), bump structural design/material selection, polymer-cored ball application, and PCB design/material selection are studied. The investigation focuses on four different WLP technologies: standard WLP (ball on I/O WLP), ball on polymer WLP without under bump metallurgy (UBM) layer, ball on polymer WLP with UBM layer, and encapsulated copper post WLP. Ball on I/O WLP, in which solder balls are directly attached to the metal pads on silicon wafer, is used as a benchmark for the analysis. 3-D finite element modeling is performed to investigate the effects of WLP structures, UBM layer, polymer film material properties (in ball on polymer WLP), and encapsulated epoxy material properties (in copper post WLP). Both ball on polymer and copper post WLPs have shown great reliability improvement in thermal cycling. For ball on polymer WLP structures, polymer film between silicon and solder balls creates a ‘cushion’ effect to reduce the stresses in solder joints. Such cushion effect can be achieved either by an extremely compliant film or a ‘hard’ film with a large coefficient of thermal expansion. Encapsulated copper post WLP shows the best thermo-mechanical performance among the four WLP structures. Furthermore, for a fan-out WLP, it has been found that the critical solder balls are the outermost solder balls under die-area, where the maximum thermal mismatch takes place. In a fan-out WLP package, chip size, other than package size, determines the limit of solder joint reliability. This paper also discusses the polymer-cored solder ball applications to enhance thermo-mechanical reliability of solder joints. Finally, both experimental and finite element analysis have demonstrated that making corner balls non-electrically connected can greatly improve the WLP thermomechanical reliability. 2009 Elsevier Ltd. All rights reserved.

Introducing Expected Returns into Risk Parity Portfolios : A New Framework for Tactical and Strategic Asset Allocation <=SEP=> Risk parity is an allocation method used to build diversified portfolios that does not rely on any assumptions of expected returns, thus placing risk management at the heart of the strategy. This explains why risk parity became a popular investment model after the global financial crisis in 2008. However, risk parity has also been criticized because it focuses on managing risk concentration rather than portfolio performance, and is therefore seen as being closer to passive management than active management. In this article, we show how to introduce assumptions of expected returns into risk parity portfolios. To do this, we consider a generalized risk measure that takes into account both the portfolio return and volatility. However, the trade-off between performance and volatility contributions creates some difficulty, while the risk budgeting problem must be clearly defined. After deriving the theoretical properties of such risk budgeting portfolios, we apply this new model to asset allocation. First, we consider long-term investment policy and the determination of strategic asset allocation. We then consider dynamic allocation and show how to build risk parity funds that depend on expected returns.

wradlib – An Open Source Library for Weather Radar Data Processing <=SEP=> Observation data from weather radars is deemed to have great potential in hydrology and meteorology for forecasting of severe weather or floods in small and urban catchments by providing high resolution measurements of precipitation. With the time series from operational weather radar networks attaining lengths suitable for the long term calibration of hydrological models, the interest in using this data is growing. There are, however, several challenges impeding a widespread use of weather radar data. The first being a multitude of different file formats for data storage and exchange. Although the OPERA project [1] has taken steps towards harmonizing the data exchange inside the European radar network [2], different dialects still exist in addition to a large variety of legacy formats. The second barrier is what we would like to describe as the product dilemma. A great number of potential applications also implies a great number of different and often competing requirements as to the quality of the data. As an example, while one user might be willing to accept more false alarms in a clutter detection filter, e.g. to avoid erroneous data assimilation results, another might want a more conservative product in order not to miss the small scale convection that leads to a hundred year flood in a small head catchment. A single product from a radar operator, even if created with the best methods currently available, will never be able to accommodate all these needs simultaneously. Thus the product will either be a compromise or it will accommodate one user more than the other. Often the processing chain needs to be in a specific order, where a change of a certain processing step is impossible without affecting the results of all following steps. Sometimes some post-processing of the product might be possible, but if not, the user’s only choice is to either take the product as is or leave it. If a user should decide that he would take a raw radar product and try to do customized corrections, he is faced with basically reinventing the wheel, writing routines for data I/O, error corrections, georeferencing and visualization, trying to find relevant literature and to extract algorithms from publications, which, often enough, do not offer all implementation details. This takes a lot of time and effort, which could be used much more effectively, if standard algorithms were available in a well documented and easy to use manner. wradlib is intended to provide this collection of routines and algorithms in order to facilitate and foster the use of weather radar data in as many disciplines as possible including research, application development and teaching.

A survey of hardware Trojan threat and defense <=SEP=> Hardware Trojans (HTs) can be implanted in security-weak parts of a chip with various means to steal the internal sensitive data or modify original functionality, which may lead to huge economic losses and great harm to society. Therefore, it is very important to analyze the specific HT threats existing in the whole life cycle of integrated circuits (ICs), and perform protection against hardware Trojans. In this paper, we elaborate an IC market model to illustrate the potential HT threats faced by the parties involved in the model. Then we categorize the recent research advances in the countermeasures against HT attacks. Finally, the challenges and prospects for HT defense are illuminated. & 2016 Elsevier B.V. All rights reserved.

An overview of Fog computing and its security issues <=SEP=> Fog computing is a paradigm that extends Cloud computing and services to the edge of the network. Similar to Cloud, Fog provides data, compute, storage and application services to end users. In this article, we elaborate the motivation and advantages of Fog computing and analyse its applications in a series of real scenarios, such as Smart Grid, smart traffic lights in vehicular networks and software defined networks. We discuss the state of the art of Fog computing and similar work under the same umbrella. Distinguished from other reviewing work of Fog computing, this paper further discloses the security and privacy issues according to current Fog computing paradigm. As an example, we study a typical attack, man-in-the-middle attack, for the discussion of system security in Fog computing. We investigate the stealthy features of this attack by examining its CPU and memory consumption on Fog device. In addition, we discuss the authentication and authorization techniques that can be used in Fog computing. An example of authentication techniques is introduced to address the security scenario where the connection between Fog and Cloud is fragile. Copyright © 2015 John Wiley & Sons, Ltd.

Investigating the Impact of Data Volume and Domain Similarity on Transfer Learning Applications <=SEP=> Transfer learning allows practitioners to recognize and apply knowledge learned in previous tasks (source task) to new tasks or new domains (target task), which share some commonality. The two important factors impacting the performance of transfer learning models are: (a) the size of the target dataset, and (b) the similarity in distribution between source and target domains. Thus far, there has been little investigation into just how important these factors are. In this paper, we investigate the impact of target dataset size and source/target domain similarity on model performance through a series of experiments. We find that more data is always beneficial, and model performance improves linearly with the log of data size, until we are out of data. As source/target domains differ, more data is required and fine tuning will render better performance than feature extraction. When source/target domains are similar and data size is small, fine tuning and feature extraction renders equivalent performance. Our hope is that by beginning this quantitative investigation on the effect of data volume and domain similarity in transfer learning we might inspire others to explore the significance of data in developing more accurate statistical models.

Determining the effectiveness of prompts for self-regulated learning in problem-solving scenarios <=SEP=> Cognitive scientists have studied internal cognitive structures, processes, and systems for decades in order to understand how they function in human learning. In order to solve challenging tasks in problem situations, learners not only have to perform cognitive activities, e.g., activating existing cognitive structures or organizing new information, they also have to set specific goals, plan their activities, monitor their performance during the problem-solving process, and evaluate the efficiency of their actions. This paper reports an experimental study with 98 participants where effective instructional interventions for self-regulated learning within problemsolving processes are investigated. Furthermore, an automated assessment and analysis methodology for determining the quality of learning outcomes is introduced. The results indicate that generic prompts are an important aid for developing cognitive structures while solving problems.

Synchronising movements with the sounds of a virtual partner enhances partner likeability <=SEP=> Previous studies have demonstrated that synchronising movements with other people can influence affiliative behaviour towards them. While research has focused on synchronisation with visually observed movement, synchronisation with a partner who is heard may have similar effects. We replicate findings showing that synchronisation can influence ratings of likeability of a partner, but demonstrate that this is possible with virtual interaction, involving a video of a partner. Participants performed instructed synchrony in time to sounds instead of the observable actions of another person. Results show significantly higher ratings of likeability of a partner after moving at the same time as sounds attributed to that partner, compared with moving in between sounds. Objectively quantified synchrony also correlated with ratings of likeability. Belief that sounds were made by another person was manipulated in Experiment 2, and results demonstrate that when sounds are attributed to a computer, ratings of likeability are not affected by moving in or out of time. These findings demonstrate that interaction with sound can be experienced as social interaction in the absence of genuine interpersonal contact, which may help explain why people enjoy engaging with recorded music.

KeyGraph: Automatic Indexing by Co-Occurrence Graph Based on Building Construction Metaphor <=SEP=> In this paper, we present an algorithm for extracting keywords representing the asserted main point in a document, without relying on external devices such as natural language processing tools or a document corpus. Our algorithm KeyGraph is based on the segmentation of a graph, representing the co-occurrence between terms in a document, into clusters. Each cluster corresponds to a concept on which author's idea is based, and top ranked terms by a statistic based on each term's relationship to these clusters are selected as keywords. This strategy comes from considering that a document is constructed like a building for expressing new ideas based on traditional concepts. The experimental results show that thus extracted terms match author's point quite accurately, even though KeyGraph does not use each term's average frequency in a corpus, i.e., KeyGraph is a contentsensitive, domain independent device of indexing.

Community detection using a neighborhood strength driven Label Propagation Algorithm <=SEP=> Studies of community structure and evolution in large social networks require a fast and accurate algorithm for community detection. As the size of analyzed communities grows, complexity of the community detection algorithm needs to be kept close to linear. The Label Propagation Algorithm (LPA) has the benefits of nearly-linear running time and easy implementation, thus it forms a good basis for efficient community detection methods. In this paper, we propose new update rule and label propagation criterion in LPA to improve both its computational efficiency and the quality of communities that it detects. The speed is optimized by avoiding unnecessary updates performed by the original algorithm. This change reduces significantly (by order of magnitude for large networks) the number of iterations that the algorithm executes. We also evaluate our generalization of the LPA update rule that takes into account, with varying strength, connections to the neighborhood of a node considering a new label. Experiments on computer generated networks and a wide range of social networks show that our new rule improves the quality of the detected communities compared to those found by the original LPA. The benefit of considering positive neighborhood strength is pronounced especially on real-world networks containing sufficiently large fraction of nodes with high clustering coefficient.

Fast Supervised Discrete Hashing <=SEP=> Learning-based hashing algorithms are “hot topics” because they can greatly increase the scale at which existing methods operate. In this paper, we propose a new learning-based hashing method called “fast supervised discrete hashing” (FSDH) based on “supervised discrete hashing” (SDH). Regressing the training examples (or hash code) to the corresponding class labels is widely used in ordinary least squares regression. Rather than adopting this method, FSDH uses a very simple yet effective regression of the class labels of training examples to the corresponding hash code to accelerate the algorithm. To the best of our knowledge, this strategy has not previously been used for hashing. Traditional SDH decomposes the optimization into three sub-problems, with the most critical sub-problem - discrete optimization for binary hash codes - solved using iterative discrete cyclic coordinate descent (DCC), which is time-consuming. However, FSDH has a closed-form solution and only requires a single rather than iterative hash code-solving step, which is highly efficient. Furthermore, FSDH is usually faster than SDH for solving the projection matrix for least squares regression, making FSDH generally faster than SDH. For example, our results show that FSDH is about 12-times faster than SDH when the number of hashing bits is 128 on the CIFAR-10 data base, and FSDH is about 151-times faster than FastHash when the number of hashing bits is 64 on the MNIST data-base. Our experimental results show that FSDH is not only fast, but also outperforms other comparative methods.

Vascular structures in dermoscopy* <=SEP=> Dermoscopy is an aiding method in the visualization of the epidermis and dermis. It is usually used to diagnose melanocytic lesions. In recent years, dermoscopy has increasingly been used to diagnose non-melanocytic lesions. Certain vascular structures, their patterns of arrangement and additional criteria may demonstrate lesion-specific characteristics. In this review, vascular structures and their arrangements are discussed separately in the light of conflicting views and an overview of recent literature.

Audio-visual emotion recognition : A dynamic , multimodal approach <=SEP=> Designing systems able to interact with students in a natural manner is a complex and far from solved problem. A key aspect of natural interaction is the ability to understand and appropriately respond to human emotions. This paper details our response to the continuous Audio/Visual Emotion Challenge (AVEC'12) whose goal is to predict four affective signals describing human emotions. The proposed method uses Fourier spectra to extract multi-scale dynamic descriptions of signals characterizing face appearance, head movements and voice. We perform a kernel regression with very few representative samples selected via a supervised weighted-distance-based clustering, that leads to a high generalization power. We also propose a particularly fast regressor-level fusion framework to merge systems based on different modalities. Experiments have proven the efficiency of each key point of the proposed method and our results on challenge data were the highest among 10 international research teams. Key-words Affective computing, Dynamic features, Multimodal fusion, Feature selection, Facial expressions. ACM Classification

Lane detection using Fourier-based line detector <=SEP=> In this paper a new approach to lane marker detection problem is introduced as a significant complement for a semi/fully autonomous driver assistance system. The method incorporates advance line detection using multilayer fractional Fourier transform (MLFRFT) and the state-of-the-art advance lane detector (ALD). Experimental results have shown a considerable reduction in computational complexity.