Query: StarCraft Micromanagement With Reinforcement Learning and Curriculum Transfer Learning

Relevant passages:
Deep Learning <=SEP=> Machine-learning technology powers many aspects of modern society: from web searches to content filtering on social networks to recommendations on e-commerce websites, and it is increasingly present in consumer products such as cameras and smartphones. Machine-learning systems are used to identify objects in images, transcribe speech into text, match news items, posts or products with users’ interests, and select relevant results of search. Increasingly, these applications make use of a class of techniques called deep learning. Conventional machine-learning techniques were limited in their ability to process natural data in their raw form. For decades, constructing a pattern-recognition or machine-learning system required careful engineering and considerable domain expertise to design a feature extractor that transformed the raw data (such as the pixel values of an image) into a suitable internal representation or feature vector from which the learning subsystem, often a classifier, could detect or classify patterns in the input. Representation learning is a set of methods that allows a machine to be fed with raw data and to automatically discover the representations needed for detection or classification. Deep-learning methods are representation-learning methods with multiple levels of representation, obtained by composing simple but non-linear modules that each transform the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level. With the composition of enough such transformations, very complex functions can be learned. For classification tasks, higher layers of representation amplify aspects of the input that are important for discrimination and suppress irrelevant variations. An image, for example, comes in the form of an array of pixel values, and the learned features in the first layer of representation typically represent the presence or absence of edges at particular orientations and locations in the image. The second layer typically detects motifs by spotting particular arrangements of edges, regardless of small variations in the edge positions. The third layer may assemble motifs into larger combinations that correspond to parts of familiar objects, and subsequent layers would detect objects as combinations of these parts. The key aspect of deep learning is that these layers of features are not designed by human engineers: they are learned from data using a general-purpose learning procedure. Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years. It has turned out to be very good at discovering intricate structures in high-dimensional data and is therefore applicable to many domains of science, business and government. In addition to beating records in image recognition and speech recognition, it has beaten other machine-learning techniques at predicting the activity of potential drug molecules, analysing particle accelerator data, reconstructing brain circuits, and predicting the effects of mutations in non-coding DNA on gene expression and disease. Perhaps more surprisingly, deep learning has produced extremely promising results for various tasks in natural language understanding, particularly topic classification, sentiment analysis, question answering and language translation. We think that deep learning will have many more successes in the near future because it requires very little engineering by hand, so it can easily take advantage of increases in the amount of available computation and data. New learning algorithms and architectures that are currently being developed for deep neural networks will only accelerate this progress.

Markov Games as a Framework for Multi-Agent Reinforcement Learning <=SEP=> In the Markov decision process (MDP) formalization of reinforcement learning, a single adaptive agent interacts with an environment defined by a probabilistic transition function. In this solipsistic view, secondary agents can only be part of the environment and are therefore fixed in their behavior. The framework of Markov games allows us to widen this view to include multiple adaptive agents with interacting or competing goals. This paper considers a step in this direction in which exactly two agents with diametrically opposed goals share an environment. It describes a Q-learning-like algorithm for finding optimal policies and demonstrates its application to a simple two-player game in which the optimal policy is probabilistic.

Mastering the game of Go with deep neural networks and tree search <=SEP=> The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

Prioritized Experience Replay <=SEP=> Experience replay lets online reinforcement learning agents remember and reuse experiences from the past. In prior work, experience transitions were uniformly sampled from a replay memory. However, this approach simply replays transitions at the same frequency that they were originally experienced, regardless of their significance. In this paper we develop a framework for prioritizing experience, so as to replay important transitions more frequently, and therefore learn more efficiently. We use prioritized experience replay in the Deep Q-Network (DQN) algorithm, which achieved human-level performance in Atari games. DQN with prioritized experience replay achieves a new state-of-the-art, outperforming DQN with uniform replay on 42 out of 57 games.

Deep reinforcement learning with experience replay based on SARSA <=SEP=> SARSA, as one kind of on-policy reinforcement learning methods, is integrated with deep learning to solve the video games control problems in this paper. We use deep convolutional neural network to estimate the state-action value, and SARSA learning to update it. Besides, experience replay is introduced to make the training process suitable to scalable machine learning problems. In this way, a new deep reinforcement learning method, called deep SARSA is proposed to solve complicated control problems such as imitating human to play video games. From the experiments results, we can conclude that the deep SARSA learning shows better performances in some aspects than deep Q learning.

One Intelligent Agent to Rule Them All Bachelor Thesis <=SEP=> In this work, we explore the possibility of training agents which are able to show intelligent behaviour in many different scenarios. We present the effectiveness of different machine learning algorithms in the StarCraft II Learning Environment [1] and their performance in different scenarios and compare them. In the end, we recreate DeepMind’s FullyConv Agent with slightly better results.

Topological pattern recognition for point cloud data ∗ <=SEP=> 

Delay Analysis of Hybrid WiFi-LiFi System <=SEP=> Heterogeneous wireless networks are capable of effectivel y leveraging different access technologies to provide a wid e variety of coverage areas. In this paper, the coexistence of WiFi and visible light communication (VLC) is investigated as a para digm. The delay of two configurations of such heterogeneous system has been evaluated. In the first configuration, the non-aggregat ed system, any request is either allocated to WiFi or VLC. While in the se cond configuration, the aggregated system, each request is s plit into two pieces, one is forwarded to WiFi and the other is forw arded to VLC. Under the assumptions of Poisson arrival proce ss of requests and the exponential distribution of requests si ze, it is mathematically proved that the aggregated system p rovides lower minimum average system delay than that of the non-aggregate d system. For the non-aggregated system, the optimal traffic allo ation ratio is derived. For the aggregated system, an efficient sol ution for the splitting ratio is proposed. Empirical result show that the solution proposed here incurs a delay penalty (less than 3%) over the optimal result.

Characterization of defense mechanisms against distributed denial of service attacks <=SEP=> We propose a characterization of distributed denial-of-s ervice (DDOS) defenses where reaction points are network-based and attack responses are active. The purpose is to provide a framework for comparing the performance and deployment of DDOS defenses. We identify the characteristics in attack detection algorithms and a ttack responses by reviewing defenses that have appeared in the literature. We expect that this char a terization will provide practitioners and academia insights into deploying DDOS defense as network services.

Bi-objective conflict detection and resolution in railway traffic management <=SEP=> Railway conflict detection and resolution is the daily task faced by railway managers that consists of adjusting train schedules whenever disturbances make the timetable infeasible. The main objective pursued by the infrastructure managers in this task is minimization of train delays, while train operating companies are also interested in other indicators of passengers dissatisfaction. The two objectives are conflicting whenever delay reduction requires cancellation of some connections between train services, which hampers passengers transfer. In fact, the infrastructure company and the train operating companies discuss on which connection to keep or drop in order to reach a compromise solution. In this paper we face the bi-objective problem of minimizing delays and missed connections to provide a set of feasible non-dominated schedules to support this decisional process. We use a detailed alternative graph model to ensure schedule feasibility and develop two heuristic algorithms to compute the Pareto front of non-dominated schedules. Our computational study, based on a complex and densely occupied Dutch railway network, shows that good coordination of connected train services is important to achieve real-time efficiency of railway services since the management of connections may heavily affect train punctuality. The two algorithms approximate accurately the Pareto front within a limited computational time.

Beamforming for MIMO-OFDM Wireless Systems <=SEP=> The smart antennas are widely used for wireless com munication, because it has a ability to increase th coverage and capacity of a communication system. Smart anten na performs two main functions such as direction of arrival estimation (DOA) and beam forming. Using beam formi ng algorithm smart antenna is able to form main bea m towards desired user and null in the direction of i nterfering signals. In this project Direction of ar rival (DOA) is estimated by using MUSIC algorithm. Receive Beam fo r ing is performed by using LMS and LLMS algorithm .In this Paper, in order to perform secure transmission of signal over wireless communication we have used chaotic sequences. This paper evaluates the performance of B am forming with and without LMS and LLMS algorith m for MIMO-OFDM wireless system. The simulations are carr ied out using MATLAB.

A comparison of the Two One-Sided Tests Procedure and the Power Approach for assessing the equivalence of average bioavailability <=SEP=> The statistical test of the hypothesis of no difference between the average bioavailabilities of two drug formulations, usually supplemented by an assessment of what the power of the statistical test would have been if the true averages had been inequivalent, continues to be used in the statistical analysis of bioavailability/bioequivalence studies. In the present article, this Power Approach (which in practice usually consists of testing the hypothesis of no difference at level 0.05 and requiring an estimated power of 0.80) is compared to another statistical approach, the Two One-Sided Tests Procedure, which leads to the same conclusion as the approach proposed by Westlake (2) based on the usual (shortest) 1–2α confidence interval for the true average difference. It is found that for the specific choice of α=0.05 as the nominal level of the one-sided tests, the two one-sided tests procedure has uniformly superior properties to the power approach in most cases. The only cases where the power approach has superior properties when the true averages are equivalent correspond to cases where the chance of concluding equivalence with the power approach when the true averages are notequivalent exceeds 0.05. With appropriate choice of the nominal level of significance of the one-sided tests, the two one-sided tests procedure always has uniformly superior properties to the power approach. The two one-sided tests procedure is compared to the procedure proposed by Hauck and Anderson (1).

Speech recognition for vocalized and subvocal modes of production using surface EMG signals from the neck and face <=SEP=> We report automatic speech recognition accuracy for individual words using eleven surface electromyographic (sEMG) recording locations on the face and neck during three speaking modes: vocalized, mouthed, and mentally rehearsed. An HMM based recognition system was trained and tested on a 65 word vocabulary produced by 9 American English speakers in all three speaking modes. Our results indicate high sEMG-based recognition accuracy for the vocalized and mouthed speaking modes (mean rates of 92.1% and 86.7% respectively), but an inability to conduct recognition on mentally rehearsed speech due to a lack of sufficient sEMG activity.

Avoiding Traffic Jam Using Ant Colony Optimization - A Novel Approach <=SEP=> Ant colony optimization (ACO) is a meta-heuristic based on colony of artificial ants which work cooperatively, building solutions by moving on the problem graph and by communicating through artificial pheromone trails mimicking real ants. One of the active research directions is the application of ACO algorithms to solve dynamic shortest path problems. Solving traffic jams is one such problem where the cost i.e. time to travel increases during rush hours resulting in tremendous strain on daily commuters and chaos. This paper describes a new approach-DSATJ (Dynamic System for Avoiding Traffic Jam) which aims at choosing an alternative optimum path to avoid traffic jam and then resuming that same path again when the traffic is regulated. The approach is inspired by variants of ACO algorithms. Traffic jam is detected through pheromone values on edges which are updated according to goodness of solution on the optimal tours only. Randomness is introduced in the probability function to ensure maximum exploration by ants. Experiments were carried out with the partial road map of North-West region of Delhi, India, to observe the performance of our approach.

Mobile robot localization by tracking geometric beacons <=SEP=> 

A Real-time Inversion Attack on the GMR-2 Cipher Used in the Satellite Phones <=SEP=> The GMR-2 cipher is a kind of stream cipher currently being used in some Inmarsat satellite phones. It has been proven that such cipher can be cracked using only one frame known keystream but with a moderate executing times. In this paper, we present a new thorough security analysis of the GMR-2 cipher. We first study the inverse properties and the relationship of the cipher’s components to reveal a bad one-way character of the cipher. Then by introducing a new concept called “valid key chain” according to the cipher’s key schedule, we for the first time propose a real-time inversion attack using one frame keystream. This attack contains three phases: (1) table generation (2) dynamic table looks-up, filtration and combination (3) verification. Our analysis shows that, using the proposed attack, the exhaustive search space for the 64-bit encryption key can be reduced to about 2 when one frame (15 bytes) keystream is available. Compared with previous known attacks, this inversion attack is much more efficient. Finally, the proposed attack are carried out on a 3.3GHz platform, and the experimental results demonstrate that the 64-bit encryption-key could be recovered in around 0.02s on average.

Finline Ortho-Mode Transducer for Millimeter Waves <=SEP=> We evaluate the possibility of using a finline orthomode transducer (OMT) at millimeter wavelengths. A finline OMT has low loss, low cross-polarization, and good return loss over a full waveguide band. We propose a novel finline OMT structure for millimeter-wavelengths and present results at X-band.

RF modules (Tx-Rx) with multifunctional MMICs <=SEP=> Next generation RF sensor modules for multifunction active electronically steered antenna (AESA) systems will need a combination of different operating modes, such as radar, electronic warfare (EW) functionalities and communications/datalinks within the same antenna frontend. They typically operate in C-Band, X-Band and Ku-Band and imply a bandwidth requirement of more than 10 GHz. For the realisation of modern active electronically steered antennas, the transmit/receive (T/R) modules have to match strict geometry demands. A major challenge for these future multifunction RF sensor modules is dictated by the half-wavelength antenna grid spacing, that limits the physical channel width to < 12 mm or even less, depending on the highest frequency of operation with accordant beam pointing requirements. A promising solution to overcome these geometry demands is the reduction of the total monolithic microwave integrated circuit (MMIC) chip area, achieved by integrating individual RF functionalities, which are commonly achieved through individual integrated circuits (ICs), into new multifunctional (MFC) MMICs. Various concepts, some of them already implemented, towards next generation RF sensor modules will be discussed and explained in this work.

Context-Dependent Sentiment Analysis in User-Generated Videos <=SEP=> Multimodal sentiment analysis is a developing area of research, which involves the identification of sentiments in videos. Current research considers utterances as independent entities, i.e., ignores the interdependencies and relations among the utterances of a video. In this paper, we propose a LSTM-based model that enables utterances to capture contextual information from their surroundings in the same video, thus aiding the classification process. Our method shows 5-10% performance improvement over the state of the art and high robustness to generalizability.

A 3 ppm 1.5 × 0.8 mm 2 1.0 µA 32.768 kHz MEMS-Based Oscillator <=SEP=> This paper describes the first 32 kHz low-power MEMS-based oscillator in production. The primary goal is to provide a small form-factor oscillator (1.5 × 0.8 mm 2 ) for use as a crystal replacement in space-constrained mobile devices. The oscillator generates an output frequency of 32.768 kHz and its binary divisors down to 1 Hz. The frequency stability over the industrial temperature range (-40 °C to 85 °C) is ±100 ppm as an oscillator (XO) or ±3 ppm with optional calibration as a temperature compensated oscillator (TCXO). Supply currents are 0.9 μA for the XO and 1.0 μA for the TCXO at supply voltages from 1.4 V to 4.5 V. The MEMS resonator is a capacitively-transduced tuning fork at 524 kHz. The circuitry is fabricated in 180 nm CMOS and includes low power sustaining circuit, fractional-N PLL, temperature sensor, digital control, and low swing driver.

A dimmable charge-pump ZVS led driver with PFC <=SEP=> This paper presents a dimmable charge-pump driver to power light-emitting diodes (LEDs) with power factor correction (PFC) and Zero Voltage Switching (ZVS). The proposed LED driver does not utilize electrolytic capacitors, providing a high useful lifetime, and it can stabilize the output current in open loop control without needing current sensors, which reduces the cost. The output power is proportional to the switching frequency, which allows the LEDs dimming. A prototype with 22 W was implemented and experimental results were discussed. The prototype presented a power factor of 0.996 and an efficiency of 89.5 %. The driver output power was reduced by more than 40% through the switching frequency while varying of 53 kHz to 30 kHz and the converter has continues to operate in ZVS.

The Bilingual's Language Modes. <=SEP=> GROSJEAN 2 Bilinguals who have reflected on their bilingualism will often report that they change their way of speaking when they are with monolinguals and when they are with bilinguals. Whereas they avoid using their other language with the former, they may call on it for a word or a sentence with the latter or even change over to it completely. In addition, bilinguals will also report that, as listeners, they are sometimes taken by surprise when they are spoken to in a language that they did not expect. Although these reports are quite anecdotal, they do point to an important phenomenon, language mode, which researchers have been alluding to over the years. For example, Weinreich (1966) writes that when speaking to a monolingual, the bilingual is subject to interlocutory constraint which requires that he or she limit interferences (Weinreich uses this as a cover term for any element of the other language) but when speaking to another bilingual, there is hardly any limit to interferences; forms can be transferred freely from one language to the other and often used in an unadapted way. A few years later, Hasselmo (1970) refers to three sets of "norms" or "modes of speaking" among Swedish-English bilinguals in the United States: English only for contact with English monolinguals, American Swedish with some bilinguals (the main language used is Swedish), and Swedish American with other bilinguals (here the main language is English). In the latter two cases, code-switching can take place in the other language. The author also notes that there exists two extremes in the behavior of certain bilinguals, one extreme involves minimal and the other maximal code-switching. A couple of years later, Clyne (1972) talks of three communication possibilities in bilingual discourse: in the first, both codes are used by both speakers; in the second, each one uses a different code but the two understand both codes; and, in the third, only one of the two speakers uses and understands both codes whereas the other speaker is monolingual in one of the codes. Finally, Baetens Beardsmore (1982) echoes these views when he writes that bilinguals in communication with other bilinguals may feel free to use both of their language repertoires. However, the same bilingual speakers in conversation with monoglots may not feel the same liberty and may well attempt to maximize alignment on monoglot norms by consciously reducing any formal "interference" features to …

A Bottom-Up Approach to Sentence Ordering for Multi-Document Summarization <=SEP=> Ordering information is a difficult but important task for applications generating naturallanguage texts such as multi-document summarization, question answering, and conceptto-text generation. In multi-document summarization, information is selected from a set of source documents. However, improper ordering of information in a summary can confuse the reader and deteriorate the readability of the summary. Therefore, it is vital to properly order the information in multi-document summarization. We present a bottom-up approach to arrange sentences extracted for multi-document summarization. To capture the association and order of two textual segments (e.g. sentences), we define four criteria: chronology, topical-closeness, precedence, and succession. These criteria are integrated into a criterion by a supervised learning approach. We repeatedly concatenate two textual segments into one segment based on the criterion, until we obtain the overall segment with all sentences arranged. We evaluate the sentence orderings produced by the proposed method and numerous baselines using subjective gradings as well as automatic evaluation measures. We introduce the average continuity, an automatic evaluation measure of sentence ordering in a summary, and investigate its appropriateness for this task.

The Hitchhiker's Guide to Testing Statistical Significance in Natural Language Processing <=SEP=> Statistical significance testing is a standard statistical tool designed to ensure that experimental results are not coincidental. In this opinion/theoretical paper we discuss the role of statistical significance testing in Natural Language Processing (NLP) research. We establish the fundamental concepts of significance testing and discuss the specific aspects of NLP tasks, experimental setups and evaluation measures that affect the choice of significance tests in NLP research. Based on this discussion, we propose a simple practical protocol for statistical significance test selection in NLP setups and accompany this protocol with a brief survey of the most relevant tests. We then survey recent empirical papers published in ACL and TACL during 2017 and show that while our community assigns great value to experimental results, statistical significance testing is often ignored or misused. We conclude with a brief discussion of open issues that should be properly addressed so that this important tool can be applied in NLP research in a statistically sound manner1.

Mr. Bennet, his coachman, and the Archbishop walk into a bar but only one of them gets recognized: On The Difficulty of Detecting Characters in Literary Texts <=SEP=> Characters are fundamental to literary analysis. Current approaches are heavily reliant on NER to identify characters, causing many to be overlooked. We propose a novel technique for character detection, achieving significant improvements over state of the art on multiple datasets.

Cross-Media Hashing with Neural Networks <=SEP=> Cross-media hashing, which conducts cross-media retrieval by embedding data from different modalities into a common low-dimensional hamming space, has attracted intensive attention in recent years. This is motivated by the facts a) the multi-modal data is widespread, e.g., the web images on Flickr are associated with tags, and b) hashing is an effective technique towards large-scale high-dimensional data processing, which is exactly the situation of cross-media retrieval. Inspired by recent advances in deep learning, we propose a cross-media hashing approach based on multi-modal neural networks. By restricting in the learning objective a) the hash codes for relevant cross-media data being similar, and b) the hash codes being discriminative for predicting the class labels, the learned Hamming space is expected to well capture the cross-media semantic relationships and to be semantically discriminative. The experiments on two real-world data sets show that our approach achieves superior cross-media retrieval performance compared with the state-of-the-art methods.

A HOG-based Real-time and Multi-scale Pedestrian Detector Demonstration System on FPGA <=SEP=> Pedestrian detection will play a major role in future driver assistance and autonomous driving. One powerful algorithm in this field uses HOG features to describe the specific properties of pedestrians in images. To determine their locations, features are extracted and classified window-wise from different scales of an input image. The results of the classification are finally merged to remove overlapping detections. The real-time execution of this method requires specific FPGA- or ASIC-architectures. Recent work focused on accelerating the feature extraction and classification. Although merging is an important step in the algorithm, it is only rarely considered in hardware implementations. A reason for that could be its complexity and irregularity that is not trivial to implement in hardware. In this paper, we present a new bottom-up FPGA architecture that maps the full HOG-based algorithm for pedestrian detection including feature extraction, SVM classification, and multi-scale processing in combination with merging. For that purpose, we also propose a new hardware-optimized merging method. The resulting architecture is highly efficient. Additionally, we present an FPGA-based full real-time and multi-scale pedestrian detection demonstration system.

Mindful consumption : a customer-centric approach to sustainability <=SEP=> How effectively business deals with the challenges of sustainability will define its success for decades to come. Current sustainability strategies have three major deficiencies: they do not directly focus on the customer, they do not recognize the looming threats from rising global over-consumption, and they do not take a holistic approach. We present a framework for a customer-centric approach to sustainability. This approach recasts the sustainability metric to emphasize the outcomes of business actions measured holistically in term of environmental, personal and economic well-being of the consumer. We introduce the concept of mindful consumption (MC) as the guiding principle in this approach. MC is premised on a consumer mindset of caring for self, for community, and for nature, that translates behaviorally into tempering the selfdefeating excesses associated with acquisitive, repetitive and aspirational consumption. We also make the business case for fostering mindful consumption, and illustrate how the marketing function can be harnessed to successfully implement the customer-centric approach to sustainability.

An Efficient Algorithm for Calculating the Exact Hausdorff Distance <=SEP=> The Hausdorff distance (HD) between two point sets is a commonly used dissimilarity measure for comparing point sets and image segmentations. Especially when very large point sets are compared using the HD, for example when evaluating magnetic resonance volume segmentations, or when the underlying applications are based on time critical tasks, like motion detection, then the computational complexity of HD algorithms becomes an important issue. In this paper we propose a novel efficient algorithm for computing the exact Hausdorff distance. In a runtime analysis, the proposed algorithm is demonstrated to have nearly-linear complexity. Furthermore, it has efficient performance for large point set sizes as well as for large grid size; performs equally for sparse and dense point sets; and finally it is general without restrictions on the characteristics of the point set. The proposed algorithm is tested against the HD algorithm of the widely used national library of medicine insight segmentation and registration toolkit (ITK) using magnetic resonance volumes with extremely large size. The proposed algorithm outperforms the ITK HD algorithm both in speed and memory required. In an experiment using trajectories from a road network, the proposed algorithm significantly outperforms an HD algorithm based on R-Trees.

Deep Network Cascade for Image Super-resolution <=SEP=> In this paper, we propose a new model called deep network cascade (DNC) to gradually upscale low-resolution images layer by layer, each layer with a small scale factor. DNC is a cascade of multiple stacked collaborative local auto-encoders. In each layer of the cascade, non-local self-similarity search is first performed to enhance high-frequency texture details of the partitioned patches in the input image. The enhanced image patches are then input into a collaborative local auto-encoder (CLA) to suppress the noises as well as collaborate the compatibility of the overlapping patches. By closing the loop on non-local self-similarity search and CLA in a cascade layer, we can refine the super-resolution result, which is further fed into next layer until the required image scale. Experiments on image super-resolution demonstrate that the proposed DNC can gradually upscale a low-resolution image with the increase of network layers and achieve more promising results in visual quality as well as quantitative performance.