In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.

Input: Consider Input: The ability of an Evolutionary Algorithm (EA) to find a global optimal solution depends on its capacity to find a good rate between exploitation of found-so-far elements and exploration of the search space. Inspired by natural phenomena, researchers have developed many successful evolutionary algorithms which, at original versions, define operators that mimic the way nature solves complex problems, with no actual consideration of the explorationexploitation balance. In this paper, a novel nature-inspired algorithm called the States of Matter Search (SMS) is introduced. The SMS algorithm is based on the simulation of the states of matter phenomenon. In SMS, individuals emulate molecules which interact to each other by using evolutionary operations which are based on the physical principles of the thermal-energy motion mechanism. The algorithm is devised by considering each state of matter at one different exploration–exploitation ratio. The evolutionary process is divided into three phases which emulate the three states of matter: gas, liquid and solid. In each state, molecules (individuals) exhibit different movement capacities. Beginning from the gas state (pure exploration), the algorithm modifies the intensities of exploration and exploitation until the solid state (pure exploitation) is reached. As a result, the approach can substantially improve the balance between exploration–exploitation, yet preserving the good search capabilities of an evolutionary approach. To illustrate the proficiency and robustness of the proposed algorithm, it is compared to other well-known evolutionary methods including novel variants that incorporate diversity preservation schemes. The comparison examines several standard benchmark functions which are commonly considered within the EA field. Experimental results show that the proposed method achieves a good performance in comparison to its counterparts as a consequence of its better exploration–exploitation balance.

Output: An optimization algorithm inspired by the States of Matter that improves the balance between exploration and exploitation


Input: Consider Input: 1 Ranks, processes and flat grammar ....................................................................................... 2 2 The Rank-Interpretation Architecture for Multilinear Grammars ......................................... 6 2.1 Outline of the Rank-Interpretation Architecture framework .............................................. 6 2.2 A preliminary note on linearity and hierarchy at the phrasal rank ..................................... 7 2.3 Characterisation of the Rank-Interpretation Architecture .................................................. 9 2.3.1 Background ..................................................................................................................... 9 2.3.2 Formal summary ........................................................................................................... 10 2.3.3 Contrast with traditional views of language architecture .............................................. 11 2.3.4 Search for sui generis properties of ranks ..................................................................... 11 2.4 Procedural perspectives on the rank hierarchy ................................................................. 12 3 The discourse rank .............................................................................................................. 15 3.1 The primacy of discourse patterning ................................................................................ 15 3.2 Intonation of an adjacency pair ........................................................................................ 16 3.3 Chanted ‘call’ intonation .................................................................................................. 18 4 The utterance or text rank ................................................................................................... 20 5 The phrase rank ................................................................................................................... 22 5.1 Characteristics of phrasal structure .................................................................................. 22 5.2 Linear sequences, iteration: regular and subregular grammars ........................................ 23 5.3 A note on long-distance and cross-serial dependencies ................................................... 26 5.4 Prosodic-phonetic interpretation at the phrasal rank ........................................................ 27 6 The word rank ..................................................................................................................... 30 6.1 Flat words ......................................................................................................................... 30 6.2 Flat derivations ................................................................................................................. 30 6.3 Flat compounds ................................................................................................................ 31 6.4 Prosodic-phonetic interpretation at the word rank ........................................................... 31 7 Summary and conclusion .................................................................................................... 33 7.1 From Duality to Multilinear Grammar and Rank Interpretation Architecture ................. 33 7.2 Generalisation to stochastic flat linear models ................................................................. 34 7.3 Future work ...................................................................................................................... 35 8 References ........................................................................................................................... 36

Output: Multilinear Grammar: Ranks and Interpretations


Input: Consider Input: We introduce a new framework for unsupervised learning of deep representations based on a novel hierarchical decomposition of information. Intuitively, data is passed through a series of progressively fine-grained sieves. Each layer of the sieve recovers a single latent factor that is maximally informative about multivariate dependence in the data. The data is transformed after each pass so that the remaining unexplained information trickles down to the next layer. Ultimately, we are left with a set of latent factors explaining all the dependence in the original data and remainder information consisting of independent noise. We present a practical implementation of this framework for discrete variables and apply it to a variety of tasks including independent component analysis, lossy and lossless compression, and predicting missing values in data. The hope of finding a succinct principle that elucidates the brain’s information processing abilities has often kindled interest in information-theoretic ideas [1, 2]. In machine learning, on the other hand, the past decade has witnessed a shift in focus towards expressive, hierarchical models with tractable update rules, with successes driven by increasingly effective ways to leverage labeled data to learn rich models [3, 4]. Information-theoretic ideas like the venerable InfoMax principle [5, 6] can be and are applied in both contexts but they do not shed much light on the questions of when and why deep representations are useful for learning. We introduce a novel incremental and hierarchical decomposition of information and show that it defines a framework for unsupervised learning of deep representations in which the contribution of each layer can be precisely quantified. Moreover, this scheme automatically determines the structure and depth among hidden units in the representation based only on local learning rules. The shift in perspective that enables our information decomposition is to focus on how well the learned representation explains multivariate mutual information in the data (a measure originally introduced as “total correlation” [7]). Intuitively, our approach constructs a hierarchical representation of data by passing it through a sequence of progressively fine-grained sieves. At the first layer of the sieve we learn a factor that explains as much of the dependence in the data as possible. The data is then transformed into the “remainder information”, which has this dependence extracted. The next layer of the sieve looks for the largest source of dependence in the remainder information, and the cycle repeats. At each step, we obtain a successively tighter upper and lower bound on the multivariate information in the data, with convergence between the bounds obtained when the remaining information consists of nothing but independent factors. Because we end up with independent factors, one can also view this decomposition as a new way to do independent component analysis (ICA) [8, 9]. Unlike traditional methods, we do not assume a specific generative model of the data (i.e., that it consists of a linear transformation of independent sources) and we extract independent factors incrementally rather than all at once. The implementation we develop here uses only discrete variables and is therefore most relevant for the challenging problem of ICA with discrete variables, which has applications to compression [10]. 1 ar X iv :1 50 7. 02 28 4v 1 [ st at .M L ] 8 J ul 2 01 5 After introducing some background in Sec. 1, we introduce a new way to iteratively decompose the information in data in Sec. 2, and show how to use these decompositions to define a practical and incremental framework for unsupervised representation learning in Sec. 3. We demonstrate the versatility of this framework by applying it first to independent component analysis (Sec. 4). Next, we use the sieve as a lossy compression to mimic the traditional strengths of generative models including in-painting and generating new samples (Sec. 5). Finally, we cast the sieve as a lossless compression and show that it beats standard compression schemes on a benchmark task (Sec. 6). 1 Information-theoretic learning background Using standard notation [11], capital Xi denotes a random variable taking values in some domain and whose instances are denoted in lowercase, xi. In this paper, the domain of all variables are considered to be discrete and finite. We abbreviate multivariate random variables, X ≡ X1:n ≡ X1, . . . , Xn, with an associated probability distribution, pX(X1 = x1, . . . , Xn = xn), which is typically abbreviated to p(x). We will index different groups of multivariate random variables with superscripts, X, as defined in Fig. 1. We let X denote the original observed variables and we often omit the superscript in this case for readability. Entropy is defined in the usual way as H(X) ≡ EX [log 1/p(x)]. We use base two logarithms so that the unit of information is bits. Higher-order entropies can be constructed in various ways from this standard definition. For instance, the mutual information between two groups of random variables, X and Y can be written as the reduction of uncertainty in one variable, given information about the other, I(X;Y ) = H(X)−H(X|Y ). The “InfoMax” principle [5, 6] suggests that for unsupervised learning we should construct Y ’s to maximize their mutual information with X , the data. Despite its intuitive appeal, this approach has several potential problems (see [12] for one example). Here we focus on the fact that the InfoMax principle is not very useful for characterizing “deep representations”, even though it is often invoked in this context [13]. This follows directly from the data processing inequality (a similar argument appears in [14]). Namely, if we start with X , construct a layer of hidden units Y 1 that are a function of X , and continue adding layers to a stacked representation so that X → Y 1 → Y 2 . . . Y , then the information that the Y ’s have about X cannot increase after the first layer, I(X;Y ) = I(X;Y ). From the point of view of mutual information, Y 1 is a copy and Y 2 is just a copy of a copy. While a coarse-grained copy might be useful, the InfoMax principle does not quantify how or why. Instead of maximizing I(X;Y ), the recently introduced principle of total Correlation Explanation (CorEx) [15, 16] suggests to construct Y ’s that explain the multivariate dependence in X according to a multivariate measure of mutual information first introduced as “total correlation” [7]. TC(X) ≡ DKL ( p(x)|| n ∏


Output: The Information Sieve