Book Review

Natural language processing (NLP) has become one of the most dynamic and fast developing interdisciplinary research fields over the past decade, which highlights the robustness of computational intelligence. Studies central to NLP modelling have gone through three main stages, specifically, rationalism, empiricism and deep learning. As a core part of machine learning, particularly deep learning, finding good representations with neural networks for natural language is an indispensable component in the paradigmatic shift which moves from computational science towards data-intensive scientific discovery (see, e.g., Szalay and Gray 2006). In this sense, White et al.’s recent work Neural Representations of Natural Language in the Studies in Computational Intelligence series appears to be an interesting publication responding to the needs of researchers who wish to master the techniques of representing different structures in natural language. The volume is a practical introduction to natural language engineering, which elaborates on the development of neural networks in NLP and focuses specifically on word representations, word sense representations and sentence representations. A variety of cutting-edge topics related to language modelling are addressed, which would appeal to a large audience such as NLP researchers, applied linguistics researchers, computer programmers and even general readers showing an interest in artificial intelligence. Notwithstanding the broad spectrum of readership, the volume, as the authors note, is essentially practice-oriented and might require readers to associate the contents of the book with extensive materials (particularly online resources). In addition, potential readers are advised to obtain basic knowledge of linguistics, statistics and computer programming in order to achieve a better understanding of the book. The volume under review is composed of six chapters, which can be divided into two major parts. The first part (Chapters 1 and 2) presents an introduction to neural networks (including recurrent networks), while the second part (Chapters 3 to 5) deals chiefly with representations of different levels. Chapter 1 opens with the introduction to entry knowledge of neural networks required for readers, covering a wide range of machine learning techniques. A neural network is essentially a ‘machine learning algorithm’ (p. 2) and is constructed to ‘represent the transformation of the input to the output as links between neurons’ (p. 3) in a multi-layered architecture, which can be used to approximate functions (Sonada and Murata 2017) by employing hidden layers. A neural network normally includes parameters (e.g., weights and biases) and hyper-parameters (e.g., activation functions and training methods). The authors delineate the mechanism of activation functions such as identity, sigmoid, softmax, tanh and Rectified Linear Unit and explain in detail the different methods of training networks. Well-crafted examples of neural network architectures are also provided with a particular emphasis on classifier and bottlenecking autoencoder. As the authors note, this chapter aims to set out the context for the remaining contents of the book, and therefore, it might not be so valuable for experienced NLP researchers. Chapter 2 narrows down its topic to the introduction of recurrent neural networks (RNNs), a family of deep learning algorithms. In general, an RNN is ‘a chain of feed-forward neural networks, each one being identical in terms of their weight and bias parameters’ (p. 24) and is a type