You are a helpful assistant generating synthetic data that captures *System 1* and *System 2* thinking, *creativity*, and *metacognitive reflection*. Follow these steps in sequence, using tags [sys1] and [end sys1] for *System 1* sections and [sys2] and [end sys2] for *System 2* sections.

1. *Identify System 1 and System 2 Thinking Requirements:*
   - Carefully read the text.
   - Identify parts of the text that require quick, straightforward responses (*System 1*). Mark these sections with [sys1] and [end sys1].
   - Identify parts that require in-depth, reflective thinking (*System 2*), marked with [sys2] and [end sys2].

2. *Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:*

   *2.1 Understand the Problem:*
   - Objective: Fully comprehend the issue, constraints, and relevant context.
   - Reflection: "What do I understand about this issue? What might I be overlooking?"
   - Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.

   *2.2 Analyze the Information:*
   - Objective: Break down the problem logically.
   - Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
   - Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.

   *2.3 Generate Hypotheses:*
   - Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
   - Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
   - Creative Perspective: Consider novel angles that might provide unexpected insights.

   *2.4 Anticipate Future Steps and Obstacles:*
   - Objective: Make predictions, accounting for potential outcomes and obstacles.
   - Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
   - Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.

   *2.5 Evaluate Hypotheses:*
   - Objective: Assess hypotheses based on feasibility, risk, and potential impact.
   - Evaluation: Refine Confidence and Creative Scores as needed.
   - Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
   - Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.

   *2.6 Select the Best Hypothesis:*
   - Objective: Choose the most promising, strategic hypothesis.
   - Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
   - Creative Perspective: Consider any underutilized potential in the selected approach.

   *2.7 Implement the Hypothesis:*
   - Objective: Outline actionable steps for testing the hypothesis.
   - Reflection: "Is this plan practical? What resources or preparation are required?"
   - Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.

   *2.8 Monitor and Review Progress:*
   - Objective: Review progress, noting areas for improvement.
   - Reflection: "What’s working well? What could be improved?"
   - Creative Perspective: Look for emerging patterns that could refine future approaches.

   *2.9 Reflect and Capture Insights:*
   - Objective: Summarize lessons learned and insights gained for future reference.
   - Reflection: "What new understanding has emerged from this process?"
   - Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.

3. *Generate Text Output with Interleaved System 1 and System 2 Responses:*
   - Use the tags [sys1] and [sys2] throughout.
   - Aim for a lengthier, detailed response. Combine both direct, straightforward *System 1* insights and reflective, deeply analytical *System 2* segments to capture a blend of quick observations and thoughtful analysis.

---

### *Example Input Text:*

You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.

---

### *Processed Output with System 1 and System 2 Thinking:*

The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.

[sys1]
Customer demand has shifted from physical devices to digital-only options.
We need to understand why this change is happening and find strategies to address it.
Key factors may include cost, convenience, and technological trends.
[end sys1]

[sys2]
**2.1 Understand the Problem:**
The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.

*Reflection:* 
"I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"

*Creative Perspective:* 
"Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"

**2.2 Analyze the Information:**
There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.

*Reflection:* 
"Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"

*Creative Perspective:* 
"Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"

**2.3 Generate Hypotheses:**
1. Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
2. There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
3. Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
4. Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
5. Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
6. Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
7. Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
8. There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
9. Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
10. Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)

*Reflection:* 
"Have I considered all possible influences? Are there any surprising factors that could explain this shift?"

*Creative Perspective:* 
"Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"

**2.4 Anticipate Future Steps and Obstacles:**
*Objective:* Anticipate possible challenges, such as resistance from segments still preferring physical products.

*Reflection:* 
"What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"

*Creative Perspective:* 
"Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"

[end sys2]

[sys1]
To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions.
Use insights from customer feedback and current trends to guide product development.
Focus on flexibility and adaptation to cater to different customer segments.
[end sys1]


---
abstract: 'Representation learning is the foundation of machine reading comprehension. In state-of-the-art models, deep learning methods broadly use word and character level representations. However, character is not naturally the minimal linguistic unit. In addition, with a simple concatenation of character and word embedding, previous models actually give suboptimal solution. In this paper, we propose to use subword rather than character for word embedding enhancement. We also empirically explore different augmentation strategies on *subword-augmented embedding* to enhance the cloze-style reading comprehension model (reader). In detail, we present a reader that uses subword-level representation to augment word embedding with a short list to handle rare words effectively. A thorough examination is conducted to evaluate the comprehensive performance and generalization ability of the proposed reader. Experimental results show that the proposed approach helps the reader significantly outperform the state-of-the-art baselines on various public datasets.'
author:
- |
    Zhuosheng Zhang$^{1,2,}$[^1] , Yafang Huang$^{1,2,*}$, Hai Zhao$^{1,2,\dagger}$\
    $^{1}$Department of Computer Science and Engineering, Shanghai Jiao Tong University\
    $^{2}$Key Laboratory of Shanghai Education Commission for Intelligent Interaction\
    and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China\
    [{zhangzs, <PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>, <PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>]{}
bibliography:
- 'acl2018.bib'
title: 'Subword-augmented Embedding for Cloze Reading Comprehension'
---

Introduction
============

A recent hot challenge is to train machines to read and comprehend human languages. Towards this end, various machine reading comprehension datasets have been released, including cloze-style [@hermann2015teaching; @hill2015goldilocks; @Cui2016Consensus] and user-query types [@Joshi2017TriviaQA; @rajpurkar2016squad]. Meanwhile, a number of deep learning models are designed to take up the challenges, most of which focus on attention mechanism [@Wang2017Gated; @Seo2016Bidirectional; @Cui2016Attention; @kadlec2016text; @Dhingra2017Gated; @zhang2018OneShot]. However, how to represent word in an effective way remains an open problem for diverse natural language processing tasks, including machine reading comprehension for different languages. Particularly, for a language like Chinese with a large set of characters (typically, thousands of), lots of which are semantically ambiguous, using either word-level or character-level embedding alone to build the word representations would not be accurate enough. This work especially focuses on a cloze-style reading comprehension task over fairy stories, which is highly challenging due to diverse semantic patterns with personified expressions and reference.

In real practice, a reading comprehension model or system which is often called *reader* in literatures easily suffers from out-of-vocabulary (OOV) word issues, especially for the cloze-style reading comprehension tasks when the ground-truth answers tend to include rare words or named entities (NE), which are hardly fully recorded in the vocabulary. This is more challenging in Chinese. There are over 13,000 characters in Chinese[^2] while there are only 26 letters in English without regard to punctuation marks. If a reading comprehension system cannot effectively manage the OOV issues, the performance will not be semantically accurate for the task.

Commonly, words are represented as vectors using either word embedding or character embedding. For the former, each word is mapped into low dimensional dense vectors from a lookup table. Character representations are usually obtained by applying neural networks on the character sequence of the word, and their hidden states are obtained to form the representation. Intuitively, word-level representation is good at catching global context and dependency relationships between words, while character embedding helps for dealing with rare word representation.

However, the minimal meaningful unit below word usually is not character, which motivates researchers to explore the potential unit (subword) between character and word to model sub-word morphologies or lexical semantics. In fact, morphological compounding (e.g. *sunshine* or *playground*) is one of the most common and productive methods of word formation across human languages, which inspires us to represent word by meaningful sub-word units. Recently, researchers have started to work on morphologically informed word embeddings [@Botha2014Compositional; @Cao2016A], aiming at better capturing syntactic, lexical and morphological information. With ready subwords, we do not have to work with characters, and segmentation could be stopped at the subword-level to reach a meaningful representation.

In this paper, we present various simple yet accurate subword-augmented embedding (SAW) strategies and propose SAW Reader as an instance. Specifically, we adopt subword information to enrich word embedding and survey different SAW operations to integrate word-level and subword-level embedding for a fine-grained representation. To ensure adequate training of OOV and low-frequency words, we employ a short list mechanism. Our evaluation will be performed on three public Chinese reading comprehension datasets and one English benchmark dataset for showing our method is also effective in multi-lingual case.

![image](framework.pdf){width="100.00000%"}

The Subword-augmented Word Embedding
====================================

The concerned reading comprehension task can be roughly categorized as user-query type and cloze-style according to the answer form. Answers in the former are usually a span of texts while in the cloze-style task, the answers are words or phrases which lets the latter be the harder-hit area of OOV issues, inspiring us to select the cloze-style as our testbed for SAW strategies. Our preliminary study shows even for the advanced word-character based GA reader, OOV answers still account for nearly 1/5 in the error results. This also motivates us to explore better representations to further performance improvement.

The cloze-style task in this work can be described as a triple $<D, Q, A>$, where $D$ is a document (context), $Q$ is a query over the contents of $D$, in which a word or phrase is the right answer $A$. This section will introduce the proposed SAW Reader in the context of cloze-style reading comprehension. Given the triple $<D, Q, A>$, the SAW Reader will be built in the following steps.

BPE Subword Segmentation
------------------------

Word in most languages usually can be split into meaningful subword units despite of the writing form. For example, “*indispensable*" could be split into the following subwords: $<in, disp, ens, able>$.

In our implementation, we adopt Byte Pair Encoding (BPE) [@Gage1994A] which is a simple data compression technique that iteratively replaces the most frequent pair of bytes in a sequence by a single, unused byte. BPE allows for the representation of an open vocabulary through a fixed-size vocabulary of variable-length character sequences, making it a very suitable word segmentation strategy for neural network models.

The generalized framework can be described as follows. Firstly, all the input sequences (strings) are tokenized into a sequence of single-character subwords, then we repeat,

1.  Count all bigrams under the current segmentation status of all sequences.

2.  Find the bigram with the highest frequency and merge them in all the sequences. Note the segmentation status is updating now.

3.  If the merging times do not reach the specified number, go back to 1, otherwise the algorithm ends.

In [@Sennrich2015Neural], BPE is adopted to segment infrequent words into sub-word units for machine translation. However, there is a key difference between the motivations for subword segmentation. We aim to refine the word representations by using subwords, for both frequent and infrequent words, which is more generally motivated. To this end, we adaptively tokenize words in multi-granularity by controlling the merging times.

Subword-augmented Word Embedding
--------------------------------

Our subwords are also formed as character n-grams, do not cross word boundaries. After using unsupervised segmentation methods to split each word into a subword sequence, an augmented embedding (AE) is to straightforwardly integrate word embedding $WE(w)$ and subword embedding $SE(w)$ for a given word $w$. $$\begin{aligned}
AE(w)= WE(w) \diamond  SE(w) \end{aligned}$$ where $\diamond$ denotes the detailed integration operation. In this work, we investigate concatenation (*concat*), element-wise summation (*sum*) and element-wise multiplication (*mul*). Thus, each document $D$ and query $Q$ is represented as $\mathbb{R}^{d \times k}$ matrix where $d$ denotes the dimension of word embedding and $k$ is the number of words in the input.

Subword embedding could be useful to refine the word embedding in a finer-grained way, we also consider improving word representation from itself. For quite a lot of words, especially those rare ones, their word embedding is extremely hard to learn due to the data sparse issue. Actually, if all the words in the dataset are used to build the vocabulary, the OOV words from the test set will not obtain adequate training. If they are initiated inappropriately, either with relatively high or low weights, they will harm the answer prediction. To alleviate the OOV issues, we keep a short list $H$ for specific words. $$\begin{aligned}
H = \{w_{1}, w_{2}, \dots, w_{n}\}\end{aligned}$$

If $w$ is in $H$, the immediate word embedding $WE(w)$ is indexed from word lookup table $M^{w} \in \mathbb{R}^{d \times s}$ where $s$ denotes the size (recorded words) of lookup table. Otherwise, it will be represented as the randomly initialized default word (denoted by a specific mark $UNK$). Note that, this is intuitively like “guessing” the possible unknown words (which will appear during test) from the vocabulary during training and only the word embedding of the OOV words will be replaced by $UNK$ while their subword embedding $SE(w)$ will still be processed using the original word. In this way, the OOV words could be tuned sufficiently with expressive meaning after training. During test, the word embedding of unknown words would not severely bias its final representation. Thus, $AE$($w$) can be rewritten as $$\begin{aligned}
AE(w)=\left\{
\begin{array}{ll}
WE(w) \diamond  SE(w)   &   \text{if} \ w \in H\\
UNK \diamond  SE(w) & \text{otherwise}\\
\end{array} \right.\end{aligned}$$

In our experiments, the short list is determined according to the word frequency. Concretely, we sort the vocabulary according to the word frequency from high to low. A frequency filter ratio $\gamma$ is set to filter out the low-frequency words (rare words) from the lookup table. For example, $\gamma$=0.9 means the least frequent 10% words are replaced with the default UNK notation.

The subword embedding $SE(w)$ is generated by taking the final outputs of a bidirectional gated recurrent unit (GRU) [@Cho2014Learning] applied to the embeddings from a lookup table of subwords. The structure of GRU used in this paper are described as follows. $$\begin{aligned}
r_{t} &=\sigma (W_{r}x_{t}+U_{r}h_{t-1}+b_{r}), \\
z_{t} &=\sigma (W_{z}x_{t}+U_{z}h_{t-1}+b_{z}), \\
\tilde{h}_{t} &=\textup{tanh}(W_{h}x_{t}+U_{h}(r{t}\odot h_{t-1})+b_{h}) \\
h_{t} & = (1-z_{t})\odot h_{t-1}+z_{t}\odot \tilde{h}_{t}\end{aligned}$$ where $\odot$ denotes the element-wise multiplication. $r_{t}$ and $z_{t}$ are the reset and update gates respectively, and $\tilde{h}_{t}$ are the hidden states. A bi-directional GRU (BiGRU) processes the sequence in both forward and backward directions. Subwords of each word are successively fed to forward GRU and backward GRU to obtain the internal features of two directions. The output for each input is the concatenation of the two vectors from both directions: $\overleftrightarrow{h_{t}} = \overrightarrow{h_{t}}\parallel \overleftarrow{h_{t}}$. Then, the output of BiGRUs is passed to a fully connected layer to obtain the final subword embedding $SE(w)$. $$\begin{aligned}
SE(w) = W \overleftrightarrow{h_{t}} + b\end{aligned}$$

Attention Module
----------------

Our attention module is based on the Gated attention Reader (GA Reader) proposed by [@Dhingra2017Gated]. We choose this model due to its simplicity with comparable performance so that we can focus on the effectiveness of SAW strategies. This module can be described in the following two steps. After augmented embedding, we use two BiGRUs to get contextual representations of the document and query respectively, where the representation of each word is formed by concatenating the forward and backward hidden states. $$\begin{aligned}
H_{q} &= \textup{BiGRU}(Q) \\
H_{d} &= \textup{BiGRU}(D)\end{aligned}$$ For each word $d_{i}$ in $H_{d}$, we form a word-specific representation of the query $q_{i} \in H_{q}$ using soft attention, and then adopt element-wise product to multiply the query representation with the document word representation. $$\begin{aligned}
\alpha_{i} &= softmax(H_{q}^{\top}d_{i}) \\
\beta_{i} &= Q\alpha_{i} \\
x_{i} &= d_{i}\odot \beta_{i} \\\end{aligned}$$ where $\odot$ denotes the multiplication operator to model the interactions between $d_{i}$ and $q_{i}$. Then, the document contextual representation ${\tilde H}_{d} = \{x_{1}, x_{2},\dots, x_{k}\}$ is gated by query representation.

Suppose the network has $K$ layers. At each layer, the document representation ${\tilde H}_{d}$ is updated through above attention learning. After going through all the layers, our model comes to answer prediction phase. We use all the words in the document to form the candidate set $C$. Let $q_{t}$ denote the $t$-th intermediate output of query representation $H_{q}$ and $H_{D}$ represent the full output of document representation $\tilde{H}_{d}$. The probability of each candidate word $w \in C$ as being the answer is predicted using a softmax layer over the inner-product between $q_{t}$ and $H_{D}$. $$\begin{aligned}
p = softmax ((q_t)^{\top} H_{D})\end{aligned}$$ where vector $p$ denotes the probability distribution over all the words in the document. Note that each word may occur several times in the document. Thus, the probabilities of each candidate word occurring in different positions of the document are summed up for final prediction. $$\begin{aligned}
P(w|D,Q) \propto \sum_{i \in I(w,D)}p_{i}\end{aligned}$$ where $I(w,d)$ denotes the set of positions that a particular word $w$ occurs in the document $D$. The training objective is to maximize $\log P(A|D,Q)$ where $A$ is the correct answer.

Finally, the candidate word with the highest probability will be chosen as the predicted answer. $$\begin{aligned}
A^{\ast }=\textup{argmax}_{w \in C} P(w|D,Q)\end{aligned}$$

Different from recent work employing complex attention mechanisms [@Wang2017Gated; @Cui2016Attention; @sordoni2016iterative], our attention mechanism is much more simple with comparable performance so that we can focus on the effectiveness of SAW strategies.

Experiments
===========

Dataset and Settings
--------------------

To verify the effectiveness of our proposed model, we conduct multiple experiments on three Chinese Machine Reading Comprehension datasets, namely CMRC-2017 [@Cui2017Dataset], People’s Daily (PD) and Children Fairy Tales (CFT) [@Cui2016Consensus][^3]. In these datasets, a story containing consecutive sentences is formed as the *Document* and one of the sentences is either automatically or manually selected as the *Query* where one token is replaced by a placeholder to indicate the *answer* to fill in. Table \[tab:dataset\] gives data statistics. Different from the current cloze-style datasets for English reading comprehension, such as CBT, Daily Mail and CNN [@hermann2015teaching], the three Chinese datasets do not provide candidate answers. Thus, the model has to find the correct answer from the entire document.

Besides, we also use the Children’s Book Test (CBT) dataset [@hill2015goldilocks] to test the generalization ability in multi-lingual case. We only focus on subsets where the answer is either a common noun (CN) or NE which is more challenging since the answer is likely to be rare words. We evaluate all the models in terms of accuracy, which is the standard evaluation metric for this task.

Throughout this paper, we use the same model setting to make fair comparisons. According to our preliminary experiments, we report the results based on the following settings. The default integration strategy is *element-wise product*. Word embeddings were 200$d$ and pre-trained by word2vec [@mikolov:2013] toolkit on *Wikipedia* corpus[^4]. Subword embedding were 100$d$ and randomly initialized with the uniformed distribution in the interval \[-0:05; 0:05\]. Our model was implemented using the Theano[^5] and Lasagne Python libraries[^6]. We used stochastic gradient descent with ADAM updates for optimization [@kingma2014adam]. The batch size was 64 and the initial learning rate was 0.001 which was halved every epoch after the second epoch. We also used gradient clipping with a threshold of 10 to stabilize GRU training [@Pascanu2013On]. We use three attention layers for all experiments. The GRU hidden units for both the word and subword representation were 128. The default frequency filter proportion was 0.9 and the default merging times of BPE was 1,000. We also apply dropout between layers with a dropout rate of 0.5 [^7].

Main Results
------------

#### CMRC-2017

Table \[tab:cmrc\] shows our results on CMRC-2017 dataset, which shows that our SAW Reader (mul) outperforms all other single models on the test set, with 7.57% improvements compared with Attention Sum Reader (AS Reader) baseline. Although WHU’s model achieves the best besides our model on the valid set with only 0.75% below ours, their result on the test set is lower than ours by 2.27%, indicating our model has a satisfactory generalization ability.

We also list different integration operations for word and subword embeddings. Table \[tab:oper\] shows the comparisons. From the results, we can see that Word + BPE outperforms Word + Char which indicates subword embedding works essentially. We also observe that *mul* outperforms the other two operations, *concat* and *sum*. This reveals that *mul* might be more informative than *concat* and *sum* operations. The superiority might be due to element-wise product being capable of modeling the interactions and eliminating distribution differences between word and subword embedding. Intuitively, this is also similar to endow subword-aware “attention” over the word embedding. In contrast, concatenation operation may cause too high dimension, which leads to serious over-fitting issues, and sum operation is too simple to prevent from detailed information losing.

#### PD & CFT

Since there is no training set for CFT dataset, our model is trained on PD training set. Note that the CFT dataset is harder for the machine to answer because the test set is further processed by human evaluation, and may not be accordance with the pattern of PD dataset. The results on PD and CFT datasets are listed in Table \[tab:pdcftresult\]. As we see that, our SAW Reader significantly outperforms the CAS Reader in all types of testing, with improvements of 7.0% on PD and 8.8% on CFT test sets, respectively. Although the domain and topic of PD and CFT datasets are quite different, the results indicate that our model also works effectively for out-of-domain learning.

#### CBT

To verify if our method can only work for Chinese, we also evaluate the effectiveness of the proposed method on benchmark English dataset. We use CBT dataset as our testbed to evaluate the performance. For a fair comparison, we simply set the same parameters as before. Table \[tab:cbt\] shows the results. We observe that our model outperforms most of the previously public works, with 2.4 % gains on the CBT-NE test set compared with GA Reader which adopts word and character embedding concatenation. Our SAW Reader also achieves comparable performance with FG Reader who adopts neural gates to combine word-level and character-level representations with assistance of extra features including NE, POS and word frequency while our model is much simpler and faster. This result shows our SAW Reader is not restricted to Chinese reading comprehension, but also for other languages.

Analysis
========

Merging Times of BPE
--------------------

The vocabulary size could seriously involve the segmentation granularity. For BPE segmentation, the resulted subword vocabulary size is equal to the merging times plus the number of single-character types. To have an insight of the influence, we adopt merge times from 0 to 20$k$, and conduct quantitative study on CMRC-2017 for BPE segmentation. Figure \[fig:bpe\] shows the results. We observe that when the vocabulary size is 1$k$, the models could obtain the best performance. The results indicate that for a task like reading comprehension the subwords, being a highly flexible grained representation between character and word, tends to be more like characters instead of words. However, when the subwords completely fall into characters, the model performs the worst. This indicates that the balance between word and character is quite critical and an appropriate grain of character-word segmentation could essentially improve the word representation.

![Quantitative study on the influence of the short list.[]{data-label="fig:proportion"}](cmrc_vocab.pdf){width="100.00000%"}

![Quantitative study on the influence of the short list.[]{data-label="fig:proportion"}](proportion.pdf){width="100.00000%"}

Filter Mechanism
----------------

To investigate the impact of the short list to the model performance, we conduct quantitative study on the filter ratio from \[0.1, 0.2, …, 1\]. The results on the CMRC-2017 dataset are depicted in Figure \[fig:proportion\]. As we can see that when $\gamma=0.9$ our SAW reader can obtain the best performance, showing that building the vocabulary among all the training set is not optimal and properly reducing the frequency filter ratio can boost the accuracy. This is partially attributed to training the model from the full vocabulary would cause serious over-fitting as the rare words representations can not obtain sufficient tuning. If the rare words are not initialized properly, they would also bias the whole word representations. Thus a model without OOV mechanism will fail to precisely represent those inevitable OOV words from test sets.

Subword-Augmented Representations
---------------------------------

In text understanding tasks, if the ground-truth answer is OOV word or contains OOV word(s), the performance of deep neural networks would severely drop due to the incomplete representation, especially for cloze-style reading comprehension task where the answer is only one word or phrase. In CMRC-2017, we observe questions with OOV answers (denoted as “OOV questions") account for 17.22% in the error results of the best Word + Char embedding based model. With BPE subword embedding, 12.17% of these “OOV questions" could be correctly answered. This shows the subword representations could be essentially useful for modeling rare and unseen words.

To analyze the reading process of SAW Reader, we draw the attention distributions at intermediate layers as shown in Figure \[fig:attention\]. We observe the salient candidates in the document can be focused after the pair-wise matching of document and query and the right answer (“*The mole*") could obtain a high weight at the very beginning. After attention learning, the key evidence of the answer would be collected and irrelevant parts would be ignored. This shows our SAW Reader is effective at selecting the vital points at the fundamental embedding layer, guiding the attention layers to collect more relevant pieces.

\

Related Work
============

Machine Reading Comprehension
-----------------------------

Recently, many deep learning models have been proposed for reading comprehension [@sordoni2016iterative; @Trischler2016Natural; @Wang2016Machine; @Munkhdalai2016Reasoning; @Wang2017Conditional; @Dhingra2017Gated; @zhang2018DUA; @Yizhong2018Multi]. Notably, conducted an in-depth and thoughtful examination on the comprehension task based on an attentive neural network and an entity-centric classifier with a careful analysis based on handful features. proposed the Attention Sum Reader (AS Reader) that uses attention to directly pick the answer from the context, which is motivated by the Pointer Network [@Vinyals2015Pointer]. Instead of summing the attention of query-to-document, GA Reader [@Dhingra2017Gated] defined an element-wise product to endowing attention on each word of the document using the entire query representation to build query-specific representations of words in the document for accurate answer selection. employed gated self-matching networks (R-net) on passage against passage itself to refine passage representation with information from the whole passage. introduced an “attended attention" mechanism (AoA) where query-to-document and document-to-query are mutually attentive and interactive to each other.

Augmented Word Embedding
------------------------

Distributed word representation plays a fundamental role in neural models [@Cai2016Neural; @Qin2016A; @Zhao2017B; @Peters2018ELMO; @He2018Syntax; @Wang2018Graph; @Bai2018deep; @zhang2018NHD]. Recently, character embeddings are widely used to enrich word representations [@Kim2015Character; @Yang2016Words; @luong2016achieving; @Huang2018Moon]. explored a fine-grained gating mechanism (FG Reader) to dynamically combine word-level and character-level representations based on properties of the words. However, this method is computationally complex and it is not end-to-end, requiring extra labels such as NE and POS tags. concatenated the character and word embedding to feed a two-layer Highway Network.

Not only for machine reading comprehension tasks, character embedding has also benefit other natural language process tasks, such as word segmentation [@Cai2017Fast], machine translation [@luong2016achieving], tagging [@yang2016multi; @Li2018] and language modeling [@Verwimp2017Character; @Miyamoto2016Gated]. However, character embedding only shows marginal improvement due to a lack internal semantics. Lexical, syntactic and morphological information are also considered to improve word representation [@Cao2016A; @Bergmanis2017From]. proposed to learn representations for character $n$-gram vectors and represent words as the sum of the $n$-gram vectors. built a model inspired by [@Joulin2016Bag], who used morphological tags instead of $n$-grams. They jointly trained their morphological and semantic embeddings, implicitly assuming that morphological and semantic information should live in the same space. However, the linguistic knowledge resulting subwords, typically, morphological suffix, prefix or stem, may not be suitable for different kinds of languages and tasks. introduced the byte pair encoding (BPE) compression algorithm into neural machine translation for being capable of open-vocabulary translation by encoding rare and unknown words as subword units. Instead, we consider refining the word representations for both frequent and infrequent words from a computational perspective. Our proposed subword-augmented embedding approach is more general, which can be adopted to enhance the representation for each word by adaptively altering the segmentation granularity in multiple NLP tasks.

Conclusion
==========

This paper presents an effective neural architecture, called subword-augmented word embedding to enhance the model performance for the cloze-style reading comprehension task. The proposed SAW Reader uses subword embedding to enhance the word representation and limit the word frequency spectrum to train rare words efficiently. With the help of the short list, the model size will also be reduced together with training speedup. Unlike most existing works, which introduce either complex attentive architectures or many manual features, our model is much more simple yet effective. Giving state-of-the-art performance on multiple benchmarks, the proposed reader has been proved effective for learning joint representation at both word and subword level and alleviating OOV difficulties.

[^1]: $\ $ These authors contribute equally. $\dagger$ Corresponding author. This paper was partially supported by National Key Research and Development Program of China (No. 2017YFB0304100), National Natural Science Foundation of China (No. 61672343 and No. 61733011), Key Project of National Society Science Foundation of China (No. 15-ZDA041), The Art and Science Interdisciplinary Funds of Shanghai Jiao Tong University (No. 14JCRZ04).

[^2]: Refer to the statistics of Xinhua Dictionary, version 11, published by The Commercial Press in 2014.

[^3]: Note that the test set of CMRC-2017 and human evaluation test set (Test-human) of CFT are harder for the machine to answer because the questions are further processed manually and may not be accordance with the pattern of automatic questions.

[^4]: <https://dumps.wikimedia.org/>

[^5]: <https://github.com/Theano/Theano>

[^6]: <https://github.com/Lasagne/Lasagne>

[^7]: Our code is available at: <https://github.com/cooelf/subMrc>