SlowGuess/ACL_2020_OCR huggingface dataset:
7During training (with teacher forcing),  $w_{d}^{t}$  is the ground truth target token at previous time-step  $t - 1$ ; during evaluation,  $w_{d}^{t} = y^{t - 1}$ , is the prediction at the previous time-step.
7971
Kp20K	Inspec	Krapivin	NUS	SemEval
Model	R@10	R@50	R@10	R@50	R@10	R@50	R@10	R@50	R@10	R@50
CopyRNN (Meng et al., 2017)	11.5	18.9	5.1	10.1	11.6	19.5	7.8	14.4	4.9	7.5
CopyRNN* (Meng et al., 2017)	3.3	8.7	4.0	8.3	4.0	8.1	2.4	8.1	0.5	2.6
CatSeq (ours)	6.0	6.2	2.8	2.9	7.0	7.4	3.7	3.1	2.5	2.5
CatSeqD (ours)	11.7	15.1	5.2	7.1	12.0	14.5	8.4	11.0	4.6	6.3
C STACKEx Data Collection
We download the public data dump from https://archive.org/details/stackexchange, and choose 19 computer science related topics from Oct. 2017 dump. We select computer science forums (CS/AI), using "title" + "body" as source text and "tags" as the target keyphrases. After removing questions without valid tags, we collect 330,965 questions. We thus randomly select 16,000 for validation, and another 16,000 as test set. Note some questions in StackExchange forums contain large blocks of code, resulting in long texts (sometimes more than 10,000 tokens after tokenization), this is difficult for most neural models to handle. Consequently, we truncate texts to 300 tokens and 1,000 tokens for training and evaluation splits respectively.
Implementation details of our proposed models are as follows. In all experiments, the word embeddings are initialized with 100-dimensional random matrices. The number of hidden units in both the encoder and decoder GRU are 150. The number of hidden units in target encoder GRU is 150. The size of vocabulary is 50,000. In all experiments, we use a dropout rate of 0.1.
The numbers of hidden units in MLPs described in Section 3 are as follows. During negative sampling, we randomly sample 16 samples from the same batch, thus target encoding loss in Equation 2 is a 17-way classification loss. In CatSeqD, we select both  $\lambda_{\mathrm{OR}}$  and  $\lambda_{\mathrm{SC}}$  in Equation 4 from [0.01, 0.03, 0.1, 0.3, 1.0] using validation sets. The selected values are listed in Table 7.
We use Adam (Kingma and Ba, 2014) as the step rule for optimization. The learning rate is  $1e^{-3}$ . The model is implemented using PyTorch (Paszke et al., 2017) and OpenNMT (Klein et al., 2017).
For exhaustive decoding, we use a beam size of 50 and a maximum sequence length of 40.
Table 6: Performance of absent keyphrase prediction on scientific publications datasets. Best/second-best performing score in each column is highlighted with bold/underline.
Experiment Setting	λOR	λSC
Table 2	1.0	0.03
Table 3	0.03	0.1
Table 4, KP20K	Greedy	1.0	0.3
Table 4, KP20K	Top Rank	1.0	0.3
Table 4, STACKEX	Greedy	1.0	0.3
Table 4, STACKEX	Top Rank	1.0	0.3
Table 5, CatSeq + Orth. Reg.	0.3	0.0
Table 5, CatSeq + Sem. Cov.	0.0	0.03
Table 5, CatSeqD	Same as Table 2
Table 6	Same as Table 2
Table 7: Semantic coverage and orthogonal regularization coefficients.
Following Meng et al. (2017), lowercase and stemming are performed on both the ground truth and generated keyphrases during evaluation.
We leave out 2,000 data examples as validation set for both KP20K and STACKEX and use them to identify optimal checkpoints for testing. And all the scores reported in this paper are from checkpoints with best performances  $(\mathbf{F}_1@O)$  on validation set.
In Section 6.2, we use the default parameters for t-SNE in sklearn (learning rate is 200.0, number of iterations is 1000, as defined in  $^8$ ).
E Dataset and Evaluation Details
We strictly follow the data pre-processing and evaluation protocols provided by Meng et al. (2017).
We pre-process both document texts and ground-truth keyphrases, including word segmentation, lowercasing and replacing all digits with symbol  $<\mathrm{digit}>$ . In the datasets, examples with empty ground-truth keyphrases are removed.
8https://scikit-learn.org/stable/ modules/generated/sklearn.manifold.TSNE. html
7972
We evaluate models' performance on predicting present and absent phrases separately. Specifically, we first lowercase the text, then we determine the presence of each ground-truth keyphrase by checking whether it is a sub-string of the source text (we use Porter Stemmer 9). To evaluate present phrase performance, we compute Precision/Recall/F1-score (see 14-16 for formulas) for each document taking only present ground-truth keyphrases as target and ignore the absent ones.
$$
P @ k = \frac {\# (\text {c o r r e c t} @ k)}{\min  \{k , \# (p r e d) \}} \tag {14}
$$
$$
R = \frac {\# (\text {c o r r e c t} @ k)}{\# (\text {t a r g e t})} \tag {15}
$$
$$
F _ {1} @ k = \frac {2 * P @ k * R}{P @ k + R} \tag {16}
$$
where  $\# (pred)$  and  $\# (target)$  are the number of predicted and ground-truth keyphrases respectively; and  $\# (correct@k)$  is the number of correct predictions among the first  $k$  results.
We report the macro-averaged scores over documents that have at least one present ground-truth phrases (corresponding to the column #PreDoc in Table 8), and similarly to the case for absent phrase evaluation.
F Examples of KP20K and STACKEX with Model Prediction
See Table 9 and Figure 3.
$^{9}$ https://www.nltk.org/api/nltk.stem.html#module-nltk.stem.porter
7973
Dataset	#Doc	#KP	#PreDoc	#PreKP	#AbsDoc	#AbsKP
KP20K	19,987	105,181	19,048	66,595	16,357	38,586
INSPEC	500	4,913	497	3,858	381	1,055
KRAPIVIN	460	2,641	437	1,485	417	1,156
NUS	211	2,461	207	1,263	195	1,198
SEMEVAL	100	1,507	100	671	99	836
STACKEX	16,000	43,131	13,475	24,809	10,984	18,322
DUC	308	2,484	308	2,421	38	63
Table 8: Statistics on number of documents and keyphrases of each test set. #Doc#KP denotes the number of documents/ground-truth keyphrases in the dataset. #PreKP/#AbsKP denotes the number of present/absent ground-truth keyphrases, and #PreDoc/#AbsDoc denotes the number of documents that contain at least one present/absent ground-truth keyphrase.
Source	Integration of a Voice Recognition System in a Social Robot Human-robot interaction (HRI) (1) is one of the main fields in the study and research of robotics. Within this field, dialogue systems and interaction by voice play an important role. When speaking about human-robot natural dialogue we assume that the robot has the capability to accurately recognize what the human wants to transmit verbally and even its semantic meaning, but this is not always achieved. In this article we describe the steps and requirements that we went through in order to endow the personal social robot Maggie , developed at the University Carlos III of Madrid, with the capability of understanding the natural language spoken by any human. We have analyzed the different possibilities offered by current software/hardware alternatives by testing them in real environments. We have obtained accurate data related to the speech recognition capabilities in different environments, using the most modern audio acquisition systems and analyzing not so typical parameters such as user age, gender, intonation, volume, and language. Finally, we propose a new model to classify recognition results as accepted or rejected, based on a second automatic speech recognition (ASR ) opinion.This new approach takes into account the precalculated success rate in noise intervals for each recognition framework, decreasing the rate of false positives and false negatives.
CatSeq	voice recognition system ; social robot ; human robot interaction ; voice recognition ; hri ; speech recognition ; automatic speech recognition ; noise intervals ; noise ; human robot ; automatic speech ; natural language
CatSeqD	human robot interaction ; voice recognition ; social robotics ; social robots ; integration ; speech recognition ; hri ; social robot ; robotics ; voice recognition system ; recognition ; asr ; automatic speech recognition ;
Ground Truth	asr ; automatic speech recognition ; dialogue ; human robot interaction ; maggie ; social robot ; speech recognition ; voice recognition ;
Table 9: Example from KP20K validation set, and predictions generated by CatSeq and CatSeqD models.
7974
Asked 5 years, 6 months ago Active 5 years, 6 months ago Viewed 1k times
Figure 3: Example from the STACKEX dataset, we show the screenshot of the original web page to better present the example. Note the input to the model is the entire question (including the code), we removed the format information in the dataset. Also note on the bottom of the screenshot it shows the 3 keyphrases (in this example all absent) which we collected as the ground-truth keyphrases in our dataset.
Ground Truth: javascript ;jquery ; event handling
CatSeq Prediction: javascript; c#;jquery; php; linq; comparative review; ecmascript 6; asp . js; beginner; strings; performance; datetime
CatSeqD Prediction: javascript ;jquery ; performance ; event handling ; array ; twitter bootstrap ; beginner ; algorithm ; indexarray ; optimization ; event programming ; datetime ; comparative review ; ecmascript 6 ; indexof ; dry ; php ; r ; java ; coffeescript ; combinatorics ; dom ; html ; event tracking ; strings ; python ; ruby ; natural language processing ; animation ; angular . js ; homework ; parameters ;jquery ui ; functional programming ; google app engine ; . net ; python 2 . 7 ; c# ; php5 ; validation ; regex ; parsing ; formatting ; hash table ; object oriented ; web scraping ; python 3 . x ; python 3 . x programming ; python 2 . net ; python 2 . 6 ; python 2 . sql ; mysql ; object oriented design ; actionscript
Creating a function when user clicks on element to toggle another element
7975