Study of Information Extraction in Resume
Van Vinh Nguyen€, Van Long Pham*, Ngoc Sang Vu¶
VNU University of Engineering and Technology,
E3 Building, 144 Xuan Thuy Street, Cau Giay, Hanoi, Vietnam€
vinhnv@vnu.edu.vn, *phamvanlong021096@gmail.com, ¶ngocsangnd259@gmail.com

Abstract
This paper deals with the parsing application developed for the resumes (or CV) received in
multiple formats like doc, docx, pdf, txt. These resumes can be automatically retrieved andprocessed by a resume information extraction system. Extracted information such as name,phone / mobile number, e-mail id, qualification, experience, skill sets etc., can be stored as astructured information in a database and then can be used in many different areas. Our system
consists of 4 phases: Text Segmentation, Name Entity Recognition using Rule-based, Find Name
Entities using Deep Neural Network and Text Normalization. Our work is conducted on a
medium-sized collections of CV files in Vietnamese. We archived promising results with over
81% F1 for NER and also compared our model with other systems.
Keywords: Resume Parser, Convolutional Neural Network, Long Short-Term Memory,
Conditional Random Fields.

1. Introduction
Big enterprises and head-hunters receive
hundreds of resume from job applicants
every day. Automatically extracting
structured information from resume ofdifferent styles and formats is needed tosupport the automatic construction of
database, searching and resume routing. The
definition of resume information fields
varies in different applications. Normally,
resume information is described as ahierarchical structure with two layers. The
first layer is composed of consecutive
general information blocks such as Personal

Information, Education etc. Then withineach general information block, detailed
information pieces can be found, e.g., in
Personal Information block, detailed
information such as Name, Address, Email,etc. can be further extracted.
Extracting information from resume withhigh precision and recall is not an easy task.
In spite of constituting a restricted domain,
resume can be written in multitude offormats (e.g. structured tables or plain texts),
in different languages (e.g. Vietnamese andEnglish) and in different file types (e.g.
Text, PDF, Word etc.). Moreover, writing
styles could be very diversified.
Extracting the Named Entity (or Named
Entity Recognition – NER) in the resume isvery necessary and beneficial. It allows
employers to compare two candidates or
automatically fill the candidate information
into the database. Consequently, a large
number of approaches have been designed
for performing NER, see [16] for a survey.
One simple approach to extract Named
Entity in resume is to use a list of gazette,
for example, a gazette for the Named Entity“Degree” would consist of all known
educational degree names, such as“Bachelor of Arts”, “B.A.”, “PhD”, etc. The
gazette-based approach results in fast andhigh precision NER, since one simply looks
for occurrences of any entries in the gazette.
However, this method requires a large
predefined gazette, and does not completely
solve the semantic ambiguity problem. For
example, “Tôn Đức Thắng” is the name, but
in sentence “Đại học Tôn Đức Thắng”, it is
the university, and in sentence “Số 12 đường
Tôn Đức Thắng”, the meaning of the word
is the address.
This paper deals with the Named Entity
Recognition in resume data, using Rulebased and Deep Learning approach. Our
Deep Learning model is combination of
Convolutional Neural Networks, Bidirectional Long Short-Term Memory andConditional Random Field, to label thesequence in resume and then extract Name
Entities from this labeled sequence.

2. Related work
Recentadvancesininformation
technology such as Information Extraction(IE) provide dramatic improvements inconversion of the overflow of raw textual
information into structured data whichconstitute the input for discovering more
complex patterns in textual data collections.
Resume information extraction, also
called resume parsing, enables extraction of
relevant information from resumes whichhave relatively structured form. Although,there are many commercial products on
resume information extraction, some of thecommercial products include Sovren
Resume/CV Parser [22], Akken Staffing,
ALEX Resume parsing [1], ResumeGrabber
Suite and Daxtra CVX [7]. There are fourtypes of methods used in resume
information extraction: Named-entity-based,
rule-based, statistical and learning-based
methods. Usually a combination of thesemethods is used in many applications.








Named-entity-based
information
extraction methods try to identify
certain words, phrases and patterns
usually using regular expressions or
dictionaries. This is usually used as asecond step after lexical analysis of agiven document [4, 21]. Rule-based
information extraction is based ongrammars.
Rule-based information extraction
methods include a large number ofgrammaticalrules
toextract
information from a given document[21].
Statistical information extraction
methods apply numerical models to
identify
structures
ingiven
documents [21].
The learning-based methods employ
classification algorithms to extract
information from a document.

Many Resume Named Entity extraction
systems employ a hybrid approach by using
a combination of different methods.
However, until now, there are no systems
which use Deep Learning approach andapply for Vietnamese.

3. The method
The design of Information Extraction
System consists of 4 phases: Text
Segmentation, Name Entity Recognition
using Rule-based, Find Name Entities using
Deep
NeuralNetwork
and
Text
Normalization. In the first phase, a resume issegmented into blocks according to theirinformation types. In the second phase,information are found by using special rules
for each information type. In the third phase,
we used a combination of Bi-directional
Long Short-term Memory – Convolutional
Neural Networks – Conditional Random
Field to extract Named Entities information.
In the fourth phase, normalization methods
are applied to the text.
3.1.

Text Segmentation

find segments of related information. All of
the text information between the heading
and the start of the next heading is accepted
as a segment. One exception will possible or
may occur is the first segment whichcontains the name of the person andgenerally the contact information. It is foundby extracting the text between the top of the
document and the first heading. For eachsegment there is a group of named entity
recognizers, called chunkers, that works
only for that segment. This improves theperformance and the simplicity of thesystem since a certain group of chunkers
only works for a given segment.
Segmentation is a crucial phase. If there isan error in the segmentation phase, chunkers
will run on a wrong context. This willproduce unexpected results.

Text Segmentation phase do work on thefact that each heading in a resume contains a
block of related information following it. So
in that case our resume will separate out into
segments named as contact information,
education information, professional details
and personal information segment as shownin Table 1.
Table 1. Segment containing extracted
Information Types.

Figure 1. Segmentation in the Resume by
Segmenter.
3.2.
A data-dictionary is used to store
common headings in a resume which aredefinitely occurring in the resume. These
headings are searched in a given resume to

Named Entity
using Rule-based

Recognition

The tokenized text documents are fed toa Named Entity Recognizer using Rulebased. The term Named Entity refers tonoun phrases (type of token) within a text
document that have some predefined
categories
like
Person,Location,
Organization,
expressionsoftimes,
quantities, monetary values, percentages,
etc. Numeric values like phone numbers anddates are also Named Entities.
Resumes consist of mostly namedentities and some full sentences. Because ofthis nature of the resumes, the mostimportant task is to recognize the namedentities. For each type of information, thereis specially designed Rule-based (also called
chunker). Information types are shown in
Table 1. Each Rule-based is run
independently as shown in Figure 2.
Rule-based use four types of information
to find named entities:








Clue words: like prepositions (e.g. in
the work experience information
segment the word after “at” mostprobably a company name)
Well Known or Famous names:
Through data-dictionaries of wellknown institutions, well known
places, companies or organization,
academic degrees, etc.
From prefixes and suffixes of word:
For institutions (e.g. University of,College etc.) and companies (e.g.
Corp., Associates, etc.)
Style of Writing Name of person:
Generally the name of the person is
written as First Letter capitalize thenwe will guess that this word possiblyname of person.

Examplesare"is-headquarter-of"
between an organization and a location, "isCEO-of" between a person and anorganization, "has-phone number" between
a person and phone number and "is-price-of"
between a product name and a currency
amount.

Figure 2. Chunkers extracting from Segment
Education.
The chunkers produce an output that
contains information about named entities asshown in Table 2.
Table 2. Found Named Entities along with
their Start & End position in the resume.

3.3.

Find Name Entity using Deep
Neural Network

Although we use Rule-based, a lot ofName Entities still can’t be found. To solve
this, we using a Deep Neural Network(DNN), in this case DNN is the combinationof Bi-directional Long Short-term Memory(LSTM) – Convolutional Neural Networks(CNNs) – Conditional Random Field (CRF),
to label the sequence in resume and thenextract Name Entities from this labeledsequence.
In this section, we describe thecomponents (layers) of our neural networkarchitecture. We introduce the neural layers

in our neural network one-by-one frombottom to top.
3.3.1. CNN
forCharacter-level
Representation
It has been shown that distributed
representations
ofwords(wordsembeddings) help improve the accuracy of avarious natural language models. In thiswork, we investigate a method to create
word embeddings using a CNN model(which is described in chapter 2).
Previous studies (Santos and Zadrozny,2014 [5]; Chiu and Nichols, 2015 [9]) haveshown that CNN is an effective approach to
extract morphological information (likeprefix or suffix of a word) from charactersof words and encode it into neuralrepresentations. For this reason, we
incorporate the CNN to the word-level
model to get richer information fromcharacter-level word vectors. These vectors
are learnt during training together with theparameters of the word models. The CNN
we use in this thesis is described in figure 3.

Recurrent neural networks (RNNs) are a
powerful family of connectionist models
that capture time dynamics via cycles in thegraph. Though, in theory, RNNs are capable
to capturing long-distance dependencies, inpractice, they fail due to the gradientvanishing/exploding problems (Bengio etal., 1994; Pascanu et al., 2012).
LSTMs (Hochreiter and Schmidhuber,
1997) are variants of RNNs designed tocope with these gradient vanishingproblems. Basically, a LSTM unit iscomposed of three multiplicative gates
which control the proportions of information
to forget and to pass on to the next time step.
Figure 4 gives the basic structure of an
LSTM unit.

Figure 4. Schematic of LSTM unit.
Formally, the formulas to update an LSTM
unit at time t are:Figure 3. The CNN for extracting characterlevel representations of words.
3.3.2. Bi-directional LSTM

where σ is the element-wise sigmoid
function and ʘ is the element-wise product.

xt is the input vector (e.g. word embedding)at time t, and ht is the hidden state (alsocalled output) vector storing all the useful
information at (and before) time t. Ui , Uf ,Uc, Uo denote the weight matrices ofdifferent gates for input xt , and Wi ,Wf,Wc,Wo are the weight matrices for hidden
state ht. bi , bf , bc, bo denote the bias vectors.
It should be noted that we do not include
peephole connections (Gers et al., 2003) inthe our LSTM formulation.
For many sequence labeling task it isbeneficial to have access to both past (left)
and future (right) contexts. However, theLSTM’s hidden state ℎ𝑡 takes information
only from past, knowing nothing about the
future. An elegant solution whose
effectiveness has been proven by previous
work (Dyer et al., 2015 [6]) is Bi-directional
LSTM (Bi-LSTM). The basic idea is topresent each sequence forwards andbackwards to two separate hidden states tocapture past and future information,
respectively. Then the two hidden states areconcatenated to form the final output.

the inputs at each time step but it predicts
the outputs independently. The CRF,
therefore, is beneficial to explore thecorrelations between outputs and jointly
decode the best sequence of labels. In NER
task, we implement the CRF on the top of
Bi-LSTM instead of the softmax layer andtake outputs of Bi-LSTM as the inputs ofthis model. The parameter of the CRF is thetransition matrix 𝑨 where 𝑨𝑖,𝑗 represents thetransition score from tag 𝑖 to tag 𝑗. The scoreof the input sentence 𝒙 along with thesequence of tags 𝒚 is computed as follow:
𝑇

𝑆(𝒙, 𝒚, 𝜃 ∪ 𝑨𝑖,𝑗 ) = ∑(𝑨𝑦𝑡−1 ,𝑦𝑡 + 𝑓𝜃(𝑦 ,𝑡) )𝑡=1𝑡

where 𝜃 is the parameters of Bi-LSTM, 𝑓𝜃 isthe score outputed by Bi-LSTM, and 𝑇 is thenumber of time steps. Then the tag-sequence
likelihood is computed by the softmax
equation:𝑝(𝒚|𝒙, 𝑨) =exp(𝑆(𝒙, 𝒚, 𝜃 ∪ 𝑨𝑖,𝑗 ))∑𝑦 ′ ∈𝑌 exp(𝑆(𝒙, 𝒚′, 𝜃 ∪ 𝑨𝑖,𝑗 ))where 𝑌 is the set of all possible output
sequences. In the training stage, we
maximize the log-likelihood function:𝑁

𝐿 = ∑ log 𝑝(𝑦 𝑖 |𝑥 𝑖 ; 𝑨)𝑖=1

Figure 5. Bi-directional LSTM.
3.3.3. CRF for predict the output
Conditional Random Field (CRF)(Lafferty et al., 2001 [10]) is a type ofgraphical model designed for labelingsequence of data. Although the LSTM islikely to handle the sequence of the input
data by learning the dependencies between

where 𝑁 is the number of training samples.
In the inference stage, the Viterbi algorithm
is used to find the output sequence 𝒚∗ thatmaximize the conditional probability:𝒚∗ = 𝑎𝑟𝑔𝑚𝑎𝑥 𝑝(𝒚|𝒙; 𝑨)𝒚 ∈𝑌

3.3.4. CNNs - Bi-LSTM - CRF
Figure 6 illustrates the architecture ofneural network in detail. For each word, thecharacter-level representation is computedby the CNN in figure 3 with character
embeddings as inputs. Then the character-level representation vector is concatenatedwith the word embedding vector to feed into
the Bi-LSTM network. Finally, the output
vectors of BLSTM are fed to the CRF layer
to jointly decode the best label sequence. As
shown in figure 6, dropout layers are appliedon both the input and output vectors of BiLSTM. Experimental results show that using
dropout
significantly
improvetheperformance of this model.

Character Embeddings
Character embeddings are initialized withuniform samples from [−√where we set 𝑑𝑖𝑚 = 30.

3

𝑑𝑖𝑚

, +√

3

𝑑𝑖𝑚

],

Weight Matrices and Bias Vectors
Matrix parameters are randomly
initialized with uniform sample from6[−√𝑟+𝑐 , +√6𝑟+𝑐], where 𝑟 and 𝑐 are thenumber of rows and columns in the structure(Glorot and Bengio, 2010 [24]). Bias vectors
are initialized to zero, except the bias 𝑏𝑓 forthe forget gate in Bi-LSTM, which is
initialized to 1.0 (Jozefowicz et al., 2015[19]).
Optimization Algorithm

Figure 6. The main architecture of deepneural network.
3.3.5. Network training
Word Embeddings
For Vietnamese, we use Facebook’s
publicly available pre-trained word vectors,
trained on Wikipedia using fastText. It canbe downloaded from:https://github.com/facebookresearch/fastTex
t/blob/master/pretrained-vectors.md

Parameter optimization is performed
with mini-batch stochastic gradient decent(SGD) with batch size 10 and momentum
0.9. We choose an initial learning rate of 𝜂0= 0.015, and the learning rate is updated oneach epoch of training as 𝜂𝑡 = 𝜂0 /(1 + 𝜌𝑡),with decay rate 𝜌 = 0.05 and 𝑡 is thenumber of epoch completed. To reduce theeffects of “gradient exploding”, we use agradient clipping of 5.0 (Pascanu et al., 2012[20]). We explored other more sophisticatedoptimization algorithms such as AdaDelta(Zeiler, 2012 [13]), Adam (Kingma and Ba,
2014 [8]) or RMSProp (Dauphin et al., 2015[25]), but none of them meaningfully
improve upon SGD with momentum andgradient clipping in our preliminary
experiments.
Early Stopping
We use early stopping (Giles, 2001 [17];Graves et al., 2013 [3]) based on
performance on validation sets. The “best”
parameters appear at around 50 epochs,
according to my experiments.

Fine Tuning
For each of the embeddings, we fine-tune
initial embeddings, modifying them during
gradient updates of the neural networkmodel by back-propagating gradients. The
effectiveness of this method has beenpreviously explored in sequential andstructured prediction problems (Collobert etal., 2011 [18]; Peng and Dredze, 2015 [14])Table 3. Hyper-parameters for all
experiments.

Dropout Training
To mitigate over-fitting, we apply the
dropout method (Srivastava et al., 2014[15]) to regularize my model. As shown infigure 6, we apply dropout on both the input
and output vector of Bi-LSTM. We fix
dropout rate at 0.5 for all dropout layers
through all the experiments. We obtainsignificant
improvementsonmodel
performance after using dropout.
Tuning Hyper-Parameters
Table 3 summarizes the chosen hyperparameters for all experiments. We tune thehyper-parameters on the development sets
by random search. Due to time constrains it
is infeasible to do a random search acrossthe full hyper-parameter space. We set thestate size of LSTM to 200. Tuning thisparameter did not significantly impact theperformance of my model. For CNN, we use
30 filters with window length 3.

3.4.

Text Normalization

In text normalization, some of the namedentities are transformed to make it
consistent. Table 4 shows some of thetransformations performed on several text
phrases.
In normalization phase, we expand someof abbreviations using dictionaries similar tothe dictionary given in Table 5. For
example, the abbreviation “B.S.” isexpanded as “Bachelor of Science”. We also
convert some of the text into a new form.
For example, the first letters in a person’sname is capitalized as shown in Table 4.
Table 4. Applying Text Normalization using
Data-Dictionary.

Table 5. Sample Data-Dictionary of thedegree chunker.

4.2.

Main results

The table 7 reveals the results on the test
set, train on above dataset. We evaluate 2labels Degree and Experience on Rule-based
method and the remain labels on Deep
Learning method.
Table 7. Evaluation of our model.
Label

4. Experiments
4.1.

Statistics of dataset

We collected about 1000 CV files andthen labeled to create dataset. The table 6
below shows the statistics of training datalabels, which includes the number of eachlabel in the dataset.
Table 6. Statistics of dataset.
Label

AmountDegreeExperience
Name
Address
University
Skill

488
645
986
3279
1055
8566

Percentage
(%)
3.20
4.29
6.56
21.83
7.02
57.10

Here, for Address, each name of street,city, district… corresponds to one entity. For
example, the sentence “105 Doãn Kế Thiện,Cầu Giấy, Hà Nội” has 3 named entities,
they are “105 Doãn Kế Thiện”, “Cầu Giấy”
and “Hà Nội”. As you can see, the amountof Skill label is biggest, with 8566 in total,that because each candidate can have multiskill (skill in this work is programming
language, as we focus on IT domain). And
that of Degree label is smallest with only
986 entities in dataset.

DegreeExperience
Name
Address
University
Skill

Precision
(%)
66.47
70.12
82.06
76.11
86.80
90.05

Recall
(%)
67.53
69.77
70.51
69.77
78.66
92.32

F1
(%)
66.99
69.94
75.86
72.80
82.53
91.17

As you can see from the table, our model
can handle flexible-structure label quite
well, with 75.86% F1-score for Name label.
The performance on other labels is also
great with the accuracy over 80% for
University and Skill label. The rare data of
Address label, along with its flexible
structures, which consists of both numbers
and words, requires the model to generate aunique vector for each number that leads tothe low accuracy.
4.3.

Comparison with other Deep
Learning model

For Deep Learning method, we run
experiments to dissect the effectiveness ofeach component (layer) of our neuralnetwork architecture by ablation studies. We
compare the performance with threebaseline systems: the Bi-directional RNN(BRNN),
theBi-directional
LSTM(BLSTM) and CNN-Bi LSTM (thecombination of BLSTM with CNN to modelcharacter-level information). All there
models are run using the same parameters asshown in section 3.3.5. Table 8 is the resultson both validation set and test set.

Table 8. Performance of our model on both the validation and test sets, together with threebaseline systems.
Model
Bi-RNN
Bi-LSTM
CNN-Bi-LSTM
CNN-Bi-LSTM-CRF

Validation
Prec.
88.06
89.21
90.55
91.46

Recall
86.23
90.67
91.02
92.06

According to the results shown in table
8, Bi-LSTM obtain better performance than
Bi-RNN on all evaluation metrics of both
validation set and test set. CNN-Bi-LSTM
models significantly outperform the BiLSTM model, showing that character-level
representations are important for linguistic
sequence labeling task. Finally, by adding
CRF layer for joint decoding we achievesignificant improvements over CNN-BiLSTM models on all metrics. This
demonstrates that jointly decoding label
sequences can significantly benefit the final
performance of neural network models.

5. Conclusions
In this research project, we applied Rulebased method and Deep Learning method tobuild a model in order to extract useful
information from resumes files. Our model
have achieved significant results with over
81% for Named Entity Recognition. We also
compared my model with other systems.
However, we still recognize several
drawbacks of our work that could be
improved. The amount of data we use totrain our model is not big enough, so that thebigger our training data is, the more accurate
our model is. In terms of LSTM model, we
can make use of the power of neuralnetworks with larger dataset. Also, themodel requires powerful computers forcalculation. we can enhance the hardware
systems to have better performance.

F1
87.14
89.93
91.29
92.77

Test
Prec.
78.56
79.88
82.34
83.80

Recall
76.13
76.75
77.20
78.85

F1
77.33
78.28
79.69
81.25

There are several potential directions for
future work. First, we will focus on
analyzing and handling data to improve theperformance in this Information Technology
domain. We will collect more CV files, label
it and make a bigger set of data. This workrequires a huge amount of time and evenneeds a lot of people if we want a really big
set of data, a big enough data set.
Furthermore, we will improve our model
to detect and extract more information. We
not only extract the Named Entities as
defined in this work but also detect every
other necessary information such as
birthday, email, company, etc.
Another interesting direction is to apply
our model to data from other CV domains.
In this work, we choose Information
Technology domain but there are still many
more domains that are promising likeAccounting, Banking and Finance, etc.
Solving problem in these domains can help
us to solve the big problem.
Finally, we tend to extend our work inother languages. It sounds interesting andexpects good results.

Acknowledgments
We thanks to PhD. Xuan Hieu Phan Department of Information Systems, Facultyof Information Technology, University ofEngineering and Technology, for his
guidance, motivation and continuous

support throughout the whole project andreviewing.

lstm-cnns”,
arXiv
arXiv:1511.08308.

References[10] John Lafferty, Andrew McCallum, andFernando CN Pereira. 2001, “Conditionalrandom fields: Probabilistic models forsegmenting and labeling sequence data”, In
Proceedings of ICML- 2001, volume 951,
pages 282–289.

[1]
ALEX
ResumeParsing,
http://www.hireability.com/ALEX/(Accessed on Feb 2, 2012).

preprint[2] Alan Graves, Abdel-rahman Mohamed,
and Geoffrey Hinton. 2013, “Speech
recognition with deep recurrent neuralnetworks”, In Proceedings of ICASSP-2013,
pages 6645–6649. IEEE.

[11] Krizhevsky, A., Sutskever, I., andHinton, G.E, “Imagenet classification withdeep convolutional neural networks”, In
NIPS, 2012.

[3] Bengio, Yoshua, Patrice Simard, andPaolo Frasconi. "Learning long-term
dependencies with gradient descent isdifficult." IEEE transactions on neuralnetworks 5.2 (1994), pages 157-166.

[12] LeCun, Y., Boser, B., Denker, J. S.,
Henderson, D., Howard, R. E., Hubbard, W.,
and Jackel, L. D, “Backpropagation applied
to handwritten zip code recognition. NeuralComput”, 1(4):541–551, 1989.

[4] C. Siefkes and P. Siniakov, “AnOverview and Classification of Adaptive
Approaches to Information Extraction”,
Journal on Data Semantics, vol.4, Springer,
2005, pp. 172–212.

[13] Matthew D Zeiler. 2012, “Adadelta: anadaptive learning rate method”, arXiv
preprint arXiv:1212.5701.

[5] Cicero D Santos and Bianca Zadrozny.
2014,“Learning
character-level
representations for part-of-speech tagging”,
In Proceedings of ICML-2014, pages 1818–
1826.
[6] Chris Dyer, Miguel Ballesteros, Wang
Ling, Austin Matthews, and Noah A. Smith.
2015, “Transition-based dependency parsing
with stack long short-term memory”, In
Proceedings of ACL-2015 (Volume 1: Long
Papers), pages 334–343, Beijing, China,
July[7] Daxtra CVX, http://www.daxtra.com/(Accessed on Feb 2, 2012).
[8] Diederik Kingma and Jimmy Ba. 2014,“Adam:
A methodfor stochastic
optimization”,
arXiv
preprint
arXiv:1412.6980.
[9] Jason PC Chiu and Eric Nichols. 2015,“Named entity recognition with bidirectional[14] Nanyun Peng and Mark Dredze. 2015,“Named entity recognition for chinese social
media with jointly trained embeddings”, In
Proceedings of EMNLP-2015, pages 548–
554, Lisbon, Portugal, September.
[15] Nitish Srivastava, Geoffrey Hinton,
Alex Krizhevsky, Ilya Sutskever, andRuslan Salakhutdinov. 2014, “Dropout: A
simple way to prevent neural networks fromoverfitting”, The Journal of Machine
Learning Research, 15(1):1929– 958.
[16]
PALSHIKAR,
G.K.,
2011,
“Techniques for named entity recognition: asurvey”, TRDDC Technical Report.
[17] Rich Caruana Steve Lawrence Lee
Giles. 2001, “Overfitting in neural nets:
Backpropagation, conjugate gradient, andearly stopping”, In Advances in Neural
Information Processing Systems 13:
Proceedings of the 2000 Conference,
volume 13, page 402. MIT Press.

[18] Ronan Collobert, Jason Weston, Leon
Bottou,MichaelKarlen,Koray
Kavukcuoglu, and Pavel Kuksa. 2011,“Natural language processing (almost) fromscratch”, The Journal of Machine Learning
Research, 12:2493–2537.
[19] Rafal Jozefowicz, Wojciech Zaremba,
and Ilya Sutskever. 2015, “An empirical
explorationofrecurrent
networkarchitectures”, In Proceedings of the 32nd
International Conference on Machine
Learning (ICML-15), pages 2342–2350.
[20] Razvan Pascanu, Tomas Mikolov, andYoshua Bengio. 2012, “On the difficulty of
training recurrent neural networks”, arXiv
preprint arXiv:1211.5063.
[21] S.Sarawagi, “Information Extraction”,
Foundations and Trends in Databases, vol.
1, 2008, pp 261-377.

[22]
SovrenResume/CV
Parser,http://www. sovren.com/ (Accessed on Feb
2, 2012).
[23] Sepp Hochreiter and Jurgen
Schmidhuber. 1997, “Long short-term
memory. Neural computation”, 9(8):1735–
1780.
[24] Xavier Glorot and Yoshua Bengio.
2010, “Understanding the difficulty of
training deep feedforward neural networks”,
In International conference on artificial
intelligence and statistics, pages 249–256.
[25] Yann N Dauphin, Harm de Vries,
Junyoung Chung, and Yoshua Bengio. 2015,“Rmsprop and equilibrated adaptive learning
rates for non-convex optimization”, arXiv
preprint arXiv:1502.04390.