Question Analysis and Query Expansion
in CS-CS IR4QA
Maofu Liu, Fang Fang, Qing Hu, Jianxun Chen

College of Computer Science & Technology
Wuhan University of Science & Technology

Outline System Architecture Question Analysis
 Query Expansion
 Experiments
 Conclusion2

System Architecture3 Index processing
 Making segmentation for all of the documents to make word-based

index unit and construct inverted index files

 Question analysis
 Analyzing the given question to extract key terms from the question

to form the query expression for the initial retrieval

 Retrieval
 Retrieval model Vector Space Model (VSM) Two retrieval phases
 Initial retrieval
 Re-retrieval

 Question expansion
 After the initial retrieval, the PRF approach will be employed to doquestion expansion

4Question analysis
 The purpose of question analysis is to extract key terms

from the original question to make up of an initial query
expression for the initial retrieval.
 When the user gives a question, the system will segment it
into words . But there may be more than one choice for theq u e s t i o ns e g m e n t a t i o n .
 For example, “诺贝尔奖” can be segmented as “诺贝尔奖”
only as one term or “诺贝尔+奖” as two terms. The systemchose the longer terms in default to do retrieval. If thereturned documents number r is less than s, which denotes
the threshold value when use the key terms to search therelevant documents, the key terms can be segmented into
the shorter ones and retrieve the original document again.
5Question analysis (Cont’d)6

Query expansion
 An effective way to solve term mismatch problem byexpanding the key terms with a certain number of otherrelated terms in the initial query The most well-known query expansion technique is
pseudo-relevance feedback (PRF) and it is widely usedin the open information retrieval evaluation workshops
for improving the retrieval effectiveness Two approaches to implement PRF query expansion
 Co-Occurrence based query expansion approach Metric Correlation based query expansion approach7

Co-Occurrence based query expansion
approach (CO) Deriving from the same idea as LCA(Local ContextAnalysis)
 Some words appear together in a “window”, and the“window” is defined as a document unit in our system.
 We try to seek the co-occurrence of key term q in the queryand word in the top n (n is assigned to 50 in ourexperiment) documents from the initial retrieval. The cooccurrence degree of query term q and word w from eachdocument of the top n documents is calculated by thefollowing equations
cood (w, q | D) = log(tf (w | D) +1.0) × log(tf (q | D) +1.0)
8CO (Cont’d) Employing the following formula to evaluate thecooccurrence degree between the word w and the query Q:
cohd ( w, q | d ) = ∏ ( cood ( w, q | S ) + 1 .0 )q∈Q Some words may have high frequency but low

distinction, we use the inverse document frequency
logarithm to discriminate them and modify the equation
as followsf ( w, Q | C , S ) = ∑ idf ( q | C )idf ( w | C ) log(cood ( w, q | S ) + 1.0)q∈Q The system uses equation to calculate the co-occurrence

degree for each word and chooses the top m words as
expanded terms
9Metric Correlation based query
expansion approach (MC) The metric correlation emphasizes the position correlation of words

in documents.
 If two words tu and tv are in the same document, the distance
between two terms is defined as r(tu, tv ) , which denotes thenumber of words between the two words, then the correlation
between q and w can be defined as follows：1∑ d ∈ S w∑∈ d q∑∈ d r ( q , w )cood ( w , q | S ) =nThe system compute the score for each word and select the words
with higher scores as expanded words
10

Weight of the expanded terms
 A common way is applying the Rocchio equation directly to computethe weight of each term in the new query In our system, we modify this equation, for we consider the score forthe expanded term evaluated by the equation also indicates theimportance of the terms
score(q)w(q | d )w( q | Qnew ) = p ⋅ w( q | Q ) + k ⋅ avg (boost ) ⋅MaxScore Where w(q|Q) is the weight of key term q in the original query, w(q|d)

is the weight of q in document D, p and k and are experimentally
determined positive constants. Boost is one factor as a multiplierbesides the factors tf and idf to compute the weight of the query key
term in the initial query, and the avg(boost) is the average value ofthem. score(q) indicates the score value for each expanded key termand MaxScore is the maximum of them.
11Experiments
Evaluation of experiment results
RunAnalysis File

QE

Mean AP

Mean Q

Mean nDCG

01-TNoMC

0.43160.45100.6763

02-T

No

CO

0.468

0.48790.705103-T

CMUJAV-CS-CS-01-T

CO

0.44120.44520.603104-T

Apath-CS-CS-01-TMC0.42410.42980.613905-DN

CSWHU-CS-CS-03-DN

CO

0.47200.48250.6724“Run” indicates the name of the run file. The suffix “T” indicates thequestion title and “DN” is the description and the narrative of the question.
The “Analysis File” means the question analysis file offered by otherparticipators. “QE” indicates the query expansion approach. In all of therun files, we set 1000 as the limited number of document IDs for each
12question.

In order to analyze the relation between the limited number ofdocuments retrieved for each question and the evaluation result of therun file, we get the experimental data as shown in the following table
where all of the run files employ our own question analysis means andthe CO query expansion approach which the same to the run file 02-T
Evaluation results with different maximal document IDsMAX

Mean AP

Mean Q

Mean nDCG

100

0.38510.37650.55070.7

200

0.4336

0.43370.61710.6

3000.45030.45750.64970.5

4000.45790.46950.66990.4

500

0.46210.47670.68060.2

600

0.4647

0.48140.69020.1

7000.4661

0.48410.69640800

0.46690.48580.7003900

0.4676

0.48720.7034

1000

0.468

0.48790.7051

0.8

0.3

100

200

300

400

Mean AP500

600

Mean Q

700

800

900

1000

Mean nDCG
13

The evaluation results for
ACLIA1-CS-T384 and ACLIA1-CS-T385
RunKey Terms & boost

APQnDCG

CSWHU{(宇宙大爆炸:1.0), (理论:0.3)}

0.77200.83420.9387

Apath{(宇宙大爆炸理论:1.0)

0.39870.39870.5908

CMU{(宇宙:1.0),(大:1.0),(爆炸:1.0),(理论:1.0)}

0.40440.4044

0.5931

NLPAI{(宇宙大爆炸:1.0), (理论:1.0)}

0.40790.49810.7847CSWHU is much better than other approaches for different boost values
RunKey Terms & boost

APQ

nDCG

CSWHU

{(北京大学:1.0),(百年:1.0),(校庆:0.3),(大事:0.3)}

0.04370.04750.2228

NLPAI{(北京大学:1.0), (百年:1.0),(校庆:1.0), (大事:1.0)}0.4283

0.4226

0.6642CSWHU is much worse than other approaches for different boost values
The key terms are set proper scores for the boost values, itwill improve the retrieval performance. On the contray, it will
14
decrease the performance

Conclusion and Future Works The system employs the word-unit based index files and the VSM

information retrieval model. When a question is submitted, the systemanalyze the question firstly to select proper key terms and do the firstphase retrieval. Then it apply the PRF approach to do question expansion
to get more relevant key terms and do the second phase retrieval.
 By experiments, we find that these techniques can reduce the impact of
the term mismatch and enhance the retrieval performance. But there arestill some important factors which are set inappropriately in the system,
such as the better index unit for the index files, the ignorant of part-ofspeech feature and so on. Our future work will make researches on theseissues.
15

Acknowledgment
 The

work in this paper was supported partially by aSpecial Research Grant from Wuhan University ofScience and Technology (No. 50003301) and partiallyby Research Grant from Hubei Provincial Departmentof Education (No. 500064).

 For further information, please contact

liumaofu@wust.edu.cn orfangfang0402@126.com

16