1KJ.n

Summary Statistics

Summary Statistics

Run ID

1KJ.n

Task

monolingual

Relevant Level
Run Description

Relevant
automatic,short without concepts

Number of Topics

39

Run ID

1KJ.n

Task

monlingual

Relevant Level
Run Description

Partial Relevant
automatic,short without concepts

Number of Topics

Total number of documents over all topics

39

Total number of documents over all topics

Retrieved:

39000

Retrieved:

39000

Relevant:

1734

Relevant:

2101

Rel-ret:1118

Rel-ret:869

Document Level Averages
Precision
At 5 docs
0.4103
At 10 docs
0.3513
At 15 docs
0.3094
At 20 docs
0.2731
At 30 docs
0.2316
At 100 docs
0.1151
At 200 docs
0.0726
At 500 docs
0.0377
At 1000 docs
0.0223R-Presicion (presicion after
R docs retreived (where R is
the number of relevant
documnets))Exact
0.2498Difference

Precision

0.6
0.4
0.2
00

0.2

0.4 0.6
Recall0.8

Recall-Precision Curve

11
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1

1
0.8
Difference

10.8

Document Level Averages
Precision
At 5 docs
0.4513
At 10 docs
0.3949At 15 docs
0.3573
At 20 docs
0.3179At 30 docs
0.2778
At 100 docs
0.1487
At 200 docs
0.0941
At 500 docs
0.0489
At 1000 docs
0.0287
R-Presicion (presicion after
R docs retreived (where R is
the number of relevant
documnets))
Exact
0.2643

Recall Level Precision Averages
RecallPrecision
0.00
0.67420.10
0.52270.20
0.38700.30
0.32200.40
0.27050.50
0.20030.60
0.16490.70
0.13200.80
0.09830.90
0.05341.00
0.0226
Average precision over allrelevant documents
non-interpolated0.2344

Precision

Recall Level Precision Averages
RecallPrecision
0.00
0.61780.10
0.49300.20
0.36720.30
0.30100.40
0.25490.50
0.19260.60
0.14630.70
0.11700.80
0.09750.90
0.04631.00
0.0312Average precision over all
relevant documents
non-interpolated
0.2209

0.6
0.4
0.2
040

50 60
Topic

7080

Di erence from Median in
Average Precision per Topic
(all)

00.2

0.4 0.6
Recall0.8

Recall-Precision Curve

11
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-140

50 60
Topic7080

Di erence from Median in
Average Precision per Topic
(all)

1KJ

Summary Statistics

Summary Statistics

Run ID

1KJ

Task

monolingual

Relevant Level
Run Description

Relevant
automatic,short without concepts

Number of Topics

39

Run ID

1KJ

Task

monlingual

Relevant Level
Run Description

Partial Relevant
automatic,short without concepts

Number of Topics

Total number of documents over all topics

39

Total number of documents over all topics

Retrieved:

39000

Retrieved:

39000

Relevant:

1734

Relevant:

2101

Rel-ret:1094

Rel-ret:845

Document Level Averages
Precision
At 5 docs
0.4256
At 10 docs
0.3846
At 15 docs
0.3316
At 20 docs
0.2910
At 30 docs
0.2359
At 100 docs
0.1133
At 200 docs
0.0695
At 500 docs
0.0362
At 1000 docs
0.0217
R-Presicion (presicion after
R docs retreived (where R is
the number of relevantdocumnets))
Exact
0.2652Difference

Precision

0.6
0.4
0.2
00

0.2

0.4 0.6
Recall

0.8

Recall-Precision Curve

11
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1

1
0.8
Difference

10.8

Document Level Averages
PrecisionAt 5 docs
0.4769
At 10 docs
0.4333
At 15 docs
0.3795
At 20 docs
0.3462
At 30 docs
0.2838
At 100 docs
0.1462
At 200 docs
0.0901
At 500 docs
0.0474
At 1000 docs
0.0281
R-Presicion (presicion after
R docs retreived (where R is
the number of relevantdocumnets))Exact
0.2749Recall Level Precision Averages
RecallPrecision
0.00
0.69760.10
0.52780.20
0.42350.30
0.33300.40
0.27070.50
0.21220.60
0.16860.70
0.12950.80
0.08690.90
0.04881.00
0.0256Average precision over all
relevant documents
non-interpolated
0.2435

Precision

Recall Level Precision Averages
RecallPrecision
0.00
0.64220.10
0.48690.20
0.40530.30
0.32900.40
0.26570.50
0.20020.60
0.15450.70
0.12040.80
0.08430.90
0.04831.00
0.0381
Average precision over all
relevant documents
non-interpolated
0.2338

0.6
0.4
0.2
040

50 60
Topic

7080

Di erence from Median in
Average Precision per Topic
(all)

00.2

0.4 0.6
Recall0.8

Recall-Precision Curve110.8
0.6
0.40.2
0-0.2
-0.4
-0.6
-0.8
-1
40

50 60
Topic

7080

Di erence from Median in
Average Precision per Topic(all)1KJ ij

Summary Statistics

Summary Statistics

Run ID

1KJ ijTaskmonolingual

Relevant Level
Run Description

Relevant
automatic,short without concepts

Number of Topics

39

Run ID

1KJ ijTask

monlingual

Relevant Level
Run Description

Partial Relevant
automatic,short without concepts

Number of Topics

Total number of documents over all topics

39

Total number of documents over all topics

Retrieved:

39000

Retrieved:

39000

Relevant:

1734

Relevant:2101

Rel-ret:1088

Rel-ret:840

Document Level Averages
Precision
At 5 docs
0.4103
At 10 docs
0.3590
At 15 docs
0.2991
At 20 docs
0.2705
At 30 docs
0.2222
At 100 docs
0.1110
At 200 docs
0.0699At 500 docs
0.0364
At 1000 docs
0.0215R-Presicion (presicion after
R docs retreived (where R is
the number of relevant
documnets))
Exact
0.2560Difference

Precision

0.6
0.4
0.2
000.2

0.4 0.6
Recall0.8

Recall-Precision Curve

11
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6-0.8-11
0.8
Difference

10.8

Document Level Averages
Precision
At 5 docs
0.4667
At 10 docs
0.4077
At 15 docs
0.3487
At 20 docs
0.3179At 30 docs
0.2675
At 100 docs
0.1433
At 200 docs
0.0906
At 500 docs
0.0474
At 1000 docs
0.0279R-Presicion (presicion after
R docs retreived (where R is
the number of relevant
documnets))
Exact
0.2666

Recall Level Precision Averages
RecallPrecision
0.00
0.68650.10
0.50280.20
0.38930.30
0.31040.40
0.25800.50
0.19580.60
0.15650.70
0.12350.80
0.09650.90
0.05371.00
0.0230
Average precision over all
relevant documents
non-interpolated
0.2322

Precision

Recall Level Precision Averages
RecallPrecision
0.00
0.63310.10
0.48920.20
0.36810.30
0.29460.40
0.25410.50
0.18390.60
0.13860.70
0.11240.80
0.09600.90
0.05031.00
0.0345
Average precision over all
relevant documents
non-interpolated
0.2216

0.6
0.4
0.2
040

50 60
Topic7080

Di erence from Median in
Average Precision per Topic
(all)

00.2

0.4 0.6
Recall0.8

Recall-Precision Curve

110.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-140

50 60
Topic7080

Di erence from Median inAverage Precision per Topic(all)

BKJJBIDS

Summary Statistics

Summary Statistics

Run IDBKJJBIDS

Task

monolingual

Relevant Level
Run Description

Relevant
automatic,short without concepts

Number of Topics

39

Run IDBKJJBIDS

Task

monlingual

Relevant Level
Run Description

Partial Relevant
automatic,short without concepts

Number of Topics

Total number of documents over all topics

39

Total number of documents over all topics

Retrieved:

39000

Retrieved:

39000

Relevant:

1734

Relevant:

2101

Rel-ret:1226

Rel-ret:957

Document Level Averages
Precision
At 5 docs
0.4769
At 10 docs
0.4000
At 15 docs
0.3590
At 20 docs
0.3051
At 30 docs
0.2624
At 100 docs
0.1297
At 200 docs
0.0831
At 500 docs
0.0419
At 1000 docs
0.0245R-Presicion (presicion after
R docs retreived (where R is
the number of relevant
documnets))
Exact
0.2826

Difference

Precision

0.6
0.4
0.2
000.2

0.4 0.6
Recall0.8

Recall-Precision Curve

110.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1

1
0.8
Difference

10.8

Document Level Averages
Precision
At 5 docs
0.5487
At 10 docs
0.4692
At 15 docs
0.4325
At 20 docs
0.3744
At 30 docs
0.3299
At 100 docs
0.1692
At 200 docs
0.1078
At 500 docs
0.0541
At 1000 docs
0.0314R-Presicion (presicion after
R docs retreived (where R is
the number of relevant
documnets))Exact
0.3182

Recall Level Precision Averages
RecallPrecision
0.00
0.80530.10
0.61330.200.4575
0.30
0.39090.40
0.31440.50
0.26460.60
0.21080.70
0.17130.80
0.12920.90
0.05381.00
0.0268Average precision over all
relevant documents
non-interpolated
0.2888Precision

Recall Level Precision Averages
RecallPrecision
0.00
0.73720.10
0.54990.20
0.41180.30
0.34940.40
0.27700.50
0.23560.60
0.18980.70
0.14990.80
0.11240.90
0.04951.00
0.0372Average precision over all
relevant documents
non-interpolated
0.2618

0.6
0.4
0.2
040

50 60
Topic7080

Di erence from Median in
Average Precision per Topic
(all)

00.2

0.4 0.6
Recall0.8

Recall-Precision Curve

110.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8-14050 60
Topic70

80

Di erence from Median in
Average Precision per Topic(all)

BKJJBIFU

Summary Statistics

Summary Statistics

Run IDBKJJBIFU

Run ID

Task

monolingual

Task

Relevant Level

Relevant

Run Description

Partial Relevant

Run Description

39

automatic,long with concepts

Number of Topics

Total number of documents over all topics

39

Total number of documents over all topics

Retrieved:

39000

Retrieved:

39000

Relevant:

1734

Relevant:

2101

Rel-ret:1114

Rel-ret:1457

Document Level Averages
Precision
At 5 docs
0.6000
At 10 docs
0.5077
At 15 docs
0.4718
At 20 docs
0.4397
At 30 docs
0.3795
At 100 docs
0.1782
At 200 docs
0.1069At 500 docs
0.0501
At 1000 docs
0.0286
R-Presicion (presicion after
R docs retreived (where R is
the number of relevant
documnets))
Exact
0.4121

Difference

0.6
0.4
0.2
000.2

0.4 0.6
Recall

0.8

Recall-Precision Curve110.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1

1
0.8
Difference

10.8

Document Level Averages
Precision
At 5 docs
0.6923
At 10 docs
0.6128
At 15 docs
0.5658
At 20 docs
0.5269At 30 docs
0.4615
At 100 docs
0.2321
At 200 docs
0.1427
At 500 docs
0.0670
At 1000 docs
0.0374R-Presicion (presicion after
R docs retreived (where R is
the number of relevant
documnets))
Exact
0.4468

Recall Level Precision Averages
RecallPrecision
0.00
0.88830.100.8253
0.20
0.70660.300.5961
0.40
0.52100.50
0.44970.60
0.36120.70
0.28020.80
0.21860.90
0.11801.00
0.0430
Average precision over all
relevant documents
non-interpolated
0.4378Precision

Recall Level Precision Averages
RecallPrecision
0.00
0.80700.10
0.75710.20
0.64910.30
0.53980.40
0.46580.50
0.40790.60
0.34060.70
0.27460.80
0.19990.90
0.09751.00
0.0601
Average precision over all
relevant documents
non-interpolated
0.3991

Precisionmonlingual

Relevant Level

automatic,long with conceptsNumber of Topics

BKJJBIFU

0.6
0.40.204050 60
Topic7080

Di erence from Median in
Average Precision per Topic
(all)

00.2

0.4 0.6
Recall0.8

Recall-Precision Curve

110.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1
4050 60
Topic

7080

Di erence from Median in
Average Precision per Topic(all)

BKJJDCFU

Summary Statistics

Summary Statistics

Run IDBKJJDCFU

Run ID

Task

monolingual

Task

Relevant Level

Relevant

Run Description

Partial Relevant

Run Description

39

automatic,long with concepts

Number of Topics

Total number of documents over all topics

39

Total number of documents over all topics

Retrieved:

39000

Retrieved:

39000

Relevant:

1734

Relevant:

2101

Rel-ret:1293

Rel-ret:975

Document Level Averages
Precision
At 5 docs
0.5128
At 10 docs
0.4410
At 15 docs
0.3983
At 20 docs
0.3538
At 30 docs
0.3000
At 100 docs
0.1503
At 200 docs
0.0928
At 500 docs
0.0447
At 1000 docs
0.0250R-Presicion (presicion after
R docs retreived (where R is
the number of relevantdocumnets))
Exact
0.3492Difference

0.60.40.2
000.2

0.4 0.6
Recall0.8

Recall-Precision Curve

11
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1

1
0.8
Difference1
0.8

Document Level Averages
Precision
At 5 docs
0.5897
At 10 docs
0.5205
At 15 docs
0.4752
At 20 docs
0.4346
At 30 docs
0.3761
At 100 docs
0.2000
At 200 docs
0.1245
At 500 docs
0.0597
At 1000 docs
0.0332
R-Presicion (presicion after
R docs retreived (where R is
the number of relevantdocumnets))
Exact
0.3737

Recall Level Precision Averages
RecallPrecision
0.00
0.83500.10
0.70920.20
0.55480.30
0.46360.40
0.38330.50
0.31630.60
0.25320.70
0.20010.80
0.13440.90
0.05201.00
0.0202Average precision over all
relevant documents
non-interpolated
0.3329Precision

Recall Level Precision Averages
RecallPrecision
0.00
0.76860.10
0.62660.20
0.51140.30
0.42840.40
0.36290.50
0.30090.60
0.23920.70
0.18160.80
0.12490.90
0.04571.00
0.0246
Average precision over all
relevant documents
non-interpolated
0.3099Precisionmonlingual

Relevant Level

automatic,long with concepts

Number of Topics

BKJJDCFU

0.6
0.4
0.2
040

50 60
Topic

7080

Di erence from Median in
Average Precision per Topic
(all)

00.2

0.4 0.6
Recall

0.8

Recall-Precision Curve

11
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1
40

50 60
Topic

7080

Di erence from Median in
Average Precision per Topic(all)CRL15.nSummary Statistics
Run ID

Summary Statistics
CRL15.nTaskmonolingual

Relevant Level

monlingual

Relevant Levela-discription

Number of Topics

CRL15.n

TaskRelevant

Run Description

Partial Relevant

Run Description

39

ab-discription

Number of Topics

39

Total number of documents over all topics

Total number of documents over all topics

Retrieved:

39000

Retrieved:

39000

Relevant:

1734

Relevant:

2101

Rel-ret:1014

Rel-ret:1305

Document Level Averages
Precision
At 5 docs
0.4154
At 10 docs
0.3590
At 15 docs
0.3111
At 20 docs
0.2821
At 30 docs
0.2385
At 100 docs
0.1279At 200 docs
0.0836
At 500 docs
0.0438
At 1000 docs
0.0260R-Presicion (presicion afterR docs retreived (where R is
the number of relevantdocumnets))
Exact
0.2620Difference

0.6
0.4
0.2
000.2

0.4 0.6
Recall0.8

Recall-Precision Curve

1

1
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1

1
0.8
Difference1
0.8

Document Level Averages
Precision
At 5 docs
0.4513
At 10 docs
0.4000
At 15 docs
0.3504
At 20 docs
0.3179At 30 docs
0.2803
At 100 docs
0.1605
At 200 docs
0.1068
At 500 docs
0.0569
At 1000 docs
0.0335R-Presicion (presicion after
R docs retreived (where R is
the number of relevantdocumnets))
Exact0.2845

Recall Level Precision Averages
RecallPrecision
0.00
0.69040.10
0.54260.20
0.38360.30
0.32490.40
0.26730.50
0.23850.60
0.20610.70
0.17560.80
0.14260.90
0.08431.00
0.0262Average precision over all
relevant documents
non-interpolated
0.2588Precision

Recall Level Precision Averages
RecallPrecision
0.00
0.62330.10
0.51950.20
0.37880.30
0.31290.40
0.24950.50
0.21560.60
0.18270.70
0.15680.80
0.12900.90
0.06801.000.0355
Average precision over all
relevant documents
non-interpolated
0.2440Precision

Run ID

0.6
0.4
0.2
040

50 60
Topic7080

Di erence from Median in
Average Precision per Topic
(all)

00.2

0.4 0.6
Recall0.8

Recall-Precision Curve

11
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-140

50 60
Topic

7080

Di erence from Median in
Average Precision per Topic(all)

ULIS10

Summary Statistics
Run ID

Summary Statistics
ULIS10Task

monolingual

Relevant Level

monlingual

Relevant Levela-discription

Number of Topics

ULIS10

Task

Relevant

Run Description

Partial Relevant

Run Description

39

ab-discription

Number of Topics

39

Total number of documents over all topics

Total number of documents over all topics

Retrieved:

39000

Retrieved:

39000

Relevant:

1734

Relevant:

2101

Rel-ret:591

Rel-ret:787

Document Level Averages
Precision
At 5 docs
0.1641
At 10 docs
0.1564
At 15 docs
0.1538
At 20 docs
0.1385
At 30 docs
0.1231
At 100 docs
0.0636
At 200 docs
0.0424
At 500 docs
0.0246
At 1000 docs
0.0152R-Presicion (presicion after
R docs retreived (where R is
the number of relevantdocumnets))
Exact
0.1061

Difference

0.6
0.4
0.2
000.2

0.4 0.6
Recall0.8

Recall-Precision Curve

1

1
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1

1
0.8
Difference1
0.8

Document Level Averages
Precision
At 5 docs
0.2615
At 10 docs
0.2359At 15 docs
0.2308
At 20 docs
0.2064
At 30 docs
0.1812
At 100 docs
0.0941
At 200 docs
0.0608
At 500 docs
0.0337
At 1000 docs
0.0202
R-Presicion (presicion after
R docs retreived (where R is
the number of relevant
documnets))
Exact
0.1431

Recall Level Precision Averages
RecallPrecision
0.00
0.46660.10
0.28190.20
0.18360.30
0.11630.40
0.07860.50
0.05430.60
0.02440.70
0.01730.80
0.00910.90
0.00001.00
0.0000
Average precision over all
relevant documents
non-interpolated
0.0948Precision

Recall Level Precision Averages
RecallPrecision
0.00
0.39790.10
0.20440.20
0.13170.30
0.07760.40
0.04920.50
0.02530.600.0119
0.70
0.00850.80
0.00710.90
0.00411.000.0037
Average precision over all
relevant documents
non-interpolated
0.0675

Precision

Run ID

0.6
0.4
0.2
04050 60
Topic7080

Di erence from Median in
Average Precision per Topic
(all)

00.2

0.4 0.6
Recall0.8

Recall-Precision Curve

11
0.8
0.6
0.4
0.2
0-0.2
-0.4
-0.6
-0.8
-1
40

50 60
Topic7080

Di erence from Median in
Average Precision per Topic(all)