Annex to Memo CP-D/627

Automatic test of EXFOR with TALYS: Attempt 1

Arjan koning
NRG, Petten, The Netherlands
WPEC SG-30 document, July 4, 2008
The objective of WPEC Subgroup 30 Quality improvement of the EXFOR database is to make EXFOR an easy accessible database of experimental nuclear reaction data, and to systematically check andcorrect EXFOR entries for errors. In 2007-2008, significant progress has been made to make EXFOR
available as a complete computational database. One obvious advantage of a computational database isthat automatic comparisons with nuclear model codes can be made, to test the quality of either the dataor the model code. This is exactly the topic of this document.
The objective is to compare two large databases:• The entire EXFOR database in so called XC4 format, made available by the IAEA (Zerkin).
• A complete nuclear reaction database, SALTY, created by default (”blind”) TALYS calculations.
The term ”complete” in the second item above means that we have stored all the output that Talys gives
us, if we specify maximum output with the Talys keywords. In this first stage, we use only part of the
SALTY library. For the current comparison with EXFOR we consider:• Projectiles: gamma’s, neutrons, protons, deuterons, tritons, helions and alpha’s• Targets: all stable isotopes in the Z=9-83 range• Incident energies from thermal energies up to 200 MeV• Cross sections: total, elastic, non-elastic, exclusive (i.e. per MT number) and residual production
cross sections.
This exercise is an EXFOR test and a Talys test at the same time. This global comparison obviously does
not replace a ”true” evaluation for one particular isotope, which involves careful studying all experimental work, precise nuclear model fitting, etc. However, it has already been shown in many occasions that
Talys provides very reasonable ”blind” estimates for many reaction processes, with the thermal range
and fission as notable exceptions (Talys can be used for that also, but only a ”true” isotopic evaluation
brings the results somewhere near the experimental data for those processes). Hence, with the exceptionof certain reactions, Talys should be able to give a reasonably good prediction of many reaction data,and obviously we will constantly try to extend such a prediction to as many reactions as possible. At
first sight, the problem is simple: If we know that Talys is usually within e.g. 30% of the experimental
1data for a certain reaction channel, alarm bells should start ringing if the deviation for such a channel issuddenly much larger. We note that large deviations may also come from bad Talys performance, even if
the visual agreement on linear scale is good. For example, for threshold reactions the difference between
Talys and experiment may easily be a factor of 10, close to threshold, and there may be experimental
and theoretical reasons for that. In general the rule holds that the smaller the cross section, the larger the
relative error. It is therefore important to watch not only the calculation/experiment (C/E) values, but also
the absolute deviation. In several cases, it turns out that there are problems in EXFOR, and many of them
can not so easily be detected with ways other than with a model code (which is why these problems arestill in EXFOR in the first place). The problems which are easiest to detect are C/E values around 0.001
or 1000, suggesting the well known error of mixing barns and millibarns. Unfortunately, the majorityof cases is more difficult to judge. This is an attempt to categorize possible problems of EXFOR (orTalys). The current comparison may also help to solve one of the largest problems of EXFOR: reaction
identifiers which are assigned in wrong, inconsistent or even multiple ways. If Talys is expected to givea reasonably good prediction for a reaction and we obtain a large deviation, it may be that we are notcomparing the Talys result with the correct quantity.
To discover and classify problems, we use 4 goodness-of-fit estimators. If they are all very large (or in
the case of R very small), something is wrong somewhere. They are the F-factor

F = 10

s

1
N

PN
i



log

σi
T
σi
E

2

,

(1)

the C/E value, also called R,R=
the χ2 ,

N
1 X
σTi
i ,
N i σE

N
1 X
χ =
N i

i
σTi − σEi
∆σE2and the absolute deviation

∆=

(2)

!2

N
1 X
i
|σTi − σexp
|,
N i

,

(3)

(4)

where the subscript T stands for theory or Talys and E for experimental. In all cases, we average over thenumber of energy points, N, in each set. Hence, each EXFOR subentry (data set) that contains a crosssection excitation function (or only 1 point) is described by four numbers: F , R, χ 2 and ∆, while wealso keep track of all individual components F i , Ri , χ2i and ∆i for e.g. C/E plots.
The F-factor is a kind of twisted R value. In fact, each individual component of the sum inside F is equal
to R if it is larger than 1, and 1/R if it is smaller than 1. This is a helpful quantity, since averaging R
values over many points may not be very meaningful if the values cross 1 at some point. A value of
F=1.2 means that for the entire data set we are roughly 20% off on average. We use F as the leading
indicator in our statistical study, i.e. we sort our results in order of increasing F to identify the worst
cases.
In our data evaluation system, the most versatile use of experimental data is made if nuclear reactions are
stored directory-wise per reaction type. For example, all experimental data sets for (n,2n) reactions on
2

54 Fe are stored in a subdirectory n/fe/054/016, inside which there are various files, one per experiment,e.g.

......
Bahal1984.tot
Bormann1976.tot
Cross1963.tot
Csikai1965.tot......
where e.g. the Bormann1976.tot file looks as follows# Target : Z= 26 A= 54
Isomer=Projectile=n# Reaction: MT= 16 (n,2n) Isomer=# Quantity: Cross section
Frame: L# X4 ID: 20614002 C4: MF
3 MT 16 X4 Code: 26-FE-54(N,2N)26-FE-53,,SIG# Author : M.Bormann+
Year: 1976#
E(MeV)
xs(mb)
dxs(mb)dE(MeV)
TALYS
F
1.40500E+01 1.01000E+01 1.00000E-01 9.00000E-02 3.41057E+00
2.961
1.45500E+01 1.75000E+01 1.20000E-01 1.00000E-01 1.10903E+01
1.578
1.48100E+01 2.25000E+01 1.50000E-01 1.00000E-01 1.65837E+01
1.357
1.50800E+01 2.36000E+01 1.50000E-01 1.00000E-01 2.23218E+01
1.057
1.53500E+01 2.90000E+01 1.70000E-01 1.00000E-01 2.81392E+01
1.031
1.56100E+01 2.95000E+01 2.00000E-01 1.10000E-01 3.37412E+01
1.144
1.59200E+01 3.70000E+01 2.10000E-01 1.10000E-01 4.04205E+01
1.092
1.64300E+01 4.55000E+01 2.40000E-01 1.10000E-01 4.92008E+01
1.081
1.69400E+01 5.60000E+01 3.00000E-01 1.10000E-01 5.75702E+01
1.028
1.75600E+01 5.62000E+01 3.10000E-01 1.10000E-01 6.44940E+01
1.148
1.82300E+01 5.81000E+01 3.00000E-01 1.20000E-01 7.06126E+01
1.215#M.Bormann, H-K.Feddersen, H.-H.Holscher,# W.Scobel, H.Wagner#Jour. Zeitschrift fuer Physik, Section A#Vol.277, p.203, 1976#-(N,2N) EXCITATION FUNCTIONS FOR FE-54, GE-70, SE-74,#RB-85, SR-86,88, Y-89, MO-92, AND HG-204 IN THE#NEUTRON ENERGY REGION 13-18 MEV-. (GERMAN)

R(C/E)
0.3377
0.63370.73710.94580.9703
1.144
1.092
1.081
1.028
1.148
1.215

Chi-2
4475.
2853.
1556.
72.61
25.64
449.7
265.3
237.8
27.39
715.8
1740.

Delta(mb)
6.689
6.410
5.916
1.278
0.8608
4.241
3.421
3.701
1.570
8.294
12.51

For this particular study, we simply print the blind, default, unadjusted (i.e. be gentle, please) Talys
result, and the statistical factors for each point in the table, while we keep the average values for this set
in another summary table. While we make the translation from the XC4 computational database to ourown directory-structured database, we do our Talys comparison, checks and statistical analyses on the
fly. After about 10 minutes, the conversion is done and all checking and statistical results are available.
We show a few examples here. Fig. 1 shows the distribution of the F-values for all (n,2n) reactions that
we managed to get out of the XC4 database. We logged the number of cases per F-bin, whereby we
distributed the range F=1-1000 over 100 logaritmically equidistant bins. Hence, the first bin means that
Talys is inside 7% of the experimental data, the second bin between 7-15%, the third bin 15-23%, and soon. All cases with F>1000 are put in the last bin. The high peak at the lowest bins probably means goodnews for Talys. The cases with very high F-values probably mean trouble for EXFOR (or XC4). The
cases in between mean trouble for either Talys or Exfor, or both. For SG-30, the interesting cases arein the tail of the distribution and it is probably best to start checking and working on the highest values.
Note that there is always the possibility of an erroneous XC4 interpretation from my side, leading to false
alarm, and hopefully this improves over time. These distributions are available for all MT numbers, andalso for residual production cross sections. Fig. 1 also shows the result for all neutron-induced reactions.
The most interesting are of course the specific cases with large deviation. The conversion process produces a file called x4.sort which has all EXFOR entries sorted by F-value. The bottom of that list looks
3

EXFOR (n,2n) cross sections

EXFOR all n−induced cross sections

Talys vs. EXFOR

Talys vs. EXFOR

300

10000

1000

Number of cases

Number of cases

200

100

100
10

0

1

10

100

1

1000

1

10

100

F

1000

F

EXFOR (n,γ) cross sections

EXFOR (p,n) cross sections

Talys vs. EXFOR

Talys vs. EXFOR
60

700

600

50

500
Number of cases

Number of cases

40
400

300

30

20
200
10

100

0

1

10

100

1000

F

0

1

10

100
F

Figure 1: Frequency table for the F-values for (n,2n) reactions, all neutron-induced reactions, (n,γ), and(p,n) reactions.

4

1000

as follows
EXFOR.......
11491007
11590082
30106012
11491015
30152014

C4 New
1
1
1
1
1

1
1
1
1
1

#C4
1
1
1
1
1

#New

Reaction

1 25-MN-55(N,HE3)23-V-53,,SIG
1 81-TL-205(N,HE3)79-AU-203,,SIG
1 57-LA-139(N,2P)55-CS-138,,SIG
1 40-ZR-94(N,HE3)38-SR-92,,SIG
1 52-TE-130(N,HE3)50-SN-128,,SIG

F

R

chi-2

Delta

1.080E+15
1.448E+15
1.523E+15
4.892E+15
5.786E+15

9.262E-16
6.908E-16
6.568E-16
2.044E-16
1.728E-16

0.00
0.00
2.25
0.00
3.52

0.420
7.000E-02
3.000E-020.400
1.500E-02

i.e. we find F-values above 1015 , and they even are probably not EXFOR errors! (Talys is known to missthe (n,He3) threshold by at least 1 MeV, resulting in a huge deviation near threshold. Note the small
deviation in millibarns in the last column). More interesting is perhaps to filter out one kind of reaction.
If under Linux, we do grep ’N,2N’ x4.sort the bottom of that list looks as follows.
EXFOR
C4 New.......
303220122 1 122662014
1 123018004
1 123018002
1 1
23018003
1 111684003
1 123018007
1 131454005
1 1
20033003
1 122969007
1 1
23018014
1 123018005
1 123018006
1 122347006
1 1

#C4
1
6
6
6
6
1
6
1
1
4
6
6
6
3

#New

Reaction

1 49-IN-115(N,2N)49-IN-114-M1,,SIG
6 58-CE-140(N,2N)58-CE-139-M,,SIG
6 80-HG-196(N,2N)80-HG-195-G,,SIG
6 80-HG-196(N,2N)80-HG-195,,SIG
6 80-HG-196(N,2N)80-HG-195-M,,SIG
1 25-MN-55(N,2N)25-MN-54,,SIG
6 80-HG-198(N,2N)80-HG-197-G,,SIG
1 72-HF-179(N,2N)72-HF-178-M2,,SIG
1 37-RB-85(N,2N)37-RB-84,,SIG
4 72-HF-176(N,2N)72-HF-175,,SIG
6 80-HG-204(N,2N)80-HG-203,,SIG
6 80-HG-198(N,2N)80-HG-197,,SIG
6 80-HG-198(N,2N)80-HG-197-M,,SIG
3 53-I-129(N,2N)53-I-128,,SIG

F

R

chi-2

Delta

107.
140.
638.
702.
748.
786.
880.
908.
960.
961.
993.
994.
1.139E+03
1.450E+03

107.
141.
642.
703.
754.
786.
881.
908.
1.042E-03
975.
994.
995.
1.140E+03
1.454E+03

3.368E+05
1.175E+07
2.852E+07
7.502E+07
6.158E+07
0.00
8.661E+07
1.039E+07
399.
1.119E+08
4.301E+08
2.696E+08
2.872E+08
1.164E+08

1.161E+03
793.
582.
1.578E+03
996.
785.
1.007E+03290.
1.010E+06
1.194E+03
2.135E+03
2.075E+03
1.069E+03
1.640E+03

Here we see the F-values around 1000 suggesting the b-mb problem. Indeed, the (n,2n) cross sections of
the various Hg isotopes were a factor of 1000 too small and this has recently been corrected. Note that
the R-factor indicates that 37-RB-85(N,2N) concerns kilobarns instead of barns (hence the deviation of
1.010E+06 mb). Unfortunately, not all problems are that simple. There may be an inconsistency between
Talys and Exfor on the definition of an isomer, leading to large discrepancies and the list above may alsosuggest that isomeric, ground state and total cross sections may have been mixed.
After grep ’N,TOT’ x4.sort we find near the bottom
EXFOR.......
21676005
11026015
22007002
31468004
20602009
31468006
31468005
31468007
31468003
11204004.......

C4 New

#C4

#New

1
1
1
1
1
1
1
1
1
1

1
1
121
7
1
6
6
2
7
12

1 23-V-51(N,TOT),,SIG,,AV
1 23-V-0(N,TOT),,SIG
121 14-SI-0(N,TOT),,SIG
7 24-CR-0(N,TOT),,SIG
1 23-V-51(N,TOT),,SIG
6 28-NI-0(N,TOT),,SIG
6 26-FE-0(N,TOT),,SIG
2 29-CU-0(N,TOT),,SIG
7 23-V-0(N,TOT),,SIG
12 4-BE-9(N,TOT),,SIG

1
1
1
1
1
1
1
1
1
1

Reaction

F

R

chi-2

Delta

914.
920.
937.
972.
974.
1.010E+03
1.013E+03
1.031E+03
1.039E+03
1.045E+03

914.
920.
983.
973.
974.
1.010E+03
1.014E+03
1.031E+03
1.040E+03
1.058E+03

1.442E+10
4.948E+09
0.00
9.116E+08
5.870E+10
7.236E+08
8.609E+08
6.671E+08
9.202E+08
0.00

4.924E+06
4.924E+06
5.151E+05
3.790E+03
4.846E+06
3.603E+03
3.642E+03
3.442E+03
3.987E+03
1.528E+06

of which some (maybe all?) are indeed mb-b problems. Note however that at very low energies Talys
may be a factor 1000 off (while above a few MeV Talys should be within 1-3 %).
In the future, this procedure should also be applied to secondary distributions such as angular distributions and (double-)differential spectra. Before this can be done, the secondary energies and angles in the
XC4 database first need to be sorted (with the sortC4 code).
5As stated above, the first obvious use of all this information is to identify problems and to correct them.
However, with all these results available, it is now also possible to set-up some ”zeroth-order” quality
flagging. Although we can never be 100% sure, it is very probable that the subentries with small F-values(where ”small” depends on the reaction channel) represent indeed the type of quantities that are reported
in EXFOR. In other words, the reaction identifier assigned by the compiler for these subentries is correct.
Note that here I am not yet talking about renormalization of wrongly interpreted experimental conditions(there has been some recent discussion about this and that may be very valuable too). The quality flagging
mentioned above would simply ensure that e.g. a cross section reported as being (n,2n) in EXFOR isindeed just that, and not e.g. a compound (n,n’) + (n,2n) cross section (which would place it further
down the tail of our F-distribution since Talys is being compared with the wrong quantity). In this way
we would obtain a large ”verified” set of EXFOR data, while ”validation” of the data would involvea more precise study of the detailed experiment and possible renormalization. To start with, such a
quality list could exist of only the EXFOR subentry number and e.g. a 1 for a verified set and a 0 for anon-verified set. This requires some further study and discussion.
Finally, all experimental uncertainties of EXFOR were also sorted per reaction or MT number and storedin files, e.g. mterr.102. The largest errors for capture are obtained with sort -g -k 3,3 mterr.016:.............
10340007 52-TE-130(N,G)52-TE-131,,SIG
11507090 74-W-180(N,G)74-W-181,,SIG,,MXW13742004 38-SR-88(N,G)38-SR-89,,SIG,,RAW400070051 34-SE-80(N,G)34-SE-81,,SIG
40975014 68-ER-170(N,G)68-ER-171,,SIG

354.
%
400.
%
619.
%
749.
%
1.538E+03 %which may or may not be wrong, closer inspection is required. The largest errors in the entire list are.............
20668004 74-W-180(N,P)73-TA-180-M,,SIG
21264010 42-MO-92(N,N+P)41-NB-91,,SIG
22640003 8-O-18(N,P)7-N-18,,SIG
40975014 68-ER-170(N,G)68-ER-171,,SIG
21288007 20-CA-48(N,2N)20-CA-47,,SIG
20909003 6-C-12(N,TOT),,SIG
21288005 20-CA-44(N,A)18-AR-41,,SIG
41163008 52-TE-126(N,P)51-SB-126-G,,SIG
C0774002 39-Y-89(P,N)40-ZR-89,,SIG
12940004 40-ZR-0(N,TOT),,SIG

960.
%
1.344E+03 %1.478E+03 %1.538E+03 %5.565E+03 %1.629E+04 %1.714E+04 %2.000E+04 %7.334E+04 %9.071E+04 %Only results for incident neutrons were given here, but the same is done for other projectiles. Morerefined test are possible and I hope to find the time to perform them in the near future. All results
described in this document, i.e. all resulting numerical files, have been sent to the IAEA for further
inspection.

6