# Multiobject Optimization of National Football League Drafts: Comparison of Teams and Experts
## 3 Results  
### Evaluation of Experts  
Results based on the 4+ GPA conversion system are presented by Figure 1. The SRD plot compares the experts to a reference column, defined as maximum here. The rationale behind choosing the maximum column as a benchmark is equivalent to define a hypothetical best expert, who can always select the best team. The plot presents a tangent hyperbolic curve, approximating the cumulated Gaussian distribution function of the random SRD values. Rankings placed on the left side of the XX1 range (5%) mean significant rankings (the probability of random ranking is less than 5%). As we look at the plot, some differentiation can clearly be seen. There is a smaller group of experts who are more optimistic regarding the 2021 drafts: RF, RD, TB, LE, FS, and St1 when compared to the others, as their SRD scores are closer to the zero SRD point. Figure 1: The SRD values (scaled between 0 and 100) of the experts based on their ratings determined by sum of ranking differences. Maximum values were used as the reference (benchmark) column, which had the highest scores. Scaled SRD values are plotted on the \(x\) axis and left \(y\) axis, right \(y\) axis shows the relative frequencies of random ranking distribution function: black curve. The probability ranges are also given as 5% (XX1), Median (Med), and 95% (XX19). Codes of the experts are listed in the data set section. Factorial ANOVA decomposes the effects of the various factors and their couplings. The Figure 2 shows the differences in coding, the original GPA rank is inferior to the full ranking (the smaller the SRD the better). Figure 3 shows the characteristic grouping of experts. _Post hoc_ tests suggest from six ([PERSON]'s test) to nine ([PERSON] honestly significant difference test) overlapping groups, but three groups can be defined relatively easily (Figure 3): Group I (SRD < 35), Group II (37 < SRD < 43), and Group III (SRD > 43), I-group of recommendable experts, II-medium-moderate type experts, and III-not recommendable opinions. A slight difference can be observed if comparing the effect of rankings to the quality of experts (Figure 4): some experts seem to be better using GPA ranking while others do better at the re-ranking case. Figure 3: Comparison of experts (F3, Sum of Ranking Differences, SRDs are scaled between 0 and 100). The coding options are amalgamated in F3. The error bars represent 95% confidence interval. Figure 2: Comparison of coding options GPAraw–original GPA coding, GPAreRank (recoding to full rank) error bars represent 95% confidence interval. The variances are homogeneous according to the [PERSON]’s test. In some cases (e.g., RD, DK,...) the two ranking options are indistinguishable. In other cases (e.g., FS, LE, St1, BR, MM, St2,...) the difference is significant, though not very large. The conclusion is obvious, the coding options should be tailored according to the preferences (taste) of the experts. During the peer-review process, one of the unknown referees suggested to compare the obtained SRD rankings to other MCDM methods. After evaluating the existing literature, we decided to use the technique for order of preference by similarity to ideal solution (TOPSIS) method. Similar to the majority of MCDM methods, e.g., ELECTRE III, PROMETHEE methods etc., TOPSIS requires the definition of variable weights. Weights are meant to express the relative importance of the variables/criteria on the ranking of the objects/alternatives. There is no standard method for the definition of weights, rather, there are a set of approaches that could be followed [27; 28]. Usually, two groups of weighting schemes are defined: so-called subjective and objective weights [29]. Subjective weights are usually defined by the decision makers (or users), while objective weights are defined by mathematical modelling.