DOI: 10.5281/zenodo.15484550
title: Supplementary Material - Who Should Test the Requirement? A Comparative Study on Requirements Classification for Assigning Test Teams using the Pre-Trained Models

description: The supplementary material is related to the statistical tests performed to validate the results from the paper "Who Should Test the Requirement? A Comparative Study on Requirements Classification for Assigning Test Teams using the Pre-Trained Models"

The goal of the statistical test was to investigate whether there is a significant difference between the BERT, XLNet, and ELECTRA models with regard to the probability assigned to the correct class of each requirement. For the statistical test, we used a set of probability distribution to the model classify correctly each class:




Model
Probability


BERT
0,956809


BERT
0,942727


BERT
0,788117


BERT
0,922548


BERT
0,005487


BERT
0,942294


BERT
0,922247


BERT
0,115416


BERT
0,945645


BERT
0,945485


BERT
0,919123


BERT
0,946622


BERT
0,940579


BERT
0,954029


BERT
0,853265


BERT
0,846922


BERT
0,942807


BERT
0,140116


BERT
0,922081


BERT
0,917714


BERT
0,95244


BERT
0,919942


BERT
0,918669


BERT
0,963556


BERT
0,300082


BERT
0,998968


BERT
0,164602


BERT
0,998052


BERT
0,998711


BERT
0,004985


BERT
0,011434


BERT
0,002421


BERT
0,992682


BERT
0,004472


BERT
0,528385


BERT
0,021226


BERT
0,042576


BERT
0,998852


BERT
0,996028


BERT
0,13071


BERT
0,998036


BERT
0,903156


BERT
0,998949


BERT
0,036362


BERT
0,998974


BERT
0,06081


BERT
0,999034


BERT
0,99895


BERT
0,999011


BERT
0,998955


BERT
0,995763


BERT
0,042633


BERT
0,980891


BERT
0,99498


BERT
0,000471


BERT
0,995075


BERT
0,005023


BERT
0,001325


BERT
0,00048


BERT
0,000377


BERT
0,994233


BERT
0,000334


BERT
0,002129


BERT
0,955263


BERT
0,974376


BERT
0,992562


BERT
0,030702


BERT
0,004845


BERT
0,997385


BERT
0,002129


BERT
0,950778


BERT
0,99656


BERT
0,002736


BERT
0,870145


BERT
0,997288


BERT
0,99595


BERT
0,23246


BERT
0,779047


BERT
0,995905


BERT
0,995811


BERT
0,992693


BERT
0,995951


BERT
0,98359


BERT
0,995895


BERT
0,995951


BERT
0,993184


BERT
0,000285


BERT
0,992741


BERT
0,995833


BERT
0,042702


BERT
0,994797


BERT
0,994508


BERT
0,989475


BERT
0,989047


BERT
0,389256


BERT
0,995951


BERT
0,990609


BERT
0,995895


BERT
0,321135


BERT
0,993384


XLNet
0,990439


XLNet
0,986381


XLNet
0,987713


XLNet
0,992904


XLNet
0,001316


XLNet
0,986608


XLNet
0,992945


XLNet
0,628709


XLNet
0,988232


XLNet
0,99103


XLNet
0,98592


XLNet
0,991984


XLNet
0,992257


XLNet
0,989767


XLNet
0,996295


XLNet
0,917428


XLNet
0,986833


XLNet
0,996749


XLNet
0,989208


XLNet
0,988726


XLNet
0,989245


XLNet
0,985766


XLNet
0,98696


XLNet
0,536825


XLNet
0,928015


XLNet
0,999848


XLNet
0,427277


XLNet
0,99978


XLNet
0,999419


XLNet
0,00289


XLNet
0,993831


XLNet
0,013385


XLNet
0,999719


XLNet
0,78129


XLNet
0,999712


XLNet
0,980641


XLNet
0,972637


XLNet
0,999845


XLNet
0,999757


XLNet
0,979846


XLNet
0,999863


XLNet
0,999824


XLNet
0,999835


XLNet
0,000634


XLNet
0,999836


XLNet
0,99985


XLNet
0,999814


XLNet
0,999827


XLNet
0,999776


XLNet
0,999837


XLNet
0,998838


XLNet
0,000242


XLNet
0,990867


XLNet
0,998966


XLNet
0,581768


XLNet
0,997184


XLNet
9,16E-05


XLNet
0,983396


XLNet
0,916334


XLNet
0,977303


XLNet
0,969508


XLNet
0,913709


XLNet
0,8947


XLNet
9,87E-05


XLNet
0,988572


XLNet
0,997477


XLNet
0,984364


XLNet
0,000605


XLNet
0,99788


XLNet
0,8947


XLNet
0,975651


XLNet
0,997627


XLNet
0,000492


XLNet
0,035856


XLNet
0,998531


XLNet
0,996914


XLNet
0,673415


XLNet
0,992078


XLNet
0,992422


XLNet
0,997631


XLNet
0,100326


XLNet
0,997638


XLNet
0,998233


XLNet
0,997233


XLNet
0,997638


XLNet
0,998255


XLNet
0,0001


XLNet
0,997868


XLNet
0,997306


XLNet
0,006061


XLNet
0,997934


XLNet
0,997859


XLNet
0,998071


XLNet
0,997701


XLNet
0,002232


XLNet
0,997638


XLNet
0,998183


XLNet
0,997233


XLNet
0,00363


XLNet
0,99795


ELECTRA
0,994836


ELECTRA
0,909889


ELECTRA
0,735999


ELECTRA
0,98587


ELECTRA
0,001246


ELECTRA
0,995318


ELECTRA
0,786375


ELECTRA
0,013252


ELECTRA
0,993442


ELECTRA
0,984104


ELECTRA
0,987827


ELECTRA
0,992197


ELECTRA
0,991061


ELECTRA
0,994944


ELECTRA
0,991898


ELECTRA
0,988246


ELECTRA
0,991382


ELECTRA
0,243261


ELECTRA
0,790035


ELECTRA
0,989


ELECTRA
0,939608


ELECTRA
0,986161


ELECTRA
0,983756


ELECTRA
0,992828


ELECTRA
0,834312


ELECTRA
0,999558


ELECTRA
0,001553


ELECTRA
0,949012


ELECTRA
0,99927


ELECTRA
0,026367


ELECTRA
0,001708


ELECTRA
0,002642


ELECTRA
0,147262


ELECTRA
0,002407


ELECTRA
0,049383


ELECTRA
0,044357


ELECTRA
0,001551


ELECTRA
0,999395


ELECTRA
0,999389


ELECTRA
0,02797


ELECTRA
0,423016


ELECTRA
0,046879


ELECTRA
0,999569


ELECTRA
0,379341


ELECTRA
0,999565


ELECTRA
0,873975


ELECTRA
0,99949


ELECTRA
0,999564


ELECTRA
0,999223


ELECTRA
0,999566


ELECTRA
0,996647


ELECTRA
0,683473


ELECTRA
0,995532


ELECTRA
0,090389


ELECTRA
0,000536


ELECTRA
0,998183


ELECTRA
0,040802


ELECTRA
0,000281


ELECTRA
0,000366


ELECTRA
0,000945


ELECTRA
0,065098


ELECTRA
0,001892


ELECTRA
0,001633


ELECTRA
0,040839


ELECTRA
0,994601


ELECTRA
0,996824


ELECTRA
0,000888


ELECTRA
0,000283


ELECTRA
0,998762


ELECTRA
0,001633


ELECTRA
0,997606


ELECTRA
0,998009


ELECTRA
0,00106


ELECTRA
0,997959


ELECTRA
0,998995


ELECTRA
0,998202


ELECTRA
0,712276


ELECTRA
0,018504


ELECTRA
0,996856


ELECTRA
0,998098


ELECTRA
0,991718


ELECTRA
0,997952


ELECTRA
0,997871


ELECTRA
0,995693


ELECTRA
0,997952


ELECTRA
0,993072


ELECTRA
0,000149


ELECTRA
0,971749


ELECTRA
0,980839


ELECTRA
0,052526


ELECTRA
0,996982


ELECTRA
0,996549


ELECTRA
0,997504


ELECTRA
0,976286


ELECTRA
0,260132


ELECTRA
0,997952


ELECTRA
0,996947


ELECTRA
0,995693


ELECTRA
0,089357


ELECTRA
0,996514