DOI: 10.5281/zenodo.7569135
title: Data: HasBugs - Handpicked Haskell Bugs

description: <sub># HasBugs - Haskell Bugs for you and me </sub> <sub>We try to provide a fresh dataset in the fashion of [Defects4J](https://github.com/rjust/defects4j) to evaluate tasks such as fault localization, program repair and test generation.</sub> <sub>Instead of just providing the commits, we learned from some of the datasets out there, which we address in the following: </sub> <sub>- A failing CI does not mean a bug - atleast not certainly from the code. Hence, our bug-points are aknowledged by the projects through PRs and Issues. This also means that the *buggy version* of the datapoints usually compiles. <br> - Similarly, the Fix is what the maintainers consider fixed (even if that is removal of a feature).<br> - As research by [Martin Monperrus](https://link.springer.com/article/10.1007/s10664-016-9470-4) showed, not all found patches in program repair actually help despite passing the test-suite. <br> To help with inspection of potential tasks, we look into the material provided (code, issues, project readme/architecture) to give a estimate of "what is buggy".<br> We hope this helps to easier distinguish actually good result from e.g. overfitting.<br> - We express multiple bug and fix locations in the datapoint</sub> <sub>The aim of these changes is to provide a (small) dataset with bugs from real-world problems of which all steps are aknowledged by their maintainers (the bug, the test & the fix).<br> We do not try to provide a dataset in the size that you can run deep learning on it - but we want to have a quality gold standard to benchmark modern tools against.</sub> <sub>## Datapoint </sub> <sub>For every project, we provide </sub> <sub>- The buggy version<br> - The fixed version<br> - The separated Test-Patch, tested Version<br> - A datapoint for analysis (see below)</sub> <sub>A single datapoint can contain the following attributes (required attributes are marked with *, but these stars are not in the actual json!): </sub> <sub>```JSON<br> {<br> "*id":42,<br> "*repositoryurl": "https://github.com/ciselab/HasBugs",<br> "*repository":"Ciselab/HasBugs",<br> "*licence":"MIT",<br> "*faultcommit":"xxx",<br> "*fixcommit":"yyy",</sub> <sub> "ghc-version":"9.2.4",<br> "buildframework":"Cabal",<br> "testframeworks" : ["QuickCheck","Tasty"],</sub> <sub> "testpatch":true,<br> "description": "Input Sanitazion missing - divides by zero",<br> "categories" : ["Sanitazion", "DivideByZero"], </sub> <sub> "relatedissues" : ["https:www.github.com/ciselab/HasBugs/issues/1"],<br> "relatedprs" : ["https:www.github.com/ciselab/HasBugs/issues/2"],</sub> <sub> "*faultlocations" : [<br> {<br> "*startline": 5,<br> "*endline": 15,<br> "*file" : "./project/.../program.hs",<br> "module" : "Math.Divison",<br> "method": "divide"<br> },{<br> "*startline": 25,<br> "*endline": 35,<br> "*file" : "./project/.../program.hs",<br> "module" : "Math.Divison",<br> "method": "divide"<br> }<br> ],<br> "*fixlocations" : [<br> {<br> "*startline": 5,<br> "*endline": 20,<br> "*file" : "./project/.../program.hs",<br> "module" : "Math.Divison",<br> "method": "divide"<br> }<br> ]<br> }<br> ```</sub> <sub>## Inclusion Criteria </sub> <sub>- Compatible with GHC &gt;=8.10<br> - Projects run with a single `make test`, `cabal test` or `stack test` <br> - buggy version newer than 2018<br> - bug and fix aknowledged as such by the maintainer(s) through Github Issues / PRs </sub><br>