text:   inadvertently state-of-the-art in knowledge-based WSD",5,"['New paper accepted at #NLPOSS @acl2018 workshop on ""The risk of  sub-optimal use of Open Source NLP Software"" with lessons for releasing #NLProc  research software <LINK>  (thread 1/5)', 'UKB is an open source collection of programs for performing, among other  tasks, knowledge-based Word Sense Disambiguation (WSD).  https://t.co/GGePy40ZLs (thread 2/5)', 'Since it was released in 2009 it has been often used by third parties  out-of-the-box in sub-optimal settings. We show that nine years later it  is the state-of-the-art on knowledge-based WSD. (thread 3/5)', 'This case shows the pitfalls of releasing open source NLP software  without optimal default settings and precise instructions for  reproducibility.  Authors should not rely on other researchers reading  the papers with care. (thread 4/5)', 'It is in the interest  of authors to include end-to-end scripts that  download all resources, perform any necessary pre-processing and  reproduce the results. We fixed this for UKB in version 3.1  https://t.co/bvXvV1FwJ6 (thread 5/5)']",18,05,981