A Modern Replacement for BSD spell(1) - Abhinav Upadhyay
Channel: EuroBSDcon
Date: 2019-10-16

Description:

The spell(1) utility in NetBSD is quite ancient and primitive. It is several
decades old based on an implementation which came with AT&T Version 7 Unix 1975.
It falls short of expectations on multiple fronts. For instance, it is not
capable of suggesting corrections for the misspelled words, it merely checks a
dictionary to decide if a spelling is correct or not. Not only that, it does
spell checking based on a set of rules, which are pretty tightly
tied to the English language, making it unusable for text involving
other languages. Also, those rules are not very accurate and are prone
to failure in case of many common misspellings.
Because of so many glaring problems in the existing spell(1) and for the lack
of a reusable library interface for other applications to add spell checking
functionality, I started work on writing a new implementation of spell(1)
with the goal of overcoming these problems and also building a library
to add spell checking feature to apropos(1) in NetBSD.

In this presentation I will talk about:

- current implementation of spell(1) and demonstrate its short comings
spell correction algorithms – Levenshtein distance, metaphone and n-grams
work done to implement these techniques resulting in a new spell(1) with a
library interface
- performance comparison of the new implementation against against one of the
most popular open source spell checker – GNU Aspell
- Demonstrate integration of the library interface with other applications such
as apropos(1) and sh(1) to do spell correction and context sensitive text
auto-completion

Speaker biography:

Abhinav Upadhyay is a NetBSD developer and works for Reve Marketing,
a martech startup of Pramati Technologies, as a Senior Software Engineer.
Abhinav first worked for The NetBSD Foundation during Google Summer of Code 2011.
He is responsible for rewriting apropos(1) in NetBSD, implementing full text
search for man pages. He has also created man-k.org – a web interface to
NetBSD’s apropos(1). His interests lie in the areas of systems software
and machine learning. He enjoys working in the cross section of the two domains
to build novel tools and interface.