Package: slate-nlp
This is a tool for labeling text documents.
Slate supports annotation at different scales (spans of characters, tokens, and lines, or a document) and of different types (free text, labels, and links).
This covers a range of tasks, such as Part-of-Speech tagging, Named Entity Recognition, Text Classification (including Sentiment Analysis), Discourse Structure, and more.Why use this tool over the range of other text annotation tools out there?FastTrivial installationFocuses all of the screen space on annotation (good for large fonts)Terminal based, so it works in constrained environments (e.g. only allowed ssh access to a machine)Not difficult to configure and modifyNote - this repository isnotfor the "Segment and Link-based Annotation Tool, Enhanced", which can be foundhereand was first presented atLREC 2010.
See 'Citing' below for additional notes on that work.InstallationTwo options:1. Install with pippipinstallslate-nlpThen run from any directory in one of two ways:slate
python -m slate2. Or download and run without installingEither download as a zip file:curlhttps://codeload.github.com/jkkummerfeld/slate/zip/master-o"slate.zip"unzipslate.zipcdslate-masterOr clone the repository:gitclonehttps://github.com/jkkummerfeld/slatecdslateThen run with either of:python slate.py
./slate.pyTo run from another directory, use:python PATH_TO_SLATE/slate.py
PATH_TO_SLATE/slate.pyRequirementsThe code requires only Python (2 or 3) and can be run out of the box.
Your terminal must be at least 80 characters wide and 20 tall to use the tool.CitingIf you use this tool in your work, please cite:@InProceedings{acl19slate,
  title     = {SLATE: A Super-Lightweight Annotation Tool for Experts},
  author    = {Jonathan K. Kummerfeld},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations},
  location  = {Florence, Italy},
  month     = {July},
  year      = {2019},
  pages     = {7--12},
  doi       = {10.18653/v1/P19-3002},
  url       = {https://aclweb.org/anthology/papers/P/P19/P19-3002/},
  software  = {https://jkk.name/slate},
}While presenting this work at ACL I learned of another annotation tool called SLATE.
That tool was first described in "Annotation Process Management Revisited",Kaplan et al. (LREC 2010)and then in "Slate - A Tool for Creating and Maintaining Annotated Corpora",Kaplan et al. (JLCL 2011).
It takes a very different approach, using a web based interface that includes a suite of project management tools as well as annotation.
The code it available athttps://bitbucket.org/dainkaplan/slate/wiki/Home.Getting StartedNote: if you used pip to install, reaplcepython slate.pywithslateeverywhere below.Runpython slate.py <filename>to start annotating<filename>with labels over spans of tokens.
The entire interface is contained in your terminal, there is no GUI.
With command line arguments you can vary properties such as the type of annotation (labels or links) and scope of annotation (characters, tokens, lines, documents).The input file should be plain text, organised however you like.
Prepare the data with your favourite sentence splitting and/or tokenisation software (e.g.,SpaCy).
If you use Python 3 then unicode should be supported, but the code has not been tested extensively with non-English text (please share any issues!).When you start the tool it displays a set of core commands by default.
These are also specified below, along with additional commands.The tool saves annotations in a separate file (<filename>.annotationsby default, this can be varied with a file list as described below).
Annotation files are formatted with one line per annotated item.
The item is specified with a tuple of numbers.
For labels, the item is followed by a hyphen and the list of labels.
For links, there are two items on the line before the hyphen.
For example, these are two annotation files, one for labels of token spans and the other for links between lines:==> label.annotations <==
(2, 1) - label:a
((3, 5), (3, 8)) - label:a
(7, 8) - label:s label:a

==> link.annotations <==
13 0 - 
13 7 - 
16 7 -A few notes:The second label annotation is on a span of tokens, going from 5 to 8 on line 3.The third label annotation has two labels.The line annotations only have one number to specify the item.When the same line is linked to multiple other lines, each link is a separate item.TutorialsIncluded in this repository are a set of interactive tutorials that teach you how to use the tool from within the tool itself.TaskCommandNamed Entity Recognition annotationpython slate.py tutorial/ner.md -t categorical -s token -o -c ner-book.config -l log.tutorial.ner.txt -sl -smLabelling spans of text in a documentpython slate.py tutorial/label.md -t categorical -s token -o -l log.tutorial.label.txtLinking lines in a documentpython slate.py tutorial/link.md -t link -s line -o -l log.tutorial.link.txtExample WorkflowThis tool has already been used for two annotation efforts involving multiple annotators (Durrett et al., 2017andKummerfeld et al., 2018).
Our workflow was as follows:Create a repository containing (1) the annotation guide, (2) the data to be annotated divided into user-specific folders.Each annotator downloaded slate and used it to do their annotations and commit the files to the repository.Either the whole group or the project leader went through files that were annotated by multiple people, using the adjudication mode in the tool.Comparing AnnotationsTo use adjudication mode, create a file,example.txt, similar to the following (you can have as many annotators as you like):raw-text0 adjudicated-anno0 ((1000,),(1000,)) anno0.1 anno0.2 anno0.3
raw-text1 adjudicated-anno1 ((1000,),(1000,)) anno1.1 anno1.2
raw-text2 adjudicated-anno2 ((1000,),(1000,)) anno2.1 anno2.2 anno2.3 anno2.4To save time, it is best to initialiseadjudicated-annoNwith the lines everyone agreed on:for i in 0 1 2 ; do
  count=`ls anno${i}.* | wc -l`
  cat anno${i}.* | sort | uniq -c | awk -v count=$count '$1 == count' | sed 's/^ *[0-9]* *//' > matching
doneThen run the tool as if you are annotating, for example for linking lines:python ../learn-anno/slate/slate.py -d example.txt -pf -t link -s line -o -l log.adj.txt --do-not-show-linkedDetailed Usage InstructionsInvocation optionsusage: slate.py [-h] [-d DATA_LIST [DATA_LIST ...]] [-t {categorical,link}]
                [-s {character,token,line,document}] [-c CONFIG_FILE] [-l LOG_PREFIX] [-ld]
                [-sh] [-sl] [-sp] [-sm] [-r] [-o] [-ps] [-pf] [--do-not-show-linked]
                [--alternate-comparisons]
                [data ...]

A tool for annotating text data.

positional arguments:
  data                  Files to be annotated

optional arguments:
  -h, --help            show this help message and exit
  -d DATA_LIST [DATA_LIST ...], --data-list DATA_LIST [DATA_LIST ...]
                        Files containing lists of files to be annotated
  -t {categorical,link}, --ann-type {categorical,link}
                        The type of annotation being done.
  -s {character,token,line,document}, --ann-scope {character,token,line,document}
                        The scope of annotation being done.
  -c CONFIG_FILE, --config-file CONFIG_FILE
                        A file containing configuration information.
  -l LOG_PREFIX, --log-prefix LOG_PREFIX
                        Prefix for logging files
  -ld, --log-debug      Provide detailed logging.
  -sh, --show-help      Show help on startup.
  -sl, --show-legend    Start with legend showing.
  -sp, --show-progress  Start with progress showing.
  -sm, --show-mark      Start with mark showing.
  -r, --readonly        Do not allow changes or save annotations.
  -o, --overwrite       If they exist already, read and overwrite output files.
  -ps, --prevent-self-links
                        Prevent an item from being linked to itself.
  -pf, --prevent-forward-links
                        Prevent a link from an item to one after it.
  --do-not-show-linked  Do not have a special color to indicate any linked token.
  --alternate-comparisons
                        Activate alternative way of showing different annotations (one colour
                        per set of markings, rather than counts).You may also define arguments in a file and pass them in as follows:pythonslate.py@arguments.txtKeybindingsThe tool shows files one at a time in plain text. Default commands are:TypeKeyLabelling AffectLinking AffectMovementjor←move to the leftmove selected item to the leftior↑move up a linemove selected item up a lineoor↓move down a linemove selected item down a line;or→move to the rightmove selected item to the rightJor [Shift+←]go to the start of the linemove linking item to the leftIor [Shift+↑]go to first linemove linking item up a lineOor [Shift+↓]go to last linemove linking item down a line:or [Shift+→]go to the end of the linemove linking item to the rightEdit Spanmextend leftextend selected item leftkcontract left sidecontract selected item left/extend rightextend selected item rightlcontract right sidecontract selected item rightM-extend linking item leftK-contract linking item left?-extend linking item rightL-contract linking item rightLabel Annotation (default)Spacethena[un]mark this item as a-Spacethens[un]mark this item as s-Spacethend[un]mark this item as d-Spacethenv[un]mark this item as v-Link Annotationd-create a link and move right / downD-create a linkEither Annotation modeuundo annotation on this itemundo all annotations for the current itemShared commands:TypeModeKeyAffectSearchingNormal\enter query editing modeQuery?orEnterexit query editing modeQuery!orBackspacedelete last character in queryQuerycharacters except?and!add character to queryNormalpgo to previous matchNormalngo to next matchNormalPgo to previous match for linking lineNormalNgo to next match for linking lineAssigning text labelsNormaltenter label editing modeLabel?orEnterexit label editing mode and assign the labelLabel!orBackspacedelete last character in labelLabelcharacters except?and!add character to labelSaving, exiting, etcNormal]save and go to next fileNormal[save and go to previous fileNormalqsave and quitNormalssaveNormalQquitMiscNormal#toggle line numbersNormalhtoggle help info (default on)Normal{orPAGE-UPshift view up 5 linesNormal}orPAGE-DOWNshift view down 5 linesNormal>thenptoggle showing progress through filesNormal>thenltoggle showing legend for labelsNormal>thenmtoggle showing the mark on the current itemNote: special keys such asENTERandBACKSPACEmay not work on non-OS-X operating systems. That is why in all places where they are used we have an alternative as well.MiscTo annotate multiple files, specify more than one as an argument.
For greater control, provide a list of files in a file specified with--data-list/-d.
The list should be formatted as follows, where [] indicate optional values:raw_file [annotation_file [starting_position [additional_annotation_files]]]For example, these commands will create a file list, use it, then return to it later:find.-name*txt>filenames_todo
./slate.py-dfilenames_todo-ldo_later# ... do some work, then quit, go away, come back..../slate.py-ddo_later.todo-ldo_even_later-oNote, the-oflag is added so it will allow you to edit the annotations you have already created.
Otherwise the system will complain that you are overwriting existing annotation files.When theadditional_annotation_filesare included it activates an adjudication mode.
By default, all annotations that appear in all additional files are added to the current annotations.
Disagreements are coloured in the text, but will disappear once a decision is made (using the normal annotation commands).CustomisationColours and keys are customisable. For labelling, the default is:Underlined, current selected itemGreen on black, 'a' itemsBlue on black, 's' itemsMagenta on black, 'd' itemsRed on black, 'v' itemsCyan on black, multiple types for a single tokenFor linking, the default is:Underlined, current selected itemGreen on black, current linking itemBlue on black, item is linked to the current linking itemYellow on black, item is in some link, though not with the current linking itemModifying the CodeSlate has a relatively small codebase (~2,200 lines) and is designed to make adding new functionality not too hard.
The code is divided up as follows:annotate.py, the main program, this has the core loop that gets user input.config.py, contains the default configuration, including colours and keyboard bindings.data.py, classes to read, store and write data.view.py, rendering the screen.Logic for determining what colour goes where is split across two parts of the code.
Indata.py, the set of labels for an item is determined.
Inview.py, that set of labels is used to choose a suitable colour.Adding a new command involves:Adding the name and key toinput_action_listinconfig.pyAdding a mapping from the name to a function inaction_to_functioninannotate.pyAdding or modifying a function inannotate.pyModifyingdata.pyorview.pyto apply the actionChanging the label set / Adding labelsThe label set is defined in your config file (see an example confighere).See lines like this for label definitions:Label:          a                         SPACE_a greenThe format is:Label:        <label>                    <command> <colour>You can add / edit / remove these lines to define your own label scheme. For example, for NER you may want to do:Label:          O                         SPACE_a green
Label:          LOC                       SPACE_s blue
Label:          PER                       SPACE_d red
Label:          ORG                       SPACE_f yellow
Label:          MISC                      SPACE_v magentaThe current set of available colours is: [green, blue, white, cyan, magenta, red, yellow].
Note that by default white is used for regular text and cyan is used for cases where multiple labels apply to the same content.To define more colours, edit the top ofslate/config.py.
By varying both the text colour (foreground) and background colour you can achieve quite a range of variations.
You can also define any RGB colour you want using the cursesinit_colorfunction and theinit_pairfunction.QuestionsIf you have a question please either:Open an issue ongithub.Mail me atjonathan.kummerfeld@sydney.edu.au.ContributionsIf you find a bug in the code, please submit an issue, or even better, a pull request with a fix.AcknowledgmentsThis tool is based in part upon work supported by IBM under contract 4915012629, and by ONR under MURI grant N000140911081.
Any opinions, findings, conclusions or recommendations expressed are those of the authors and do not necessarily reflect the views of IBM.