Package: semtagger
SEM: a free NLP tool to tag and create annotated dataSEM (Segmenteur-Étiqueteur Markovien)is a free NLP tool relying on Machine Learning technologies, especially CRFs. SEM provides powerful and configurable preprocessing and postprocessing.SEM also has an online version.Main SEM featuresA GUI for manual annotation (requires TkInter)from terminal: runsem annotation_guifast annotation: keyboard shortcuts and document-wide annotation broadcastcan load pre-annotated filessupport for hierarchical tags (dot-separated, eg: "noun.common")handles multiple input formatexport in different formatsA GUI for easier use (requires TkInter)on Linux: double-clic on sem_gui.shon Windows: double-clic on sem_gui.batfrom terminal: runsem guisegmentationsegmentation for: French, Englisheasy creation and integration of new tokenisersfeature generationXML file to write features without coding themsingle-token and multi-token dictionary featuresRegular expression featuressequenced featurestrain/label modedisplay option for features that are useful for generation, but not needed in outputexporting outputsupported export formats: CoNLL, text, HTML (from plain text), two XML-TEI (one specific to NP-chunks and another one for the rest)easy creation and integration of new exportersextension of existing featuresautomatic integration of new segmenters and exporterssemi automatic integration of new feature functionseasy creation of new CSS formats for HTML exportsFirst steps with SEMinstall SEMseeinstall.mdIt will compile Wapiti and create necessary directories. Currently, SEM datas are located in~/sem_datarun testsrunsem --testin a terminalrun SEMrun GUI (see "main features" above) and annotate "non-regression/fr/in/segmentation.txt"or run:sem tagger resources/master/fr/NER.xml ./non-regression/fr/in/segmentation.txt -o sem_output(optional) go through theSEM tutorial(optional) go through some of the examples given in theexamplesfolder.External resources used by SEMFrench TreebankbyAbeillé et al. (2003): corpus used for POS and chunking.NER annotated French Treebank bySagot et al. (2012): corpus used for NER.Lexique des Formes Fléchies du Français (LeFFF)byClément et al. (2004): french lexicon of inflected forms with various informations, such as their POS tag and lemmatization.WapitibyLavergne et al. (2010): linear-chain CRF library.setuptools: to install SEM.Tkinter: for GUI modules (they will not be installed if Tkinter is not present).Windows only:MinGW64: used to compile Wapiti on Windows.Windows only:POSIX threads for Windows: if you want to multithread Wapiti on Windows.SEM source code vs SEM resourcesWhile SEM source code is released under the MIT licence, the resources present in this repository (including but not limited to lexica and models) have each their own licence which may (usually does) differ. SEM provides them for research, teaching and personal use, they may not be used for any other reason. If you wish to create your own models, you will have to provide your own lexica, annotate your own data (or use data annotated under a permissive licence) and train your models yourself. You have to to check every resource licence and its compatibility for your project.SEM resources are being moved to a specific repository:SEM-resources. SEM will provide means to download them on your computer.Planned changes (for latest changes, see changelog.md)add lemmatiserhave more unit testspublish some benchmarksSEM references (with task[s] of interest)DUPONT, Yoann et PLANCQ, Clément. Un étiqueteur en ligne du Français. session démonstration de TALN-RECITAL, 2017, p. 15.Online interface(best RECITAL paper award)DUPONT, Yoann. Exploration de traits pour la reconnaissance d’entités nommées du Français par apprentissage automatique. RECITAL, 2017, p. 42.Named Entity Recognition (new, please use this one)TELLIER, Isabelle, DUCHIER, Denys, ESHKOL, Iris, et al. Apprentissage automatique d'un chunker pour le français. In : TALN2012. 2012. p. 431–438.ChunkingTELLIER, Isabelle, DUPONT, Yoann, et COURMET, Arnaud. Un segmenteur-étiqueteur et un chunker pour le français. JEP-TALN-RECITAL 2012Part-Of-Speech TaggingchunkingDUPONT, Yoann et TELLIER, Isabelle. Un reconnaisseur d’entités nommées du Français. session démonstration de TALN, 2014, p. 40.Named Entity Recognition (old, please do not use)SEM references (bibtex format)@inproceedings{dupont2017etiqueteur,
    title={Un{'e}tiqueteur en ligne du fran{\c{c}}ais},
    author={Dupont, Yoann and Plancq, Cl{'e}ment},
    booktitle={24e Conf{'e}rence sur le Traitement Automatique des Langues Naturelles (TALN)},
    pages={15--16},
    year={2017}}@inproceedings{dupont2017exploration,
  title={Exploration de traits pour la reconnaissance d’entit{'e}s nomm{'e}es du Fran{\c{c}}ais par apprentissage automatique},
  author={Dupont, Yoann},
  booktitle={24e Conf{'e}rence sur le Traitement Automatique des Langues Naturelles (TALN)},
  pages={42},
  year={2017}}@inproceedings{tellier2012apprentissage,
  title={Apprentissage automatique d'un chunker pour le fran{\c{c}}ais},
  author={Tellier, Isabelle and Duchier, Denys and Eshkol, Iris and Courmet, Arnaud and Martinet, Mathieu},
  booktitle={TALN2012},
  volume={2},
  pages={431--438},
  year={2012}}@inproceedings{tellier2012segmenteur,
  title={Un segmenteur-{'e}tiqueteur et un chunker pour le fran{\c{c}}ais (A Segmenter-POS Labeller and a Chunker for French)[in French]},
  author={Tellier, Isabelle and Dupont, Yoann and Courmet, Arnaud},
  booktitle={Proceedings of the Joint Conference JEP-TALN-RECITAL 2012, volume 5: Software Demonstrations},
  pages={7--8},
  year={2012}}@article{dupont2014reconnaisseur,
  title={Un reconnaisseur d’entit{'e}s nomm{'e}es du Fran{\c{c}}ais (A Named Entity recognizer for French)[in French]},
  author={Dupont, Yoann and Tellier, Isabelle},
  journal={Proceedings of TALN 2014 (Volume 3: System Demonstrations)},
  volume={3},
  pages={40--41},
  year={2014}}