Package: segram
Segram:a framework for semantic grammar and narrative analysisNoteThis project is still in an early stage of development,
so one should expect significant changes in the future,
including backward incompatible ones. That said, the general
concepts and design principles should remain the same or be extended,
not changed or limited. Thus, the package is suitable for experimental
usage.Segramis a software implementation of a framework for automated
semantics-oriented grammatical analysis of text data. It is implemented
in Python and based on the excellentspacypackage, which is used to solve core NLP tasks such as tokenization,
lemmatization, dependency parsing and coreference resolution.Main use cases and featuresAutomated grammatical analysis in terms of phrases/clauses focused
on detecting actions as well as subjects and objects of those actions.Flexible filtering and matching with queries expressible
in terms of properties of subjects, verbs, objects, prepositions and
descriptions applicable at the levels of individual phrases and entire
sentences.Semantic-oriented organization of analyses in terms of stories
and frames.Data serialization framework allowing for reconstructing allsegramdata after an initial parsing without access to anyspacylanguage
model.Structured vector similarity model based on weighted averages of
cosine similarities between different components of phrases/sentences
(several algorithms based on somewhat different notions of what
it means for sentences or phrases to be similar are available).Structured vector similarity model for comparing documents in terms
of sequentially shifting semantics.Hypergraphical representation of grammatical structure of sentences.Core requirementsPackageVersionpython>=3.11spacy>=3.4The required Python version will not change in the future releases
for the foreseeable future, so before the package becomes fully
mature the dependency onpython>=3.11will not be too demanding
(although it may be bumped to>=3.12as the new release is expected
soon as of time of writing - 29.09.2023).Core requirements (coreference resolution)Segramcomes with a coreference resolution component based on an
experimental model provided byspacy-experimentalpackage.
However, both at the level ofsegramandspacythis is currently
an experimental feature, which comes with a significant price tag attached.
Namely, the acceptablespacyversion is significantly limited
(see the table below). However, asspacy-experimentalgets integrated
in thespacycore in the future, these constraints will be relaxed.PackageVersionspacy>=3.4,<3,5spacy-experimental0.6.3en_coreference_web_trf3.4.0a2Supported models and languagesCurrently,only English is supportedandsegramwas tested on models:en_core_web_trf>=3.4.1(transformer-based model for the general NLP)en_core_web_lgl>=3.4.1(used for context-free word vectors)en_coreference_web_trf==3.4.0a2(for coreference resolution)InstallationPyPIpipinstallsegram
python-mspacydownloaden_core_web_trf
python-mspacydownloaden_core_web_lg# skip if word vectors are not neededWith GPU support coreference resolutionpipinstallsegram[coref,gpu]# Just one of the two options can also be selected# And language modelspython-mspacydownloaden_core_web_trf
python-mspacydownloaden_core_web_lg# The last one is a special model for the coref componentpipinstallhttps://github.com/explosion/spacy-experimental/releases/download/v0.6.1/en_coreference_web_trf-3.4.0a2-py3-none-any.whlGithub (development version)pipinstallgit+ssh://git@github.com/sztal/segram.git# + downloading language modelsWith GPU and coreference resolutionpipinstall"segram[gpu,coref] @ git+ssh://git@github.com/sztal/segram.git"Dependencies for running example notebookspipinstall-rrequirements/examples.txtBasic usageimportspacynlp=spacy.load("en_core_web_trf")nlp.add_pipe("segram",config={"vectors":"en_core_web_lg"})nlp.add_pipe("segram_coref")# Get standard 'spacy' documentdoc=nlp("The merchants travelled a long way to buy spices ""and rest in our taverns.")# Convert it to segram 'grammar' documentdoc=doc._.segramdocThe code above parses the text usingspacyand additionally applies
further processing pipeline components defined bysegram. They inject
many additional functionalities into standardspacytokens.
In particular,Docinstances are enhanced with a special extension
property._.segram, which converts them tosegramgrammar documents.
Note that the printing results is different now - the output is colored!The colors denote the partition of the document intocomponents,
which are groups of related tokens headed by a syntactically and/or
semantically important token. They are divided into four distinct types
which are marked with different colors when printing to the console.
The following (default) color scheme is:$\text{\color{orange}\bf Noun components}$$\text{\color{red}\bf Verb components}$$\text{\color{violet}\bf Description components}$$\text{\color{limegreen}\bf Preposition components}$Components are further organized into phrases, which are higher-order
and more semantically-oriented units. Crucially, while components are
non-overlapping and form a partition of the sentence, the phrases
can be nested in each other and form a directed acyclic graph (DAG).Running examplesExamplesareJupyter notebookswith some sample analyses and tutorials. Below are instructions for
setting up an environment sufficient for running the notebooks.gitclonegit@github.com:sztal/segram.gitcdsegram
condaenvcreate-fenvironment-coref.yml# default env name is 'segram'# In this case the versions of all language models are fixed# so they are installed automatically with the rest of the dependenciescondaactivatesegram
pipinstall--editable.# OR to allow for GPU acceleration:pipinstall--editable.["gpu"]# Finally, install some extra dependencies used in the notebookspipinstall-rrequirements/docs.txtDevelopment and contributingSeedevelopment and contributing guidelines.FeedbackIf you have any suggestions or questions aboutsegramfeel free to email
me at<stalaga@protonmail.com>.If you encounter any errors or problems, please also let me know!Open an Issuein theGitHub repository.AuthorsSzymon Talaga,stalaga@protonmail.com