Package: catgram
catgramBasic tools for working with categorial grammarsThis is a simple Python package providing some basic tools for working with
categorial grammars.
Development is on-again, off-again.
Bug reports and feature requests are welcome—especially if it's for an item on
theTODO listbelow, as you'd be providing extra motivation! :)This package also includes a CCG dependency evaluation script that implements
decomposed scoring as specified inDecomposed scoring of CCG
dependencies.
Seebelowfor examples.
If you use decomposed scoring in your research, please cite the paper:Aditya BhargavaandGerald
Penn. 2023.Decomposed scoring of CCG
dependencies.  InProceedings
of the 61st Annual Meeting of the Association for Computational Linguistics
(Volume 2: Short Papers), pages 1030–1040, Toronto, Canada.  Association for
Computational Linguistics.The script can also do the regular CCG dependency evaluation
(examples).In general, if you use this package in your research, please include a link tothe GitHub repository, and remember
to cite any appropriate research papers depending on your usage (e.g., for
decomposed scoring as mentioned above; cite(Lewis and Steedman,
2014)if
you use this package's implementation of their head-finding rules; etc.).RequirementsPython 3.10+lambda_calculusPython package (will be installed automatically bypipcommand below)InstallationIn your environment of choice:$ pip3 install catgramIf you only want to run the evaluation script, you might want to consider usingpipxto keep the installation isolated:$ pipx install catgram
$ ccg_depeval -hOr use it to run the script in a temporary environment:$ pipx run --spec catgram ccg_depeval -hExamplesDecomposed scoringIn additional to subcategorial labelling and alignment, decomposed scoring
specifies the inclusion of root nodes.
Most parsers do not explicitly specify these, and if they do, they must be
extracted from heads specified in the.autofile (as far as I know, EasyCCG
is the only parser that does this).
Theccg_depevalscript includes the facility for extracting root dependencies
as necessary from parser.autofiles (most statistical CCG parsers at least
have the option to output these).Usage is as follows:$ ccg_depeval ground_truth_deps sys_deps ground_truth.auto sys.autowhere:ground_truth_depsis the ground-truth dependencies, usually as produced
by theparg2ccgbank_depsscript from C&C (the actual filenames will bewsj00.ccgbank_depsfor the dev set orwsj23.ccgbank_depsfor the test
set). The originalPARGfile format from CCGbank can also be used.sys_depsis the dependencies predicted by a statistical parser, usually as
produced by thegenerateprogram from C&C (I recommend using what's
available in the Java version of C&C as it is updated compared to what's in
the original C&C package).ground_truth.autois the ground-truth.autofile (e.g., straight from
CCGbank). The heads specified in this file are followed directly according
to the syntax specified in CCGbank.sys.autois the parse preidcted by the statistical parser. By default, the
head-finding rules ofLewis and Steedman
(2014)are followed to extract the root
node.Note: instead of.autofiles, you can also provide root node information
directly in a.rootsfile.
The format is as produced by theccg_rootsscript
(examples).A warning will be issued if there is no root available for a sentence, including
if the last two arguments aren't specified.
You can use the-roption to suppress this warning if you don't want to fuss
with root nodes:$ ccg_depeval -r ground_truth_deps sys_depsThis can be handy for, e.g., evaluating the Java version of the C&C parser,
which doesn't produce a.autofile and instead produces a.depsfile
directly.
Of course, omitting the root nodes will produce different scores.
See(Bhargava and Penn, 2023)and(Bhargava, 2022, chapter 5)for
examples of why you should include root nodes.Other options of the script allow you to control whether subcategorial labelling
and/or alignment are used as well, or to print per-sentence scores. See the
script's help for full details:$ ccg_depeval -hStandard CCG scoringFor convenience, you can use the-sflag when runningccg_depevalto revert
to the standard CCG scoring method:$ ccg_depeval -s ground_truth_deps sys_depsExtracting rootsThis package also includes a standalone root-extraction script.
For example:$ ccg_roots -m ls14 sys.auto
will_8 S[dcl]
is_3 S[dcl]
...It's important to use-m ls14for a.autofile generated by most statistical
parsers and-m autofile(which is the same as omitting the-moption) for
a.autofile where the heads specified as per the.autofile syntax are
indeed the desired heads.
The latter case is applicable to CCGbank's.autofiles but not those produced
by most parsers, since they do not indicate the semantic heads.
(EasyCCG is the biggest exception to this, and indeed, thels14rules are the
same as used by that parser.)
If you use thels14option for something in your research, make sure to citeLewis and Steedman (2014)as Section 3.5
of that paper is where the rules were originally specified.See the script help for full usage details:$ ccg_roots -hTODOTestsAbility to directly evaluate CCG .auto filesThis would re-implement the functionality of thegenerateprogram from
C&C (or the more directly-integrated version of this process in Java C&C)
so that new parsers wouldn't need to go back to C&C to do the evaluationsExamples for basic usage (forCategoryTreeandTermGraph)For now, take a look atdependencies.pyfor examples of how to useCategoryTree.
If examples are even slightly of interest to you, please submit a GitHub
issue asking for them as doing so will help motivate me to add them!Other tools that might be useful?Evaluation scripts (e.g., for evaluating statistical parsers)Visualization tools (CCG dependency graphs, LCG term graphs; outputs to
SVG, LaTeX...)LicenseUnless otherwise stated, all files in this package are subject to the below
copyright and license. The main exception iscandc_ignore.py, which is derived from the
original C&C package and thus covered by the C&C System Licence Agreement.
The code therein is reproduced with permission for inclusion in this package.Copyright 2023 Aditya BhargavaLicensed under the Apache License, Version 2.0 (the "License"); you may not use
the files in this repository except in compliance with the License. You may
obtain a copy of the License athttps://www.apache.org/licenses/LICENSE-2.0or
in theLICENSEfile in this repository.Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.