Package: averell
Averell, the python library and command line interface that facilitates working
with existing repositories of annotated poetry. Averell is able to download an annotated corpus and reconcile different
TEI entities to provide a unified JSON output at the desired granularity.
That is, for their investigations some researchers
might need the entire poem, poems split line by line,
or even word by word if that is available. Averell allows to specify the
granularity of the final generated dataset, which is a combined JSON with all
the entities in the selected corpora.
Each corpus in the catalog must specify the parser to produce the expected data format.Free software: Apache Software License 2.0Available corpora (version 1.1.0)idnamelangsizedocswordsgranularitylicense1Disco V2.1
(disco2_1)es22M4088381539stanza
lineCC-BY2Disco V3
(disco3)es28M4080377978stanza
lineCC-BY3Sonetos Siglo
de Oro
(adso)es6.8M5078466012stanza
lineCC-BY-NC
4.04ADSO 100
poems corpus
(adso100)es128K1009208stanza
lineCC-BY-NC
4.05Poesía Lírica
Castellana Siglo
de Oro
(plc)es3.8M475299402stanza
line
word
syllableCC-BY-NC
4.06Gongocorpus (gongo)es9.2M48199079stanza
line
word
syllableCC-BY-NC-ND
3.0
FR7Eighteenth Century
Poetry Archive
(ecpa)en2400M30842063668stanza
line
wordCC
BY-SA
4.08For Better
For Verse
(4b4v)en39.5M10341749stanza
lineUnknown9Métrique en
Ligne (mel)fr183M50811850222stanza
lineUnknown10Biblioteca Italiana
(bibit)it242M253417121246stanza
line
wordUnknown11Corpus of
Czech Verse
(czverse)cs4100M6642812636867stanza
line
wordCC-BY-SA12Stichotheque
(stichopt)pt11.8M1702168411stanza
lineUnkwownDocumentationhttps://averell.readthedocs.io/InstallationTo install averell, run this command in your terminal:pip install averellThis is the preferred method to install averell, as it will always install
the most recent stable release.If you don’t havepipinstalled, thisPython installation guidecan guide
you through the process.UsageTo show averell help:averell --helpTo list all available corpora:averell listVisualization example of one of the available corpora:id  name                 lang    size      docs    words  granularity    license
----  -------------------  ------  ------  ------  -------  -------------  -----------
   1  Disco V2.1           es      22M       4088   381539  stanza         CC-BY
                                                            linedownloadDownload desired corpora into “mycorpora” folder:averell download 2 3 --corpora-folder my_corporaExample of poem in TEI format obtained from one of the corpora:<TEIxmlns="http://www.tei-c.org/ns/1.0"><teiHeader><fileDesc><titleStmt><title>SpanishMetricalPatternsBank:GoldenAgeSonnets.</title><principal>BorjaNavarroColorado</principal><respStmt><name>MaríaRibesLafoz</name><name>NoeliaSánchezLópez</name><name>BorjaNavarroColorado</name><resp>Metricalpatternsannotation</resp></respStmt></titleStmt><publicationStmt><publisher>NaturalLanguageProcessingGroup.DepartmentofSoftwareandComputingSystems.UniversityofAlicante(Spain)</publisher></publicationStmt><sourceDesc><bibl><title>Sonetos</title>de<author>GarcilasodeLaVega</author>.<publisher>BibliotecaVirtualMigueldeCervantes</publisher>,ediciónde<editorrole="editor">RamónGarcíaGonzález</editor>.</bibl></sourceDesc></fileDesc><encodingDesc><metDeclxml:id="bncolorado"type="met"pattern="((\+|\-)+)*"><metSymvalue="+">stressedsyllable</metSym><metSymvalue="-">unstressedsyllable</metSym></metDecl><metDecl><p>Allmetricalpatternshavebeenmanuallychecked.</p></metDecl></encodingDesc></teiHeader><text><body><head><title>-XX-</title></head><lgtype="cuarteto"><ln="1"met="-++--++--+-">Contalfuerzayvigorsonconcertados</l><ln="2"met="-----+-+-+-">paramiperdiciónlosdurosvientos,</l><ln="3"met="--+--+---+-">quecortaronmistiernospensamientos</l><ln="4"met="+----++--+-">luegoquesobremífueronmostrados.</l></lg><lgtype="terceto"><ln="5"met="-++--+---+-">Elmalesquemequedanloscuidados</l><ln="6"met="---+-----+-">ensalvodeestosacontecimientos,</l><ln="7"met="-++--+---+-">quesonduros,ytienenfundamentos</l></lg></body></text></TEI>Generated example JSON file from input XML/TEI poem into
my_corpora/{corpus}/averell/parser/{author_name}/{poem_name}.json{"manually_checked":true,"poem_title":"-XX-","author":"Garcilaso de La Vega","stanzas":[{"stanza_number":"1","stanza_type":"cuarteto","lines":[{"line_number":"1","line_text":"Con tal fuerza y vigor son concertados","metrical_pattern":"-++--++--+-"},{"line_number":"2","line_text":"para mi perdición los duros vientos,","metrical_pattern":"-----+-+-+-"},{"line_number":"3","line_text":"que cortaron mis tiernos pensamientos","metrical_pattern":"--+--+---+-"},{"line_number":"4","line_text":"luego que sobre mí fueron mostrados.","metrical_pattern":"+----++--+-"}],"stanza_text":"Con tal fuerza y vigor son concertados\npara mi perdición los duros vientos,\nque cortaron mis tiernos pensamientos\nluego que sobre mí fueron mostrados."},{"stanza_number":"2","stanza_type":"terceto","lines":[{"line_number":"5","line_text":"El mal es que me quedan los cuidados","metrical_pattern":"-++--+---+-"},{"line_number":"6","line_text":"en salvo de estos acontecimientos,","metrical_pattern":"---+-----+-"},{"line_number":"7","line_text":"que son duros, y tienen fundamentos","metrical_pattern":"-++--+---+-"}],"stanza_text":"El mal es que me quedan los cuidados\nen salvo de estos acontecimientos,\nque son duros, y tienen fundamentos"}]}exportNow we can combine and join these corpora through “granularity” selection:averell export 2 3 --granularity line --corpora-folder my_corpora --filename export_1It produces an single JSON file with information about all the lines in
those corpora. Example oftworandom lines in the file mycorpora/export_1.json:{"line_number":"5","line_text":"¿Has visto que en el mismo lugar donde","metrical_pattern":"++---+--++-","stanza_number":"2","manually_checked":false,"poem_title":" - II - ","author":"Mira de Amescua","stanza_text":"¿Has visto que en el mismo lugar donde\nbordado estuvo el cristalino velo\nun bordado terliz de escarcha y hielo\nhace que el campo de verdor se monde?","stanza_type":"cuarteto"}{"line_number":"10","line_text":"el que a lo cierto no a lo incierto mira,","metrical_pattern":"---+-+-+-+-","stanza_number":"3","manually_checked":false,"poem_title":"- VIII - Considerando un sepulcro y los que están en él ","author":"Lope de Zarate","stanza_text":"De aquí si que consigue el ser dichoso\nel que a lo cierto no a lo incierto mira,\npues le adorna lo eterno fastuoso;","stanza_type":"terceto"}By default,exportwill download corpora if needed. To avoid this behaviour, the flag--no-downloadcan be passed in.Exported corpora can be easily loaded into Pandasaverellexportadso100--filenameadso100.jsonimportpandasaspdadso100=pd.read_json(open("adso100.json"))A note on IDSIDS can be numeric identifiers in theaverell listoutput, corpus shortcodes (shown between parenthesis), the speciall literalallto refer to all corpora, or two-letter ISO language codes to refer to avaliable corpora in a specific language.For example, the commandaverell export 1 bibit frwill export DISCO V2.1, the Biblioteca Italiana poetry corpus, and all corpora tagged with the French languge tag in a single file spliting poems line by line.Changelog1.2.1 (2021-07-14)Added two new readers:Stichotheque Portuguese corpusCorpus of Czech Verseexport_filenameis also returned as an output ofexport_corporaFix writing function so as not to duplicate informationChangenamekey tocorpusfor clarityFix path split on Windows systemsAdd corpus name to averell output files1.1.0 (2020-09-18)AddedBiblioteca Italiana (bibit)readerAdded Archivio Metrico Italiano info to Biblioteca Italiana readerReduced fixtures file sizeAdding a tmp file to git ignoreAdding languages and some other cosmetic changesFixing an error with the expected output of theaverell listcommandAdding slugs, langs, and ‘all’ todownloadandexportFixing coverageAdding documentation and fixing a test1.0.3 (2020-09-03)Addedexport--filenameoptionAdded two new readers:For better for verseMétrique en ligne1.0.2 (2020-06-23)Added two new readers:ECPA corpusGongocorpusMinor bug fixes1.0.1 (2020-05-18)Setting up bumbpversionIntegration with Zenodo1.0.0 (2020-04-29)Remove commits-since code blockAdding automated deployments to PyPI on tag releasesAdded menuRemove comments and cleaner code fixesFix sorted output of testsAdded proper documentation and coverage testsAdded tests forexportfunctionAddedexportfunctionAddedTEI_NAMESPACEas a constantFixed docs. Fixed loads withPath. Fixed logging errorsAdded tests0.0.1 (2020-01-08)First release on PyPI.