internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: cltk/cltk

Issue:
Compile Poeti d’Italia for CLTK
http://www.mqdq.it/mqdq/poetiditalia/indice_autori_alfa.jsp?scelta=AZ&path=metri_opere



Discussion:
This site will be challenging to crawl. I don't want to invest the time right now to get this data.

That said, we should keep this site in mind as a good project for a non-programmer to essentially do a copy-paste walk-through of the site, making an index along the way. There's a lot of unusual stuff in here.

I agree.. lots of unusual material and with metrical patterns! My only
concern is I don't see any open access statement of cc license, only "all
rights reserved", what shall we do?  Also I don't know if the site is
mulfunctioning, but i only could get an index in pdf both in "deoque" and
in "poeti di italia". In the latter some texts can be seen and eventually
copy pasted as you suggest...

2014-10-14 12:11 GMT-03:00 Kyle P. Johnson notifications@github.com:

> This site will be challenging to crawl. I don't want to invest the time
> right now to get this data.
> 
> That said, we should keep this site in mind as a good project for a
> non-programmer to essentially do a copy-paste walk-through of the site,
> making an index along the way. There's a lot of unusual stuff in here.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kylepjohnson/cltk/issues/46#issuecomment-59061742.

@marpozzi Would you have any interest in writing to the owner of these sites  at mqdq@memoratapoetis.it ? I think this would be a good first step.

@souravsingh has obtained permission for Poeti d’Italia. Assigning to him.

@souravsingh How much work to finish this issue on the  Poeti d’Italia corpus?

The work is being done in the repository here-https://github.com/souravsingh/latin_text_poeti_ditalia

Pinging @souravsingh Is this corpus ready to move over? I took a look again and it looks pretty good

Sorry for being inactive @kylepjohnson. I need to properly format the JSON files before it is pushed to upstream branch. Might take a day or two.

Ok, thank you for the fast response. Get it as good as you can, but this doesn't need to 100% perfect.

@kylepjohnson I have transferred the repository to cltk latin team. 


Resolving patch:
diff --git a/cltk/corpus/latin/corpora.py b/cltk/corpus/latin/corpora.py
--- a/cltk/corpus/latin/corpora.py
+++ b/cltk/corpus/latin/corpora.py
@@ -88,5 +88,9 @@
      {'location': 'remote',
      'type': 'text',
      'name': 'latin_text_corpus_grammaticorum_latinorum',
-     'origin': 'https://github.com/cltk/latin_text_corpus_grammaticorum_latinorum.git'}
+     'origin': 'https://github.com/cltk/latin_text_corpus_grammaticorum_latinorum.git'},
+    {'location': 'remote',
+     'type': 'text',
+     'name': 'latin_text_poeti_ditalia',
+     'origin': 'https://github.com/cltk/latin_text_poeti_ditalia.git'}
 ]


Test patch:
Repository: cltk/cltk

Issue:
Regex lemmatizers in backoff.py run slowly
RegexLemmatizer (and related PPLemmatizer,  RomanNumeralLemmatizer, etc.) run slowly because they iterate unnecessarily over both all unlemmatized tokens *and* all regex patterns. Yet the lemmatizer only returns a lemma in a small number of cases. Proposed solution: limit the number of iterations over regex patterns.


Discussion:

Resolving patch:
diff --git a/cltk/lemmatize/latin/backoff.py b/cltk/lemmatize/latin/backoff.py
--- a/cltk/lemmatize/latin/backoff.py
+++ b/cltk/lemmatize/latin/backoff.py
@@ -475,8 +475,9 @@ class BackoffLatinLemmatizer(object):
     ### For comparison, there is also a TrainLemmatizer that replicates the
     ###    original Latin lemmatizer from cltk.stem
     """
-    def __init__(self, train):
+    def __init__(self, train, seed=3):
         self.train = train
+        self.seed = seed
         
         rel_path = os.path.join('~/cltk_data/latin/model/latin_models_cltk/lemmata/backoff')
         path = os.path.expanduser(rel_path)
@@ -531,8 +532,9 @@ def __init__(self, train):
             self.latin_pps = {}
             print('The file %s is not available in cltk_data' % file)  
 
-        def _randomize_data(train):
+        def _randomize_data(train, seed):
             import random
+            random.seed(seed)
             random.shuffle(train)
             pos_train_sents = train[:4000]
             lem_train_sents = [[(item[0], item[1]) for item in sent] for sent in train]
@@ -541,7 +543,7 @@ def _randomize_data(train):
 
             return pos_train_sents, train_sents, test_sents
 
-        self.pos_train_sents, self.train_sents, self.test_sents = _randomize_data(self.train)
+        self.pos_train_sents, self.train_sents, self.test_sents = _randomize_data(self.train, self.seed)
 
     def _define_lemmatizer(self):
         backoff0 = None
@@ -551,7 +553,7 @@ def _define_lemmatizer(self):
         backoff4 = UnigramLemmatizer(self.train_sents, backoff=backoff3)        
         backoff5 = RegexpLemmatizer(self.latin_misc_patterns, backoff=backoff4)
         backoff6 = TrainLemmatizer(model=self.LATIN_MODEL, backoff=backoff5)        
-        backoff7 = BigramPOSLemmatizer(self.pos_train_sents, include=['cum'], backoff=backoff6)
+        #backoff7 = BigramPOSLemmatizer(self.pos_train_sents, include=['cum'], backoff=backoff6)
         #lemmatizer = backoff7
         lemmatizer = backoff6
         return lemmatizer


Test patch:
Repository: cltk/cltk

Issue:
Move 3 Arabic corpora to CLTK organization
@LBenzahia : I have merged your corpus code for Arabic.

https://github.com/cltk/cltk/blob/master/cltk/corpus/arabic/corpora.py

Please transfer ownership of these three libraries to the `cltk` organization.

Then one of use can update the above `corpora.py`.

Thank you!


Discussion:

Resolving patch:
diff --git a/cltk/corpus/arabic/alphabet.py b/cltk/corpus/arabic/alphabet.py
--- a/cltk/corpus/arabic/alphabet.py
+++ b/cltk/corpus/arabic/alphabet.py
@@ -1,12 +1,10 @@
-"""Arabic alphabet
-source 1 : pyarabic 'https://github.com/linuxscout/pyarabic'
-source 2 : arabicstemmer 'https://github.com/assem-ch/arabicstemmer/blob/master/algorithm/stemmer.sbl'
+"""
+   Arabic alphabet
+   source 1 : pyarabic 'https://github.com/linuxscout/pyarabic'
+   source 2 : arabicstemmer 'https://github.com/assem-ch/arabicstemmer/blob/master/algorithm/stemmer.sbl'
 """
 __author__ = 'Lakhdar Benzahia <lakhdar[.]benzahia[at]gmail[.]com>'
 
-#Arabic digits from 0 to 9
-DIGITS = ['۰', '۱', '۲', '۳', '٤', '۵', '٦', '۷', '۸', '۹']
-
 # Arabic letters
 
 # Hamza letter
@@ -253,9 +251,7 @@
 FULL_STOP = u'\u06d4'
 BYTE_ORDER_MARK = u'\ufeff'
 
-# Diacritics
-SHORT_VOWELS = ['َ', 'ِ', 'ُ']
-
+#Diacritics
 FATHATAN = u'\u064b'
 DAMMATAN = u'\u064c'
 KASRATAN = u'\u064d'
@@ -464,11 +460,6 @@
 
 PUNCTUATION_MARKS = [COMMA, SEMICOLON, QUESTION]
 
-# The Persian Alphabet
-LONG_VOWELS = ['ا', 'و', 'ي']
-CONSONANTS = ['ب', 'ت', 'ث', 'ج', 'ح', 'خ', 'د', 'ذ', 'ر', 'ز', 'س', 'ش', 'ص', 'ض', 'ط', 'ظ', 'ع', 'غ', 'ف', 'ق', 'ک', 'ل', 'م', 'ن', 'ه']
-# Special Characters
-SPECIAL = ['‎ّ', 'ﻻ']
 
 WESTERN_ARABIC_NUMERALS = ['0','1','2','3','4','5','6','7','8','9']
 
diff --git a/cltk/corpus/arabic/corpora.py b/cltk/corpus/arabic/corpora.py
old mode 100644
new mode 100755
--- a/cltk/corpus/arabic/corpora.py
+++ b/cltk/corpus/arabic/corpora.py
@@ -1,22 +1,24 @@
 """Arabic corpora available for download."""
 
+__author__ = 'Lakhdar Benzahia <lakhdar[.]benzahia[at]gmail[.]com>'
+
 ARABIC_CORPORA = [
     {'name': 'arabic_text_perseus',
-     'markup': 'xml',
-     'origin': 'https://github.com/LBenzahia/arabic_text_perseus',
+     'markup':'xml',
+     'origin': 'https://github.com/cltk/arabic_text_perseus',
      'location': 'remote',
      'type': 'text',
      'RomanizationType': 'Buckwalter',
      },
      {'name': 'quranic-corpus',
-      'markup': 'xml',
-      'origin': 'https://github.com/LBenzahia/quranic-corpus',
+      'markup':'xml',
+      'origin': 'https://github.com/cltk/arabic_text_quranic_corpus',
       'location': 'remote',
       'type': 'text',
       'RomanizationType': 'none',
       },
       {'name': 'quranic-corpus-morphology',
-       'origin': 'https://github.com/LBenzahia/quranic-corpus-morphology',
+       'origin': 'https://github.com/cltk/arabic_morphology_quranic-corpus',
        'location': 'remote',
        'type': 'text',
        'RomanizationType': 'Buckwalter',


Test patch:
Repository: cltk/cltk

Issue:
Double-check code and data for new French PR
This issue is for @nat1881 to follow up on her large PR #571 for Old and Middle French.

Natasha, I would like you to do the following steps, to be certain that the code works as you intended:

* Start a brand new clone of (this) cltk repo.
* Make a new virtual env
* Mk source tarball and install (this should install all dependencies, too): `python setup.py sdist install`
* Temporarily rename your `~/cltk_data` dir (eg, `mv ~/cltk_data ~/cltk_data_backup`)
* Import the french corpora and make sure they appear as they should
* Check in ipython all of your commands that you have added to the docs. Copy-paste these exactly as they are in the docs.
* Follow up on any bugs in your own updated branch ([this is what I recommend for updating your branch](https://github.com/cltk/cltk/wiki/Example-Git-and-Python-workflow))
* Bump the version in `setup.py` and make PR for this
* Then @diyclassics or I will push the code to PyPI

You may be tired of this, but you're getting close! :weary:

cc @mlj 


Discussion:
Corpora download in the right place, and ipython tests all work (I've had to add a line to the stop filtering docs so the output prints). Should I add French to the the docs index?
> Should I add French to the the docs index?

Yes, good idea.
Also, I see that some of the text here is not formatted as you wanted: http://docs.cltk.org/en/latest/french.html#lemmatizer

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -36,7 +36,7 @@
     name='cltk',
     packages=find_packages(),
     url='https://github.com/cltk/cltk',
-    version='0.1.63',
+    version='0.1.64',
     zip_safe=True,
     test_suite='cltk.tests.test_cltk',
 )


Test patch:
Repository: cltk/cltk

Issue:
Latin Syllabifier Fails on Macrons
The Latin syllabifier seems to fail on words with macrons. For example, `audītū` becomes `['aud', 'ītū']` rather than `['au', 'dī', tū']`.  

It looks like we can fix this by just adding the macronized vowels to the  [vowels property](https://github.com/cltk/cltk/blob/master/cltk/stem/latin/syllabifier.py#L27).

@kylepjohnson I can submit a PR for this if it sounds reasonable.


Discussion:
Definitely reasonable! Pinging @lukehollis who was the module's author. One trap to watch out for is detecting [combined](https://en.wikipedia.org/wiki/Combining_character) or [precomosed](https://en.wikipedia.org/wiki/Precomposed_character) characters.

Here's one fix that we threw in to normalize input: https://github.com/cltk/cltk/blob/master/cltk/corpus/utils/formatter.py#L119

I forget what the options mean (in fact, this really needs a docstring!), but it may help.



Awesome ok! Thanks for the heads up on combined/precomposed characters.

Re: normalize input, yeah I'm not too sure what NFC/NFKC means but I'll look into that.
Sure, thank you!

If you get tangled up in the normalization stuff, just reach out and I can dig up my notes on why we made this as we did. KJ

Resolving patch:
diff --git a/cltk/stem/latin/syllabifier.py b/cltk/stem/latin/syllabifier.py
--- a/cltk/stem/latin/syllabifier.py
+++ b/cltk/stem/latin/syllabifier.py
@@ -27,6 +27,7 @@
          'vowels': [
              "a", "e", "i", "o", "u",
              "á", "é", "í", "ó", "ú",
+             "ā", "ē", "ī", "ō", "ū",
              "æ", "œ",
              "ǽ",  # no accented œ in unicode?
              "y"


Test patch:
diff --git a/cltk/tests/test_stem.py b/cltk/tests/test_stem.py
--- a/cltk/tests/test_stem.py
+++ b/cltk/tests/test_stem.py
@@ -201,6 +201,19 @@ def test_latin_syllabifier(self):
         syllables = syllabifier.syllabify(word)
         target = ['si', 'de', 're']
         self.assertEqual(syllables, target)
+        # tests for macronized words
+        macronized_word = 'audītū'
+        macronized_syllables = syllabifier.syllabify(macronized_word)
+        macronized_target = ['au', 'dī', 'tū']
+        self.assertEqual(macronized_syllables, macronized_target)
+        macronized_word2 = 'conjiciō'
+        macronized_syllables2 = syllabifier.syllabify(macronized_word2)
+        macronized_target2 = ['con', 'ji', 'ci', 'ō']
+        self.assertEqual(macronized_syllables2, macronized_target2)
+        macronized_word3 = 'ā'
+        macronized_syllables3 = syllabifier.syllabify(macronized_word3)
+        macronized_target3 = ['ā']
+        self.assertEqual(macronized_syllables3, macronized_target3)
 
     def test_syllabify(self):
         """Test Indic Syllabifier method"""
Repository: cltk/cltk

Issue:
Unused imports in word tokenizer module
The word tokenizer module has a large number of imports from NLTK that are not used anywhere in the module. Removing them 1. cleans up the code, and 2. speeds up testing.


Discussion:

Resolving patch:
diff --git a/cltk/tokenize/word.py b/cltk/tokenize/word.py
--- a/cltk/tokenize/word.py
+++ b/cltk/tokenize/word.py
@@ -1,40 +1,19 @@
-# -*-coding:utf-8-*-
 """Language-specific word tokenizers. Primary purpose is to handle enclitics."""
 
-import re
+__author__ = ['Patrick J. Burns <patrick@diyclassics.org>', 
+              'Kyle P. Johnson <kyle@kyle-p-johnson.com>', 
+              'Natasha Voake <natashavoake@gmail.com>']
+# Author info for Arabic, Old Norse?
 
-from nltk.tokenize.punkt import PunktLanguageVars
-from nltk.tokenize.punkt import PunktSentenceTokenizer, PunktParameters
+__license__ = 'MIT License. See LICENSE.'
 
 import re
 
-# Cleanup these imports—most are not used!
-from nltk.data              import load
-from nltk.tokenize.casual   import (TweetTokenizer, casual_tokenize)
-from nltk.tokenize.mwe      import MWETokenizer
-from nltk.tokenize.punkt    import PunktSentenceTokenizer
-from nltk.tokenize.regexp   import (RegexpTokenizer, WhitespaceTokenizer,
-                                    BlanklineTokenizer, WordPunctTokenizer,
-                                    wordpunct_tokenize, regexp_tokenize,
-                                    blankline_tokenize)
-#from nltk.tokenize.repp     import ReppTokenizer
-from nltk.tokenize.sexpr    import SExprTokenizer, sexpr_tokenize
-from nltk.tokenize.simple   import (SpaceTokenizer, TabTokenizer, LineTokenizer,
-                                    line_tokenize)
-from nltk.tokenize.stanford import StanfordTokenizer
-from nltk.tokenize.texttiling import TextTilingTokenizer
-#from nltk.tokenize.toktok   import ToktokTokenizer
-from nltk.tokenize.treebank import TreebankWordTokenizer
-from nltk.tokenize.util     import string_span_tokenize, regexp_span_tokenize
-from nltk.tokenize.stanford_segmenter import StanfordSegmenter
+from nltk.tokenize.punkt import PunktLanguageVars
+from nltk.tokenize.punkt import PunktSentenceTokenizer, PunktParameters
 
 import cltk.corpus.arabic.utils.pyarabic.araby as araby
 
-__author__ = ['Patrick J. Burns <patrick@diyclassics.org>', 'Kyle P. Johnson <kyle@kyle-p-johnson.com>',
-              'Natasha Voake <natashavoake@gmail.com>']
-__license__ = 'MIT License. See LICENSE.'
-
-
 class WordTokenizer:  # pylint: disable=too-few-public-methods
     """Tokenize according to rules specific to a given language."""
 


Test patch:
Repository: cltk/cltk

Issue:
Greek not among word tokenizers
Hi! I attempted to use the Greek word tokenizer for a really quick search of a plain text file, as follows:

<pre>
from cltk.tokenize.word import WordTokenizer

word_tokenizer = WordTokenizer('greek')

for word in word_tokenizer.tokenize('files/plain_text/1-23_1.1-42.txt'):
	if word.endswith('εαι'):
		print(word)
</pre>

When I tried to call the script, I got the following error:

`AssertionError: Specific tokenizer not available for 'greek'. Only available for: '['arabic', 'latin', 'french', 'old_norse']'.`

Thanks!


Discussion:

Resolving patch:
diff --git a/cltk/corpus/kannada/alphabet.py b/cltk/corpus/kannada/alphabet.py
--- a/cltk/corpus/kannada/alphabet.py
+++ b/cltk/corpus/kannada/alphabet.py
@@ -20,14 +20,14 @@
 
 YOGAVAAHAKAS = ['ಅಂ', 'ಅಃ']
 
-STRUCTUREDCONSONANTS = [
+STRUCTURED_CONSONANTS = [
     'ಕ', 'ಖ', 'ಗ', 'ಘ', 'ಙಚ',
     'ಚ', 'ಛ', 'ಜ', 'ಝ', 'ಞ',
     'ಟ', 'ಠ', 'ಡ', 'ಢ', 'ಣ',
     'ತ', 'ಥ', 'ದ', 'ಧ', 'ನ',
     'ಪ', 'ಫ', 'ಬ', 'ಭ', 'ಮ']
 
-UNSTRUCTURESCONSONANTS = [
+UNSTRUCTURED_CONSONANTS = [
     'ಯ', 'ರ', 'ಱ', 'ಲ', 'ವ', 'ಶ',
     'ಷ', 'ಸ', 'ಹ', 'ಳ', 'ೞ']
 
@@ -37,7 +37,7 @@
     '೦', '೧', '೨', '೩', '೪',
     '೫', '೬', '೭', '೮', '೯']
 
-VOWELSIGNS = [
+VOWEL_SIGNS = [
     '', 'ಾ', 'ಿ', 'ೀ', 'ು',
     'ೂ', 'ೃ', 'ೆ', 'ೇ', 'ೈ',
     'ೊ', 'ೋ', 'ೌ', 'ಂ', 'ಃ']


Test patch:
Repository: cltk/cltk

Issue:
Greek Scansion class: small bug in prosody/greek/scanner.py
In line 141 of `scanner.py` the class method `_long_by_nature` checks only if a character is a single consonant, but not if it is a double consonant:

```python
elif char not in self.sing_cons:
     vowel_group += char
```
vowel_group is then used to check if a group is a diphthong. The result is that a syllable with a double consonant and a diphthong is not recognized as "long by nature". E.g.:

```python

>>> from cltk.prosody.greek.scanner import Scansion
>>> scanner = Scansion()
>>> scanner.scan_text("Ζεὺς.")
['˘']

```


Discussion:
Thanks for finding this bug! I'll try to push out a fix this evening.
Hello, if it has not been fixed yet, can I work on this?
@kylepjohnson @TylerKirby 
Hey @siddharthkv7 that would be great. This seems like a pretty small fix. We just need to be checking for double consonants (ξ , ψ , and ζ)

Resolving patch:
diff --git a/cltk/prosody/greek/scanner.py b/cltk/prosody/greek/scanner.py
--- a/cltk/prosody/greek/scanner.py
+++ b/cltk/prosody/greek/scanner.py
@@ -138,7 +138,7 @@ def _long_by_nature(self, syllable):
             print
             if char in self.long_vowels:
                 return True
-            elif char not in self.sing_cons:
+            elif char not in self.sing_cons and char not in self.doub_cons:
                 vowel_group += char
 
         if ''.join(vowel_group) in self.diphthongs:


Test patch:
Repository: cltk/cltk

Issue:
Fix broken NER doctest
Open to anyone.

This simple NER doctest: https://github.com/cltk/cltk/blob/9b9cdb42dcc1c707ab3db3ef8214837bb7c262b5/cltk/tag/ner.py#L78

is all of a sudden failing (example: https://travis-ci.org/cltk/cltk/jobs/525125856#L1935 ).

The test expects 4 padded spaces on the left.

I have no idea why this would break all of a sudden.




Discussion:
Investigating... The doctest passes locally with ```python -m doctest -v ner.py```.
This doctest can be considered redundant as there is a unittest that tests the same functionality—going to remove.
Separate (and for a different issue)—looks like this module could be updated/refactored/revisited in general. Will add to my to-do list...

Resolving patch:
diff --git a/cltk/tag/ner.py b/cltk/tag/ner.py
--- a/cltk/tag/ner.py
+++ b/cltk/tag/ner.py
@@ -74,10 +74,6 @@ def _check_latest_data(lang):
 
 def tag_ner(lang, input_text, output_type=list):
     """Run NER for chosen language.
-    Choosing output_type=list, returns a list of tuples:
-    
-    >>> tag_ner('latin', input_text='ut Venus, ut Sirius, ut Spica', output_type=list)
-    [('ut',), ('Venus',), (',',), ('ut',), ('Sirius', 'Entity'), (',',), ('ut',), ('Spica', 'Entity')]
     """
 
     _check_latest_data(lang)
@@ -134,4 +130,3 @@ def tag_ner(lang, input_text, output_type=list):
         return string
 
     return ner_tuple_list
-


Test patch:
diff --git a/cltk/tests/test_corpus/test_corpus.py b/cltk/tests/test_corpus/test_corpus.py
--- a/cltk/tests/test_corpus/test_corpus.py
+++ b/cltk/tests/test_corpus/test_corpus.py
@@ -710,6 +710,7 @@ class TestFilteredCorpus(unittest.TestCase):
         def setUpClass(cls):
             try:
                 corpus_importer = CorpusImporter('latin')
+                corpus_importer.import_corpus('latin_models_cltk')
                 corpus_importer.import_corpus('latin_text_latin_library')
             except:
                 raise Exception('Failure to download test corpus')
Repository: cltk/cltk

Issue:
Lexicon process for Latin fails on regex special characters
LatinLexiconProcess fails when regex special characters, e.g. single open parenthesis (i.e. ```(```) are included in tokenized input. Occurred while running MacOS 11.4; Python 3.9.5; CLTK 1.0.15; regex 2021.4.4 (but should fail in any case when this input is passed to the regex module). The solution is to escape the input ```lemma``` before running ```regex.match``` at https://github.com/cltk/cltk/blob/5dbfcf6fccade146d322cae036b35533aec59286/src/cltk/lexicon/lat.py#L70

I have written the patch and will make a PR soon.

Example and traceback:

```
from cltk import NLP
text = "Omnes igitur partes mundi (tangam autem maximas) calore fultae sustinentur." # Cic. Nat. D. 2.25
cltk_nlp = NLP(language="lat")
cltk_doc = cltk_nlp.analyze(text=test)
```

```
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    cltk_doc = cltk_nlp.analyze(text=text)
  File "[PATH]/lib/python3.9/site-packages/cltk/nlp.py", line 142, in analyze
    doc = a_process.run(doc)
  File "[PATH]/lib/python3.9/site-packages/cltk/lexicon/processes.py", line 45, in run
    word.definition = lookup_algo.lookup(word.lemma)
  File "[PATH]/lib/python3.9/site-packages/cltk/lexicon/lat.py", line 70, in lookup
    matches = [key for key in keys if regex.match(rf"^{lemma}[0-9]?$", key)]
  File "[PATH]/lib/python3.9/site-packages/cltk/lexicon/lat.py", line 70, in <listcomp>
    matches = [key for key in keys if regex.match(rf"^{lemma}[0-9]?$", key)]
  File "[PATH]/lib/python3.9/site-packages/regex/regex.py", line 253, in match
    pat = _compile(pattern, flags, ignore_unused, kwargs, True)
  File "[PATH]/lib/python3.9/site-packages/regex/regex.py", line 532, in _compile
    raise error(caught_exception.msg, caught_exception.pattern,
regex._regex_core.error: missing ) at position 9
```


Discussion:
The solution is to refuse special characters for regular expressions in the `lemma` string.

Resolving patch:
diff --git a/src/cltk/lexicon/lat.py b/src/cltk/lexicon/lat.py
--- a/src/cltk/lexicon/lat.py
+++ b/src/cltk/lexicon/lat.py
@@ -47,6 +47,8 @@ def lookup(self, lemma: str) -> str:
         >>> lll = LatinLewisLexicon()
         >>> lll.lookup("clemens")[:50]
         'clēmēns entis (abl. -tī; rarely -te, L.), adj. wit'
+        >>> all(word in lll.lookup("levis") for word in ["levis","lēvis"]) # Test for concatenated entries
+        True
         >>> lll.lookup("omnia")
         ''
         >>> lll.lookup(".")
@@ -55,6 +57,8 @@ def lookup(self, lemma: str) -> str:
         ''
         >>> lll.lookup("175.")
         ''
+        >>> lll.lookup("(") # Test for regex special character
+        ''
         """
         if not self.entries:
             raise CLTKException(
@@ -64,7 +68,7 @@ def lookup(self, lemma: str) -> str:
         if regex.match(r"^[0-9\.\?,\:;\!\<\>\-]*$", lemma) is not None:
             return ""
 
-        lemma = lemma.lower()
+        lemma = regex.escape(lemma.lower())
 
         keys = self.entries.keys()
         matches = [key for key in keys if regex.match(rf"^{lemma}[0-9]?$", key)]


Test patch: