internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: acl-org/acl-anthology

Issue:
More open graph headers
We should add open graph headers to volume and event pages, perhaps summarizing how many papers are available, etc. This would make for much nicer presentation [via tweet](https://twitter.com/read_papers/status/1178724409332252678), for example, and the computed information could also be used when we get around to generating an XML feed (#358).


Discussion:

Resolving patch:
diff --git a/bin/anthology/data.py b/bin/anthology/data.py
--- a/bin/anthology/data.py
+++ b/bin/anthology/data.py
@@ -25,6 +25,7 @@
 
 ANTHOLOGY_URL = ANTHOLOGY_PREFIX + '/{}'
 ANTHOLOGY_PDF = ANTHOLOGY_PREFIX + '/{}.pdf'
+ANTHOLOGY_THUMBNAIL = ANTHOLOGY_PREFIX + '/thumb/{}.jpg'
 
 ATTACHMENT_URL = ANTHOLOGY_PREFIX + '/attachments/{}'
 
diff --git a/bin/anthology/papers.py b/bin/anthology/papers.py
--- a/bin/anthology/papers.py
+++ b/bin/anthology/papers.py
@@ -119,10 +119,10 @@ def from_xml(xml_element, *args):
                 del paper.attrib["pages"]
 
         if 'author' in paper.attrib:
-            # for x in paper.attrib['author']:
-            #     print('X', x[0].full)
             paper.attrib["author_string"] = ', '.join([x[0].full for x in paper.attrib["author"]])
 
+        paper.attrib["thumbnail"] = data.ANTHOLOGY_THUMBNAIL.format(paper.full_id)
+
         return paper
 
     def _interpret_pages(self):


Test patch:
Repository: acl-org/acl-anthology

Issue:
Presentation link not working
https://www.aclweb.org/anthology/people/t/tuhin-chakrabarty/

https://www.aclweb.org/anthology/N19-1054/

Presentation : https://github.com/tuhinjubcse/tuhinjubcse.github.io/blob/master/IMHO%20Fine-Tuning%20Improves%20Claim%20Detection%20.pdf


Discussion:
Thanks for reporting. I corrected the file, and you can download it here: https://www.aclweb.org/anthology/attachments/N19-1054.Presentation.pdf

The problem is twofold:

1. This user had uploaded a Github URL that wasn't downloaded correctly by our automated attachment ingestion.

2. In the Anthology build, we are apparently appending attachment URLs instead of calling `.format()`. See this line in the generated YAML:

```yaml
  attachment:
  - filename: N19-1054.Presentation.pdf
    type: presentation
    url: https://www.aclweb.org/anthology/attachments/{}/N19-1054.Presentation.pdf
```

Resolving patch:
diff --git a/bin/anthology/data.py b/bin/anthology/data.py
--- a/bin/anthology/data.py
+++ b/bin/anthology/data.py
@@ -22,13 +22,13 @@
 ################################################################################
 
 ANTHOLOGY_PREFIX = "https://www.aclweb.org/anthology"
+ATTACHMENT_PREFIX = ANTHOLOGY_PREFIX + '/attachments'
 
+# These should be renamed as "templates" so it's clear how they should be used
 ANTHOLOGY_URL = ANTHOLOGY_PREFIX + '/{}'
 ANTHOLOGY_PDF = ANTHOLOGY_PREFIX + '/{}.pdf'
 ANTHOLOGY_THUMBNAIL = ANTHOLOGY_PREFIX + '/thumb/{}.jpg'
 
-ATTACHMENT_URL = ANTHOLOGY_PREFIX + '/attachments/{}'
-
 # Names of XML elements that may appear multiple times
 LIST_ELEMENTS = ("attachment", "author", "editor", "video", "revision", "erratum")
 
diff --git a/bin/anthology/utils.py b/bin/anthology/utils.py
--- a/bin/anthology/utils.py
+++ b/bin/anthology/utils.py
@@ -134,7 +134,7 @@ def infer_attachment_url(filename, parent_id=None):
                 parent_id, filename
             )
         )
-    return infer_url(filename, data.ATTACHMENT_URL)
+    return infer_url(filename, data.ATTACHMENT_PREFIX)
 
 
 def infer_year(collection_id):


Test patch:
Repository: acl-org/acl-anthology

Issue:
Reverse-order the full bibtex?
ACL 2020 will include the entire Anthology bibtex in the default overleaf template. Overleaf doesn't autocomplete all the keys from a file that is so large, but seems to stop after reading some number of them. For this reason, it would be nice to reorder the bib file, perhaps reverse chronologically, or putting *ACL venues first, or something like that.


Discussion:
I had a look at the anthology bibtex today and I agree that a reverse sorting by publication date would be a good idea.  This would also automatically group by conference and should be the easiest to implement.

Resolving patch:
diff --git a/bin/anthology/utils.py b/bin/anthology/utils.py
--- a/bin/anthology/utils.py
+++ b/bin/anthology/utils.py
@@ -146,7 +146,7 @@ def infer_year(collection_id):
     """
     assert (
         len(collection_id) == 3
-    ), "Couldn't infer year: unknown volume ID format"
+    ), f"Couldn't infer year: unknown volume ID format '{collection_id}' ({type(collection_id)})"
     digits = collection_id[1:]
     if int(digits) >= 60:
         year = "19{}".format(digits)
diff --git a/bin/create_bibtex.py b/bin/create_bibtex.py
--- a/bin/create_bibtex.py
+++ b/bin/create_bibtex.py
@@ -36,8 +36,20 @@
 import os
 
 from anthology import Anthology
-from anthology.utils import SeverityTracker
+from anthology.utils import SeverityTracker, deconstruct_anthology_id, infer_year
 from create_hugo_pages import check_directory
+from operator import itemgetter
+
+
+def volume_sorter(volume_tuple):
+    """
+    Extracts the year so that we can sort by the year and then
+    the collection ID.
+    """
+    volume_id = volume_tuple[0]
+    collection_id, year, _ = deconstruct_anthology_id(volume_id)
+    year = infer_year(collection_id)
+    return year, volume_id
 
 
 def create_bibtex(anthology, trgdir, clean=False):
@@ -51,7 +63,7 @@ def create_bibtex(anthology, trgdir, clean=False):
     with gzip.open(
         "{}/anthology.bib.gz".format(trgdir), "wt", encoding="utf-8"
     ) as file_full:
-        for volume_id, volume in tqdm(anthology.volumes.items()):
+        for volume_id, volume in tqdm(sorted(anthology.volumes.items(), key=volume_sorter, reverse=True)):
             volume_dir = trgdir
             if not os.path.exists(volume_dir):
                 os.makedirs(volume_dir)


Test patch:
Repository: acl-org/acl-anthology

Issue:
Names in BibTeX not rendered properly in PDF output
The first entry in our `anthology.bib.gz` is [W19-8500](https://www.aclweb.org/anthology/W19-8500/):

```bibtex
@proceedings{ws-2019-international-resources,
    title = "Proceedings of the Second International Workshop on Resources and Tools for Derivational Morphology",
    author = "{\v{S}}ev{\v{c}}{\'\i}kov{\'a}, Magda  and
      {\v{Z}}abokrtsk{\'y}, Zden{\v{e}}k  and
      Litta, Eleonora  and
      Passarotti, Marco",
    month = "19--20 " # sep,
    year = "2019",
    address = "Prague, Czechia",
    publisher = "Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics",
    url = "https://www.aclweb.org/anthology/W19-8500",
}
```

The names are

> Magda Ševčíková, Zdeněk Žabokrtský, Eleonora Litta, Marco Passarotti

but in an ACL 2020 Overleaf document I have renders like this:

![image](https://user-images.githubusercontent.com/455256/68898906-71449000-06fe-11ea-98b2-9373e2ad6c97.png)



Discussion:
`author` is not the correct field for a `@proceedings` entry; it should be `editor`.

The culprit is here, where the `editor` field is explicitly changed to `author`:
https://github.com/acl-org/acl-anthology/blob/48be50a6935139ce7b6139d0ee016c02056a700b/bin/anthology/papers.py#L107-L109

This is presumably done for having it render correctly on the website, but produces the wrong result in this case, as it is the "front matter paper" from which the volume BibTeX entry is generated.

We can add an exception to BibTeX generation that changes `author` back to `editor` when the bibentry is `@proceedings`, which is a two-line fix, but feels a bit messy. Alternatively, we could change the site rendering to use the `editor` field when `author` is empty, which is potentially quite a bit more involved.

Resolving patch:
diff --git a/bin/anthology/formatter.py b/bin/anthology/formatter.py
--- a/bin/anthology/formatter.py
+++ b/bin/anthology/formatter.py
@@ -86,6 +86,8 @@ def bibtex_convert_month(text):
 def bibtex_make_entry(bibkey, bibtype, fields):
     lines = ["@{}{{{},".format(bibtype, bibkey)]
     for key, value in fields:
+        if key == "author" and bibtype == "proceedings":
+            key = "editor"
         if key in ("author", "editor") and "  and  " in value:
             # Print each author on a separate line
             value = "  and\n      ".join(value.split("  and  "))
diff --git a/bin/anthology/volumes.py b/bin/anthology/volumes.py
--- a/bin/anthology/volumes.py
+++ b/bin/anthology/volumes.py
@@ -69,8 +69,6 @@ def from_xml(volume_xml,
         front_matter_xml = volume_xml.find('frontmatter')
         if front_matter_xml is not None:
             front_matter = Paper.from_xml(front_matter_xml, volume, formatter)
-            front_matter._id = '0'
-            front_matter.is_volume = True
             volume.add_frontmatter(front_matter)
 
         return volume


Test patch:
Repository: acl-org/acl-anthology

Issue:
`similar` field in name_variants.yaml should be symmetric and transitive
This comes up in #615.


Discussion:
I mean that if author A has `similar: [B]` and author B has `similar: [C]`, this should implicitly create links from C to B, C to A, and B to A.

Resolving patch:
diff --git a/bin/anthology/index.py b/bin/anthology/index.py
--- a/bin/anthology/index.py
+++ b/bin/anthology/index.py
@@ -109,6 +109,22 @@ def load_variant_list(self, directory):
                     self.comments[id_] = entry["comment"]
                 if "similar" in entry:
                     self.similar[id_].update(entry["similar"])
+                    for other in entry["similar"]:
+                        if id_ not in self.similar[other]:
+                            log.debug('inferring similar name {} -> {}'.format(other, id_))
+                        self.similar[other].add(id_)
+                        
+        # form transitive closure of self.similar
+        again = True
+        while again:
+            again = False
+            for x in list(self.similar):
+                for y in list(self.similar[x]):
+                    for z in list(self.similar[y]):
+                        if z != x and z not in self.similar[x]:
+                            self.similar[x].add(z)
+                            log.debug('inferring similar name {} -> {}'.format(x, z))
+                            again = True
 
     def _is_stopword(self, word, paper):
         """Determines if a given word should be considered a stopword for


Test patch:
Repository: acl-org/acl-anthology

Issue:
I08-2 front matter links to whole PDF
The link from here https://www.aclweb.org/anthology/events/ijcnlp-2008/ that is normally the front matter for volume 2 links instead to the full PDF for all of volume 2.



Discussion:
That's weird. From the XML, it shouldn't be generating a link at all, since there is no `<url>` tag for the frontmatter:

```xml
  <volume id="2">
    <meta>
      <booktitle>Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-<fixed-case>II</fixed-case></booktitle>
      <url>I08-2</url>
    </meta>
    <frontmatter/>
```
This is a deeper problem, I suspect with `create_hugo_yaml.py`.

Resolving patch:
diff --git a/bin/anthology/papers.py b/bin/anthology/papers.py
--- a/bin/anthology/papers.py
+++ b/bin/anthology/papers.py
@@ -54,6 +54,7 @@ def __init__(self, paper_id, ingest_date, volume, formatter):
                 "venues",
                 "meta_date",
                 "url",
+                "pdf",
             ):
                 continue
 


Test patch:
Repository: acl-org/acl-anthology

Issue:
ACL 2019 videos are available
They can be [browsed here](vimeo.com/user/46432367/folder/1250297).

They video company provided a CSV file linking Anthology IDs to Vimeo URLs. Does anyone want to do the work of ingesting this? [02 ACL2019.xlsx](https://github.com/acl-org/acl-anthology/files/4098244/02.ACL2019.xlsx)

I also have a list of plenary talks and videos that are not currently linked from the Anthology, but should be (#298): [01 ACL 2019 - All Speeches.xlsx](https://github.com/acl-org/acl-anthology/files/4098219/01.ACL.2019.-.All.Speeches.xlsx)



Discussion:
I can look at this next week or so, I wanted to update my linking script
anyway to link the naacl 2019 videos.

The good news is that video companies from hereonout are following [the guidelines](https://github.com/acl-org/acl-anthology/wiki/Contract-Guidelines-for-Recording-Talks). This means we should get a CSV always, but also that the videos will have a backpointer into the Anthology. So the script could work on either data source (and could also double-check).

Resolving patch:
diff --git a/bin/anthology/utils.py b/bin/anthology/utils.py
--- a/bin/anthology/utils.py
+++ b/bin/anthology/utils.py
@@ -331,6 +331,9 @@ def parse_element(xml_element):
         elif tag == "video":
             # Treat videos the same way as other attachments
             tag = "attachment"
+            # Skip videos where permission was not granted (these should be marked as private anyway in Vimeo)
+            if element.get("permission", "true") == "false":
+                continue
             value = {
                 "filename": element.get("href"),
                 "type": element.get("tag", "video"),


Test patch:
Repository: acl-org/acl-anthology

Issue:
Recaser bug: adding fixed-case inside tex-math markup
`<tex-math><fixed-case>O</fixed-case>(<fixed-case>M</fixed-case>(n^2))</tex-math>` caused the build to fail in #892


Discussion:

Resolving patch:
diff --git a/bin/fixedcase/protect.py b/bin/fixedcase/protect.py
--- a/bin/fixedcase/protect.py
+++ b/bin/fixedcase/protect.py
@@ -24,7 +24,7 @@
 # recursive helper called by protect
 # protect text of "node", including children, and tails of children
 def protect_recurse(node, recased):
-    if node.tag == "fixed-case":  # already protected
+    if node.tag in ("fixed-case", "tex-math"):  # already protected text, or math
         newnode = copy.deepcopy(node)  # don't need to modify descendents
         newnode.tail = None  # tail will be protected by caller
         return newnode


Test patch:
Repository: acl-org/acl-anthology

Issue:
Volume links are 404
The "URL" field on volume pages is 404, e.g., https://www.aclweb.org/anthology/2020.acl-main.

![image](https://user-images.githubusercontent.com/455256/93599234-cccd5300-f98b-11ea-838c-fd6a0a13c3de.png)

We need to either (a) update `.htaccess` to recognize and redirect volume links or (b) update the URL to https://www.aclweb.org/anthology/volumes/2020.acl-main


Discussion:

Resolving patch:
diff --git a/bin/anthology/papers.py b/bin/anthology/papers.py
--- a/bin/anthology/papers.py
+++ b/bin/anthology/papers.py
@@ -19,6 +19,7 @@
 from .utils import (
     build_anthology_id,
     parse_element,
+    infer_url,
     infer_attachment_url,
     remove_extra_whitespace,
     is_journal,
@@ -79,6 +80,10 @@ def from_xml(xml_element, *args):
             paper.attrib["xml_title"] = paper.attrib["xml_booktitle"]
             paper.attrib["xml_title"].tag = "title"
 
+        # Create URL field if not present. But see https://github.com/acl-org/acl-anthology/issues/997.
+        if "url" not in paper.attrib:
+            paper.attrib["url"] = infer_url(paper.full_id)
+
         # Remove booktitle for frontmatter and journals
         if paper.is_volume or is_journal(paper.full_id):
             del paper.attrib["xml_booktitle"]


Test patch:
Repository: acl-org/acl-anthology

Issue:
Metadata Correction to Anthology ID 2020.findings-emnlp.118
**Please be sure to add the Anthology ID in the title**

[See here](https://www.aclweb.org/anthology/info/corrections/) to read about the three types of corrections.

## Metadata correction: please describe the issue here**

Metadata title should match the final title of the paper and the name in [Findings proceedings](https://2020.emnlp.org/papers/findings)

Original Title: Parsing with Multilingual BERT, a _Small Treebank, and a Small Corpus_
Updated Title: Parsing with Multilingual BERT, a **Small Corpus, and a Small Treebank**

Thank you!


Discussion:

Resolving patch:
diff --git a/bin/ingest.py b/bin/ingest.py
--- a/bin/ingest.py
+++ b/bin/ingest.py
@@ -162,7 +162,13 @@ def bib2xml(bibfilename, anthology_id):
             else:
                 continue
 
-            make_simple_element(field, text=value, parent=paper)
+            try:
+                make_simple_element(field, text=value, parent=paper)
+            except:
+                print(
+                    f"Couldn't process {bibfilename} for {anthology_id}", file=sys.stderr
+                )
+                sys.exit(2)
 
     return paper
 
@@ -309,12 +315,11 @@ def correct_caps(person, name_node, anth_id):
         if name.islower() or name.isupper():
             # capitalize all parts
             corrected = " ".join(list(map(lambda x: x.capitalize(), name.split())))
-            choice = input(
-                f"({anth_id}): Author '{person}': Change '{name}' to '{corrected}'?\n(Return for yes, any text for no): "
+            print(
+                f"-> Correcting capitalization of '{name}' to '{corrected}'",
+                file=sys.stderr,
             )
-            if choice == "":
-                print(f"-> Correcting {name} to {corrected}")
-                name_node.text = corrected
+            name_node.text = corrected
 
     def disambiguate_name(node, anth_id):
         name = PersonName.from_element(node)


Test patch:
Repository: acl-org/acl-anthology

Issue:
dataset for ACL 2020 paper giving 404 error
The dataset for this paper is giving a 404 error:

https://www.aclweb.org/anthology/2020.acl-main.84/


Discussion:
Hmm, it seems that every attachment just links to `https://www.aclweb.org/anthology/attachments`. I suspect that this has to do with #1124, which reworked the prefixes a little bit. I'll fix this a bit later tonight. Thank you for reporting, and apologies for the trouble.

Resolving patch:
diff --git a/bin/anthology/data.py b/bin/anthology/data.py
--- a/bin/anthology/data.py
+++ b/bin/anthology/data.py
@@ -37,6 +37,7 @@
     ANTHOLOGY_PREFIX = "https://www.aclweb.org/anthology"
 
 ATTACHMENT_PREFIX = ANTHOLOGY_PREFIX + "/attachments"
+ATTACHMENT_TEMPLATE = ATTACHMENT_PREFIX + "/{}"
 
 PDF_LOCATION_TEMPLATE = ANTHOLOGY_PREFIX + "/{}.pdf"
 PDF_THUMBNAIL_LOCATION_TEMPLATE = ANTHOLOGY_PREFIX + "/thumb/{}.jpg"
diff --git a/bin/anthology/utils.py b/bin/anthology/utils.py
--- a/bin/anthology/utils.py
+++ b/bin/anthology/utils.py
@@ -231,6 +231,11 @@ def infer_url(filename, template=data.CANONICAL_URL_TEMPLATE):
     """If URL is relative, return the full Anthology URL.
     Returns the canonical URL by default, unless a different
     template is provided."""
+
+    assert (
+        "{}" in template or "%s" in template
+    ), "template has no substitution text; did you pass a prefix by mistake?"
+
     if urlparse(filename).netloc:
         return filename
     return template.format(filename)
@@ -246,7 +251,7 @@ def infer_attachment_url(filename, parent_id=None):
                 parent_id, filename
             )
         )
-    return infer_url(filename, data.ATTACHMENT_PREFIX)
+    return infer_url(filename, data.ATTACHMENT_TEMPLATE)
 
 
 def infer_year(collection_id):


Test patch:
Repository: acl-org/acl-anthology

Issue:
Adding Anthology ID 2021.naacl-srw.21
## Revision or erratum: please add the following information**

- [x] I have attached the revised PDF or erratum to this issue

Hi! I (one of the NAACL SRW chairs) mistakenly omitted this paper when compiling the proceedings. Would it be possible to add it as a new entry `2021.naacl-srw.21` ?

The paper is attached to this issue.
The title is: `Towards Multi-Modal Text-Image Retrieval to improve Human Reading`
The authors are: `Florian Schneider, Özge Alacam, Xintong Wang, Chris Biemann`
[37_Final_Paper_PDF.pdf](https://github.com/acl-org/acl-anthology/files/6704101/37_Final_Paper_PDF.pdf)

Thank you so much!


Discussion:

Resolving patch:
diff --git a/bin/likely_name_split.py b/bin/likely_name_split.py
--- a/bin/likely_name_split.py
+++ b/bin/likely_name_split.py
@@ -24,6 +24,13 @@
 from anthology.people import PersonName
 
 
+def log0(x):
+    if x == 0:
+        return -inf
+    else:
+        return log(x)
+
+
 class NameSplitter:
     def __init__(self, anthology=None, anthology_dir=None):
         # counts of how often each name appears
@@ -105,18 +112,21 @@ def best_split(self, name):
             # max of log prob of "Maria Victoria" and
             # log prob of "Maria" + log prob of "Victoria"
             first_probs = [
-                log((self.first_count[x] + 0.01) / self.first_total) for x in words[0:i]
+                # more smoothing for first than last name,
+                # so that default is one-word last name when all counts are zero
+                log((self.first_count[x] + 0.1) / self.first_total)
+                for x in words[0:i]
             ]
             first_score = max(
-                log((self.first_full_count[first] + 0.000001) / self.first_total),
+                # no smoothing for multiword name: log(0) => -inf
+                log0((self.first_full_count[first]) / self.first_total),
                 sum(first_probs),
             )
             last_probs = [
                 log((self.last_count[x] + 0.01) / self.last_total) for x in words[i:]
             ]
             last_score = max(
-                log((self.last_full_count[last] + 0.000001) / self.last_total),
-                sum(last_probs),
+                log0((self.last_full_count[last]) / self.last_total), sum(last_probs)
             )
 
             if first_score + last_score > best_score:


Test patch:
Repository: acl-org/acl-anthology

Issue:
Make volumes adhere to the flat directory structure
All our files are relatively flat, but we still have nested structures for volumes (in the build dir under `anthology/papers/`).

This looks like an oversight from a previous change and noone has complained yet.
Make volumes adhere to the flat directory structure
All our files are relatively flat, but we still have nested structures for volumes (in the build dir under `anthology/papers/`).

This looks like an oversight from a previous change and noone has complained yet.


Discussion:
Here's the precise reason why that still happens:

https://github.com/acl-org/acl-anthology/blob/6df3f486e8f3cdeffd47593e15f1d65ccc683fb1/bin/create_hugo_pages.py#L116-L118

Maybe that alias can be removed, and a flat permalink for volumes be added in `hugo/config.toml`. I can test that when I have some time.
Here's the precise reason why that still happens:

https://github.com/acl-org/acl-anthology/blob/6df3f486e8f3cdeffd47593e15f1d65ccc683fb1/bin/create_hugo_pages.py#L116-L118

Maybe that alias can be removed, and a flat permalink for volumes be added in `hugo/config.toml`. I can test that when I have some time.

Resolving patch:
diff --git a/bin/create_hugo_pages.py b/bin/create_hugo_pages.py
--- a/bin/create_hugo_pages.py
+++ b/bin/create_hugo_pages.py
@@ -108,14 +108,10 @@ def create_volumes(srcdir, clean=False):
     for anthology_id, entry in data.items():
         with open("{}/content/volumes/{}.md".format(srcdir, anthology_id), "w") as f:
             print("---", file=f)
-            paper_dir = "/papers/{}/{}/".format(anthology_id.split("-")[0], anthology_id)
             yaml.dump(
                 {
                     "anthology_id": anthology_id,
                     "title": entry["title"],
-                    "aliases": [
-                        paper_dir,
-                    ],
                 },
                 default_flow_style=False,
                 stream=f,


Test patch: