Repository: freelawproject/courtlistener
Issue category: Feature Request

Issue:
Clarify number of references on Authorities Page vs. Search Results
Noticed after doing #4134

On the “Authorities” page for a given docket, we display the references to opinions in the docket's documents. However, each "authority" in the list is not actually an opinion, but an element of a `Docket`'s `authorities`:

```python
    @property
    def authorities(self):
        """Returns a queryset that can be used for querying and caching
        authorities.
        """
        return OpinionsCitedByRECAPDocument.objects.filter(
            citing_document__docket_entry__docket_id=self.pk
        )
```

```python
class OpinionsCitedByRECAPDocument(models.Model):
    citing_document = models.ForeignKey(
        RECAPDocument, related_name="cited_opinions", on_delete=models.CASCADE
    )
    cited_opinion = models.ForeignKey(
        Opinion, related_name="citing_documents", on_delete=models.CASCADE
    )
    depth = models.IntegerField(
        help_text="The number of times the cited opinion was cited "
        "in the citing document",
        default=1,
    )
```

This means an opinion cited by multiple documents in the same docket will be listed multiple times, like the authorities in [this docket](https://www.courtlistener.com/docket/4214664/authorities/national-veterans-legal-services-program-v-united-states/). Notice `Norman v. United States, 429 F.3d 1081 (Fed. Cir. 2005)` being listed 14 times, where [all links point to the same search](https://www.courtlistener.com/?type=r&q=docket_id%3A4214664%20AND%20cites%3A(211579)) which yields 14 docket entries.
![Image](https://github.com/user-attachments/assets/99b998fa-4920-44a8-aa30-d66ada80e38c)
![Image](https://github.com/user-attachments/assets/1e355972-932a-48ba-851f-723fe5518215)


This also means the counts are confusing. Each "depth" represents only the number of times the opinion is cited in one citing document, not the total number of references across all documents in the docket. We could:
1. Make the counts less confusing by making the authorities list include the citing document and changing the text to something like `5 references to <CITED_OPINION> in <CITING_DOC>`, with a link to the citing doc.
2. Group authorities by opinion, then aggregate the depth. This way we would get the summary of all references to a given opinion in a given docket, instead of having the same opinion repeated several times. This does sound like it could be a lot more work, but potentially more informative if we could also include a sub-list of all citing docs with their links.

    So instead of:

    > - 5 references to Norman v. United States, 429 F.3d 1081 (Fed. Cir. 2005)
    > Court of Appeals for the Federal Circuit Nov. 18, 2005
    > - 5 references to Norman v. United States, 429 F.3d 1081 (Fed. Cir. 2005)
    > Court of Appeals for the Federal Circuit Nov. 18, 2005
    > - 2 references to Norman v. United States, 429 F.3d 1081 (Fed. Cir. 2005)
    > Court of Appeals for the Federal Circuit Nov. 18, 2005

    It would be:

    > - 12 references to Norman v. United States, 429 F.3d 1081 (Fed. Cir. 2005)
    > Court of Appeals for the Federal Circuit Nov. 18, 2005
    >   - 5 references in citing doc 1
    >   - 5 references in citing doc 2
    >   - 2 references in citing doc 3

3. Update the search results to display the depth of treatment when a cites query is made, so that each result shown says something like, "22 references to case XYZ".

Whatever the option, we should make sure that the opinions' authorities page doesn't get broken since `authorities_list.html` is used in both docket and opinion authorities, which are not `OpinionsCitedByRECAPDocument` but `OpinionsCited` so we don't always have the same attributes available.


Hints:
Fun, so this issue is indeed worse than we thought. I didn't realize that we're repeating the same case many times in the list of authorities. That's not great. 

The easiest thing is to aggregate on the depth and have it say something like:

> 3 filings make 14 references to XYZ

We don't need to say which filings do that on this page, so we can spare ourselves that nested layout you describe.

Aggregating the depth should be easy. I can't think offhand how to get the filing count, but I'm guessing it's not too hard either and can be done at the query-level as well.

If we do the above, I think that fixes the confusion issue too, since it says the number of documents and then when you click on it, that's how many show up in the search results. 

One last thing: We're introducing a second count to this page. Currently, it's ordered by the citation depth, but I think if we make this change, we should order by the number of filings citing a document instead. I guess a later feature could be to allow users to choose which ordering they prefer.
lol sup, i'm gonna look at this soon
@tactipus, we're planning to fix this in our coming sprint, which starts Monday. Are you still thinking about helping with this one? 
> [@tactipus](https://github.com/tactipus), we're planning to fix this in our coming sprint, which starts Monday. Are you still thinking about helping with this one?

yeah. just wanted to know if elisa still wanted to do paired programming so it can be scheduled. i'm looking at the Docket class rn
Up to you guys. I'll get out of the way. :) @elisa-a-v? 
Oh I'd love to! @tactipus if you're down, let me know when would be a good time for you and I can probably adjust my schedule :)
@elisa-a-v I am good after 17:00 EST, usually. i can also do 13:00 to 16:00 EST
@tactipus that's great, 13:00 EST generally works for me as well, but I think we should probably move this discussion over to email so we don't keep cluttering up the issue thread 😅 my address is elisa@free.law—feel free to reach out!
Good morning,

This comment is more a record than anything. Per our conversation yesterday, we will focus on views.py & map out the implementation @mlissner discussed. That way, we can avoid tinkering too much with the models.py.

The solution is to map out the references using an array or a query set IIRC. It was all @elisa-a-v's idea, I just took notes ;p

```python
async def docket_authorities(
    request: HttpRequest,
    docket_id: int,
    slug: str,
) -> HttpResponse:
    docket, context = await core_docket_data(request, docket_id)
    if not await docket.ahas_authorities():
        raise Http404("No authorities data for this docket at this time")

    context.update(
        {
            # Needed to show/hide parties tab.
            "parties": await docket.parties.aexists(),
            "docket_entries": await docket.docket_entries.aexists(),
            "authorities": docket.authorities_with_data,
            
        }
    )
    return TemplateResponse(request, "docket_authorities.html", context)
```
That's correct, we basically need two things:
1. Map the `authorities` in the `context` of that view so that it's an iterable of `Opinion` instances instead of `OpinionsCitedByRECAPDocuments`, with the number of filings (how many `RECAPDocuments` cite this `Opinion` in this `Docket`) and the number of references (the aggregated `depth` of all those `OpinionsCitedByRECAPDocument` instances)
    - We now list `Opinions` instead of `OpinionsCitedByRECAPDocuments` so we only list each opinion once.
    - I believe the iterable could either be a `QuerySet` if you know your way around annotations, or simply a list. I imagine something like:
      ```python
      >>> context["authorities"]
      [
          {
              "opinion": Opinion1,
              "filings": OpinionsCitedByRECAPDocument.filter(cited_opinion=Opinion1, ...).count(),
              "references": OpinionsCitedByRECAPDocument.filter(cited_opinion=Opinion1, ...).aggregate(...)["sum"],
          },
          {
              "opinion": Opinion2,
              "filings": OpinionsCitedByRECAPDocument.filter(cited_opinion=Opinion2, ...).count(),
              "references": OpinionsCitedByRECAPDocument.filter(cited_opinion=Opinion2, ...).aggregate(...)["sum"],
          },
      ]
      ````
2. Update the template to display the info in the new format without breaking the document authorities view (they both use the same `authorities_list.html` template) which should probably remain unchanged:
![Image](https://github.com/user-attachments/assets/322739b5-eb8d-441e-9cc1-f2663c2c700d)

@mlissner I did notice a small thing when looking through the `authorities_list.html` template: in [line 14](https://github.com/freelawproject/courtlistener/blob/main/cl/opinion_page/templates/includes/authorities_list.html#L14) it says `{% if authority.blocked %}`, but I don't see how that's ever `True`. I wonder if maybe that's a mistake and it should be `authority.cited_opinion.cluster.blocked` instead?

```html
<a href="{{ authority.cited_opinion.cluster.get_absolute_url }}{% querystring %}" {% if authority.blocked %}rel="nofollow" {% endif %}>
```
I just emailed @elisa-a-v about this. To recap that email, I suggested that we use `annotate()` to get counts. `annotate()` returns QuerySet objects, which are iterable while `aggregate()` provides a dictionary, which is iterable but...not as flexible I believe.
> I wonder if maybe that's a mistake and it should be authority.cited_opinion.cluster.blocked instead?

That's certainly possible. The idea here is to tell crawlers not to waste their time crawling pages that we already know are blocked. Google and other crawlers give you a budget of links that they'll follow (called the "Crawl Budget"), so it's important not to send them to pages they can't index anyway.

I think you can get all the information you need from the queryset with something like this:

```
cited = (
    OpinionsCitedByRECAPDocument.objects
        .filter(citing_document__docket_entry__docket_id=4214664)
        .values("cited_opinion_id")
        .annotate(opinion_count=Count('cited_opinion_id'), total_depth=Sum('depth'))
)
```

That returns objects like:

```
{'cited_opinion_id': 9339585, 'opinion_count': 1, 'total_depth': 1}
```

And if you iterate over the entire thing you get results like:

```
In [27]: for c in cited:
    ...:     print(f"citation_count: {c["opinion_count"]}; total depth: {c["total_depth"]}")
    ...
    citation_count: 1; total depth: 1
    citation_count: 4; total depth: 35
    citation_count: 1; total depth: 1
    citation_count: 1; total depth: 1
    citation_count: 1; total depth: 2
    ...
```

Does this work as a solution (if you pick the fields carefully, do the prefetches, etc)?
> That's certainly possible. The idea here is to tell crawlers not to waste their time crawling pages that we already know are blocked. Google and other crawlers give you a budget of links that they'll follow (called the "Crawl Budget"), so it's important not to send them to pages they can't index anyway.

I understand, well from what I saw I don't think the `authority` actually has any `blocked` attribute so I'd bet it's not working, unless I'm missing something 🤔 

> Does this work as a solution (if you pick the fields carefully, do the prefetches, etc)?

Yes that makes a lot of sense to me!

Patch:
diff --git a/cl/opinion_page/templates/docket_authorities.html b/cl/opinion_page/templates/docket_authorities.html
index 05e3154e38..7390f2ef36 100644
--- a/cl/opinion_page/templates/docket_authorities.html
+++ b/cl/opinion_page/templates/docket_authorities.html
@@ -13,12 +13,12 @@
 {% block tab-content %}
 <div class="flex flex-column justify-content-center col-xs-12 col-md-10 col-lg-12">
   <div class="col-xs-12 col-md-10 col-lg-10">
-    {% with authorities_count=docket.authority_count %}
+    {% with authorities_count=opinions_cited.count %}
       <h3>The filings in this docket cite {{authorities_count|intcomma}} opinion{{authorities_count|pluralize}}</h3>
     {% endwith %}
     <hr>
     {# Westlaw's metadata organization: Checkbox, Treatment, Name + citation + summary, Date, Type, Depth #}
-    {% include 'includes/authorities_list.html' with authorities=authorities %}
+    {% include 'includes/docket_authorities_list.html' %}
   </div>
 </div>
 {% endblock %}
diff --git a/cl/opinion_page/templates/includes/docket_authorities_list.html b/cl/opinion_page/templates/includes/docket_authorities_list.html
new file mode 100644
index 0000000000..4ffdfeaeb1
--- /dev/null
+++ b/cl/opinion_page/templates/includes/docket_authorities_list.html
@@ -0,0 +1,18 @@
+{% load text_filters %}
+
+<ul>
+  {% for opinion in opinions_cited %}
+    <li>
+      <a href="/?type=r&q=docket_id%3A{{ docket.pk }}%20AND%20cites%3A({{ opinion.cluster.sub_opinions.all|OR_join }})">
+        {{ opinion.num_filings }} filing{{ opinion.num_filings|pluralize }}
+      </a>
+      make{% if opinion.num_filings == 1 %}s{% endif %} {{ opinion.total_depth }} reference{{ opinion.total_depth|pluralize }} to
+      <a href="{{ opinion.cluster.get_absolute_url }}{% querystring %}" {% if opinion.cluster.blocked %}rel="nofollow" {% endif %}>
+        {{ opinion.cluster.caption|safe|v_wrapper }}
+      </a>
+      <br />
+      <span class="bullet-tail">{{ opinion.cluster.docket.court }}</span>
+      <span class="bullet-tail">{{ opinion.cluster.date_filed }}</span>
+    </li>
+  {% endfor %}
+</ul>
diff --git a/cl/opinion_page/templates/includes/authorities_list.html b/cl/opinion_page/templates/includes/rd_authorities_list.html
similarity index 63%
rename from cl/opinion_page/templates/includes/authorities_list.html
rename to cl/opinion_page/templates/includes/rd_authorities_list.html
index ed9fa3a942..279cd34df2 100644
--- a/cl/opinion_page/templates/includes/authorities_list.html
+++ b/cl/opinion_page/templates/includes/rd_authorities_list.html
@@ -3,14 +3,7 @@
 <ul>
   {% for authority in authorities %}
   <li>
-    {% if docket %}
-    <a href="/?type=r&q=docket_id%3A{{ docket.pk }}%20AND%20cites%3A({{ authority.cited_opinion.cluster.sub_opinions.all|OR_join }})">
-    {% endif %}
-      {{ authority.depth }} reference{{ authority.depth|pluralize }}
-    {% if docket %}
-    </a>
-    {% endif %}
-    to
+    {{ authority.depth }} reference{{ authority.depth|pluralize }} to
     <a href="{{ authority.cited_opinion.cluster.get_absolute_url }}{% querystring %}" {% if authority.blocked %}rel="nofollow" {% endif %}>
       {{ authority.cited_opinion.cluster.caption|safe|v_wrapper }}
     </a>
diff --git a/cl/opinion_page/templates/recap_authorities.html b/cl/opinion_page/templates/recap_authorities.html
index 7a69d4b0fe..1980313837 100644
--- a/cl/opinion_page/templates/recap_authorities.html
+++ b/cl/opinion_page/templates/recap_authorities.html
@@ -13,7 +13,7 @@ <h3>This document cites {{ authorities_count|intcomma }} opinion{{ authorities_c
       {% endwith %}
       <hr>
       {# Westlaw's metadata organization: Checkbox, Treatment, Name + citation + summary, Date, Type, Depth #}
-      {% include 'includes/authorities_list.html' with authorities=authorities %}
+      {% include 'includes/rd_authorities_list.html' with authorities=authorities %}
     </div>
   </div>
 {% endblock %}
diff --git a/cl/opinion_page/views.py b/cl/opinion_page/views.py
index 32fee0691b..a2d062e530 100644
--- a/cl/opinion_page/views.py
+++ b/cl/opinion_page/views.py
@@ -543,7 +543,7 @@ async def docket_authorities(
             # Needed to show/hide parties tab.
             "parties": await docket.parties.aexists(),
             "docket_entries": await docket.docket_entries.aexists(),
-            "authorities": docket.authorities_with_data,
+            "opinions_cited": docket.authority_opinions.distinct(),
         }
     )
     return TemplateResponse(request, "docket_authorities.html", context)
diff --git a/cl/search/models.py b/cl/search/models.py
index 51084008e3..851d756e23 100644
--- a/cl/search/models.py
+++ b/cl/search/models.py
@@ -12,7 +12,8 @@
 from django.contrib.postgres.indexes import HashIndex
 from django.core.exceptions import ValidationError
 from django.db import IntegrityError, models, transaction
-from django.db.models import Q, QuerySet
+from django.db.models import Prefetch, Q, QuerySet
+from django.db.models.aggregates import Count, Sum
 from django.db.models.functions import MD5
 from django.urls import NoReverseMatch, reverse
 from django.utils import timezone
@@ -870,6 +871,22 @@ def authorities_with_data(self):
         """
         return build_authorities_query(self.authorities)
 
+    @property
+    def authority_opinions(self):
+        """Return a queryset of Opinions cited by this docket's RDs.
+
+        This is similar to the authorities property, but instead of
+        listing OpinionsCitedByRECAPDocuments, it lists Opinions.
+
+        The returned queryset is annotated with the number of filings
+        and the total depth across references, and optimized by prefetching
+        related data.
+        """
+        opinions = Opinion.objects.filter(
+            citing_documents__citing_document__docket_entry__docket_id=self.pk
+        )
+        return build_authority_opinions_query(opinions)
+
     def add_idb_source(self):
         if self.source in self.NON_IDB_SOURCES():
             self.source = self.source + self.IDB
@@ -3290,6 +3307,51 @@ class Meta:
         indexes = [models.Index(fields=["depth"])]
 
 
+def build_authority_opinions_query(
+    base_queryset: QuerySet[Opinion],
+) -> QuerySet[Opinion]:
+    """Annotates a queryset of Opinions and optimizes related queries.
+
+    The annotated queryset includes the number of filings (total number of
+    RECAPDocuments that cite each opinion) and the total depth (total
+    number of references across documents).
+    """
+    return (
+        base_queryset.select_related("cluster__docket__court")
+        .prefetch_related(
+            "cluster__citations",
+            "citing_documents",
+            Prefetch(
+                "cluster__sub_opinions",
+                queryset=Opinion.objects.only("pk", "cluster_id"),
+            ),
+        )
+        .annotate(
+            num_filings=Count(
+                "citing_documents__citing_document", distinct=True
+            ),
+            total_depth=Sum("citing_documents__depth"),
+        )
+        .only(
+            "cluster__id",
+            "cluster__blocked",
+            "cluster__case_name",
+            "cluster__case_name_full",
+            "cluster__case_name_short",
+            "cluster__docket_id",
+            "cluster__docket__court__full_name",
+            "cluster__docket__docket_number",
+            "cluster__docket__court_id",
+            "cluster__docket__court__citation_string",
+            "cluster__slug",
+            "cluster__citation_count",
+            "cluster__docket_id",
+            "cluster__date_filed",
+        )
+        .order_by("-num_filings", "-total_depth")
+    )
+
+
 def build_authorities_query(
     base_queryset: QuerySet[OpinionsCitedByRECAPDocument],
 ) -> QuerySet[OpinionsCitedByRECAPDocument]:


Test patch: