Repository: mir-evaluation/mir_eval
Issue: [QUESTION] Why leave out query frame during _gauc computation in hierarchy module?

## Description of Status Quo:
The `_gauc` routine is used in both hierarchy metrics. It iterates through query frames `q` and look for how relevant every other frame is with q in two different annotations.

The routine then count ranking violations between the annotations' relevance score for all frames regarding the query frame q.
This is currently implemented in the following lines in hierarchy.py

https://github.com/mir-evaluation/mir_eval/blob/94581744a9a3afccc0f3d6ac928281736fb77c91/mir_eval/hierarchy.py#L303C1-L314C10

Here, the relevance score explicitly leaves out the q frame itself.

## Question:
I was wondering why this is decided (explicitly leaves out the q frame itself) and why not include the relevance score for q frame (max depth) in the result before counting ranking inversions?

## Challenge to Status Quo
Here's a couple reasons favoring including the query frame itself:
- when we take the limit with frame_size approaching 0, the length of frame q would be 0 and including q or not including q would be technically the same?
- it sort of simplifies the code a bit...
- The q frame will receive the highest relevance score in both est and ref annotations (because it's the same frame!), but the score on that frame still depends on how many other frames in the annotations meets with q at maximum depth.... There's no particularly good reason to throw this info away, as far as I can tell?
- related to the first point again: trying to envision a version of these metrics that doesn't rely on frame_size but instead just do sections defined by boundaries and weight the scores' by section length, effectively taking the frame_size to the smallest limit. In this conceptualization, the ignoring query frame q step is kind of not possible?

Hope this is intelligible... Happy to clarify on any of the above points, but just want to get an initial sense on why the decision on leaving out the query frame during gauc calculation? @bmcfee 

> I was wondering why this is decided (explicitly leaves out the q frame itself) and why not include the relevance score for q frame (max depth) in the result before counting ranking inversions?

The idea here is that `(q, q)` is a non-informative comparison in almost all cases: for meet matrices derived from segment label agreement, it will always have a maximal value in both ref and estimate.  If you then take any arbitrary frame `j` and ask `(q, q) >= (q, j)?` the answer is always `True`, so there is nothing to be learned about agreement between ref and est by looking at the `i=q` case.

More generally, you can think of this as deriving from k-nearest neighbor retrieval, where the query point itself is never included in the returned list of nearest neighbors.


> * when we take the limit with frame_size approaching 0, the length of frame q would be 0 and including q or not including q would be technically the same?

I'm not sure about that.  As noted above, the q,q comparison inflates the score (by including trivial agreements), and if you sample the q's more densely, the inflation may be exacerbated. (I don't know for sure, I haven't thought about it deeply, but it's not obvious.)



> it sort of simplifies the code a bit...

Yes it would, but it's computing something different.



> * The q frame will receive the highest relevance score in both est and ref annotations (because it's the same frame!), but the score on that frame still depends on how many other frames in the annotations meets with q at maximum depth.... There's no particularly good reason to throw this info away, as far as I can tell?

I don't think we're throwing this information away.  We still look for each q at all other frames i and j.



> * related to the first point again: trying to envision a version of these metrics that doesn't rely on frame_size but instead just do sections defined by boundaries and weight the scores' by section length, effectively taking the frame_size to the smallest limit. In this conceptualization, the ignoring query frame q step is kind of not possible?

Ehh, it's possible, but the notation gets messy.  Basically you'd define a measure (probability density) over the continuous time interval, and then argue that we don't need to worry about self-comparison because any particular point q has measure 0 and can thus be omitted from the calculation without affecting the result.

Just did some testing... and you are right. 
See experiments at this colab notebook:
https://colab.research.google.com/drive/1DowXsaxecNmtGySx5Pw7wGZlVtQtbWz3?usp=sharing

- If I keep decreasing the frame_size (I got to 0.001), the two versions converges and produce scores that's very close / identical.
- Without taking out the q, scores are higher.