In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Q: We therefore use reduction in softmax probability of the correct relation as our signaling strength metric for the model. We call this metric ${\Delta }_s$ (for delta-softmax), which can be written as:

where $rel$ is the true relation of the EDU pair, $t_i$ represents the token at index $i$ of $N$ tokens, and $X_{mask=i}$ represents the input sequence with the masked position $i$ (for $i \in 1 \ldots N$ ignoring separators, or $\phi $, the empty set).
Question: How is the delta-softmax calculated?
A:
Answer with content missing: (Formula) Formula is the answer.