TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
PROBLEM: In this work, we introduce a new logical inference engine called MonaLog, which is based on natural logic and work on monotonicity stemming from vanBenthemEssays86. Since our logic operates over surface forms, it is straightforward to hybridize our models. We investigate using MonaLog in combination with the language model BERT BIBREF20, including for compositional data augmentation, i.e, re-generating entailed versions of examples in our training sets. We perform two experiments to test MonaLog. We first use MonaLog to solve the problems in a commonly used natural language inference dataset, SICK BIBREF1, comparing our results with previous systems. Second, we test the quality of the data generated by MonaLog. To do this, we generate more training data (sentence pairs) from the SICK training data using our system, and performe fine-tuning on BERT BIBREF20, a language model based on the transformer architecture BIBREF23, with the expanded dataset.
Question: How do they combine MonaLog with BERT?

SOLUTION: They use Monalog for data-augmentation to fine-tune BERT on this task

PROBLEM: Proposed Fusion Techniques ::: Step-Wise Decoder Fusion
Our first proposed technique is the step-wise decoder fusion of visual features during every prediction step i.e. we concatenate the visual encoding as context at each step of the decoding process. Proposed Fusion Techniques ::: Multimodal Attention Modulation
Similar to general attention BIBREF8, wherein a variable-length alignment vector $a_{th}(s)$, whose size equals the number of time steps on the source side, is derived by comparing the current target hidden state $h_{t}$ with each source hidden state $\overline{h_{s}}$; we consider a variant wherein the visual encoding $v_{t}$ is used to calculate an attention distribution $a_{tv}(s)$ over the source encodings as well. Proposed Fusion Techniques ::: Visual-Semantic (VS) Regularizer
In terms of leveraging the visual modality for supervision, BIBREF1 use multi-task learning to learn grounded representations through image representation prediction.
Question: What are 3 novel fusion techniques that are proposed?

SOLUTION: Step-Wise Decoder Fusion Multimodal Attention Modulation Visual-Semantic (VS) Regularizer

PROBLEM: We can identify three main cases for which the model produces an error:

the correct class can be directly inferred from the text content easily, even without background knowledge

the correct class can be inferred from the text content, given that event-specific knowledge is provided

the correct class can be inferred from the text content if the text is interpreted correctly
Question: What type of errors do the classifiers use?

SOLUTION:
correct class can be directly inferred from the text content easily, even without background knowledge correct class can be inferred from the text content, given that event-specific knowledge is provided orrect class can be inferred from the text content if the text is interpreted correctly