In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

[Q]: Yet, we noticed a common misbehavior for all our multimodal models: if the attention loose track of the objects in the picture and "gets lost", the model still takes it into account and somehow overrides the information brought by the text-based annotations. The translation is then totally mislead.
Question: What misbehavior is identified?
[A]: if the attention loose track of the objects in the picture and "gets lost", the model still takes it into account and somehow overrides the information brought by the text-based annotations

[Q]: The reviews we used have been extracted from TripAdvisor and originally proposed in BIBREF10 , BIBREF11 .
Question: Where are the hotel reviews from?
[A]: TripAdvisor

[Q]: Following BIBREF3, we hypothesize that a sentence representation in mBERT is composed of a language-specific component, which identifies the language of the sentence, and a language-neutral component, which captures the meaning of the sentence in a language-independent way. We assume that the language-specific component is similar across all sentences in the language.

We thus try to remove the language-specific information from the representations by centering the representations of sentences in each language so that their average lies at the origin of the vector space. We then analyze the semantic properties of both the original and the centered representations using a range of probing tasks.
Question: How they show that mBERT representations can be split into a language-specific component and a language-neutral component?
[A]:
We thus try to remove the language-specific information from the representations by centering the representations of sentences in each language so that their average lies at the origin of the vector space.