# _Applied Sciences_
## 3 ES-Contradiction: A New Spanish Contradiction Dataset
### Dataset Annotation Stages  
The dataset was built in four stages, subsequently outlined and detailed: (1) Extracting information from data source, (2) modifying news headline according to the different types of contradictions, (3) classifying the relationship between headline and body text (Compatible or Contradiction), and (4) randomly mixing headlines and body texts (Unrelated). 1. Extracting information from data source: The headline, body text, and date of the news item are extracted from a reliable data source. In this case, the news agency EFE was used ([[https://www.efe.com/efe/espana/1](https://www.efe.com/efe/espana/1)]([https://www.efe.com/efe/espana/1](https://www.efe.com/efe/espana/1)), accessed on 22 March 2021). The news extracted belongs to the political and economic domains, assuming that the headlines and body texts are compatible, although in the third stage this relationship is verified. 2. Modifying news headlines: The aim of this stage is to make the news headline contradictory to the body text by including simple alterations to the headline structure. The changes to the headline together with some examples are (examples given in Spanish and translated into English for clarification) given as follows: * NEGATION (Con_Neg): This alteration consists of negating the headline of the news item. 1. Original headline: _\"El comite de empresa_ debatira _manana la \"propuesta final\" de Alcoa\"_ (\"Union representatives _will discuss_ Alcoa's 'final proposal' tomorrow\") 2. Modified headline: _\"El comite de empresa_ no debatira _manana la \"propuesta final\" de Alcoa\"_ (\"Union representatives _will not discuss_ Alcoa's 'final proposal' tomorrow\") * ANTONYM (Con_Ant): This transformation consists of replacing the verb denoting the main event of the headline with an antonym. 1. Original headline: _\"El Goberno se compromete a_ subir _los salarios a los empleados pablicos tras los comicios\"_ (\"The Government pledges to _raise_ public employees' salaries after the elections\") 2. Modified headline: _\"El Goberno se compromete a_ bajar _los salarios a los empleados pablicos tras los comicios\"_ (\"Government pledges to _cut_ public employees' salaries after the elections\") * NUMERIC (Con_Num): This amendment consists of changing numbers, dates, or times appearing in the headline. 1. Original headline: _\"La economia brithatica ha crecido un \(3\%\) metos por el brexit, segin S&P\"_ (\"UK economy has grown by \(3\%\) less due to Brexit, says S&P\") 2. Modified headline: _\"La economia brithatica ha crecido un \(5\%\) metos por el brexit, segin S&P\"_ (\"UK economy has grown by \(5\%\) less due to Brexit, says S&P\") 3. STRUCTURE (Con_Str): This modification consists of changing the position of one word for another or substituting words in the sentence. 1. Original headline: \"[PERSON] _sustiturd a_ [PERSON] _como consejero delegado de IBM\"_ (\"_[PERSON]_ will replace _[PERSON]_ as IBM's CEO\")* Modified headline: \"[PERSON] _sustituird a_ [PERSON] _como consejero delegado de IBM\" (\"[PERSON]_ will replace _[PERSON]_ as IBM's CEO\") These alterations will change the semantic content of the sentence, making it contradictory to the previous headline and body text. The annotation process was carried out by two independent annotators that were trained by an expert annotator. * Classifying the relationship between the headline and the body text: The semantic relationship between the headline and the body text was annotated in two phases: The first phase consisted of classifying the information into Compatible (compatible information) or Contradiction (contradictory information). In the second phase, in the case of Contradiction, the type of contradiction was also annotated (Negation, Antonym, Numeric, Structure). This stage involved four annotators who are trained to detect semantic relationships between pairs of texts. * Aleatory mixing headline and body text: The news items reserved in the first stage were used to generate unrelated examples (Unrelated).