K-pop and fake facts: from texts to smart alerting for maritime security (2023)

Abstract: Maritime security requires full-time monitoring of the situation, mainly based on technical data (radar, AIS) but also from OSINT-like inputs (e.g., newspapers). Some threats to the operational reliability of this maritime surveillance, such as malicious actors, introduce discrepancies between hard and soft data (sensors and texts), either by tweaking their AIS emitters or by emitting false information on pseudo-newspapers. Many techniques exist to identify these pieces of false information, including using knowledge base population techniques to build a structured view of the information. This paper presents a use case for suspect data identification in a maritime setting. The proposed system UMBAR ingests data from sensors and texts, processing them through an information extraction step, in order to feed a Knowledge Base and finally perform coherence checks between the extracted facts.

## introduction one of the main challenges in the maritime domain is to ensure safety and security of ships: in and around harbors but also when they are offshore for several days. the security aspect has benefited from a renewed interest recently, due to piracy and trafficking. most harbor administrations rely today on ai-powered investigation tools to perform a number of checks on each entering vessel: comparing the declared status of the ship, aggregating sensor data, and even searching the web for news. among the organizations that collect and disseminate information about maritime events, the maritime information cooperation and awareness center (mica) collects and relays useful information to all actors in the field of maritime industry. its purpose is to process maritime security data worldwide. the 2022 annual report 1 summarizes the reports regularly sent to the maritime industry and analyses the trends observed as well as the evolution of modes of action. the sensor data mainly consists of radar and ais signals. every ship must emit its identity, speed, position and course at short time intervals. this information is received by all other vessels in reach as well as dedicated receiving stations, on the coast and in space. based on sensor data, alerts related to the behavior of vessels are raised automatically: abnormal position, sudden change of direction, etc. this ensures a quick and efficient reaction of the police/security agents. relevant information about ships and maritime events also occurs in a non-technical way, through the news (so-called "soft data"). accidents, illegal events, presence of a vessel in blockade-regulated areas and even modifications in the financial structure of the proprietary company are highly susceptible to increase the risk of a ship entering a harbor. malicious actors may use a large variety of techniques in order to perform covert operations, including trafficking, illegal fishing, piracy and smuggling. ais are easy to tamper with, as ships may -illegally-decide to modify their identity, declare a false destination or even cease to emit. another case of concern occurs when civil vessels are the main object of a crisis between international powers, such as the stena impero near iran in 2019 or the wheat vessels in the black sea in 2022: different newspapers may diffuse contradictory information about the same ships, which may in turn be inconsistent with technical data. in these cases, we believe that smart tools are needed in order to refine information and help the analysts build a clear picture of the situation. to tackle these security challenges, we propose umbar, a system to automatically collect, analyze and compare information from a variety of sources of data, resulting in a risk assessment that is practical for a security operator in the maritime domain. ## related works we structure our review of the literature along three streams: first, the identification of lies or manipulation on structured data. detecting fake news has recently received a lot of attention, combining facts and language @xcite ; here we focus on the identification of dissimilarity between facts such as stored in knowledge bases (attributes, properties, relations). for media analysis purposes, this falls under the topic of "automatic fact-checking" @xcite . in the maritime use case, expert systems for alert raising are common to detect a change of destination for a commercial vessel, or even a change of shipowner or flag can usually be observed (alaeddine and ray, 2022). ais systems can also be hacked to disseminate false information manufactured. the objective of these false messages (e.g. distress signals, false vessel locations, etc.) is to attract attention and trap the targeted vessels @xcite . these operations of disinformation and deception are very dangerous: it is essential to identify them. second, we focus our research on reasoning on facts in knowledge bases (kb), extracting the information using knowledge base population (k-pop), to automatically compute dissimilarity between text-extracted small knowledge graphs (kg) and to enable relation prediction; these applications are considered relevant for maritime security @xcite . @xcite focus on the link prediction task with complex linked datasets. their approach successfully captures crossover interactions between entities and relations when modeling @xcite propose an approach based on semantic similarity for generating explanations to link prediction problems on knowledge graphs. @xcite propose a model based on a graph transformer that learns entity embeddings by iteratively aggregating information from neighboring nodes to tackle the problem in the case of graphs that evolve over time. finally, our goal is to detect changes which occur over time and to evaluate information that evolves over time. thus, this technique will identify a large number of alerts linked to a normal evolution of the characteristics of an entity. dealing with temporal kb is still nothing trivial and mainly dealt with for question-answering where the relevant answers is dependent on time @xcite . reasoning on such facts with intelligent systems is pretty much novel @xcite , and still mainly dealt with by expert rules in operational systems. ## system breakdown in this section, we first sketch a view of umbar, then detail its two pillars: k-pop to extract the information, and coherence checks to perform the verification. ## expert opinion and maritime security qualitative analyses on maritime security use cases are still on progress. for production-grade real word applications, the extracted information from ais messages as presented in the figure 1 will be processed so as to check the coherence of information over time and to raise alerts in case suspect information is spotted. once combined, the aforementioned methods present features allowing to deal with actual data and information coming from heterogeneous sources, on a massive scale. it encompasses techniques to better correlate and assess information with different timings, from both texts and sensor data domains, so that trusted kbs can be populated with a user-defined reliability. obviously, human intervention cannot be eliminated, but adding this process to ensure maritime security will help a lot, and will increase the performance of detecting false information or attempts to manipulate information. ## conclusion this paper allowed us to present our semiautomatic end-to-end processing chain for information extraction and misinformation detection applied to a maritime surveillance use case. maritime security being a central sector susceptible to false information leading to disastrous consequences. although there is still room for improvement, our information extraction system and inconsistency detection provide support and alleviate the task of the operational staff in charge of monitoring. future work will focus on improving the kbp pipeline to move towards fully automatic extraction, conducting an evaluation and further study on the detection of erroneous information. the use of umbar in a representative setting is planned in order to evaluate the system and qualify it for its future operational deployment.