O cean GPT : A Large Language Model for Ocean Science Tasks (2024)

Abstract: Ocean science, which delves into the oceans that are reservoirs of life and biodiversity, is of great significance given that oceans cover over 70% of our planet’s surface. Recently, advances in Large Language Models (LLMs) have transformed the paradigm in science. Despite the success in other domains, current LLMs often fall short in catering to the needs of domain experts like oceanographers, and the potential of LLMs for ocean science is under-explored. The intrinsic reason may be the immense and intricate nature of ocean data as well as the necessity for higher granularity and richness in knowledge. To alleviate these issues, we introduce OceanGPT, the first-ever LLM in the ocean domain, which is expert in various ocean science tasks. We propose DoInstruct, a novel framework to automatically obtain a large volume of ocean domain instruction data, which generates instructions based on multi-agent collaboration. Additionally, we construct the first oceanography benchmark, OceanBench, to evaluate the capabilities of LLMs in the ocean domain. Though comprehensive experiments, OceanGPT not only shows a higher level of knowledge expertise for oceans science tasks but also gains preliminary embodied intelligence capabilities in ocean technology.

## introduction ocean science, which delves into the intricacies of oceans that cover over 70% of our planet's surface, is essential not only for understanding the rich reservoirs of life and biodiversity but also for recognizing their pivotal role in regulating the global climate and supporting economies @xcite @xcite . recently, advances in large language models (llms) @xcite @xcite @xcite have what are the fishery resources in south china sea? the following species of commercial and recreational importance can be found among ... south china sea: tuna, mackerel sardines squid crabs shrimps lobsters oyster shellfish pearls etc. in addition to these commercially ... ## oceangpt to obtain oceangpt, we firstly construct the training corpus for ocean science and pre-train an ocean llm based on llama-2 @xcite in section 3.1. then we propose doin-struct, an automated framework for domain instruction generation to build an ocean domainspecific instruction dataset. our framework leverages multi-agent collaboration and utilizes ocean literature to automatically generate a large volume of domain-specific instructions for ocean science tasks (section 3.2). the overview training procedure of our oceangpt is shown in figure 2 . to pre-train the foundation model for ocean science tasks, it is essential to construct the pretraining corpus specific to ocean science. therefore, we firstly collect a raw corpus of 67,633 documents from open-access literature. for the specific volumes we choose, we prefer to consider publications from recent years to ensure the inclusion of the latest research and developments. at the same time, we select some historically significant literature to help the llm understand the developmental history of the field. for diversity, we choose articles from different sources to ensure coverage of various research perspectives and methods. specifically, we utilize the python package pdfminer to convert the content of literature files into plain text. to ensure the quality and consistency of the data, further processing of the collected dataset is necessary. we apply regular expressions to filter out figures, tables, headers, footers, page numbers, urls and references. additionally, any extra spaces, line breaks, and other non-text characters are removed. the processed documents cover various aspects of ocean science such as ocean physics, ocean chemistry, ocean biology, geology, hydrology, etc. it is important to note that special characters, emoticons, and garbled characters are also replaced or eliminated during this process. we also employ hash-based methods to de-duplicate the data, which helps reduce the risk of over-fitting during pre-training and enhances its generalization capability. ## benchmarking ocean science tasks we provide detailed explanations of the experimental setup and the baseline models in section 4.1. in section 4.1, we construct an ocean-specific benchmark oceanbench foot_0 to evaluate the capabilities of our oceangpt. we describe the automatic and human evaluation in section 4.1. ## conclusion in this paper, we introduce oceangpt, the firstever oceanographic pre-trained language model, which is expert in various ocean science tasks. to alleviate the difficulties for obtaining ocean data, we propose an domain construction framework called doinstruct, which constructs the ocean instruction dataset by multi-agent collaboration. each agent in our designed framework is considered as an expert in a specific topic and is responsible for generating the corresponding data. our generated dataset consists of diverse instructions to align the desired behaviors in ocean science issues. though comprehensive analysis, we observe that oceangpt not only demonstrates a higher level of knowledge expertise for oceans science tasks but also gains preliminary embodied intelligence capabilities in ocean engineering. we will continue to improve oceangpt by training on larger corpus with larger models (e.g., 30b, 70b) and maintain oceanbench by adding new data and tasks.