IUGS-sponsored meeting on Large Language Models in
the Geological Sciences – for attendees
Note the meeting followed Chatham House Rules: participants are free to use the information
received, but neither the identity nor the a=iliation of the speaker(s), nor that of any otherparticipant, may be revealed.
Date: Tuesday 16 July 2024
Location: Geological Society of London and online.

Attendees
15 in-person and 44 virtual, spanning interested organisations across DDE, IUGS, geoscience
societies and publishers, academia, industry and governmental organisations.

Agenda and notes
1. An introduction on the objectives of the meeting was given.
2. Round table introductions from all attendees (in person and virtual).
3. How do geological science entities see Large Language Models (LLMs) in the future?
a. Presentations and statements by a number of participants were given outlining theirexperience and observations of the ethical development and future of LLMs.
b. An open discussion was undertaken which touched on a variety of key themes:i. There was positivity that this constructive meeting had been organised
between key entities and that discussion regarding development of LLMs
within the geosciences was in early stages, but critical for maintaining
integrity of the science.
ii. Common themes were the importance of using authoritative, unbiased
data and trusted sources to maintain scientiﬁc research credibility andensuring attribution to these. The importance of respecting intellectual
property rights was raised and concern around the erosion of criticalthinking.
iii. It was suggested that geoscience has not engaged with AI systems suchas LLMs as much as it could and may risk being left behind, however thisis true in other disciplines. Industry is in some cases more advanced inits use than academia/governmental organisations.
iv. Examples were given of other geoscience data projects that did not haveclear use cases and were considered to under-deliver. The importanceof having a clear use case for GeoGPT was stressed.
v. The role of societies and publishers in delivering high-quality, trusted
and specialised content was highlighted. There are hidden costs toensuring the integrity, usability and discoverability of publishedresearch, and many societies are active in curating, maintaining andencouraging FAIR (Findable, Accessible, Interoperable, Reusable) data.
This is a valuable source of training material for a geoscience-speciﬁc
LLM but also important for the ﬁnancial sustainability of societies,
allowing them to advance the science and to deliver their charitableobjectives. There wasthe general view that this copyright and attribution of authors,
researchers and publishers must be respected.
vi. Whilst this meeting was focussed on LLMs in the geological sciences,these by their very nature focus on language which is an interpretation ofgeology as a physical system. It was suggested that LLMs may not be themost e\ective model and thought should be given to physical models.
vii. Finally, the use of AI in delivering insights, more e\icient workﬂows
and new discoveries is inevitable and valuable. It is imperative that weﬁnd collaborative routes to deﬁning ethics and frameworks, so we canutilise these opportunities.
4. Members of the GeoGPT and DDE team presented the current state and future plans for
GeoGPT. Participants were pleased that this included the most transparent view of themodel to date, including the shift in technology from a generative model to a RAG(Retrieval Augmented-Generation) model. The latest version of GeoGPT was presented
including the choice of use of multiple foundational models (Qwen-2, LLAMA-3 andMistral).
a. There was an open discussion around several themes:i. There was a strong recommendation that the geoscience corpus used tofurther train GeoGPT from its base model is transparent and be madeavailable to the community. The ability to interrogate the model andunderstand data sources is necessary in research since not all sources
are made equal, and transparency will build trust within the researchcommunity and ultimately improve GeoGPT’s e\ectiveness. Thisincludes:
1. The Geoscience Q&A pairs, as demonstrated by theStanford Question Answering Dataset (SQuAD, a commonly usedbase for Natural Language Processing). There was also therecommendation that groups involved in the Q&A process arebroadened to be more globally representative to avoid bias.
2. The make-up of data sources e.g. X% Wikipedia, X% CommonCrawl, X% Published Open Access content.
3. The geoscience-speciﬁc training data to the article level. This
could be crowd-sourced 'checked' by society publishers andother parties to assess the ethics and biases of the content used
as well as the legalities, to build broad trust around a number ofareas including existing copyright concerns and a mechanism
put in place to remove IP infringing works.
ii. Whilst general and commercial LLMs do not make their sources
transparent for competitive reasons, GeoGPT is to be not-for-proﬁt andfreely available to all researchers around the world and therefore should
not rely on these practices. It was felt that GeoGPT should be setting thegold standard of transparency and ethics.
iii. As an international project, the governance, nature of the organisationand funding and potential government control of GeoGPT (currentlylargely funded by the Zhejiang Lab, China) must be transparent.
iv. iv. There was discussion around the ambition of GeoGPT, its strategy
and intended use cases which were felt to be a little unclear. There hasbeen quick progress, but development of these models is resource
intensive
and there should be focus on understanding the key audience and theirneeds, then deﬁning the strategy to ensure development is aligned anda\ordable.
v. The mutual beneﬁts of LLM development in the geosciences wasdiscussed. Societies and publishers have valuable training sources butmay also beneﬁt from the development of geoscience-speciﬁc LLMs. A
prerequisite for any content licensing agreement is trust, transparency
and e\ective governance to ensure terms are adhered to andrenegotiated when development advances.
5. There was then a discussion regarding governance of GeoGPT and some draft
recommendations for next steps were proposed by the Chair.
a. It was suggested that no governance model for GeoGPT is required whilst theIUGS review of DDE begins.
b. Development of GeoGPT can continue, with community engagement and testingat IGC in August but not for public launch.
c. There will be a reconvening of this group in Oct or Nov where a progress reportwill be given on GeoGPT development and delivery against therecommendations. More industry involvement will be sought.
d. Approval on the nature of involvement of IUGS will be required before public
launch of GeoGPT. IUGS will separately determine its own endorsement of DDE.