Operationalizing AI for Good: Spotlight on Deployment and Integration of AI Models in Humanitarian Work (2025)

Abstract: Publications in the AI for Good space have tended to focus on the research and model development that can support high-impact applications. However, very few AI for Good papers discuss the process of deploying and collaborating with the partner organization, and the resulting real-world impact. In this work, we share details about the close collaboration with a humanitarian-to-humanitarian (H2H) organization and how to not only deploy the AI model in a resource-constrained environment, but also how to maintain it for continuous performance updates, and share key takeaways for practitioners.

## introduction the last ten years have seen a surge in ai and natural language processing research to address real world problems that have a social good impact @xcite . many of these problems align with the united nations sustainable development goals (unsdg) 1 . this has also led to a surge in publications in this space to the point that even prominent ai research conferences have special tracks and themes related to social good (ie. aaai, acl-ijcnlp in 2021 @xcite ) and many targeted venues to tackle this topic such as the nlp for positive impact workshop series foot_1 . @xcite describe four different stages of ai for good tasks: 1. fundamental theories, 2. building block tools, 3. applicable tools and 4. deployed applications. while there have been a lot of publications in this space (for example @xcite found that just over 13% of all papers in the acl anthology map to one of the unsdgs), most published ai for good work has tended to focus more on the first three stages: specifically on analysis of the problem area, building a dataset, or building a model. however, there is comparatively very little published work on the fourth stage: on how these models fare when deployed in the real world and how they align with the expectations of the social good organization. in fact, for the acl-ijcnlp 2021 special theme of "nlp for social good", only one of the twelve accepted papers mentioned deployment. in addition, there has been very little work that discusses the collaboration process between a humanitarian organization and ai practitioners where a model is built to be used by the partner organization. the closest works are @xcite and @xcite , which highlight how ai teams should approach and undertake ai4sg projects -but do not mention any details about development and deployment process. in this short paper, we present our experience with working with insecurity insight foot_2 , a humanitarian-to-humanitarian organization (h2h), to bring an nlp model into the real world and provide impact to that organization and the aid community it supports. this work builds upon our previous research @xcite , in which we developed a multilingual dataset of news articles in english, french, and arabic, annotated with various types of violent incidents categorized by the humanitarian sectors they affect-such as aid security, education, food security, health, and protection. we also evaluated a range of deep learning architectures and techniques to tackle the associated task-specific challenges. in this paper, we take the next step by addressing the critical final stage: model deployment. in particular, we discuss not only the technical and process aspects of deploying a model in a resource-constrained environment, but also how to maintain it for continuous performance updates. we conclude with key takeaways and best practices for both ai model developers and humanitarian experts around technical topics, collaboration and processes. while this is just one example of a deployment, we hope this paper will encourage others to share their experiences and lessons learned. ## implementation and deployment following standard ml ops practices @xcite we split the model development into three stages: offline experimentation, staging deployment calibration and deployment monitoring (as presented in figure 1 ). ## discussion developing and deploying ai strategies for "ai for good" projects presents unique opportunities and challenges for ai practitioners and ngos. ensuring a sustainable and impactful deployment requires a collaborative approach that bridges technical expertise and domain-specific knowledge. below we outline key takeaways from our collaboration with insecurity insight. t1. understanding the problem: before developing ai models, practitioners must deeply understand the problem they want to address. this requires a thorough stakeholder engagement, data assessment, and problem scoping. during the early phase of the project, we gathered crucial domain knowledge from domain experts and engineers in insecurity insight to get a deep understanding of their current service and system, impact measurement, specific needs with priorities, resource and operational constraints, data availability and technical stack. this helps inform our key decisions in the steps of data collection, model selection and deployment. t2. data availability and quality: both parties must assess the availability, reliability and bias of data sources. available data may be noisy or limited in scope, thus requiring new data collection methods or annotation. data quality could be an ongoing issue, and thus it is important to start early and iterate: practitioners should work with domain experts to come up with clear annotation guidelines. in this particular study, we found it is essential to be mindful of the domain expert's time (operational cost). this requires both teams to setup realistic and meaningful plans and schedules. t3. capacity building: for ai solutions to be sustainable, partner organizations must have the capacity to use and maintain them. it is important to keep the partner in the loop throughout the development process and establish support mechanisms for model updates, debugging, and continuous improvement. t4. model performance mismatch awareness: both parties should be aware of potential discrepancies between offline evaluations and real-world ai performance (as we saw with our food security results). establishing a staged testing environment helps validate and refine ai solutions before deployment, reducing unexpected behaviors in pro-duction. both parties should be flexible in adjusting metrics to better fit real-world needs (e.g., optimize for precision instead of recall). ## limitations we acknowledge that this is just one example of an ai deployment in a humanitarian setting. ideally, we would present several examples of such deployments to paint a more robust picture of the different decisions partners can make, and the associated challenges. however, that is outside the scope of this short paper. we hope that by going into the details of this deployment process and showing the real-world impact will encourage others to publish their findings as well. another aspect we want to acknowledge is that there are many different types of ai for good projects and deployments. a group of ai scientists partnering with a humanitarian organization is just one configuration. ## ethical considerations the dataset is constructed from publicly available news articles, ensuring that no contractual agreements were violated in the data acquisition process. our web scraper strictly accessed openly available content, excluding any material behind paywalls. for the annotation process, we engaged internal humanitarian experts from the partnering organization. these experts were fairly compensated as part of their professional, paid employment.