###### Abstract  
This study investigates the capabilities of open-source Large Language Models (LLMs) in automating GDPR compliance documentation, specifically in generating data categories--types of personal data (e.g., names, email addresses)--for processing activity records, a document required by the General Data Protection Regulation (GDPR). By comparing four state-of-the-art open-source models with the closed-source GPT-4, we evaluate their performance using benchmarks tailored to GDPR tasks: a multiple-choice benchmark testing contextual knowledge (evaluated by accuracy and F1 score) and a generation benchmark evaluating structured data generation. In addition, we conduct four experiments using context-augmenting techniques such as few-shot prompting and Retrieval-Augmented Generation (RAG). We evaluate these on performance metrics such as latency, structure, grammar, validity, and contextual understanding. Our results show that open-source models, particularly Owen2-7B, achieve performance comparable to GPT-4, demonstrating their potential as cost-effective and privacy-preserving alternatives. Context-augmenting techniques show mixed results, with RAG improving performance for known categories but struggling with categories not contained in the knowledge base. Open-source models excel at structured legal tasks, although challenges remain in handling ambiguous legal language and unstructured scenarios. These findings underscore the viability of open-source models for GDPR compliance, while highlighting the need for fine-tuning and improved context augmentation to address complex use cases. large language model; GDPR documentation; natural language processing +
Footnote †: journal:  
0000-0001-9991-4000-0000-0000-0000-0000-0000-0000-0000-00000-0000-00000-00000-00000-00000-00000-00000-00000-00000-00000-000000-00000-00000-000  
Given these extensive and precise documentation requirements, organizations face growing challenges in achieving efficiency and accuracy in GDPR compliance, making LLMs a promising tool to partially automate compliance tasks. However, leveraging LLMs for GDPR compliance introduces significant challenges. GDPR documentation often relies on precise interpretations of legal terms such as \"necessary\", \"appropriate\", or \"adequate\", which are inherently ambiguous and context-dependent. This creates challenges for LLMs to generate legally valid text [2]. Additionally, the static knowledge bases of LLMs make it difficult for them to reflect real-time updates to the GDPR or evolving regulatory requirements [2]. This further complicates their application to compliance tasks. As LLMs have been designed to predict the most probable next token, they are prone to create hallucinated information [2], which can be particularly problematic when precision and adherence to strict legal standards are essential. To be effective with respect to GDPR compliance, LLMs must meet strict requirements. These include ensuring data security, preserving privacy (Art. 32 GDPR [3]), providing transparency in data processing (Recital 58 GDPR [3]), and achieving high accuracy in generating compliance documentation. Furthermore, the GDPR mandates robust control over data handling (Art. 5, 24, 25, 32 GDPR [3]), which poses challenges for models hosted externally, as organizations cannot guarantee compliance if they lack insights into how the data are processed. For these reasons, the choice between open-source and closed-source LLMs is critical. Closed-source models such as GPT-4, though being powerful, raise data security and compliance concerns due to their API-based access and lack of visibility into pretraining data. Open-source models, on the other hand, allow for on-premise deployment, giving organizations full control over their infrastructure and data, which is crucial to meet the strict standards set out in the GDPR.