# MA-1062.042024: MyCERT Advisory - Guidance on Deploying AI Systems Securely

**Date:** 19 Apr 2024  
**Source:** https://www.mycert.org.my/portal/advisory?id=MA-1062.042024  
**Advisory ID:** MA-1062.042024

---

**1.0 Introduction**
The widespread integration and utilization of AI technologies present lucrative opportunities for cyber attackers. These individuals, who traditionally exploit data theft for their own gain, may now aim to manipulate deployed AI systems for malicious purposes.
When targeting AI systems, malicious actors may employ specialized attack methods alongside conventional IT techniques. Given the multitude of potential attack avenues, defenses must be both varied and thorough. Experienced attackers often blend different strategies to bypass layered security measures effectively.
To safeguard AI deployments, organizations should adhere to several key practices. These include securing the deployment environment, implementing continuous protection measures for AI systems, and ensuring secure operation and maintenance protocols are in place.
**2.0 Scope and Audience**
The term "AI systems" in this document specifically refers to artificial intelligence systems based on machine learning (ML).
These recommended practices are particularly relevant for organizations implementing externally developed AI systems in on-premises or private cloud setups, especially in environments with heightened risks and significant assets at stake. They may not be suitable for organizations that are not directly deploying AI systems but rather utilizing AI systems deployed by others.
Not all of the suggested guidelines will directly apply to every organization or scenario. The sophistication level and attack methods employed will vary depending on the adversaries targeting the AI system. Therefore, organizations should assess the advice in conjunction with their specific use cases and threat landscape.
**3.0 Recommendations3.1 Secure the deployment environment.**
Before implementing AI systems, organizations should confirm that their existing IT infrastructure adheres to strong security principles, including effective governance, well-structured architecture, and secure configurations. It's essential, for instance, to assign cybersecurity responsibility to the same individual who oversees the organization's overall cybersecurity.
The security measures and standards applicable to IT environments also extend to AI systems. The following best practices are especially crucial for both AI systems and the IT environments where they are deployed.
**3.1.1 Manage deployment environment governance**
- If a non-IT department within the organization is deploying or managing the AI system, collaborate with the IT service department to ensure that the deployment environment aligns with the organization's IT standards.
- Assess the organization's overall risk level and ensure that the AI system and its usage fall within the organization's risk tolerance and that of the specific IT environment hosting the AI system. Evaluate and document relevant threats, potential impacts, and risk acceptance.
- Define the roles and responsibilities of each stakeholder and establish accountability measures, particularly if the organization manages its IT environment separately from the AI system.
- Determine the security boundaries of the IT environment and how the AI system fits within them.
- Require the primary developer of the AI system to create a threat model for their system.
- Utilize the threat model provided by the AI system developer to guide the implementation of security best practices, assess potential threats, and develop mitigation strategies.
- When drafting contracts for AI system products or services, incorporate requirements 
- Foster a collaborative culture among all involved parties, with a focus on encouraging open communication among the data science, infrastructure, and cybersecurity teams to address any risks or concerns effectively.
**3.1.2 Ensure a robust deployment environment architecture**
- Establish security measures to safeguard the interfaces between the IT environment and the AI system.
- Identify and rectify any blind spots in boundary protections and other security-critical areas highlighted in the threat model of the AI system. For instance, implement an access control system for managing access to the AI model weights, restricting access to authorized users through two-person control (TPC) and two-person integrity (TPI) mechanisms.
- Identify and secure all proprietary data sources utilized for training or refining AI models. Evaluate the list of data sources used by externally trained models, if available, and maintain a repository of trusted and validated data sources to mitigate risks like data poisoning or backdoor attacks. When acquiring data from third parties, consider incorporating contractual or service level agreement (SLA) provisions as suggested safeguards.
- Implement secure by design principles and Zero Trust (ZT) frameworks within the architecture to effectively manage risks associated with the AI system.
**3.1.3 Harden deployment environment configurations**
Apply established security best practices to the deployment environment, including isolating the environment where machine learning (ML) models run within secure containers or virtual machines (VMs), monitoring network traffic, configuring firewalls with approved lists, and adhering to other recommended practices such as those outlined in the NSA's Top Ten Cloud Mitigation Strategies for cloud setups.
- Regularly review guidance and notifications from hardware vendors (e.g., GPUs, CPUs, memory) and apply software patches and updates to mitigate the risk of vulnerabilities being exploited, ideally through the Common Security Advisory Framework (CSAF).
- Protect sensitive AI data (e.g., model weights, outputs, logs) by encrypting data at rest and storing encryption keys in a hardware security module (HSM) for secure on-demand decryption.
- Implement robust authentication mechanisms, access controls, and secure communication protocols, such as employing the latest version of Transport Layer Security (TLS) for encrypting data during transit.
- Utilize phishing-resistant multifactor authentication (MFA) to secure access to information and services, and actively monitor and respond to fraudulent authentication attempts.
- Understand and address how malicious actors exploit weak security controls by adopting mitigations outlined in documents like “Weak Security Controls and Practices Routinely Exploited for Initial Access.”
**3.1.4 Protect deployment networks from threats**
Embrace a Zero Trust (ZT) mindset, which operates under the assumption that a security breach is either imminent or has already occurred. Implement detection and response capabilities to swiftly detect and mitigate compromises.
- Utilize reliable and well-tested cybersecurity solutions to efficiently identify unauthorized access attempts and enhance incident assessment speed and accuracy.
- Incorporate an incident detection system to assist in prioritizing incidents. Additionally, integrate mechanisms to promptly block access for suspected malicious users or to sever inbound connections to AI models and systems in the event of a significant incident requiring immediate response.
**3.2 Continuously protect the AI system3.2.1 Validate the AI system before and during use**
- Utilize cryptographic methods, digital signatures, and checksums to verify the origin and integrity of each artifact, safeguarding sensitive information from unauthorized access during AI processes.
- Generate hashes and encrypted duplicates of each release of the AI model and system for secure storage in an unalterable location. Safeguard hash values and/or encryption keys within a secure vault or hardware security module (HSM) to prevent simultaneous access to both encryption keys and encrypted data and models.
- Store all types of code (e.g., source code, executable code, infrastructure as code) and artifacts (e.g., models, parameters, configurations, data, tests) in a version control system with appropriate access controls to ensure usage of only validated code and track any modifications.
- Conduct comprehensive testing of the AI model for resilience, accuracy, and potential vulnerabilities following modifications. Utilize techniques such as adversarial testing to assess the model's ability to withstand compromise attempts.
- Implement automated rollback capabilities and employ advanced deployment methods with human oversight to enhance reliability, efficiency, and enable continuous delivery of AI systems. Automated rollback capabilities ensure swift restoration to a known good state in case of issues or compromises.
- Assess and secure the supply chain for external AI models and data, ensuring adherence to organizational standards and risk management policies, and prioritizing those developed according to secure by design principles. Address risks for parts of the supply chain unable to meet organizational standards and policies.
- Prior to tuning, training, and deployment, carefully scrutinize models, particularly imported pre-trained models, within a secure development environment. Utilize organization-approved AI-specific scanners, if available, to detect potential malicious code and ensure model validity before deployment.
- Explore automation of detection, analysis, and response capabilities to enhance efficiency of IT and security teams, enabling rapid and targeted responses to potential cyber incidents. Conduct continuous scans of AI models and their hosting IT environments to detect potential tampering.
- When considering the use of additional AI capabilities to improve automation efficiency, carefully evaluate risks and benefits, and ensure human oversight where necessary.
**3.2.2 Secure exposed APIs**
- If the AI system offers application programming interfaces (APIs), ensure their security by incorporating authentication and authorization mechanisms to control API access. Utilize secure protocols like HTTPS, which includes encryption and authentication.
- Introduce validation and sanitization procedures for all input data to mitigate the possibility of unwanted, suspicious, incompatible, or malicious input reaching the AI system, such as vulnerabilities like prompt injection attacks.
**3.2.3 Actively monitor model behaviour**
- Gather logs to encompass inputs, outputs, intermediate stages, and errors; automate notifications and triggers.
- Regularly monitor the architecture and configuration settings of the model to detect any unauthorized alterations or unexpected adjustments that could jeopardize its performance or security.
- Be vigilant for any efforts to access or extract data from the AI model or collect aggregate inference responses.
**3.2.4 Protect model weights**
- Strengthen interfaces for accessing model weights to raise the difficulty level for potential adversaries attempting to extract them. This can involve ensuring that APIs only provide essential data necessary for the task, hindering attempts at model inversion.
- Implement hardware safeguards for storing model weights where feasible. This may include deactivating unnecessary hardware communication functionalities and safeguarding against emanation or side channel techniques.
- Take aggressive measures to isolate weight storage. Options include storing model weights in a secure storage vault, within a highly restricted zone (HRZ) like a dedicated enclave, or utilizing a hardware security module (HSM).
**3.3 Secure AI operation and maintenance3.3.1 Enforce strict access controls**
- Ensure unauthorized access or manipulation of the AI model is prevented. Employ role-based access controls (RBAC), or preferably attribute-based access controls (ABAC) when possible, to restrict access solely to authorized individuals.
- Differentiate between regular users and administrators. Mandate multifactor authentication (MFA) and the use of privileged access workstations (PAWs) for administrative access.
**3.3.2 Ensure user awareness and training**
Provide training to users, administrators, and developers regarding security best practices, including effective password management, methods to prevent phishing attacks, and secure handling of data. Foster a culture that prioritizes security awareness to reduce the likelihood of human error. Consider utilizing a credential management system, if available, to control, oversee, and track credential usage for additional risk mitigation.
**3.3.3 Conduct audits and penetration testing**
- Involve external security professionals to perform audits and penetration testing on AI systems ready for deployment. This assists in uncovering any overlooked vulnerabilities or weaknesses internally.
**3.3.4 Implement robust logging and monitoring**
- Implement strong monitoring and logging mechanisms to track the behavior, inputs, and outputs of the system, enabling the detection of abnormal behavior or potential security incidents. Keep an eye out for data drift or frequent and repetitive inputs, as these may indicate model compromise or automated breach attempts.
- Set up alert systems to promptly notify administrators of potential adversarial compromise attempts, security breaches, or anomalies resembling oracle-style attacks. Timely identification and response to cyber incidents are vital for protecting AI systems.
**3.3.5 Update and patch regularly**
- Before redeploying, thoroughly evaluate the updated model to ensure that accuracy, performance, and security tests meet acceptable standards.
**3.3.6 Prepare for high availability (HA) and disaster recovery (DR)**
- Employ an immutable backup storage system, tailored to the needs of the system, to guarantee that all objects, particularly log data, remain unchangeable and cannot be altered.
**3.3.7 Plan secure delete capabilities**
- Conduct automated and irreversible removal of components, like training and validation models or cryptographic keys, ensuring no retention or remnants remain once any process involving data and models is completed.
**4.0 References**
- <https://www.cisa.gov/news-events/alerts/2024/04/15/joint-guidance-deploying-ai-systems-securely>
- <https://media.defense.gov/2024/Apr/15/2003439257/-1/-1/0/CSI-DEPLOYING-AI-SYSTEMS-SECURELY.PDF>