Package: blind_llama
BlindLlamaMaking AI Confidential & TransparentTable of ContentsAbout the projectUse casesGetting startedAdvanced securityVision and roadmapAbout usContact📜 About the projectIntroduction🛠️BlindLlamamakes it easy to use open-source LLMs by usingZero-trust AI APIsthat abstract all the complexity of model deployment while ensuringusers’ data is never exposedto us thanks to end-to-end protection withsecure hardware.🔐 To provide guarantees to developers that data sent to our managed infrastructure is not exposed, we have developed aZero-trust architecture to serve AI models.We currently serveLlama2but will be making more open-source models available in the near future!Our backend has two key properties:Confidentiality: Your data isnever accessible to us. We serve AI models insidehardened environmentsthat do not expose data even to our admins. All points of access, such as SSH, logs, networks, etc., are blocked to ensure the isolation of data.Transparency: We provide you with verifiablecryptographic proofthat these controls are in place, thanks to the use ofTrusted Platform Modules (TPMs).WarningBlindLlama is stillunder developmentand does have the full security features.Do not test our API with confidential information... yet!You can follow our progress towards the next beta and 1.0 versions of BlindLLama on ourroadmap.We welcome contributions to our project from the community! Don't hesitate toraise issueson GitHub,reach out to usor see our guide on how to audit BlindLlama (coming soon!).(back to top)ArchitectureBlindLlama is composed of two main parts:Anopen-source client-side Python SDKthat verifies the remote Zero-trust AI models we serve are indeed guaranteeing data sent is not exposed to us.Anopen-source serverthat serves models without any exposure to us as the server is hardened and removed potential leakage channels from network to logs, and provides cryptographic proof those privacy controls are indeed in place using TPMs.The server combines a hardened AI server with attested TLS usingTPMs.The client performs two main tasks:Verifying that the server it communicates with is the expected hardened AI serverusing attestation.Securely sending datato be analyzed by a remote AI model using attested TLS to ensure data is not exposed.The server has two main tasks:Itloads a hardened AI serverwhich is inspected to ensure no data is exposed to the outside.Itserves models using the hardened AI server that can be remotely verifiedusing attestation.Trust modelOn this page, we will explain more precisely what components/parties have to be trusted when using BlindLlama.To understand better which components and parties are trusted with BlindLlama, let’s start by examining what is trusted with regular AI services.To do so, we will use the concept of aTrusted Computing Base (TCB), which refers to the set of all hardware, firmware, and software components that are critical to a system's security.Trusted Computing Base with regular AI providersWe can imagine that an AI provider serves AI APIs to their users using a Cloud infrastructure. Then the parties to be trusted are:The AI provider: they provide the software application that is in charge of applying AI models to users’ data.The Cloud provider: they provide the infrastructure, Hypervisor, VMs and OS, to the AI provider.The hardware providers: they provide the lowest physical components, CPU, GPU, TPMs, etc. to the Cloud provider who then manages those to resell infrastructure to the AI providers.The higher the party in the stack, the closer they are to the data, and the more they are in a position to expose data.In most scenarios today, there is often blind trust in the AI provider, aka we send data to them without any technical guarantees that they will do what they said they would do. For instance, the AI provider could say they just do inference on data, while they could actually train models on users’ data.For privacy-demanding users that require more technical guarantees, they often choose not to send data to AI providers as they cannot trust them with their confidential data.Trusted parties with BlindLlamaWith BlindLlama, we remove the AI provider from the list of trusted parties. When models are served with BlindLlama, users' data cannot be seen by the AI provider because we use a Zero-trust AI infrastructure that removes the service/AI provider from the trust base. We can prove such controls are in place usingTPM-based attestation.👩🏻‍💻 Use casesBlindLlama is meant tohelp developers working with sensitive data to easily get started with LLMsby usingmanaged AI APIsthat abstract the hardware and software complexity of model deployment while ensuring their data remains unexposed.Several scenarios can be answered by using BlindLlama, such as:Benchmarking the best open-source LLMs against one’s private data to find out which one is the most relevant without having to do any provisioningStructuring medical documentsAnalysis or auto-completion of a confidential code base✅ When should you use BlindLlama?You want to get started with LLMs that are complex to deploy, such as Llama 2 70BYou don’t want to manage that infrastructure as it requires too much time, expertise and/or budgetYou don’t want to expose your data to a third party AI provider that manages the infrastructure for you due to privacy/compliance issues❌ What is not covered by BlindLlama?BlindLlama is simply a drop-in replacement to query a remotely hosted model instead of having to go through complex local deployment. We do not cover training from scratch, but we will cover fine-tuning soon.BlindLlama allows you to quickly and securely leverage models which are open-source, such as Llama 2, StarCoder, etc.Proprietary models from OpenAI, Anthropic, and Cohere are not supportedyet as we would require them to modify their backend to offer a Zero-trust AI infrastructure like ours.BlindLlama’s trust model implies some level of trust in Cloud providers and hardware providerssince we leverage secure hardware available and managed by Cloud providers (see ourtrust model sectionfor more details).BlindLlama virtually provides the same level of security, privacy, and control as solutions provided by Cloud providers like Azure OpenAI Services.(back to top)🚀 Getting startedCheck out ourQuick tour, which will enable you to play with an example using theLlama 2model while ensuring your data remains private and without the hassle of provisioning!Find out more aboutHow we protect your dataRefer to ourConceptsguide for more information on key conceptsLearn more about BlindLlama's design with ourBlindLlama 101guide(back to top)📚 Advanced securityWe created the BlindLlama whitepaper to cover the architecture and security features behind BlindLLama in greater detail.The whitepaper is intended for an audience with security expertise.You can read or download the whitepaperhere!🎯 RoadmapThere are three key milestones planned for the BlindLlama project.BlindLlama Alpha (non-secure) launch:The alpha launch of BlindLlama provides an API for the Llama2-70b model which you can query with our python SDK.Users can test out and query our APIbut should not yet send any confidential data to the APIas it is does not yet have full implementation of security features.The server-side code includes the backbones for our attestation feature (which means we can prove the server is deploying the expected code to end users) but this feature will be fully launched in the following beta phase.Expected launch date: week ending 08/09/2023BlindLlama Beta launch:The beta version adds the full implementation of TPM-based attestation, meaning our API can be fully verified remotely. This version will not yet have full hardening of server-side environment or audit and thus is not yet recommended in production!Provisional launch date: week ending 06/10/2023BlindLlama 1.0 audit-ready launch:A fully-secure version of BlindLlama ready for audit, with a fully hardened server environment.Provisional launch date: week ending 08/12/2023You can check out more details about these stages and our progress to achieveing these milestones on ourofficial roadmap.(back to top)🦙 Who made BlindLlama?BlindLlama is developed byMithril Security, a startup focused ondemocratizing privacy-friendly AI using secure hardware solutions.We have already had our first project,BlindAI, an open-source Rust inference server that deploys ONNX models on Intel SGX secure enclaves, audited byQuarkslab.BlindLlama builds on the foundations of BlindAI but provides much faster performance and focuses on serving managed models directly to developers instead of helping AI engineers to deploy models.(back to top)📇 Contact(back to top)