Security Operations on the Data Lakehouse: Hunters SOC Platform is now available for Databricks customers - The Databricks BlogSkip to main contentPlatformThe Databricks Lakehouse PlatformDelta LakeData GovernanceData EngineeringData StreamingData WarehousingData SharingMachine LearningData SciencePricingMarketplaceOpen source techSecurity and Trust CenterWEBINAR May 18 / 8 AM PT
Goodbye, Data Warehouse. Hello, Lakehouse.
Attend to understand how a data lakehouse fits within your modern data stack.
Register nowSolutionsSolutions by IndustryFinancial ServicesHealthcare and Life SciencesManufacturingCommunications, Media & EntertainmentPublic SectorRetailSee all IndustriesSolutions by Use CaseSolution AcceleratorsProfessional ServicesDigital Native BusinessesData Platform MigrationNew survey of biopharma executives reveals real-world success with real-world evidence.
See survey resultsLearnDocumentationTraining & CertificationDemosResourcesOnline CommunityUniversity AllianceEventsData + AI SummitBlogLabsBeaconsJoin Generation AI in San Francisco
June 26–29
 
Learn about LLMs like Dolly and open source Data and AI technologies such as Apache Spark™, Delta Lake, MLflow and Delta SharingExplore sessionsCustomersPartnersCloud PartnersAWSAzureGoogle CloudPartner ConnectTechnology and Data PartnersTechnology Partner ProgramData Partner ProgramBuilt on Databricks Partner ProgramConsulting & SI PartnersC&SI Partner ProgramPartner SolutionsConnect with validated partner solutions in just a few clicks.Learn moreCompanyCareers at DatabricksOur TeamBoard of DirectorsCompany BlogNewsroomDatabricks VenturesAwards and RecognitionContact UsSee why Gartner named Databricks a Leader for the second consecutive yearGet the reportTry DatabricksWatch DemosContact UsLoginJUNE 26-29REGISTER NOWCategoriesAll blog postsCompanyCultureCustomersEventsNewsPlatformAnnouncementsPartnersProductSolutionsSecurity and TrustEngineeringData Science and MLOpen SourceSolutions AcceleratorsData EngineeringTutorialsData StreamingData WarehousingData StrategyBest PracticesData LeaderInsightsIndustriesFinancial ServicesHealth and Life SciencesMedia and EntertainmentRetailManufacturingPublic SectorSecurity Operations on the Data Lakehouse: Hunters SOC Platform is now available for Databricks customersby Lipyeow Lim and Guy YasoorMarch 27, 2023 in Platform BlogShare this postCybersecurity is a big data problem: The growing volume and complexity of data flowing in and out of enterprises have created new cybersecurity challenges. Existing SIEM solutions cannot scale with the velocity of data growth without taxing security budgets and draining existing resources.Today, cybersecurity company Hunters is announcing the availability of its SOC Platform for Databricks customers. For the first time, Databricks customers will be able to attain an end-to-end, security operations platform on their own Databricks Lakehouse Platform deployments, while keeping the flexibility of owning all the data and having the power to build their own additional security analytics on the Lakehouse.Hunters SOC Platform is a modern SIEM alternative that ingests, normalizes and analyzes data from all security data sources of an organization, including endpoint telemetry, network traffic, identity management, and cloud infrastructure. Not only does Hunters provide a wider variety of security-related data integrations, but the platform also identifies threats in real-time across the attack surface and provides security teams prioritized incidents to handle, reducing the time needed to contain and remediate threats to the organization.What can Databricks customers do with Hunters to create even more value?Build a Security Data LakeOne of the biggest burdens of security teams today is managing the ingestion of terabytes of data from dozens of security products. Hunters eases this process with a state-of-the-art engine that provides scalable ingestion, monitoring and optimization. Moreover, it comes prebuilt with a large library of off-the-shelf integrations that can be set up in minutes.Hunters SOC Platform ingests and performs the ETL of all security-related data into the customer's Databricks Lakehouse using the customer's cloud storage: the customer retains ownership of all the security data. The Hunters ETL follows the Databricks' Medallion Architecture model storing the raw data and also normalizing the data into a unified schema that will facilitate further analysis. While Hunters already provides a rich set of analytical capabilities, customers with advanced cybersecurity analytics teams can augment the Hunters capabilities by leveraging Databricks Data Science and Machine Learning capabilities and the partner technologies in the Databricks ecosystem. For example, many customers have AI/ML models for detecting threats that are highly specific and customized to their specific organizational context (eg. insider threats). Such detections are so specific that it does not make sense for a vendor like Hunters to build into their product. Hunters provides the flexibility for customers to leverage the Databricks lakehouse for such use cases.Detect and Investigate IncidentsHunters provides a library of hundreds of built-in detection rules that cover the majority of the threat landscape, mapped onto a common industry framework (the MITRE ATT&CK). This allows customers to visualize coverage and understand their security gaps. All detection rules are pre-verified on real-world customer data to minimize false positives and excessive alerting. The detection rules are deployed directly to all customer tenants without requiring any action or tweaking, thereby automatically reducing the cybersecurity risk with little operational overhead.Each alert also passes through an investigation engine, where it is automatically enriched with contextual information from various sources, and sophisticated dynamic scoring is applied to it to reduce alert fatigue. Not all signals from the same detection logic require the same urgency. For example, alerts that involve sensitive assets (e.g., C-level, domain servers, etc.) are prioritized, and risk for known benign behaviors is lowered (e.g., an executable IoC signed by Microsoft). Addressing the priority of alerts or incidents with dynamic scoring helps security teams manage their SOC workloads more efficiently.When the SOC analyst gets to an alert, all contextual information is provided in a single pane of glass to expedite triage and investigation. The contextual information goes beyond 'simple' enrichment of IP addresses with threat intelligence feeds, to deep correlation such as linking the user name in a CrowdStrike EDR alert with login records from the Okta authentication logs. Hunters' deep correlation capability is powered by a graph correlation engine: Alerts across entities and attack surfaces are automatically correlated on a graph. This graph correlation capability enables Hunters to highlight high-fidelity threat activity and provides analysts the flexibility to leverage low-fidelity signals that are often overlooked without generating more noise.For incidents requiring investigations from multiple organizations including third party service providers and/or government agencies, Databricks provides cleanrooms where collaborating investigators can jointly investigate an incident using the relevant subsets of data and the custom analytics that can be proprietary to different organizations.Search & Incident ResponseHaving all of your security data stored in a modern data lake has great advantages for incident responders, and anyone who wants to gain insights on massive amounts of data.Using Hunters and Databricks, customers can not only store petabytes of data, but also make use of them in their day-to-day investigations and in their most critical incidents. Some capabilities that support this are the following:IOC Search: Purpose-built search capability, allows responders to search all organizational data ingested by Hunters that resides on the Lakehouse for IOCs (IP, domain, hash) in seconds within the SOC Platform itself.Entity Search: Makes it easy to see all information about an entity in the environment in a centralized place. For example, from one suspicious login alert, customers can easily pivot to see the latest logins of the user in question across all endpoints, cloud infrastructure, and SaaS providers. In the same user interface, a responder can observe which alerts a user in question was involved in, and what is their role in the organization. Entity related views create huge efficiencies and productivity for security teams.Raw data access: all of your security data is available for you to dive into as you see fit, both from within the Hunters console, and from your familiar Databricks interface. You can run queries on months of data to find that needle in a haystack, create operational dashboards that help expedite investigations, and run your own AI/ML models.The openness of the Hunters and Databricks integration encourages security teams to innovate in their fight against cyber criminals. The Hunters SOC Platform not only helps security teams to do their day-to-day job more efficiently and effectively, but also provides all the data in a Databricks lakehouse where they can experiment, create, and test their own security analytics and AI/ML models and contribute these back to the cybersecurity community at large. Cybersecurity is a team sport. Let a thousand flowers bloom.If you want to try Hunters out on your Databricks Lakehouse, please request a demo!Try Databricks for freeGet StartedRelated postsCybersecurity in the Era of Multiple Clouds and RegionsAugust 30, 2022 by Zafer Bilaloglu and Lipyeow Lim in Product
In 2021, more than three quarters of all enterprises have infrastructure in multiple clouds. This trend shows no signs of slowdown with hybrid...
Cybersecurity in ManufacturingMarch 1, 2023 by Lipyeow Lim and Bala Amavasai in Manufacturing
With the increasing push toward smart manufacturing, cybersecurity has taken center stage in the operational risk profile of manufacturers. A study by Deloitte...
Analyzing Okta Logs With Databricks Lakehouse Platform to Detect Unusual ActivityApril 7, 2022 by Arun Pamulapati in Product
With the recent social media reports of an Okta incident through a third party contractor, security teams ran to their logs and asked...
See all Platform Blog postsProductPlatform OverviewPricingOpen Source TechTry DatabricksDemoProductPlatform OverviewPricingOpen Source TechTry DatabricksDemoLearn & SupportDocumentationGlossaryTraining & CertificationHelp CenterLegalOnline CommunityLearn & SupportDocumentationGlossaryTraining & CertificationHelp CenterLegalOnline CommunitySolutionsBy IndustriesProfessional ServicesSolutionsBy IndustriesProfessional ServicesCompanyAbout UsCareers at DatabricksDiversity and InclusionCompany BlogContact UsCompanyAbout UsCareers at DatabricksDiversity and InclusionCompany BlogContact UsSee Careers
at DatabricksWorldwideEnglish (United States)Deutsch (Germany)Français (France)Italiano (Italy)日本語 (Japan)한국어 (South Korea)Português (Brazil)Databricks Inc.
160 Spear Street, 13th Floor
San Francisco, CA 94105
1-866-330-0121© Databricks 2023. All rights reserved. Apache, Apache Spark, Spark and the Spark logo are trademarks of the Apache Software Foundation.Privacy Notice|Terms of Use|Your Privacy Choices|Your California Privacy Rights