# Application of Workflow Technology for Big Data Analysis Service
## 3 Case Study
### Network Security Situation Analysis  
Network security situation analysis uses big data analysis and data mining technology to analyze security logs and network traffic information to guarantee network security situations. For network operators, network businesses are subject to several security threats, such as zombie Trojan-worm attacks, malicious URLs, distributed denial-of-service (DDoS) attacks, and abnormal bot behavior. Discovering these malicious attacks and monitoring the real-time network flow to intercept the malicious attacks can greatly improve the quality of network services to avoid the malicious attack invasion. Network security situation analysis mainly includes two tasks: malicious attack behavior analysis and network traffic real-time analysis.  
Malicious attack behavior analysis: The core ideas of big data analysis on security logs and network traffic are to integrate multiple data for correlation analysis and discover rule sets of malicious behavior. The security logs mainly include web, intrusion detection system (IDS) equipment web attack, international Data Corporation (IDC), server, database, network management, DNS, and firewall logs. Rule correlation analysis, attack behavior discovery, scenario correlation analysis, and historical traceability methods are used to analyze web attacks, SQL (Structured Query Language) injection, sensitive information leak, data pack download transmission, cross-site scripting (XSS) attacks, and password-cracking attack applications. Batch analysis of network traffic data and logs using parallel MapReduce includes four parts: NetFlow collection, dependent graph construction, PageRank algorithm application, and cluster model (DBSCAN) establishment. The detailed analysis process is shown in Figure 19. The analysis can build analysis models and discover the malicious attack behaviors. Table 1 demonstrates part of the behavior features of web attacks. In our experiments, we employed actual operational data, including 720 million records, approximately 77 GB, and continuous 23 h. The experimental data generated a dependent graph including 16 billion nodes and 57 billion links.  
Network traffic real-time analysis: Deploying bypass network flow monitoring at the Internet exit and using Hadoop for storage and Storm for streaming analysis, the detecting model--trained in the batch analysis step--can detect the security threats and risks in the network flow deeply. The network traffic analysis mainly includes collecting NetFlow data, router configuration data, zombie Trojan-worm attacks, and malicious URL attack events; as well as employing dimensional analysis, behavior pattern analysis, fingerprint analysis, outlier detection, and protocol revert methods to mine the malicious behavior model  
Figure 19: Malicious attack behavior discovery.  
and rules. The model could carry out web vulnerability discovery and detection of command-and-control attacks, suspicious scanning, malicious bot behavior, advanced persistent threat (APT) attacks, and DDoS attacks on network traffic and intercept the suspected actions.