Position: Critical Incident Response (CIR) Manager
Company: ZONE3000
Category: Lead
Experience required: 5y
English level: upper
Published: 2020-09-01T00:00:00+03:00

Critical Incidents (CI) are any type of unscheduled events that have potential to cause significant harm to customers, organization and employees (financial or otherwise). Examples of CIs are - downtime and outages of systems, DDOS attacks, 3rd party caused outages, cybersecurity attacks, unplanned system updates due to 0-day vulnerabilities disclosures,  etc. 

In this role you will be working with all of technology (Engineering, Infrastructure (TechOps, NetOps, DCOps, SRE & Devops, IT, Architecture etc.) and other business units (Product, Customer Support, BI etc.)  to drive communication and coordination efforts to react to and resolve critical incidents and to learn from and improve both processes and technical platforms, so we can react to further incidents in a more effective way. 

This role requires the ability to be available and react to a CIs on a 24x7 availability basis (by direct reaction or being brought online via a paging system by a 24x7 on duty SRE team). You will be able to schedule your working hours and working locations freely in order to meet this requirement, as well as build a CIR team to meet the operational needs. As a leader, you will be a single point of contact for coordination efforts in case of a CIR event.

You will not only react to ongoing incidents but work with teams/organizations to preventively improve monitoring and detection of critical incidents to decrease the number of incidents and financial losses. 

Requirements 
— Experience and skill in risk evaluation / calculation
— Experience with & strong ability in communication (verbal, written, synchronous and asynchronous) with a broad range of communication partners: Customers, Stakeholders, Team members, Senior Leaders etc.
— Experience working effectively both on your own and as a part of a team
— Experience in adapting to changing environment
— Experience making both fast and high impact decisions
— Experience of working effectively under pressure
— Strong Logical Thinking. Experience in logical analysis and persuasion via logical argumentation. Knowledge and experience with Thinking Processes of Theory of Constraints is a big plus.
— Experience responding to Critical Incidents / Security events, including front-line analysis and escalation
— Experience prioritizing multiple high importance streams in a fast pacing situation
— Experience delivering your work in a customer oriented way.
— Experience with self motivation, flexibility
— Experience in result oriented delivery
— Experience in root cause analysis techniques
— Strong organizational, coordination and mediation skills to drive decision making
— Upper-Intermediate written and verbal English at the least, advanced / fluent is ideal
— Experience with system/application log analysis, alerting and data flow, and SIEM-based workflows
— Understanding and practical experience with at least 2 (or more) of the following systems / methods:
• Grafana logging or similar
• Kibana logging or similar
• VLAN Segmentation and NAT
• VMware (or other virtualization platforms)
• Basics of AWS or OpenShift
• NGINX filtration
• F5 filtration
• Cloudflare filtration
• Communication Networks / HTTP Protocol (understanding of networking and communication patterns)
• Knowledge of top software security vulnerabilities and understanding patterns of protection against them
— Extensive theoretical and practical knowledge with networking and application layers

As a plus
— Scrum master abilities / understanding
— Experience in process design and implementation
• Knowledge of BPMN or other chart methods
— Experience in Project Management body of knowledge and tools
— Smartsheet, Google docs
• PMBOK or similar
• JIRA / Confluence or similar tracking systems
— Experience in applying Theory of Constraints
— Experience in organizational design
— Experience with agile methodologies SAFe / Scrum or others

Education as a plus:

— Undergraduate degree or direct experience in information security, information systems, computer science or other fields of engineering
— Other technical security certifications or academic background is a plus

We offer
— High & competitive salary
— Challenging work in an international professional environment
— Opportunity to influence software development process, to be the owner of the product in your field of expertise
— Opportunity to apply SAFe methodology
— Flexible management
— Flexible / Casual Leave
— Relocation Bonus when moving from a different city / country
— Full benefits package: paid vacation and sick leave
— Continuous professional development (free internal and external professional trainings)
— Free English classes in the company office
— Free use of the services provided by Namecheap (for non-commercial purposes)
— Quarterly teambuilding activities and company corporate events
— RDX gym membership
— Coffee, tea, fruits

Responsibilities
— Lead & coordinate team of teams to achive incident resolution during a crisis in a timely and effective manner 24×7
• Including after-hours support on an on-demand basis
— Act as an independent escalation and coordination point for critical incidents. I.e. can deal with a CIR without involving CTO / CTO Group in most cases.
— Create and enforce positive feedback loop targeted at improving organization’s ability to deal with CIRs via live and simulated experience.
— Drive and establish clear and concise communication with a variety of stakeholders, providing transparent updates to internal customers during critical incident
— Drive and coordinate retrospective events (retros) on past incidents to come up with process and technical improvements
— Drive implementation of lessons learn and changes needed based on previous CIRs.
— Drive estimation and accounting of CIR triggered financial and other losses. Improve this metric over time.
— Analyze past incidents and improve technology processes and flows to improve ability to react to CIR
— Create FAQs, Cheat Sheets and other forms of documentation to help preserve and reuse knowledge about how to best act within a certain kind of CIR
— Ensure adherence to quality/security standards
— Work with teams/individuals to minimize the incidents.
— Organize education for Technology teams of processes and reaction to critical incidents. Hold drills to improve readiness to incidents.
— Weekly and monthly status reports to senior leadership (Heads, Directors, CTO & CTOGroup etc.)
— Work with technology branches (Engineering, Infrastructure (TechOps, NetOps, DCOps, SRE & Devops, IT, Architecture etc.) to develop detection patterns across a broad range of technologies and log sources