SRE: Building a Culture of Reliability, Resiliency and Risk Management

By: Biswajit Mohapatra

Site Reliability Engineering (SRE) propagates a culture of building and operating reliable, resilient, risk managed software systems. SRE looks at operations through the lens of software engineering practices.

Traditional software development models typically address reliability at the beginning of the design phase. As a result, changes to the functionality in later stage impacts previously considered reliability requirements. Non-functional requirements are not reviewed as often. Quality of Service (QoS) parameters are most widely overlooked during the software development life cycle leading into operational issues subsequently.

The goal is surely not here to push the software to the production but to run and manage efficiently and effectively once it is live.  SRE bridges this gap leveraging a well-defined set of practices, principles and culture built on DevOps foundations with strong emphasis on engineering capabilities.

SRE sets measurable engineering objectives mapping to Service Level Objective (SLO) and enables monitoring and tracking of QoS parameters such as:

  1. Reliability – Ability of the system to function correctly, failure free software operation
  2. Availability – System response to disruption and fault tolerance, avoid down-time, Stateless application design, fail forward database design
  3. Recoverability – System ability to recover from incidences through actionable alerts and next-gen automation
  4. Serviceability – Speed with which system can be repaired, System health assessment, monitoring and logging mechanism, end user experience
  5. Elasticity – System scalability and performance with reference to data, traffic, peak load and response time
  6. Resiliency – System ability to withstand potential failure, focus on Mean Time to Repair (MTTR) over Mean Time Between Failures (MTBF)
  7. Risk Budgeting – Ongoing process of risk measurement, attribution and allocation. Optimal risk allocation to maximize expected return

It’s of paramount importance to standardize SLO, identify KPIs, create balanced scorecards and continuously drive measurement, monitoring and tracking. Measurable Service Level Indicators (SLI) will determine success or failure of a change in production. Error Budget will act as an explicit quantitative measurement parameter in your Service Level Agreement (SLA) that can connect feature planning to service reliability. 

How you balance change velocity vs. availability, reliability, security and other operational attributes is the key question to be answered. Implementation of continuous delivery, continuous integration, continuous testing, continuous release and deployment coupled with collaboration will drive the required cultural change. The system must recover from failure by automation.

Your SRE team needs to be responsible for the system design and development, release management, capacity management, change management, incidence management, automation, availability, latency, performance, security and monitoring of their services.

SRE will deliver differentiated value  proposition towards your digital reinvention journey by providing fast and uninterrupted services through resilient systems, drive operational excellence and cost optimization by adopting automation and best practices, adopt risk management frameworks to address risk tolerance of services and bridge the relationship gap between development and operations teams and enable them to communicate with cost of reliability.

Leveraging SRE to design, build, operate and enhance software systems is critical for the future of business. Every CIO is today looking at SRE to strengthen their digital business foundation. It’s time now to build the culture of risk managed, reliable and resilient digital footprint and SRE is at the heart of all these happenings.

LINK TO ORIGINAL ARTICLE

Become Free Member