The Power Of A Strategic Pivot
Behind every successful innovation, there are multiple tests, failures, and learnings. Today, one
would not think twice about the capability of an aircraft while traveling by air. There were
multiple versions of flying machines that were tried and tested until a successful working model was
achieved. Attempts to get these machines off the ground often ended with the pilot getting a few
feet into the air and then crashing gracefully. These repeated smash-ups by aviation pioneers were
the only way to learn what worked and what didn’t in the air. We must owe every smooth flight today
to those successful experiments and their negative results.
"Strength does not come from never failing, but from rising every time we fall"
This story highlights the complexities of engineering and its relevance to daily software
development. Despite product and engineering discussions, rigorous testing, verifications, and
demos, crashes and outages can still occur when code is deployed to production. This leads to
penalties for failing to meet agreed-upon standards with customers.
Impact of Outage for Customer:
- Financial ramifications of customer contracts
- Customers lose money as users churn due to poor reliability
Impacts of Outage for SB&A/Comcast:
- Tarnish reputation and brand of Comcast
- Challenge in acquiring new customers
- Penalties paid to customers
- The toll on teammates
The time, money, energy spent is not anything less than 14+ days resolving customer
concerns:
- Engineering - Root cause write ups
- Product - Reprioritize, reshuffle work, disruption
- OPS - Customer communications and meetings with each customer impacted
- Sales - Constant reminders of how good the service is, fighting negative
impressions
- SLT - Customer meetings, explanations to Ken and Charlie, brand repair
How we can try and prevent Outages ?
- Take ownership, responsibility and foster accountability to break the cycle of stress.
- Teams must ask questions to reveal unknown or overlooked issues to avoid unforeseen risks.
- Register risks and address the highest-priority ones to eliminate or reduce failures.
- Ensure confidence in a successful deployment by using a deployment readiness checklist
"Remember, every small effort contributes to the bigger picture. Just as every drop
makes an
ocean, Together, we can achieve Greatness, one step at a time"
Resiliency
Navigating Uncertainty: Building Reliable & Resilient Systems 3 mins
read
In recent months, CVP faced turbulent waters, encountering infrastructure outages and
stability concerns across the services. From Seattle data center power disruption to..
Read More
ReSILENCY- DT Case Study
The Euro Cup 2024, a prestigious football tournament’s broadcasting infrastructure provided by
"SB&A" 3 mins read
Imagine being engrossed in an intense IPL final of CSK vs Gujarat
Titans, every moment charged with anticipation, when suddenly, the
stream stutters and..
Read More
LRm EKS Migration Case Study
Diving Deep, Rising High - Conquering EKS Migration Without Boundaries 2
mins read
Resiliency is choosing to rise above adversity, standing tall in challenges, and
demonstrating innovation even amidst system pressure.
Read More
ENGINEERING PREMIER LEAGUE
Success Story on Performance improvement ft. ALEXANDRiA Team! 3 mins
read
Optimizing database timeouts is key to ensuring seamless User experiences and
efficient system performance.
Read More
Teams Achievement
Optimizing PR Aesthetics: The Power of Decoration 3 mins read
As part of our continuous effort to enhance code quality and streamlining the development
process, several teams within the CVP have begun adopting SonarQube as a key component of
their
validation workflow...
Read More
Teams Achievement
Innovating with Copilot: Team Hali’s Experience 3 mins read
In the ever-evolving landscape of software development, finding tools that enhance
productivity
and efficiency is crucial...
Read More
RISK REGISTER ANALYSIS
Identifying risks is a way of uncovering opportunities 3 mins
read
We welcome E1, E2, and E3 to participate in an awareness questionnaire!
Read More
KNOW YOUR TEAM
Hear It From Our Engineering Managers 3 mins read
Art of unwinding: Offbeat methods used by our Managers to rejuvenate their Engineering
Spirit.
Read More
Rethinking Work
The 70-Hours A Week Myth 3 mins read
Everyone must have heard Narayana Murthy’s statement about working 70 hours a week, making
one
wonder, "How is this possible?" It seems excessive, but understanding his
statement requires a
deeper look...
Read More