Comcast Technology Solutions Logo
Issue 2 | July, 2024

Video Harold

A Newsletter from CVP & AdFusion Engineering

Know your team

EMs decorative image

Hear It From Our Enggineering Managers: Mantra for Resiliency in Our Teams

Naveen (Giza): I believe the resilience of our software systems is in a way a by-product of the processes & culture that we build to ensure we have quality checks & sign-offs at every stage of software release cycle. While it is a common practise to review the code changes, I think it is equally important to brainstorm on the integration test scenarios & the test results, without any developer bias, to ensure we ask the necessary "whys" & "why nots" keeping the customer workflows.

Hannah (Alexandria): I believe in the principle of "Begin with the end in mind", which helps organize thoughts and focus on what is important. Implementing a code cut-off, where no PRs are merged during the last two days of the sprint. These days are dedicated to thorough testing and bug fixes. Deploying the stories developed in each sprint and promoting the thoroughly tested version to the preview environment, which will then be promoted to production

Sridevi (Halicarnassus & Zeus): It is important to monitor the stability of critical services on weekly basis. Capacity analysis for critical services based on past few weeks load Identify and log critical application metrics and ensure integration with slack channels for continuous monitoring. Custom Datadog dashboard and alerting metrics out of Splunk queries

Kailash (Rhodes): I am of the opinion that focusing on periodic capacity review of their services with support from OPS SRE. We also perform log replay testing to play out most of the customer scenarios in lower environment. Utilising an โ€˜Internal Deployment Checklist" to verify code promotion, type of tests performed & deployment readiness. It is also important to review PS scripts consuming our services prior to execution, wherever possible. Brainstorm with the team & incorporate valid suggestions.

Benito (Olympia): I feel it is essential to conduct load testing to understand the system’s breaking points and ensure it can handle peak loads. Design the system to scale horizontally to manage increased traffic and load. Implement regular data backups and ensure that recovery processes are tested and efficient. Develop and maintain a comprehensive disaster recovery plan.

Lakshmi (Artemis): I firmly believe in taking one step at a time. Despite our regular testing and review process, unless absolutely necessary, I prefer not to bundle multiple features into a single deployment. A significant strength of Artemis is our approach to player testing. The team has invested considerable effort in learning various packaging methods for different renditions and key types and has explored different players to effectively test multi-DRM support.

Raja (Babylon): I believe it is good to divide applications into smaller, independent services. This reduces the impact of a failure in one service on the entire system. Keep components loosely connected to decrease dependency and the danger of cascading failures. Use circuit breaker patterns to prevent a breakdown in one portion of the system from affecting other parts by temporarily suspending requests to the faulty component.

Art of unwinding: Offbeat methods used by our Managers to rejuvenate their Engineering Spirit

Lakshmi: From childhood, I've always cherished studying, working, and playing on the terrace ๐Ÿ˜Š. I even typed this message here, where my thoughts flow more freely ๐Ÿ˜‰. I feel rejuvenated by the fresh air, the open sky, my plants, and the birds visiting my small garden of just ten pots.

Raja: Meditation, mindfulness exercises, and yoga can all help you reduce stress and enhance your focus. Book genres such as science fiction, historical novels, and biographies can provide a mental respite and new insights.

Naveen: If I feel the need to recharge myself, I would like to travel to any destination which has some sand & water and I always believe there is no better place in the earth other than the place where the land & water meets!

Hannah: Good sleep, playing board games and UNO, family get togethers, reading self-help books, a time just to zone out ๐Ÿ˜Š (Me Time), gardening, writing, short vacations.

Benito: I indulge in a range of offbeat activities that bring a refreshing break from my professional routine. Cooking used to be one of my favorite pastimes, as it allowed me to relax and express some creativity in a unique way, still brings me joy, although I do it less frequently now. Lately, on weekends or during my free time, I enjoy watching movies and spending quality moments with my family, which helps clear my mind and restore balance.

Kailash: Playing Badminton & gardening helps relieve stress. For lazier days, movies & binge watching help unwind.

Previous Next