Comcast Technology Solutions Logo
Issue 2 | July, 2024

Video Harold

A Newsletter from CVP & AdFusion Engineering

LRM EKS MIGRATION CASE STUDY

DIVING DEEP, RISING HIGH - CONQUERING EKS MIGRATION WITHOUT BOUNDARIES

AWS EKS migration

The LRM Engineering team is a quintessential "two-pizza" team comprised of 7 engineers. They are responsible for all Golang & Java development, testing and deployments for the LRM platform. The diverse technology stack of LRM calls for bold engineers who are ready to venture into new areas and collaborate closely with their peers to support the platform.

The micro-service based LRM (Linear Rights Metadata Manager) platform has two customer facing environments: Production and Preview. Preview (a.k.a. Lab) enables quick feedback on new features and customer acceptance testing prior to Production deployments.

THE CHALLENGE:

Following catastrophic loss of the Kubernetes control plane for the LRM Preview environment, migration to EKS became a top priority since one errant scaling increase could topple Production or Preview Bravo clusters. Recovery of the Preview environment proved essential team preparation for migration to EKS.

The LRM team seized the opportunity and transitioned swiftly through planning to independent action since the SRE team was fully engaged in supporting CVP migrations to EKS.

The migration workload was daunting:
  • 104 K8S pods across multiple AWS accounts and two regions
  • Integrations with 98 Lambda functions and 14 AWS services

Despite no prior experience, the LRM team independently managed successful EKS migrations through just-in-time learning, strategic experimentation, and skillful execution.

Highlights of LRM Team Accomplishments:
  • EKS familiarization through experimentation in Dev and Preview/Lab environments.
  • Designed, developed, tested and performed multi-phase migration plans.
  • Improved deployment and test automation and expanded system monitors.
Outcome:
  • LRM Preview environment migrated to EKS (cts-lab) - January 2024
  • LRM Production environment migrated to EKS in 4 stages - April 2024
  • No service interruptions

-Jonathan Eid, Director, Software Engineering

Previous Next