mytf1

Three years ago, mytf1, the leading private TV channel in France, embarked on a project to modernize its broadcasting platform, recognizing the shifting landscape towards digital platforms like smart TVs and mobile devices. This shift led to a significant portion of TF1’s audience consuming content online, prompting the need for a robust and scalable digital architecture.

Thierry Bonhomme, CTO of mytf1, revealed in a recent interview that digital broadcasting now comprises 20–25 percent of TF1’s total audience. This digital shift brought forth unique technical and economic challenges, including managing peak traffic during live events and delivering targeted advertisements to diverse audiences across various devices and network conditions.

To address these challenges, mytf1 prioritized scalability and performance in its new technical architecture. Scalability was crucial for handling peaks in concurrent viewership, while performance was essential for ensuring quick video startup times and uninterrupted playback. To achieve these objectives, TF1 partnered with AWS to enhance its system’s scalability, performance, and security.

The redesigned architecture included implementing robust metrics to monitor system performance and establishing clear processes and roles within the team. Additionally, TF1 focused on optimizing its system to deliver seamless viewing experiences, with video streams starting within three seconds and minimal re-buffering.

By prioritizing scalability and performance in its digital architecture, TF1 positioned itself to meet the evolving demands of its audience and deliver high-quality content across a diverse range of devices and network conditions. This proactive approach ensures that viewers can enjoy a seamless viewing experience, whether they’re tuning in from the comfort of their home or on the go.

The Technology

Starting in 2019, mytf1 embarked on a comprehensive redesign of its digital broadcasting architecture, rewriting key components like the back-end API and front-end applications. They adopted a microservices architecture deployed on Amazon Elastic Kubernetes Service (Amazon EKS) and leveraged the Go programming language for optimal performance. TF1 also implemented a set of REST and GraphQL APIs to define communication between front and back-end applications and employed an event-driven architecture with Apache Kafka for scalability. Multiple content delivery networks, including Amazon CloudFront, were utilized to ensure reliable distribution of video streams to client devices.

a major sporting event in August 2020, TF1 had the opportunity to test their new platform at scale. The process begins when high-quality video streams are captured at the mytf1 tower in Paris and encoded using AWS Elemental Live hardware encoders. These encoders generate various encoding formats tailored to different devices and network conditions. The encoded videos are then sent to AWS Elemental Media Package for packaging before being distributed to CDNs, where player applications fetch the video segments. Player applications dynamically select the best video encoding based on device size and available network bandwidth.

In preparation for the 2022 World Cup final, TF1 identified risks and areas for improvement based on past experiences. They anticipated significant traffic spikes, particularly if the French national team progressed through the tournament. TF1 aimed to accommodate up to 4.5 million concurrent viewers accessing the platform 15 minutes before a game, necessitating readiness not only from mytf1 team but also from external systems like AWS cloud services and CDNs.

Viewer arrivals trigger authentication processes, API calls to fetch content, and connections to CDNs to download video segments. Smooth delivery of video streams without interruptions or re-buffering is crucial for ensuring an optimal viewer experience. TF1’s meticulous preparation and robust technical architecture are essential for delivering high-quality content to millions of viewers during major events like the World Cup final.

The Preparation

Six months before France’s historic showdown against Argentina in the World Cup final, TF1 began collaborating closely with their vendors, including AWS, to establish requirements, reserve capacity, and develop test and execution plans. mytf1 engaged with AWS Infrastructure Event Management, a program within the AWS enterprise support plan, to receive architecture guidance and operational support during the preparation and execution of major events like the World Cup.

A key focus during this preparation phase was testing the scalability of TF1’s API. The TF1 team developed a load-testing engine to simulate user interactions with the platform, including authentication and video streaming. To simulate real traffic, TF1 utilized another hyperscale cloud provider to send requests to their AWS infrastructure. However, the initial load testing attempt resulted in the crash of the entire backend system, highlighting the need for optimization.

TF1 used this setback as an opportunity to fine-tune their system over the following 2–3 weeks. They optimized client applications to reduce redundant API calls and implemented aggressive caching strategies. Through iterative testing and deployment, TF1 learned how to effectively scale their backend platform to handle high traffic loads and identify key performance metrics.

After several backend deployments and updates to their mobile apps, TF1 successfully passed the load test one month before the start of the World Cup. At this point, TF1 decided to freeze all new developments or deployments until the start of the event, unless critical bugs were discovered. This meticulous preparation and optimization ensured that TF1’s platform was ready to deliver a seamless viewing experience to millions of viewers during the World Cup final.

Monitoring and Planning

In addition to building a robust technological platform, TF1 implemented comprehensive monitoring and preparedness measures to ensure a smooth viewing experience during the World Cup event. They designed metric dashboards using Datadog and Grafana to monitor key performance indicators and detect anomalies in real time. Thierry emphasized the importance of observing percentile values, such as P95, to understand the experience for a significant portion of users, especially in large-scale events with millions of viewers.

TF1 also prepared contingency plans for worst-case scenarios, such as widespread service outages. They involved community managers and social media owners early on to prepare press releases and social media messages for various scenarios. Additionally, TF1 established a “war room” during each game, bringing together key team members, including AWS technical account managers and CDN vendors, to facilitate rapid communication and response to any issues that arose.

AWS contributed to TF1’s preparedness efforts by providing on-call engineers from service teams and premium support teams to monitor the health of AWS services and respond promptly to any incidents. This collaborative approach ensured that TF1 was well-equipped to address any challenges and deliver a seamless viewing experience to millions of viewers during the World Cup event.

The Attacks Weren’t Just on the Field

During the World Cup tournament, TF1 faced several challenges and incidents that tested the resilience of their digital broadcasting platform. Three key moments at the start of the tournament, including the opening ceremony and the first games, provided opportunities to test the platform in real-world scenarios. As the tournament progressed, TF1 continuously reevaluated its traffic estimates and conducted debriefs after each game to optimize performance.

However, the team encountered a significant challenge during the quarter-finals when they detected unusual activity from a wide range of distributed IP addresses, indicating a large distributed denial of service (DDOS) attack. Despite the attack’s intensity, TF1’s dashboard helped identify traffic patterns in real-time, allowing them to mitigate the incident swiftly. Services such as AWS Shield and AWS Web Application Firewall played a crucial role in neutralizing the attack without impacting the viewer experience. The TF1 security team, along with AWS experts, conducted further analysis to proactively block malicious traffic for subsequent games.

Although the intensity of the attacks escalated during the semi-finals and final game, peaking at 40 million requests within a ten-minute period, TF1’s preparedness and collaboration with AWS proved effective. The team in the war room adeptly detected and countered new attack strategies, dynamically updating filtering rules to safeguard the platform. Thanks to their thorough preparation and swift response, TF1 successfully mitigated the attacks without any adverse consequences, ensuring uninterrupted service for millions of viewers.