The pandemic has been the time of record highs for video consumption. With many staying at home, movie theaters closed, and limited restaurants to dine in, people have been spending more of their lives online. But what has changed more over the last 5-10 years is the way people consume these videos now. Compared to earlier days, where people primarily consumed videos from their television set, today they consume it from varied forms of digital devices (e.g., smart TV, streaming box/stick, gaming consoles, DVR/STB, tablet, computer, mobile), supporting assorted flavors of operating systems, browsers, and network connectivity. Recent ComScore OTT (Over The Top) state report clearly shows the growing penetration of different digital devices among U.S households.
Such evolving landscape makes it difficult for Digital Service Providers (DSPs) to roll out video services to their customers. This requires humongous amount of testing to ensure compatibility of the video service with a broad range of device types, different flavors of operating systems, browsers, and network connectivity. But at the same time, the market demands these services just in time.
Delivering such seamless video services with high quality needs extensive testing, which is not possible to achieve with the traditional manual approach. This mandates DSPs to technically upgrade their way of working, the testing process, and existing release platforms. In this article, we explore four key enablers, which if successfully implemented can empower DSPs to rollout video streaming services with flawless quality and faster time to market.
1. Customer Critical Service Indexing (CCSI) orchestrator – Map video services with dependent features, back-end components, and corresponding test scenarios
The traditional approach considers testing video services as a black box without peering into its flow, internal structures, or workings. This leads to a lack of visibility into changed components and corresponding test cases. As there is no visibility into which individual component is failing, it is difficult to trace the root cause of failure. Also, in this scenario, even for a small component change, the tester needs to execute the entire regression suite, that delays the time-to-market of new features.
Building Customer Critical Service Indexing (CCSI) enables DSPs to map critical services with the back-end components and test scenarios to cover E2E system integration flow.
CCSI provides clear visibility into changed components, which enables faster test execution and Root Cause Analysis (RCA) of any failure. This can help DSPs to achieve 3X acceleration in time to roll out video services.
2. CI/CD framework & release cadence – Integrate with CCSI orchestrator for faster automated testing and rollout of video services
Once CCSI is integrated with CI/CD framework, it helps to orchestrate the entire test pipeline execution based on the user story and changed components.
Key Recommendations
- Use CPE (customer premise equipment) firmware management tool to remotely configure test environment setup – With this tool, once the release is available, engineers can remotely upgrade/downgrade devices, thereby ensuring zero-latency and zero-dependency. This can be made available with simple UI (User Interface) (easy drop-down selection capturing details like ID, owner, location, user, environment, and product type).
- Automate test suite execution using a keyword-driven approach – Execute test cases using Robot framework where keywords can be quickly edited and further configured. This helps to write the test cases in a user-friendly approach. Also, parallel executor can be used here to speed up test execution.
CCSI integrated with CI/CD framework can reduce testing efforts by 80%.
3. Setup continuous monitoring platform for proactive management of KPIs driving customer experience
Video experience testing is a continuous process, which cannot be claimed to be perfect by testing one feature in one attempt. Whereas it requires DSPs to run test cases in a continuous loop to analyze the stability of the current release.
Key Recommendations
- Integrate CCSI with a monitoring platform to fast-track RCA of any failure– With this, the triage dashboard not only shows which test case has failed but also why it has failed. This is feasible with the clear mapping of components in CCSI and automating the following steps:
- Traceback the request through the components involved and gather the status
- Track down the component, where the flow breaks causing the feature to stop working
- Create alert mechanism – Provide a mechanism to configure critical cases. Such test cases, if failed should trigger alarms to the respective team.
Embracing these steps can reduce the Turn Around Time (TAT) to fix critical issues by 40%. These recommendations also drastically reduce the overall backlog of failed test cases with faster resolution.
4. Use ML (Machine Learning) models to predict release stability and behavior of services when rolled out
Having insights-rich information on predicted behavior of services to be rolled out, enables the execution of timely decisions.ML-based release stability prediction empowers the release managers to analyze the behavior of the upcoming release, compare it with the release in production and make judicious go/no-go decisions. A ML based continuous monitoring and prediction platform can be setup to understand the stability score of the upcoming release and forecast the stability score for individual features.
Results achieved by a leading DSP (Digital Service Provider) in Europe by leveraging key enablers mentioned in this article
The DSP sought to build a unified video service delivery platform to support its next-generation media services delivery catering to multiple affiliates across the pan-European region. For this, it required a robust framework to automate E2E compatibility testing to deliver a consistent user experience across multiple platforms and form factors.
Benefits Achieved
- 3X acceleration in time to roll out video services with the help of the CCSI orchestrator. Parallel execution achieved with multiple devices & software/firmware version
- Reduced the Turn Around Time (TAT) to fix critical issues by 40%
- Regression and sanity automation using CCSI reduced testing effort by 82%
- Predicting release stability using ML models enabled the DSP to take corrective action ahead of rollout, thereby avoiding any impact on customer experience
I am grateful to my colleagues Sekar K, Senior Project Manager, Kaarthick R – Senior Technical Lead and Sumit Thakur – Senior Manager, Strategic Insights, for their contribution in research and help in developing the insights presented in this article.
About Author:Eswara Moorthy Logaraj Senior Director - Delivery, Prodapt Solutions Eswara Moorthy Logaraj has 21+ years of IT (Info Tech) Service Delivery Experience focused on leading the global, diverse delivery teams and programs for multiple Telco / Cable Service Providers across Wholesale, Enterprise, Mobile, Cable, and Entertainment LOBs with the focus on Customer Satisfaction and Capex / OpEx (operating expenses) optimization achieved through Agile, DevOps Practices, and Automation. Eswara Moorthy Logaraj is Senior Director - Delivery at Prodapt, a two-decade-old consulting & managed services provider, singularly focused on Telecom, Entertainment and Technology companies that helps clients transform their IT, products, operations, and networks to meet their strategic objectives. Prodapt's business consultants enable DSPs on their transformation journey at several layers, including cloud, customer experience, business outcome-focused initiatives, Capex, and OpEx optimization programs.
