By Ross Cassan - Director of Product Marketing - Mobility Infrastructure at Spirent Communications.
Welcome to the third installment of our 5G blog series: The 5G Transformation Needs Automated Assurance Now. In our first blog in the series, we described the challenges 5G brings, the need for automated networks and assurance and some key automation use cases to get started with: service activation and turn-up, service quality monitoring, change management and fault isolation. In the second blog, we drilled into the service activation and quality monitoring use cases with a focus on 5G network slicing. In this blog, we’ll examine the critical role change management plays in dynamic 5G networks.
Manual change management can’t keep up with 5G
What happens when the network changes faster than the changes can be validated? Problems ranging from isolated customer issues to national outages will occur – it’s just a matter of time. That’s not an educated guess about the future, it’s a story we’ve heard from multiple operations executives talking about the networks they’re running today. One operations executive for a tier 1 provider recently explained to us that her inability to comprehensively validate core network changes was the biggest driver of Enterprise customer issues and inefficiency for her team.
In her case, changes to the core network were made during a tight maintenance window. But due to a highly manual validation process, only a small subset of her total Enterprise mobility VPN customer base could be checked during the maintenance window. The team approached the day after every upgrade with trepidation – they knew that eventually an undetected issue would trigger a cascade of customer complaints. These complaints would lead to an all-hands-on-deck emergency fallback to a previous release, resulting in network downtime, wasted effort and a flood of angry calls.
Providers roll the dice when changes are only validated for a portion of the network.
Another operations executive we spoke with described the flip side of this challenge. In his case, upgrades of certain 4G evolved packet core functions needed to be exhaustively tested before going live. His team couldn’t validate the entire national network within a reasonable time period, so they would upgrade changes in regional sub-networks one at a time. As a result, any change to the entire national network would take weeks to roll out and only two major releases could be completed each year. The provider’s ability to innovate was effectively held hostage by the slow pace of network upgrades.
Rollout of critical EPC upgrades may take months to complete due to slow, manual validation processes.
Change management is already a huge challenge in today’s 4G networks and now, with 5G, we’re introducing a new technology designed to be ultra-configurable to support smart factories, augmented reality, autonomous vehicles and more. 5G brings a rapid-fire stream of changes to network slices, virtualized network functions, multi-access edge compute nodes and more. For change management, 5G isn’t going to be the straw that breaks the camel’s back – it’s more like a massive hay bale!
Automated Change Management for 5G
We know 5G will bring more devices, more cell sites and more configurability. At the same time, providers are shifting to agile network update schedules that occur every six weeks instead of every six months. Change management must be radically accelerated to keep up with this pace and that can only be achieved with automation.
To automatically validate changes for 5G networks, we first need the ability to deploy virtualized Test Agents throughout the end-to-end 5G network. These Test Agents are instantiated as needed by orchestration functions and then destroyed when no longer needed. A Controller configures and manages the Test Agents to perform tests and deliver test results to Analytics functions (now being standardized as the 3GPP NWDAF node). This automated assurance framework, consisting of Test Agents, Controller and Analytics components, provides the core foundation for automation of Change Management (see the figure below).
Automated assurance platform for change management of 5G networks: Analytics, Controller and Test Agents.
Example: Validating a UPF Change
Let’s examine automation of 5G change management in more detail by looking at a hypothetical change to a 5G User Plane Function (UPF). The UPF handles data plane connectivity between the gNB (i.e. 5G base station) and the DN (Data Network) including packet routing and forwarding, data session mobility functions, application detection and per-flow QoS handling. As such the UPF is a critical element for implementing 5G network slicing and has a direct impact on the performance of services. For our hypothetical change, let’s assume a performance improvement has been made to the UPF and it’s now ready for operational deployment.
Once the updated UPF becomes available, Test Agents are instantiated and the Controller is triggered to begin validation tests. The Controller directs the Test Agents to perform two types of validation tests: end-to-end performance tests that evaluate the updated UPF as part of the broader network and wrap-around tests that evaluate the UPF function in isolation. In both cases the Test Agents emulate other parts of the 5G network as needed including the AMF, SMF, gNB and DN. In addition, 5G devices running real-world applications are emulated to enable validation of the UPF as if it were deployed in an actual operational network. If the UPF passes the validation tests, the orchestration layer is informed that the new UPF is ready to begin carrying traffic in the operational network. If the validation fails, tests results can be passed to the Analytics layer to determine root causes and enable rapid fixes.
Test Agents emulate the 5G network and devices to validate the updated UPF before it goes live.
Change Management at the Speed of 5G
The change management approach outlined above takes only minutes to perform – not weeks or months. That’s fast enough to keep up with the rapid pace of change we will see in 5G networks. Spirent has already implemented a similar approach for change management of LTE evolved packet core networks which has successfully been deployed by multiple tier-1 providers. If you’re interested to learn how Spirent is helping service providers adopt automation to deliver on their 5G promises, please contact us to set up a meeting with one of our automated assurance experts.