XLAs: Experience Level Agreements in IT sourcing

XLAs are a trend which is beginning to take hold. It is based on the recognition of traditional Service Level Agreements (SLAs) falling short because they do not cover customer experience - their focus is on technical and organisational metrics, such as server availability and service desk uptime. This allows customers to determine whether and to what extent their requirements are being met by the supplier. In addition, financial compensation can be linked to the agreed metrics. SLAs are primarily designed to meet the expectations of service or product managers.

Until now, little attention has been paid to users' perceptions - the "perceived quality" of IT services - and these have rarely been used as an evaluation criterion. After all, they have to work with the services. IT organisations traditionally help themselves with systematic customer satisfaction surveys, for example with the low-threshold Net Promoter Score (NPS) or the UMUX Usability Metric for User Experience (more on customer satisfaction and the listening strategy of IT organisations in our blog).

XLAs: between output and result

This is where XLAs come in. They are designed to help determine the perceived user experience and measure the vendor against it. The 'watermelon effect' is often used to justify this new perspective: On the outside, everything is green for the service manager, but on the inside, the traffic light is red for the user. Users have an individual perception of IT performance - what they perceive as inadequate may well be the agreed service level. It’s the difference between output (application availability = green) and outcome (slow application = red).

XLAs ask for service value

XLAs focus on customer experiences and are designed to ensure all interactions with a service are taken into account to measure it against an agreed service level. So rather than replacing SLAs, XLAs complement them: did the provided availability help the customer achieve his objectives? In any case, the holistic approach to evaluation means a service provider needs to understand the objectives of a customer persona or role, what they expect from the service and what criteria make them happy. However, this information cannot be picked up in passing; the feedback in turn requires new interactions with the customer.

Challenges in XLAs

As customer feedback is the most important metric in XLAs, it is important the information is collected as easily and smoothly as possible - no user is going to click "five stars" several times a day when asked about the perceived quality of their ERP software. In addition, subjective experiences can distort the picture in the long term - for example, if the service is not available in a critical situation. Experience shows users are more reluctant to praise than to criticise. The rating system of the XLAs must therefore be well thought out and designed, otherwise the provider will always be running against the wind.

New penalty grids

This also applies to penalties or service credits: IT organisations and vendors need to consider the performance (not met/exceeded) for both service and experience levels. This requires a special sense of proportion to balance the interests of both sides. SLA penalties encourage providers to optimise processing at the expense of experience. In addition, the customer and the provider need to define user's touch points with different services and find a solution for individual feedback and XLAs cannot be evaluated with an annual survey; the cycles are much shorter or linked to events.

Conclusion: XLAs are a paradigm shift

XLAs do not make the evaluation of IT services easier or more efficient. Moreover, they cannot be rolled out on demand with a template, and some preparatory work is required in any case. After all, this is not about metrics, but about good collaboration. However, when this approach is used, it changes not only the relationship with a provider, but also with your own IT users. The view from their perspective shows where IT can actually improve - in terms of experience and the achievement of business goals. In this respect, it is somewhat surprising that XLAs are only gradually becoming common.