{"id":5764,"date":"2024-04-13T13:55:11","date_gmt":"2024-04-13T13:55:11","guid":{"rendered":"https:\/\/www.horsesforsources.com\/?p=5764"},"modified":"2024-09-11T11:48:33","modified_gmt":"2024-09-11T11:48:33","slug":"assuring_generative_enterprise_041324","status":"publish","type":"post","link":"https:\/\/www.horsesforsources.com\/assuring_generative_enterprise_041324\/","title":{"rendered":"To assure the journey toward the Generative Enterprise\u2122, organizations must show humility for the unknown"},"content":{"rendered":"
Operations leaders face unprecedented challenges. They have to manage the new complexity of becoming cloud native and anticipate the implications of GenAI. If that\u2019s not enough, they also have to find answers to the Digital Dichotomy, balancing the macroeconomic \u201cslowdown\u201d with the \u201cbig hurry\u201d to innovate to keep up with innovation pacesetters. Yet, it is not a question of doing one or the other\u2014they must address all those challenges simultaneously.<\/p>\n
Against this background, quality assurance (QA), or simply \u201ctesting,\u201d as it was called in the old days, can no longer be a reactive afterthought coupled with an unwillingness to invest in quality. Instead, it must become an integral part of the software development lifecycle (SDLC) and take on a much more holistic responsibility for assuring transformational outcomes.<\/p>\n
This is the context for HFS\u2019s seminal study on assuring the Generative Enterprise\u2122 (HFS Horizons: Assuring the Generative Enterprise\u2122, 2024<\/em><\/a>). We sought to understand better where organizations are with their QA efforts and how they are trying to ensure transformational outcomes. We also explored how they assure change agents such as automation, AI, and blockchain. Finally, we looked at the adoption of GenAI through the lens of QA. In the following, we share the insights gleaned from our study.<\/p>\n Finding your bearings on all things QA is difficult because there is a massive gap between the aspirations (and lip service) for QA and QA functions\u2019 maturity and enterprises\u2019 willingness to invest in them. While the mature end of the market is pivoting toward quality engineering (QE) with a focus on achieving continuous testing, data-driven decision-making, and cross-functional collaboration, two-thirds of the market is stuck at a lower maturity level, often still working with a Waterfall methodology, as Exhibit 1 depicts. Simply put, most organizations are on the left-hand side of this infographic.<\/p>\n Yet, we might finally see organizations embracing the transformation of their quality assurance functions. According to our study\u2019s reference clients, 95% described the primary value delivered by their service provider today as the ability to drive functional optimizations with selective quality assurance capabilities. Simply put, they expect services on the left-hand side of Exhibit 1. However, in two years, most of these organizations expect a transformation of their quality assurance functions, intending to drive experience-led outcomes and stakeholder experiences while creating new sources of value through ecosystem synergy.<\/p>\n Discussing these issues with QA leaders, technology partners, and service providers could crystalize more nuanced market dynamics. Most organizations have embraced \u201cshift left\u201d principles emphasizing the early and proactive involvement of quality assurance activities in software development. We are seeing mature QA functions augment this with \u201cshift right\u201d principles advocating quality activities also at the later stages of the SDLC.<\/p>\n Regarding organizations\u2019 QA priorities, four clusters jump out for us: First, carving out a budget for QE architecture modernization and transformation. This goes back to the discussion of the Digital Dichotomy and the need to self-fund innovation. Second, solving the conflict of production quality versus speed to market. Not least, with innovation cycles dramatically compressed with the ascent to GenAI, this is a hard nut to crack. Third, overcoming a new complexity to provide integrated assurance for apps, infrastructure, and platforms or progressing toward the OneOffice\u2122, as HFS would put it. And fourth, driving change management to foster transformation. Yet, large chunks of the QA community are stuck in a tools and technology mindset.<\/p>\n Let\u2019s zoom in on the topic that comes up in every discussion we have, regardless of the context. While many providers hype use cases around domain knowledge and code creation, grown-up discussions about how to assure GenAI are sparse. As such, Cognizant\u2019s Artificial Intelligent Lifecycle Assurance (AILA) and Infosys\u2019 AI Assurance Platform are more exceptions than rules. This provides indicators for the enterprise adoption of GenAI. We are still at the very beginning of enterprise adoption. The core value proposition of GenAI use cases in QA is having a higher accuracy with less data. Thus, providers can drive new levels of automation to generate test scenarios and test cases.<\/p>\n While we had many discussions on the infusion of QA with GenAI, deliberations on governance on GenAI are still nascent. Where we had honest discussions, providers reflected that large language models (LLMs) are largely unfamiliar entities. One executive framed this aptly: In the context of GenAI, we must show humility to the unknown. Thus, the predominant way to engage is through experimentation. However, there is no beating around the bush that we were a tad underwhelmed by the discussions of assuring GenAI outcomes; outside of the service providers, we had some stimulating conversations. For instance, MunichRe provides insurance for AI<\/a>, while German start-up QuantiPi<\/a> blends quality assurance with governance for GenAI.<\/p>\n Beyond the broader pivot to QE, what have we learned about the evolution of quality requirements? Like cloud-native operations are shifting toward persona-based solutions, QE is shifting toward persona-based testing, requiring capabilities to support better specific stakeholders such as product teams, site reliability engineering (SRE), DevOps engineers, and beyond. But to be clear, this is the North Star, and only a few organizations have it on their roadmap. Looking at these requirements in the context of GenAI, the key is blending prompt engineering with specific persona scenarios. Using prompts to generate automation scripts such as Selenium can significantly enhance the scale of automation. Lastly, using the input from one prompt to generate another prompt can leverage GenAI to industrialize offerings.<\/p>\n Last but by no means least, congratulations to the Horizon 3 market leaders. These leaders\u2019 shared characteristics include blending a compelling vision of transformational QA with nuanced approaches to assure change agents such as GenAI. The wheat separates from the chaff when providers ensure transformation outcomes are enabled rather than depicting functional testing and an overreliance on tools and technology. The leaders are pushing the envelope on transformation with new themes such as cross-functional testing, zero-touch testing, black-box testing, and beyond. Exhibit 2 outlines the detailed rankings of our research.<\/p>\n <\/a><\/span><\/p>\n Accenture and Wipro stand out, clearly outlining the evolution to QE. They shift the focus of their narratives by depicting transformational journeys and outcomes rather than getting stuck in tools and technology. Infosys supports clients\u2019 pivots to product-centric delivery and intelligent ecosystems, while TCS infuses emerging technologies into QE and embeds them at the core of transformation. Capgemini surprised us with a nuanced and thoughtful narrative on adopting GenAI. Cognizant was demonstrating test automation chops and strongly emphasized customer experience assurance. Perhaps surprisingly to some, Persistent is going deep on cloud-native transformation by investing ahead of the market in digital engineering and GenAI capabilities.<\/p>\n The innovation delivered by the QA community continues to be stupendous. Yet, the community does a modest job articulating the goals and outcomes those change agents achieve. Without snapping out of this tool- and solution-centric view of QA, it is difficult to articulate a more value-driven approach, where QA executives get the limelight they crave to discuss and decide the significant sourcing issues. Stepping up to QE transformation and putting QE at the heart of transformation and software development is an opportunity for the community to emancipate itself from being boxed in a technology and tools mindset. Discussions with the market leaders in Horizon 3 gave us some hope that we are getting closer to this.<\/p>\n Operations leaders face unprecedented challenges. They have to manage the new complexity of becoming cloud native and anticipate the implications…<\/p>\n","protected":false},"author":9,"featured_media":5766,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[867,1092,1073,975,1050],"tags":[968,143,1061,1055,619,1209,1207,1208,1163,1210,784],"organization":[],"ppma_author":[39,900,923,19],"class_list":["post-5764","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence","category-genai","category-generative-enterprise","category-hfs-horizons","category-the-generative-enterprise","tag-ai","tag-artificial-intelligence","tag-ecosystem","tag-generative-enterprise","tag-phil-fersht","tag-qa","tag-quality-assurance","tag-quality-engineering","tag-sdlc","tag-testing","tag-transformation"],"yoast_head":"\nLofty aspirations for quality assurance<\/span><\/span><\/span><\/span><\/strong><\/span><\/h3>\n
Exhibit 1: Pivoting to quality engineering is about aligning quality assurance to customer journeys<\/span><\/h3>\n
<\/a><\/span>Source: HFS Research, 2024
\n<\/em><\/strong><\/h3>\n\u201cShift right\u201d is starting to augment \u201cshift left\u201d<\/span><\/span><\/span><\/span><\/strong><\/span><\/h3>\n
There is much noise but little assurance on GenAI<\/span><\/span><\/span><\/span><\/strong><\/span><\/h3>\n
North Star (autonomous) persona-based testing<\/span><\/span><\/span><\/span><\/strong><\/span><\/h3>\n
Horizon 3 market leaders blend a compelling vision of transformation QA with nuanced approaches to assure change agents such as GenAI\u00a0<\/span><\/span><\/span><\/span><\/strong><\/span><\/h3>\n
Exhibit 2: The vanguard of the quality assurance services ecosystem<\/span><\/h3>\n
The Bottom Line: The QA community needs to emancipate itself<\/span><\/span><\/span><\/strong><\/span><\/h3>\n
HFS subscribers can download the report and find many more details here<\/span><\/a>.<\/span><\/h3>\n","protected":false},"excerpt":{"rendered":"