This Market Impact Report is for IT leaders and technology executives seeking to scale GenAI initiatives from proof-of-concept to enterprise-wide production.
Generative AI (GenAI) has the power to revolutionize industries, but scaling it is no walk in the park. The pressure is building for technology leaders to turn GenAI projects into scalable solutions for their businesses. Yet to succeed with GenAI it requires bold leadership, relentless modernization, and a willingness to confront harsh truths about the organization’s readiness. Only by shifting from pet projects to enterprise-grade initiatives can IT leaders begin to deliver the promise of GenAI to reinvent and, in many cases, reimagine their business.
In today’s world, not doing AI is simply not an option.
— VP and head of applications strategy, financial investment firm
HFS Research conducted over two dozen interviews and discussions with IT leaders to understand their journey from proof-of-concept to delivering at scale for their enterprises. Their interests in GenAI span a myriad of aspects, from boosting productivity to staying competitive in an evolving market. Here’s what’s motivating organizations to dive headfirst into this transformative technology:
AI-driven chat agents are now table stakes in our industry.
— Global head IT, banking and investment firm
It’s a carrot for those eager to experiment with cutting-edge tech.
— CIO, global financial firm
GenAI is leading to a transition from people-driven uses of technology and services to solve problems to technology-driven solutions. This new era of technology arbitrage is based on the maturing of technology and associated skills to fully realize the benefits of automation, process mining, and artificial intelligence. The technology arbitrage era is based on firms using generative AI projects to rethink their workforce, differentiate products, and expedite how data enhances organizational decision-making. As illustrated in Exhibit 1, this inflection point drives enterprises (and their partners) to create value for the foreseeable future.
Source: HFS Research, 2024
As part of the new ‘S-curve,’ many firms are quickly evolving their data, analytics, and automation projects, including GenAI. However, when HFSW asked a select group of IT leaders about the maturity of their GenAI solutions, fewer than 25% of technology leaders indicated deployment of production-ready scalable enterprise solutions (see Exhibit 2). To date, the majority have not realized any significant impact on their enterprise.
This survey shows that nearly 70% of these projects stall at proofs-of-concept (POC) and pilots. When asked why, many executives express challenges in identifying realistic use cases beyond how GenAI enhances productivity. Too often, GenAI projects have focused on capturing value lost with inefficiencies in existing processes or allowing teams to allocate ‘just a bit more time’ to other projects.
Source: HFS Research, 2024: n=26 IT executives currently leading GenAI programs
But don’t abandon hope; even in these early days of GenAI’s promises, some firms are beginning to move past the pilot stage toward production use cases at scale. These are the ones that should be of interest to IT leaders who are being held accountable for the past 24 months of investments.
For example, a large financial firm is increasing its adoption of GenAI by using private data centers and a personalize large language model (LLM) to manage costs and risks. This firm is seeing results where GenAI is integrating into the workflows and data sources for existing platforms and adding internal chat-based applications that allow large amounts of client and market data to be contextualized as a new product. These new offerings use enormous amounts of historical and new data to create tailored investor profiles with real-time market indicators. By using GenAI for what it does best, crunch limitless data sources and figures, but contextualize it into information human agents can use to better advise their clients, this firm has differentiated their services at scale.
However, in many cases, the transition from POC to scalable solutions is generally more cautious and controlled. A large North American healthcare payer that is investing in GenAI to fulfill payment obligations requires its IT and business teams to demonstrate how risk has been mitigated as part of the ROI. For this firm, it’s the sensitivity of its data that is stalling the POC. While the pilot remains limited in production, its commercial viability is questioned until risks, reporting, and regulatory concerns are solved.
Even with POC failures, it is still possible to find the right path. Another firm was forced to cancel its GenAI practice due to cost overruns. Yet they believed in the potential value, and thus a reset of the POC with a new services partner is gaining traction and has started delivering the expected gains. This again shows how the inflection point cited in Exhibit 1, may force many organizations to rethink if their current partners are right for their future plans.
One thing is clear: No one is willing to give up on these new technologies. As cited by a CIO of a global financial services firm, the journey to deliver AI-driven business transformation is actively underway. While companies are seeing results in POCs and scaling solutions in some enterprise programs, this is only the beginning.
We’re not at the very beginning, but by no means are we mature. I would say we’re somewhere in the middle because we’re past proofs-of-concept. We have GenAI use cases deployed. I’m not going to declare victory yet.
— CIO, global financial firm
Scaling GenAI from POC to enterprise-wide deployment is fraught with challenges beyond technology. In an additional survey of 2,300 IT and business leaders, 90% believe legacy adds to the costs.
Source: HFS Research, NTT DATA, 2024; n=2300
Here’s a deep dive into the most significant hurdles:
Hidden costs: Scaling GenAI is expensive, and unforeseen costs can derail even the most well-planned projects. Without meticulous cost planning, organizations risk ballooning expenses that undermine ROI.
We wasted $10 million on a vendor that didn’t understand our business, an expensive lesson in choosing the right partner.
— VP, Apps, financial investment firm
Garbage in, garbage out: Data is the lifeblood of GenAI, but poor data quality and fragmented sources create significant bottlenecks. GenAI models produce unreliable results without clean, well-governed data, leading to mistrust and failed initiatives.
We had to rebuild several database tables to ensure accurate insights, delaying deployment and adding costs.
— Information technology infrastructure manager, North American manufacturer
Legacy IT strikes again: Legacy systems are ill-equipped to handle the demands of GenAI, from data processing to real-time analytics. Scaling GenAI requires robust, scalable infrastructure, but legacy systems often slow progress.
Maintaining legacy and GenAI systems in parallel has significantly increased our costs.
— CIO, North American healthcare firm
One breach away from disaster: Handling sensitive data in GenAI projects introduces heightened security risks and regulatory scrutiny. Balancing innovation with security and compliance is a tightrope, especially in regulated industries.
Our GenAI efforts vary by region to comply with stricter laws like GDPR and California’s CCPA.
— IT director, North American retail firm
The human factor: GenAI requires not only technical expertise but also cultural alignment and workforce buy-in. Without the right skills and a supportive culture, even the best GenAI strategies will falter.
We’re learning as we go because we don’t have enough in-house expertise to guide us.
— VP, Advanced Product and Technology Strategy, financial investment firm
Through our in-depth interviews, HFS uncovered seven pillars of change that IT leaders must consider harnessing GenAI without falling for the usual traps.
Source: HFS Research, 2024
GenAI initiatives thrive on decisive leadership. Someone needs to own the strategy, whether it’s a chief AI officer or a cross-functional coalition. While the IT leadership is held accountable, companies where the CEO is investing and driving the transition of company culture toward understanding how GenAI is crucial to their operations are most likely to move from POC to enterprise scale. A sizeable financial group’s CIO remarked on their CEO’s passion for internally and externally discussing how GenAI tools will reshape their business.
In our research across IT leaders, it was insightful to see how fragmented GenAI leadership can be. In our interviews and the roundtable discussion, HFS found that 50% of companies have a leadership committee or a team led by a line-of-business (LOB) leader. Meanwhile, 42% lean on the CIO or CTO to drive the AI efforts. After a lengthy debate, IT agreed on how it plays a crucial role in delivery. Still, adoption at scale across the business must depend on the business leadership showing a passion for its application and their willingness to own the vision.
Source: HFS Research, 2024 n=26
Why CEOs are talking about GenAI and their business in their financial and investor reporting but not often showing up as an internal champion? An insight came to light during our research/roundtable when a participant stated that “CEOs don’t do POCs.” This simple statement reflects an underlying challenge: As their companies become more mired in POCs, the less likely they’ll get the executive sponsorship needed to scale. Therefore, the onus is on IT and business to partner effectively on skills, technologies, and objectives.
Our new CEO’s vision for the company is to bring us to the forefront of technology, cultivate ourselves as leaders, and emerge to the future. We’ve always been an organization that follows trends. So, with the CEO’s vision, we hope to get out front. For GenAI, we have teams internally, and we’ve built our own internal LLM that helps us navigate going from POC to scale within our organization.
— COO, North American insurance firm
In a global financial management firm, the CEO was responsible for large-scale education and advocacy of GenAI. This solution allows teams to co-create solutions, decide which aren’t scalable, and develop cross-functional training to ensure multiple groups see how advancement benefits their workflows.
No more of us versus them; it’s all hands-on deck under our CEO’s sponsorship.
— CIO, North American financial firm
In some cases, firms told HFS they are creating a chief AI officer—a departure from the trend of appointing chief data officers in the 2010s. In the chief AI office, the leader must be very aware of the business operations, processes, and products. They see data and AI from a businessperson’s point of view, not that of a technologist.
If your IT team still thinks their job is to “keep the lights on,” you’re doomed. IT must evolve from a support function to a strategic powerhouse. Make IT a co-leader in your GenAI journey as its IT that holds the keys to your data kingdom and should know how to wield them to help the business leaders achieve the outcomes they desire.
IT is foundational in enabling GenAI projects by providing infrastructure, data management, and security. For instance, a VP of IT corporate technology noted: “Our responsible AI program is headed by IT but works closely with privacy and legal teams to ensure compliance.” IT collaborates with business units to deliver GenAI solutions.
As one CIO commented: “We’ve conquered the buy versus build debate; IT partners with third-party vendors to ensure scalability and integration.” IT often provides the technical backbone for deploying and scaling GenAI. In another firm, the global head of essential technology and innovation developed an internal GenAI platform, with IT ensuring data security and cost efficiency. IT departments frequently act as gatekeepers, ensuring GenAI initiatives align with organizational standards.
IT was ahead of the business on GenAI, guiding them on what’s possible.
— IT director, Global consumer products
Legacy IT systems pose a barrier to scaling GenAI. As part of their GenAI efforts, IT leaders must adopt a phased approach to modernize legacy systems, prioritizing data readiness and workforce training. As there is often resistance to change among resources accustomed to legacy systems, necessitating the support of executives and employees will be crucial while transitioning to new technologies. Additionally, these leaders must be prepared to deal with regulatory challenges related to data privacy and jurisdiction, requiring mechanisms for data tracing and security enhancements.
We have mainframes. Like every financial company, and yeah, we have systems that are old and older and all that. Our GenAI projects are forcing us to speed up modernization efforts as we need more robust data, security, and cloud capabilities.
— VP, IT – corporate technology, financial investment firm
As shared by IT leaders in the finance, insurance, and healthcare industries, legacy systems are often embedded in specific company needs. GenAI’s advantages are seen in customer-related systems such as CRM and new tools. Security is a significant concern, with firms anticipating stricter regulatory demands for AI solutions, necessitating adherence to bias and transparency controls while recognizing the likely need to evolve security to include AI itself.
A large financial firm, for instance, realized success through creating new revenue streams by providing data in a format its customers can directly access and use without extensive data manipulation. This approach is a potential growth opportunity because it saves customers time and effort, making them willing to accept a premium price for better services.
Our firm is adopting a data strategy it calls ‘ready data.’ We believe that our customers can consume the data directly into their models without spending too much time finagling it or massaging it. That is where we think there’s an opportunity for us to generate revenue with a premium service, as this is a huge amount of savings for the customer.
— Strategy director, global food and beverage firm
When asked how important it was to ensure their data can leverage the scalability of both public and private clouds as part of their GenAI efforts, the overwhelming response was “very important.” Many executives indicate that their data may never be perfect. So, rather than continuing this quixotic mission, they focus on getting good data in critical business processes that GenAI could significantly improve.
Source: HFS Research, 2024
An important discussion point among many IT leaders is about deploying and scaling GenAI in a hybrid environment. All technology leaders agree that enterprise data must be able to transit seamlessly across on-premises, private, and public cloud architectures.
The desire for AI and data to be available on a private cloud was assumed to be very important. However, our roundtable discussions revealed a general lack of interest in a dedicated private cloud for their GenAI project. While they agreed on the importance of a firm leveraging a hybrid cloud model to scale GenAI, in general, IT leaders are pushing to get their infrastructure extensively into the public cloud. While valid, HFS believes that it remains important for a company to ensure their data is private. Thus, a private instance to develop a POC, in some cases to run LLM/GenAI should be part of the strategy—whether hosted or on-premises. Interviews with executives indicate they increasingly likely to lean on their IT services partners’ capabilities in data, cloud, and cloud-integrated services as their experiments with GenAI mature.
For example, a large retail firm cited a successful hybrid cloud strategy was a crucial part of its GenAI project development. This approach allows the firm to balance the benefits of scalability and cost-effectiveness offered by hyperscalers, providing public and private cloud offerings with the control and security offered by a hybrid model.
We must have a hybrid cloud strategy that can scale based on our GenAI project and data needs. Our partner helps us gauge the POC using private cloud and our data, and then our CEO steers us toward the best long-term platform.
— IT director, North American retail firm
Given its choice to use a hybrid cloud architecture, the firm balances on-premises cloud capabilities to run POCs and sensitive data processes while ensuring data security and compliance. It then leverages hyperscaler clouds for scaling and integrating broader datasets. Additionally, having mechanisms to migrate between cloud providers within a multi-cloud infrastructure creates manageable overhead, facilitating the desired flexibility and efficiency in deploying GenAI solutions. With their partners’ support, this strategy also helps mitigate computing costs, scaling, and data protection uncertainties as projects move from POCs to enterprise production.
Implementing an AI council or a similar governance body that includes representatives from various business units, IT, risk and compliance, and legal departments is essential to baking GenAI into the business. The role of this council is to oversee the development and enforcement of guardrails for safe, secure, and compliant GenAI use. They ensure that humans remain in the loop and that there are programs to address security, bias, and usage.
The AI Center of Excellence (COE) is responsible for anticipating and preparing for any regulatory demands that might develop as the technology and industry oversight mature by ensuring that GenAI systems adhere to existing IT general controls and potential new controls related to AI advancement, such as those addressing bias and transparency in financial markets. It is also crucial that these groups ensure that business and technology teams are engaging in ongoing testing and oversight to identify and mitigate potential vulnerabilities, especially in multi-cloud environments. These efforts often include cybersecurity measures and regular audits to ensure compliance with security standards.
For a long time, we have focused on building data and information into organizational culture. Systems and applications were put in place to create an order of operations. This culture of effectiveness and efficiency is the bedrock of a ‘well-oiled’ machine. Scaling GenAI requires a willingness to foster a company culture that embraces AI. Providing education, frameworks, and recognition to employees is overcoming resistance to adopting AI projects.
Wall Street is looking at us. Are we embracing technology in the right way, and can we demonstrate the benefits we’re gaining from it? We believe committing to using GenAI as part of how we work helps attract, retain people, and build the right talent.
— CIO, Global financial firm
GenAI is expected to positively impact people and culture within organizations. For instance, it could help attract and retain talent by demonstrating technological advancement and offering career growth opportunities and exposure. This positive perception is also expected to extend to shareholders and regulators as firms embrace technology in beneficial ways. Additionally, cultural change management challenges are anticipated, as employees need to adapt to new technologies and processes, which includes setting realistic expectations about what GenAI can achieve and addressing any skills gaps.
In our interviews, executives shared the crucial “dos” and “don’ts” with their peers, as shown in Exhibit 7. Based on feedback from our executives, firms should have strong data, clear outcomes in mind, and active engagement from business partners. While these seem like motherhood and apple pie, the real insights come from the “don’ts” shared by the executives.
As many of them find themselves caught in POCs, what they’ve learned is very telling and they have great insights for their peers. Three important insights that a business or technology leader should take to heart include:
Source: HFS Research, 2024
Many leaders indirectly communicated a crucial insight during our interviews: they are often challenged to recognize how GenAI POC delivers value capture versus value creation outcomes. HFS recommends that as technology and business leaders begin to ask whether POCs can be scaled, they must answer capture versus creation. Making this distinction early allows for planning and executing the right resources and investments and setting goalposts.
In many cases, capturing value can be pivoted to creating value only after broader adoption. Thus, wins in productivity, effectiveness, efficiency, and improved data and insights (capturing) will often be the precursor to the transformative, experience-changing solution that creates new value in the markets and with the customers you serve.
Register now for immediate access of HFS' research, data and forward looking trends.
Get Started
If you don't have an account, Register here |
With the exception of our Horizons reports, most of our research is available for free on our website. Sign up for a free account and start realizing the power of insights now.
Our premium subscription gives enterprise clients access to our complete library of proprietary research, direct access to our industry analysts, and other benefits.
Contact us at [email protected] for more information on premium access.
