Saurabh Gupta — HFS Research[00:21]
Hi, everyone, and welcome to today’s edition of HFS Unfiltered. We’re going to be talking about generative AI. And to answer some of these tough questions, I have Manish Goyal from IBM. I lead HFS’s research and advisory services, and I’ll be your host today. Manish, why don’t you quickly introduce yourself?
Manish Goyal — Senior Partner, IBM Consulting[00:46]
So, I’m Manish, a senior partner in IBM Consulting. I lead our AI and analytics business and consulting globally. And like you, my last 18 months or so have been nothing but about generative AI, going through the cycle of what is it, to what are we going to do with it, to now much more real impact that businesses are seeing. So really, really exciting times, and looking forward to a really fun chat with you.
Saurabh Gupta — HFS Research[00:53]
Yeah, so, Manish, let me ask you, what’s been your favorite use case for generative AI? And let’s talk about a favorite use case in the business context versus a B2C context.
Manish Goyal — Senior Partner, IBM Consulting[01:23]
There’s a ton that’s happening in terms of driving productivity, but if I look at some of the major things that have happened because of this technology that weren’t possible previously — if you think of the work that Google did with AlphaFold in terms of being able to predict the 3D structure of proteins. I mean, it’s amazing. It was a very, very hard technical scientific problem that wasn’t easily solved, and now, with what they’ve done, we can actually do it. And IBM Research is doing something similar with what we have put out as a molecular foundation model called MoLFormer. And that was used for building out broad-spectrum antimicrobials, for example, for COVID. And it took us 5 to 6 weeks, with a 10 to 50% success rate, compared to 2 to 4 years and less than 1% success rate. So again, the impact of this across industries is pretty massive. And then the third one I’ll take — and I’ve got more — but the third one that we are very involved in with clients is if you think about industrial processes. The biggest polluters are heavy industries; 25% of global CO2 is produced by these industries. But what if you could apply foundation models to optimize the processes? If you take a cement mill as an example, each mill, and for each grade of cement, you have multiple models running, and it takes about 6 to 7 months to tune for one mill. Taking what IBM Research has been doing, and our teams, now we can run an entire cement mill on two foundation models — one for process, one for quality. And the impact of that is we’ve gone from like 24 to 30 models to 2 models. Obviously that’s very, very useful, but we’ve also gone from like 6 months to 8 to 10 weeks to bring up a mill. But the more important thing is we’re getting a 7x improvement in accuracy. That’s a 7 to 10% reduction in greenhouse gases for that mill, multiplied by thousands of mills. You’re really making a dent in the greenhouse gas emitted by these things.
Saurabh Gupta — HFS Research[03:53]
Yeah, this is fascinating, Manish. And one thing that I noticed in all those three examples is you didn’t use the word productivity in any of those. But if you look at the broader construct of use cases, everybody’s telling me, I’m expecting 30% or more productivity through this. The narrative around GenAI, at least within the IT operations and business operations world, has been that this will take us to a new productivity curve. Now, how do you react to that? One, is that 30% productivity, or 30% and more? Is that realistic? Is that actually happening? And two, is productivity the only measure that we should use?
Manish Goyal — Senior Partner, IBM Consulting[05:01]
Yeah. So, listen, like you, that is absolutely the focus for the C-suite. Again, because this has become a general-purpose technology — a different spin on GPT — right? Everybody’s using it. The C-suite is using it, everybody’s played around with ChatGPT or Gemini or Claude or whatever, and everybody’s seeing, okay, I can do my things differently and faster. So productivity is absolutely top of mind, and yes, we are seeing numbers of that sort, 30% or higher. But the caveat I would put is, is it across the board 30%? No. It is task-specific, interspersed with a whole bunch of different things that they do, even in the workflow. So you’ve got to really look at what is the process and task that you’re looking at and what’s the productivity that’s going to drive. For example, if you look at the software development life cycle, on task-specific things you can see even higher numbers than 30%, but that is very task-specific. A developer is not spending 100% of their time just writing code; there are other things that happen as well. So will the overall productivity be at a high level or will it be low? It’ll be slightly lower, but on average, yes, you’re going to see in that kind of range. In other areas — customer service, I’ll take a couple of examples of work that we have done — we’re seeing even higher numbers, but again it is very specific to what an agent does after a call. So if you look at the after-call work, where they’ve got to type up notes, we can generate the summary, we can have it consistent in a format that can go back into the CRM. So yes, it drives a lot of productivity for the agent, and the agent can handle other calls or interactions and spend less time on this, and you can also drive a bunch of consistency. But just in that example — because now I’m doing this for all interactions — I’m generating a consistent way of summarizing the interaction with the customer, pushing that back into the CRM system. I have now used that interaction to create data points that my upstream product teams, service teams, marketing teams can use to understand what’s happening better, to improve their products and services. So in that sense, it is not just productivity that you should be looking at. The impact of this is much more than that. Yes, it is an important measure for sure; it is going to fund a lot of the business cases upfront. But you really want to be taking a more holistic view in terms of what it is going to do for improving the velocity of innovation for whatever products and services your enterprise is building. Because across that digital product engineering life cycle, being able to test out ideas faster, to generate more ideas for your products and services, to test them out, to implement them — that entire process, there is productivity in there, but that’s driving innovation velocity for you to test out things and build out your competitive advantage. It can’t be just a singular conversation around productivity.
Saurabh Gupta — HFS Research[08:27]
Which, I think, is a fair comment. I think the biggest question now is, how do I go about using this technology? And at least our research seems to suggest that it’s not just a technology implementation, it is a much bigger people, process, technology, data, cultural transformation. But I would love to hear your thoughts on — as you’ve, over these 18 months, worked with, I’m assuming, multiple clients, some successfully and some unsuccessfully — what’s your words of wisdom on the how?
Manish Goyal — Senior Partner, IBM Consulting[09:28]
Yeah, so absolutely. You’ve gone through the same cycle of educating, then helping with initial pilots and where to focus, building out the pilots, and now saying, okay, now what? How do we scale and actually get value? So a couple of things. Because anybody could experiment with this — you didn’t really need to bring your IT team in to test out a bunch of stuff — there was a lot of proliferation, a lot of experimentation that happened. And in one way it was good, because a lot of people got direct interaction with those things, saying, okay, there’s something here. But at the same time, for a lot of our clients, there was this lack of control on what was happening. And there are clear issues and concerns with the technology as well. In an enterprise setting, there are liability things, you know, is our data being sent out, etc. So a lot of different things were going on. But now, where we are with clients and what we are advising them on — in general, our view, and our surveys and conversations with clients, say that north of 70%, 75% of the C-suite have basically made this a priority. It’s baked into their strategy and what they’re focused on. How do you actually convert that into value? Our view there is that you really want to take an enterprise-wide approach to this. It needs to be cross-functional teams, a coordinated AI strategy, governance, how are you going to share learnings, data, models. What are you going to use? Closed source? Open source? A combination? Are you using SaaS services? Are you building your own? So you want some level of coordination around this so that it’s not all over the map and then you’re trying to corral it post facto. So you want to set some direction. And in some ways it is similar to traditional ML, but in some ways it’s not. It has new ways of working, new things that you need to be aware of in terms of how you observe the model behaviors for large language models or foundation models versus traditional models. Previously you trained the models, you knew what the training data was — even if you may not have done a great job of all of the MLOps stuff, at least it was your data that you trained the models on. Here it is sort of black-box-ish, and you’re adding your data to it. So there are different concerns here. So you really want to have some engineering best practices in terms of how we are going to do this as an enterprise. And it has to be agile, because fortunately or unfortunately, the technology is moving so quickly that you just have to bake that in. So that’s the approach in terms of where do you start. The proliferation was fine when you were experimenting, but you really want to now start picking what is the highest value for us, for our enterprise. Is it marketing that we want to go after? Is it customer service, is it operations, back office, whatever it might be? And it’ll be different for different enterprises. So making sure that you have the right talent pool — and it is a challenge. Whether you are using consulting firms like ourselves, or building, or hiring, whatever that combination is, you can’t not be focused on that, not just for now but for the long term, because this is a skill set that you are going to need in the enterprise. But in terms of driving the adoption, there are two things: you’re doing this to improve your products and services, but you’re also driving it back to your productivity question. And that comes from usage — usage across the enterprise, getting that right, and that comes back to process and culture, to be thoughtful and intentional on how you’re going to roll this out, how it is baked into the workflow of the different functions and the different folks across the enterprise. And then finally, what we advise clients on is to look at the impact of this holistically, beyond just productivity metrics. Is our innovation velocity or development velocity faster because we are using this? You really have to be measuring this across — whether it’s marketing, whether it’s customer service — you want to be measuring the right KPIs that you believe would be impacted by the rollout of this for that function in that workflow. So you want to have that broad thing that you’re looking at, and then aggregate that up and say, yeah, we are making progress towards the goal of using this technology. So that’s how we are advising clients where they need to be moving to.
Saurabh Gupta — HFS Research[15:18]
Yeah, so there’s so much to unpack there, Manish. Well, let me pick on a couple of things that you mentioned. There’s one very interesting comment that you made on open versus closed — open versus closed technologies, open source versus closed. And that conversation reminds me a lot of the cloud conversation. Remember when cloud first came into being? The whole thing was, it needs to be private cloud. Then suddenly we moved to a public cloud or multi-cloud, and I think we’re settling on a hybrid cloud, a sort of mix of both worlds. Is that how you see this evolving? Where do you think this open versus closed debate will go?
Manish Goyal — Senior Partner, IBM Consulting[16:10]
Yeah, it’s really, really interesting. Because obviously, if you look at the frontier models, they are still closed source. But if you look at the absolutely amazing innovation that’s happening in the open-source world, they’re catching up. And in our conversations with clients, our view is that clients want choice. They want open for a variety of reasons — for preventing lock-in, for preventing regulatory capture. And like a lot of open-source projects, they also want to contribute back. So I think our view is open, but at the end of the day it’s going to be hybrid, because there’s going to be a mix of things. I think there’s no debate at this point in the cycle that there is not going to be a single model to rule them all. It just doesn’t make sense, from cost, latency, a whole variety of reasons. So you are going to have different models for different tasks and use cases. Will they all be open, will they all be closed, or will it be a hybrid? It’ll definitely be a hybrid. There’s a tremendous amount of work being done — just last week, there were three small language models that were released by Microsoft, by Meta, and others. So you have much smaller models, which are more cost-efficient, better for the task that you’re putting them against, and use less energy, which is important when you look at the cost of these models from a training and inferencing perspective. And just recently IBM, on the thread of open source — which is our point of view on this — IBM Research put out something called InstructLab. So you take an open-source model, but the thing is, if I contribute something back to it, do I get a different model or do I improve the same model? So with InstructLab, and a language model dev kit, anybody can contribute knowledge and skills to the same model, and it could be an open-source or a closed-source model. So we are really open-sourcing that technology to drive that kind of innovation, so that you can make the same models better, and a lot of people can contribute without having to retrain the base model. So in my view, you’ll end up with a hybrid situation. There’ll be certain SaaS applications that an enterprise may say, you’re just going to use that, and that comes with a closed-source model, and a whole variety of other use cases across different functions will end up with open-source or closed-source models — in many cases smaller models — that they have trained on their data and put together. And at the end of the day, you’re going to have enough choice of models. The thing that drives competitive advantage is not necessarily the models, unless you are going to be able to build your own model — you have such proprietary data of such quality and volume that you can build your own. But that’s going to be few and far between. The bulk of it is going to be closed or open-source models coupled with your proprietary data that drives a competitive advantage for you. And how you use that, combined with ways of working, product, innovation, and so on — combining all of that together is where enterprises are going to build up their competitive advantage.
Saurabh Gupta — HFS Research[19:49]
Yeah, so it’s not one versus the other, it’s one and the other. It’s always the ‘and’ construct. It’s the same cloud story — it is hybrid. And that’s where we’re going to end up with this as well. I think you mentioned machine learning as traditional machine learning, and machine learning is not that old as a concept, especially in the B2B world. And now we’re calling it traditional, which highlights the speed of technological innovation that we are seeing. Till yesterday, almost, it was machine learning. Now it’s LLMs. Now we’re starting to see large action models, contextual language models, all kinds of variations. If you put your technologist hat on, what’s the most exciting innovation that you’re looking for? What’s the one thing that you’re really excited by?
Manish Goyal — Senior Partner, IBM Consulting[21:09]
So, I think we’re also living in a time period — I call it a general-purpose technology, driven by what we’re doing now — but there’s also a convergence happening, because of the capabilities we have with the foundation models. And you see what is happening with, for example, robotics, or even software agents, and the agentic workflow, where you can give it a problem, it plans it out, it solves it — the agents talk to each other and they solve the problem. Early days, but a tremendous amount of work happening there. It’s going to have a profound impact in terms of how you configure what you do, and how quickly you can do this. But at the same time, you can also use some of those things to train models that provide you training data for robotics. And what is happening between 3D printing, robotics, coupled with large language models, or the ability to just tell the robot — in fact, I just read this yesterday in the MIT Technology Review — you can now start talking to a robot: hey, pick up from this basket only the tennis balls and move them to the other thing. And it can now start doing what I was just talking about, saying, okay, I’m having trouble gripping the balls, and you can say, well, instead of using 6 suction cups, use 3, because maybe that’s the problem. It’ll adjust automatically. You’re not programming this. So that, coupled with the advances happening with 3D printing and how that is going to revolutionize manufacturing — all of this combined, I think it is going to be absolutely fantastic in terms of the innovation and products that you’re going to see, and the speed with which this is going to happen. So, very, very optimistic about the impact all of this is going to have in terms of value creation. But at the same time, I think it’s also going to have a profound impact on society. And I don’t know if society, governments are planning well enough ahead to look at that displacement and what’s going to happen with that, because there is going to be impact, and this is going to be impact on all kinds of jobs.
Saurabh Gupta — HFS Research[23:37]
So does that keep you awake at night? Amidst all this excitement, is there anything that keeps you awake?
Manish Goyal — Senior Partner, IBM Consulting[23:37]
Yes. So I talked about one of them, which is, where does this all end up from a societal perspective — displacement of jobs? You know, what do my kids need to be learning? I’ve got two teenagers; what should they be learning, and what are the right career choices for them? So that’s one aspect. But the other aspect is, all of this fantastic innovation comes with those risks. The level of misinformation that is going to be created using these technologies — it’s already happening, and there’ll be more of it over the next few months as the elections come up. I’m really worried about that. I’m worried about how the same technologies can be used for security breaches, whether it’s impersonating me. We’re going to be on this vlog, my voice is there, my video is going to be there, so somebody cloning me — I mean, this is getting really, really good. So what does that mean from a cybersecurity perspective? The same technologies that are being used to build out those proteins and medicines and materials, or to recycle plastic better — all those things can also be done for negative uses. So, very, very excited, but these are things that are real concerns. At heart I’m an optimist, so I think as a society we’ll work it out, we’ll build up the defenses, and we will cooperate, so that it all works out for the betterment. But yeah, are there concerns? Absolutely.
Saurabh Gupta — HFS Research[25:23]
This was a fascinating conversation. Thank you so much for spending some time with us. I really think we need to do more of these conversations, because in the next 3 months things will have changed — hopefully for the better. But this seems to be a space where we take two steps forward, one step backward, and what I’ve realized is there’s no other way than to just keep talking, because there is no template for success. Nobody can give you the other 10 things to do to be successful. We are all figuring it out together, and I hope this podcast, this videocast, amongst other things that HFS and IBM are doing, sort of helps us move those two steps forward — at least learn from each other’s mistakes, if not successes. So thanks a lot, Manish, for taking the time. This was a fantastic conversation.
Manish Goyal — Senior Partner, IBM Consulting[26:23]
Oh, thank you, Saurabh. I had a great time. Thank you for inviting me.
