David Cushman — HFS Research[00:22]
Hello, you’ll find me today with Premkumar Balasubramanian, who’s the Chief Technology Officer at Hitachi Digital Services. He oversees technology and innovation and is primarily responsible for strategizing and supporting go-to-market pursuits, offering shaping, and architecting repeatable solutions, and providing technology and thought leadership in the areas of cloud, data, IoT, Gen AI, and I’m guessing quite a lot of agentic these days, Prem. First of all, thank you for joining us today. I know that you guys have been doing quite a lot of work in actually delivering agentic into both your own organization and increasingly into others, and I think it’s interesting to see the shift, if you like, of need in the enterprise. Initially it was a lot of education, a lot of why should you do this? And so there’s been lots of work done there, and I think quite a lot of investment. And then there’s been a lot of, well, what can you actually do? And there was lots of work on use cases. I think we’ve built up something like 1,000 use cases now, and they spread across every industry. But increasingly the tough bit is how we do this. And I wanted to talk to you to understand a little bit about how you guys had gone about it, and I’m thinking particularly, in the first instance at least, of how you treated yourselves as client zero, particularly around the work that you did in HR. To put it in a kind of strange nutshell, perhaps, what’s it like having an agent running HR for 30,000 people, and how could you get there?
Premkumar Balasubramanian — Chief Technology Officer, Hitachi Digital Services[02:04]
First of all, thank you so much for having me here, David. It’s nice to be here, and always with HFS it’s fun because you always ask really tough questions, and more practical ones as well. So, we’ve done it. When we started thinking about this, there were two things. One is how do we apply this internally and then take knowledge to our customers. So it’s kind of tasting your own medicine. When we thought it through, there were three areas, in fact. One is the HR that you mentioned, and there are two other areas that we are currently actively finalizing rollout. While the HR is rolled out, it was slightly an elevated initiative by Hitachi Digital across the group companies. The second area is marketing. Like for HR we work with a startup, for marketing we’re working with another, where we’re going to have marketing agents, specifically focused a lot on our case studies, our ebook creation, content and stuff like that. And then the third piece is a lot more internal and very interesting. This is in our procurement business itself, where we’re starting to focus heavily on automating our invoices and purchase order process. Each of these are slightly different use cases. So if you think about the HR use case, it’s rolled out at scale. It’s more starting out like a chatbot, so you’re asking it questions and it gives you policies, anything related to our entities, our company and our sister concerns, and depending on who you are, it can give you that information. As we speak, we’re working on upgrading it into the further capabilities of agents, as we call it. It’s truly not an HR replacement at this time, but it’s still an HR bot that we built together. The second aspect, like I said, the procurement part is a lot more agentic workflow. So that’s like agents putting our invoices, making sure it’s reconciled, and saving it in our target systems, and focusing on building purchase orders, with a lot of consistency. A lot of data issues that we’ve had in the past, we’re fixing with the agentic workflow. And the marketing is a slightly smaller way — the number of marketing people who would use it is less, but its reach is significant. So the HR bot is more us using it, but the marketing bot is us publishing content through it, so it has a more profound impact on our business. Because we want to make sure it does the right thing, we’re taking more cycles to get it to production, but obviously it is very promising and quite close to its first release. Those are the three key use cases, David, that we thought we’ll first try, and there’s a ton of learnings that we’ve now packed into our own offerings that we would be taking to our customers as they go with us on this agentic journey.
David Cushman — HFS Research[05:21]
So you mentioned the difference maybe between doing something that you’ve got to do to yourselves — and so I could ask you what it’s like being managed by an agent. There’s a little bit of that spookiness in the internal HR role, but I suspect we could always iron things out internally. We don’t have to do that in public, right, and no one gets hurt. Well, maybe people get hurt, but we can manage it. When you’re externally facing like the marketing story you’re telling there, suddenly the risks are significantly greater, aren’t they? So you mentioned you learned some lessons. I know from our own experience, data and getting the metadata right seems incredibly important in being able to actually deliver real value with the agentic. But I wonder what else you’ve learnt from your internal experiences that you’re now taking to the market, particularly in regards to things that are repeatable — frameworks and guardrails, how they are helping your clients actually adopt this new wild west that’s being thrust upon them.
Premkumar Balasubramanian — Chief Technology Officer, Hitachi Digital Services[06:33]
One of the things that we’ve really, really nailed down on is what we term as the minimum viable agent story, right? Everybody seems today to call everything an agent. A year back, everything was a Gen AI application, and today every Gen AI application is an agent in some form or shape. So we’ve started defining what that minimum viable agent should be, so that when people, even within Hitachi, come to us and say, hey, I’ve released this agent, here are like six things we ask them on. Hey, does it do A, B, C, and D? If it doesn’t, then it’s a chatbot. Leave it with that — there’s more work for you to make it an agent. The reason is the level of autonomy that you want to put on an agent. Especially if you think about our marketing, it’s an agent because it’s collecting data from people, from SMEs. So if I’m writing an ebook, it’s actually working with an SME collecting all the knowledge, and then packaging it into an ebook. And we have an evaluator, which is again another agent within that framework — it’s a multi-agent framework, as you would know. So it leverages more of a generator-evaluator type pattern, right? We want to make sure it evaluates and reconciles that with some of the facts that we have, and if it sees something not right or something novel that is in the ebook that it doesn’t see in some of those artifacts that we’ve given, it reaches out to a marketing person. So the evals are pretty solid in that, like I said, because it is external facing. So we want to make sure the content we publish is absolutely making sense to the industry. It’s not a hallucinated piece of stuff that an LLM broke. So there’s a lot of work that we learned on how to manage it, right, from getting the prompts nailed down, to having a multi-agent framework, to putting in evals, and then having human in the loop. So none of this is autonomous end to end. There is a human in the loop all the time. So that’s a big learning for us, David, right? When you think of an agent, with an agent comes a certain level of autonomy that you want to give, but this is not autonomy, autonomy — it’s not replacing our entire marketing team to just keep publishing content out to the web, right? So that’s a big, big learning, which is how do you continue to run it. The second learning is training the internal folks on prompting and understanding how to prompt and work with the agents. It took us a bit of a cycle, but it was a small group to be trained, and our marketing team is now kind of pros in this, right? They can actually go and have a good conversation with these agents that we built to really get meaningful content out in the world. Third is the data, the grounding, right? Even though it’s content that is getting created, we want to make sure that it’s created from the content base that we have created, the knowledge base. So keeping the knowledge base updated and constantly feeding to the knowledge base the right set of documents becomes extremely critical. So I think those are some of our learnings in this space. There is a lot more. Today we’re working with the startup to actually have them expose their metrics in OpenTelemetry. They don’t do it by default. Many of these companies don’t, right? None of the companies I’ve seen are doing it. You have to go to their dashboard to get whatever data they show you, but with this company we’re working with them to really give us the data, because we as a company have started building what we call an agent management system. So this is across the multiple startups and the procurement work that we are doing — we want to make sure we get a single pane of glass view, because that’s another critical piece of work from our own end, to get control of where we are spending. Because like container proliferation, agent proliferation is very real. So we want to get slightly ahead of that, and then we can take that and give it to our customers too. That’s kind of our endeavor, right? When we build it, we build it for ourselves, test it, and we can expose it to our customers that come and leverage HARC services or managed services.
David Cushman — HFS Research[11:06]
And you mentioned this multi-agent piece, and managing multiple agents, orchestrating them, controlling them, and at least having a view of them. It seems that agents, much as Gen AI has, are very good at offering individual productivity hikes. We’ve all played with agents, got them doing some work for us in the background while we get on with something else. I think we’ll have seen ChatGPT’s rolled out its agent mode, in at least the $20 version, very recently, and just to watch that work is quite interesting in itself — to see what it’s actually doing and when it calls on you to join in and hand over your credentials, so that it’ll access stuff that you really need to make things happen. You’re seeing Copilot, it’s got some models that look very much like agents too. So we’re all getting used to getting the individual benefits, the productivity benefits. But I think the next challenge is now, how do you scale that across an enterprise? Because the reality is, if we all get a bit of productivity benefit, where does that end up going? There has to be something more coordinated about it. So I know you’ve got some plans around scaling your agentic AI offering, which will be coming up next month by the time we publish this — so we’re talking about September, I believe. Can you tell us a little bit about that and how that’s working to standardize and scale agentic AI across industries?
Premkumar Balasubramanian — Chief Technology Officer, Hitachi Digital Services[12:36]
Yeah, yeah, David. This is a really deeper question because it will take a long time and I can go on and on with this. But what we are doing in September is we’re launching our agentic framework, which we call HARC agents — Hitachi Application Reliability Center-based agents. The idea behind that is, when you think of agents in production, as an example of the marketing work we are doing — for us to knock up a prototype, it’s pretty straightforward. It’s easy with LLMs. That whole process has been shrunk in terms of timeline, it’s very quick. But when you want to productionize it, we always believe there are four parameters, right? It has to be reliable, which means same question, same answer. It has to be responsible, so the RAI guidelines that we spoke about — not publishing toxic content, not publishing content that is not real, hallucinating, groundedness with the data, bias management — so that sits within that framework. So it has to be reliable, responsible, and the third piece is observable, which is what I meant when we are working with the startups. Our idea is we need to get the explainability clear. We need to know, if it reasoned something, what has it reasoned, why did it arrive at a conclusion, even if you want to do a post facto analysis of what happened. And finally, the most critical aspect at scale is when you will feel the pain of this — it has to be optimal. For every time I run an agent, do I know how much am I spending? And is that the right amount of money I should spend for that workflow? So when we think about invoice processing, we worked with a customer even, right, where we actually measured how much it was with their original RPA and how much it is with an agentic workflow. And it is 10x ROI. It was like $5 — it was $4.94, and now it’s $0.052 or something like that in terms of the difference — but that was measured. We spent six weeks in production measuring that number. But that’s kind of how we think before we start scaling. Today we’re doing a bigger program with them around agentic, so they want a lot more of their processes to go, but for them to be convinced, for us to be convinced — the token usage, how much it uses, because in the agent it’s easier for you to get lost in the loops, in the number of calls. So if you don’t have control over what’s going on, it’s very hard, and when you scale this, it will hit you even harder. So as a part of what we do within HARC agents, we do leverage our R2O2 AI framework that focuses on, when you build an agent, how do you ensure reliability, responsibility, observability, and optimality of your solution. And that’s kind of the view, David, and we’ve taken all those learnings and put it into the HARC agents platform. So every agent that we’re going to release on top of HARC agents — think of it as a production-ready jumpstart. We’re not saying, hey, hire an agent from us; we’re not that company that’s building agents that you can just hire instead of humans. It is more about, hey, work with us on your agentic journey, but here are a bunch of jumpstart agents that we can expose, that we can then take and make yours. The code, the container, everything can be deployed in our customer’s environment, and we can build on top of it with the customer. And the frameworks and the SDKs that we’re putting together will just accelerate how they build it, with RAI guardrails that you spoke about, with red teaming and security for agents that’s been built into this framework, and with observability under the agent management system. So that’s kind of how we’re trying to ensure we can take it and scale it for our customers, as much as the work that we’re doing with them to scale these across a multitude of platforms, if I politely put it.
David Cushman — HFS Research[17:00]
It’s good to hear. I’m wondering, maybe just as a wrap-up question, a bit of a punt into the future. I know you will be aware that we talk about a series of debts that need to be paid down by all organizations to be able to reap the benefits of AI. Tech is one of the debts, skills is one of the debts, data is one of the debts. I just wonder whether we are risking creating more tech debt when we create, spin up loads and loads of agents. What’s the plan? What are we going to do with them all?
Premkumar Balasubramanian — Chief Technology Officer, Hitachi Digital Services[17:42]
Very interesting question, David, and I’ve been talking to a few customers about this as well. There is a real risk of us just building so much more tech debt. While vibe coding, while agentic development, all of that is definitely true, forward-looking, and I think it’s inspirational to use it — I use it in my day to day. But I also see the amount of code it just generates within very little time that we work with, and the efficiency of the code is always in question. So I think the gotcha for most of the companies at an enterprise level is not to be worried about the tech debt, but to embrace the technology — so that is one — but find ways to make sure this is not a runaway problem, in the sense that it just becomes huge before you really realize it, because then it’s hard to manage it. So I think that balance is extremely important. Second is the ROI measurement. There is no right answer to ROI measurement today, because even with Copilots, people are struggling. Everybody wants to use Copilot, and I’m thinking Microsoft Copilot right as I speak, because many of our customers are on the Microsoft stack. But the question is, you and I use Copilot — how much productivity have we gained by that is not clearly known. But it definitely makes my life easier. In a day I use Copilot a lot. But what does it mean to my company, except for the $10 per month per seat that the company pays? It’s hard to measure. So I think those are two things that every company should start thinking about — tech debt and ROI. Companies that are leveraging AI in their software engineering need to have a clear view of how much code it’s generating and how they manage it in the future. And companies that are using AI for productivity enhancements need to start thinking about — it is going to proliferate within my company, how do I get a handle on it? How do I understand the spend and ROI? And how do I really look at and hone in on areas where it makes sense for me to use? Just because it’s available everywhere, you don’t have to use it everywhere.
David Cushman — HFS Research[20:12]
That’s great. I think there is a risk that we’re all going to be — everyone will use the tools that are available to us, and it’ll become very commoditized, and we’re going to have to think very hard about how we differentiate in those circumstances. But there’s no doubt that you don’t get to compete unless you’re in the same boat as everyone else at the moment. So I imagine the message must be, you’re going to have to bite the bullet, join in this, because otherwise you’re going to be last out of the port and first to sink potentially. So this is, I think, a really important area for firms to get excited about. I’m glad, Prem, you can help to excite them, and I’d like to thank you for your time. If you have any final message, please share that now.
Premkumar Balasubramanian — Chief Technology Officer, Hitachi Digital Services[21:04]
No, I think you summarized it very well, David. My final message is, this is real, it is happening. We need to find the best ways as enterprises to embrace it, and make sure we gain control of what’s going on, right? I think that’s kind of my message, and all of the work that we are doing is to empower our customers to do exactly that. With that, thank you for the opportunity.
David Cushman — HFS Research[21:32]
Thank you very much indeed. Take care.
