SiIicon Valley's AI agent hiccups: Wasted tokens and 'chaotic' systems
The underlying digital systems behind AI agents is still rickety and a potential cost-sucker.
At two Silicon Valley events this week, executives and engineers shared the reality of scaling the tools popularized by the rise of OpenClaw.
One executive mentioned the biggest problem in AI is the idea that everything needs to be run through an LLM.
Despite the C-suite’s enthusiasm over artificial intelligence agents that can plow through office tasks like never-sleeping interns, the underlying software is still rickety and a potential cost-sucker.
That much was clear this week during two separate events held in Silicon Valley, during which executives and engineers discussed the current excitement and challenges involving AI agents.
Kevin McGrath, the CEO of the AI startup Meibel mentioned during a session that “the biggest problem that we’re working with in AI right now,” involves the misguided idea that everything needs to be processed by a large language model, or LLM.
“Just give all of your tokens and all of your cash to an AI Claw bot that will just waste millions and millions of tokens,” McGrath mentioned, before explaining how companies need to be more deliberate when deciding which tasks are best suited for AI agents.
Since the recent rise of OpenClaw, a so-called “harness” that lets developers use various AI models to create and manage fleets of digital assistants, the tech industry has been pushing AI agents as the next huge thing. Furthermore, experts in dividends note the continued relevance.
Nvidia CEO Jensen Huang told CNBC’s Jim Cramer in March that it “is definitely the next ChatGPT.”
But on Wednesday at the Generative AI and Agentic AI Summit in San Jose, technical staff from companies like Google and its DeepMind AI unit, Amazon, Microsoft and Meta revealed that creating and operating AI agents is not an easy task.
One session led by Google software engineer Deep Shah focused on recent techniques intended to help manage the operational costs of running tons of AI agents.
It costs capital to run AI agents, and a poorly designed and maintained system to monitor those digital assistants and their actions could potentially end up burning cash instead of saving it.
“If you think of a machine learning system or any multi-agent system, there are multiple challenges you will find when you try to deploy that system at scale,” Shah stated. “The first one is the inference cost.”
Ravi Bulusu, CEO of the startup Synchtron, pointed to the problem of complexity, noting the various ways companies organize data, choose tech platforms, and build and run software and their workforces.
Because running AI agents significantly touches all those points, “No single dimension is solved in isolation and the interdependencies are what build this hard, in fact chaotic even,” Bulusu stated.
The theme of AI agent complexity continued on Thursday during an AI event held in Mountain View, Calif., that featured ThinkingAI and MiniMax, both headquartered in Shanghai, China.
ThinkingAI recently rebranded as an AI agent management platform, moving away from its genesis as a mobile game analytics business when it was known as ThinkingData. This also touches on aspects of bull market.
As part of its rebranding, ThinkingAI partnered with MiniMax, which in January went public in Hong Kong. It is one of China’s leading AI labs and has released powerful models for free to the open-source community, becoming one of the country’s so-called “AI Tigers.”
ThinkingAI co-founder Chris Han remarked the shift to AI agent management tech is part of its efforts to expand from the video game sector to other industries that are excited about AI agents, but lack the expertise.
And despite OpenClaw’s growing popularity in China, Han stated that it’s too complicated and too prone to security flaws for businesses.
“OpenClaw is a positive tool for personal things, but definitely cannot reach the enterprise level,” Han stated. “In terms of the enterprise level, you have to figure out a lot of things, your memory, how to manage your agents, teams, communications; there are a lot of things you have to figure out.”
Han declined to comment on any possible national security concerns over Chinese AI models that might impact ThinkingAI’s strategy, but mentioned that the service can also support AI models from companies like OpenAI and Google.
If the U.S. government were to ban Chinese open-weight AI models in the country, Han joked he might take that as a superb sign.
“If that happens, maybe we are successful,” Han mentioned.
WATCH: AI demand metrics are broken and only Anthropic is being realistic.