Why Intelligent Agents are the Future of Efficient Business Architectures

The Limited Value of Large Language Models in Business Applications

The power of large language models (also known as generative AI language models) such as OpenAI GPT-4 has been increasing at a dramatic rate. So too has the hype about the revolutionary improvements in workforce efficiency that these models are predicted to bring about. Interest by governments and the private sectors has been intense, but to date, examples of successful enterprise-wide implementations are curiously sparse. While over 81% of CIOs in a recent poll believe that generative AI holds promise in gaining efficiencies within their organizations, less than 22% report having successfully integrated generative AI in their core business processes. So why isn’t reality matching the hype?

The answer is that LLMs alone are not equipped to provide human-like assistance with real-world business processes. Out-of-the-box, they can respond to direct user prompts but only using the data they have been trained on. Ask a chatbot how to develop a plan for your new project and it will give you a detailed but generic answer. Yet ask it to plan specific project activities or update your project status report and it will draw a blank. While chatbots are trained on a great deal of project management knowledge and practice, they have no knowledge of your particular project. Their knowledge is static and limited to what they were provided during training. Moreover, out-of-the-box LLMs retain no memory of what you may have told them about your project in previous interactions. Neither do they understand your goal in interacting with them.

To be useful as intelligent assistants, LLMs need to display the same sorts of characteristics that would make a good human assistant. Specifically, you would want an effective assistant to have the following traits:

• Goal - Orientation. An intelligent assistant should pursue specific goals or objectives and be able to understand whether they are currently on the path to achieving their goal.

• Autonomy of Action. An intelligent assistant should be able to make independent choices and decisions based on their understanding of their goal and the actions necessary to achieve it.

• Perception. An intelligent assistant should be able to see the effect of its actions on its environment and evaluate the effect of these actions relative to their goal.

• Knowledge and Memory. An intelligent assistant should be equipped with good domain knowledge to equip them to achieve their goal as well as the ability to recall details from past interactions and actions.

• Reasoning and Decision Making. Intelligent assistants should be able to analyze situations, weigh options, and use logic and reasoning to make decisions that make progress towards the goal.

Large Language Model to Intelligent Agent

 An intelligent agent is what results when you give LLMs the ability to sense and interact with the outside world. Developers provide this ability through layering on functionality such as the addition of external memory, access to tools, and cleverly designed prompts that instruct the LLM to process the information it receives in specific ways. With this added functionality, LLMs are transformed into intelligent agents that can sense and act outside of their static model and learn from their interactions. With cleverly designed add-ons, they can be given roles as analyst, editor, planner, or researcher and can be given specific goals to complete. These goals may be routine and administrative with a high level of autonomy, such as waiting for new information and then parsing it into given categories. Intelligent agents may be designed to act in a higher-level capacity such as a strategist that is required to gather and analyze multiple variables in preparation for creating an organization's action plan. Such agents would typically be designed to achieve their goal incrementally through many interactions with a human counterpart. 

Other uses include cognitive-repetitive processes that involve a chain of humans who each use their individual knowledge and experience to analyze information before composing conclusions and passing the results to the next step. These can be expensive and time-intensive processes that are dependent on experienced human workers each completing a combination of skilled and mundane work. Intelligent agents have the potential to assist the human component by, for example, completing the mundane portions automatically while working interactively with the user to complete the higher-skilled portions.

Overall, the benefits of intelligent agents are compelling. Human experts would no longer be limited by time-consuming bottlenecks in their business processes, and businesses could benefit across several metrics such as:

• Increased Efficiency and Productivity:  Intelligent agents can handle routine and repetitive tasks, freeing up human workers to focus on areas where human insight and thinking are needed.

• Faster Decision Making: Intelligent agents can process large volumes of data quickly and provide insights and recommendations, accelerating decision-making processes.

• Resource Optimization: Intelligent agents can mitigate resource gaps or weakness in business workflows maintain efficiency.

• Cost Savings: Intelligent agents can reduce operational budget through cost reductions from reduction in errors and repeat work.

• Scalability:  Intelligent agents can scale existing operations without a corresponding increase in costs, allowing businesses to grow more efficiently or for governments to expand the scope of service offerings.

• Enhanced Accuracy and Consistency: Intelligent agents can help ensure consistent application of business rules, processes, and compliance standards leading to more reliable outcomes

Where we Are Now

 There is an immense amount of effort focused on productizing agents for the workplace. In recent weeks, vendors such as OpenAI, Anthropic, Microsoft, and Google have each made major upgrades to their base LLM offerings to make integration easier with intelligent agents. With OpenAI, users can build custom "Assistants" that can supplement GPT-4 with additional knowledge in the form of documents and data. OpenAI Assistants can also be instructed to customize their responses by giving them explicit descriptions of the role they are to play, such as data analyst or researcher. OpenAI also allows their models to operate in different modes depending on whether their responses are for the purpose of issuing instructions to a tool or communicating with a human user.

There are also open-source software frameworks such as LangChain, AutoGen, CrewAI, and others. These frameworks enable developers to use pre-built functions and LLM interactions to assemble sophisticated agent structures that they can integrate with existing corporate applications and security.

Despite this activity, issues with intelligent agents remain. Chief among these are reliability and accuracy. While intelligent agents achieve good accuracy on short one- or two-step processes, they currently struggle with successful goal completion as the number of process steps increases. In multi-step processes where human completion accuracy is around 78%, the best intelligent agents currently achieve about 34%. A higher level of accuracy is still needed to evoke the confidence needed for most enterprise use cases. 

The Path Forward

 Intelligent agent research is making rapid progress. Vendors are increasingly incorporating agentic design elements such as reasoning, planning, memory, and multi-modal capabilities directly into their LLM platforms. For example, OpenAI's latest model, GPT-4o, has a desktop mode in which GPT and the user can interactively cooperate to analyze elements on the user's desktop such as graphs, documents, and slides. And Microsoft has just announced an updated version of its AI CoPilot product that promises to simplify the prototyping of intelligent agents by simply observing a human expert at work.

A key takeaway is that AI agent capabilities are moving rapidly toward human-level competence while at the same time being made easier to adapt to individual use cases. Within a relatively short timeframe, it is likely that intelligent agents will be as indispensable a part of the workplace as computers themselves. AI luminaries such as Sam Altman, Elon Musk, and Bill Gates have individually commented on intelligent agents as logical extensions of current AI technology. What does seem certain is that organizations looking to maintain their value propositions into the future are best advised to begin studying scenarios for intelligent agents within their organizations now. Forward-looking organizations that take proactive steps today will be better positioned to leverage these advanced tools effectively, ensuring they remain competitive and efficient in an increasingly AI-driven world.

Originally published on LinkedIn, May 30, 2024.