Agentplex Weekly - Issue #8
What is an AI Agent, Jul 2024? Building a Auto-Analyst Agent. Reliable AI Agents with LangGraph. Mixture of Agents Beats GPT-4o. The New llama-agents. Deep Dive into AutoGen.
What is an AI Agent, July 2024? People ask me this frequently; a question very relevant for even experienced AI/ ML engineers and devs who are sceptical about AI Agents. It’s a very valid, broad, not simple question because it involves many key aspects of AI. And the answer is: I’m not sure… there isn’t a clear cut answer. There are several definitions and many interpretations of what’s an AI agent is. And the term is constantly evolving… Let’s see:
Early AI days: Agents, sensors & actuators. In the early days of AI, first Alan Turing -in the Turing Test- tried to define an AI [agent] as a machine that has intelligent behaviour indistinguishable from a human. This is still an ongoing challenge but some frontier models are claiming this has been achieved in several benchmarks.
Norbert Weiner in his influential book Cybernetics (free MIT pdf edition) and other cybernetics researchers like Stafford Beer, defined an agent as a model for an autonomous system capable of producing itself.
Then years later, Russell & Norvig in their authoritative, text book AI: A Modern Approach (pdf, 1.5k pages), defined intelligent agents as:
An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors
Well, if you think about it, a thermostat would qualify as an agent under that definition!
Cybernetic agents in the early 2000’s. Several cybernetics researchers continued to define many AI systems patterns like: control feedback, sense/actuate, black-box, feedforward, self-reflection, autopoiesis, viable autonomous systems… patterns which today are at the core of “modern AI” and AI Agents. In the early 2000’s, a group of researchers introduced a cybernetic architecture of practical reasoning agents.
AI Reinforcement Learning (RL) Agents and DeepMind, 2017. RL Agents are based on training a program that make decisions based on reward, search, optmisation and Markov algorithms. RL Agents don’t have a priori world knowledge, and mimic the human trial-and-error learning & self-teaching processes. RL Agents are very good in environments like games. DeepMind made some notable AI breakthroughs by combining RL Agents with DeepNNs and Monte-Carlo Tree search algos winning at chess and go games.
Since 2017: AI Agents based on LLMs/ LMMs. AI has been totally dominated by massive R&D and investment in transformers, Large Language Models (LLMs) and Large-Multimodal Models (LMMs). And so when people refer to AI Agents these days, they mainly mean LLM/LMM-based agents. Due to the way LLMs & LMMs work, several new core elements are introduced like for example: Prompt engineering.
Unlike RL Agents, these models require to be instructed with natural language prompts. Also unlike RL Agents, LM-based agents can perform external search in DBs, function calls with other external agents or programs, and run many different tasks based on pre-defined promtps like CoT, ReAct, Sul-Reflect, ToT… If you want to develop these type of AI agents you should master prompt engineering. Checkout this massive survey with +50 prompts and patterns used in AI agents: The Prompt Report: A Systematic Survey of Prompting Techniques.
Recently: Agentic systems instead of agents. Within the context of LLMs and LMMs, several researchers have tried to formalise the AI community’s popular understanding of what is an AI agent. Andrew Ng recently posted about agentic systems, agentic workflows, and the agentic spectrum instead of a binary definition of an agent.
Andrew didn’t go deep into the detail, so: What defines the levels of an agentic system?
• Environment and goals. The more complex the environment in which the agent operates (e.g. random changes, disparate tasks and domains, long time planning horizon ) the more agentic
• UI and supervision. AI systems that can be instructed in natural language (prompts) and act autonomously are more agentic. The less user supervision needed, the more agentic the AI system is
• AI system design: Systems that use tools (e.g., web search, DBs, code plug-ins) and follow AI patterns (e.g. CoT, ReAct, ToT) are more agentic. Systems whose control flow is dynamically driven/ routed by an LLM are more agentic
July 2024: What’s an AI Agent? The CEO @LangChain responds. LangChain is quickly pivoting to graph-based, LLM-based agents with LangGraph. Harrison says that he gets asked this question almost daily! His controversial technical answer:
An agent is a system that uses an LLM to decide the control flow of an application.
Some hardcore AI/ ML engineers and devs went up in arms, LOL! He admits his definition is not perfect. He argues people in the AI community have different perceptions of what an agent should be capable of, and often think of agents as advanced, autonomous, and human-like. He also explains why he likes the terms agentic and agentic systems. Read more here: What is an agent? Introducing a new series of musings on AI agents.
In any case, whether it’s an AI Agent, a RL Agent, a cybernetic agent, or an LLM-based agent, I guess the most important thing is to build reliable, productive agents that improve our daily lives and work, and contribute to a better future of humanity.
Announcing our first AI Agents meetup in London, July 17. We are hosting our first meetup at UCL London on July 17! Come and join us to learn about Camel & AutoGen AI Agents frameworks, and the AI Agents Global Challenge. If you are interested in giving a talk or do a demo at our meetups, please contact Carlos here.
Hands-on, tutorials and practical guides
Building Auto-Analyst Agent. This is a technical guide on how to build a data analytics AI agent with DSPy. It shows how a planner agent orchestrates the data processing, ML, stats, and dataviz agents to perform auto data analysis
How to build and test reliable AI agents. A nice video tutorial on how to use LangGraph to build agents, and a comparison between a custom LangGraph agent to a ReAct agent for RAG
How? Mixture of Agents (MoAs) faster, better than GPT-4o. A great video tutorial on how to implement open-source MoAs from TogetherAI using super-fast inference with Groq.ai
Tools, platforms, and frameworks
llama-agents just released! (repo & examples.) LlamaIndex just released this async-first framework for building, iterating, and productionising multi-agent systems, including multi-agent communication, distributed tool execution, human-in-the-loop, and more! Key point: Each agent is now a micro-service
A platform for organising the world's AI Agents. According to Hyperspace, aiOS is the world's largest peer-to-peer AI network for agents. Apparently, you can start earning “points” with your agent if you publish it in this platform
A practical deep dive into the AutoGen platform. This blog post goes deep into reviewing the AutoGen paper and the AutoGen project. The blog provides conversational coding, RAG, and multi-agent examples wit AutoGen
Startups & VC Investments
AI Agents are having a ‘ChatGPT moment’ as investors look for what’s next after chatbots
Norm.ai - AI agents for regulatory compliance
Rafa.ai - Build wealth with a team of AI agents
Hippocratic.ai - Role-based GenAI agents for healthcare
Altera.ai - We build AI agents for gaming, simulation and beyond
Research papers
Tree search algo for LM-based agents beats GPT-4o. Researcher at CMU propose a new inference-time search algo that addresses the planning challenges of LM agents. The algo performs exploration and multi-step planning in interactive web environments. Researchers claim that the algo beats GPT-4o in several benchmarks. Paper &r repo: Tree Search for Language Model Agents.
Solving the TSP with multi AI agents, vision reasoning, and GPT-4oV. Fascinating paper. Can we solve the TSP with vision reasoning agents? The Traveling Salesperson Problem is a classic, NP-hard problem in combinatorial & graph optimisation. In this paper, researchers introduce a new multi-agent AI model that improves the TSP solution. Paper: Visual Reasoning and Multi-Agent Approach in Multimodal LLMs (MLLMs): Solving TSP and mTSP Combinatorial Challenges.
An update on The AI Agents Global Challenge (AAGC) (link)
Based on the feedback received from participants, we’re announcing new important details:
New deadline: November 15, 2024. Remember: You can keep refining and updating your application until the deadline.
Re-allocation of the $1M Prize pool into uncapped SAFE investments plus adding free compute credits. You don’t need a company to submit an application
Introduction to Agentplex Ventures team for possible additional investment
A better definition of the challenge and 5 business domains
A list of 45 tentative application areas for developing AI Agents within the above domains (see FAQs)
A new set of 8 clearly defined judging evaluation criteria
New FAQs answering all the questions we have received
New AI Agents Global Challenge website with improved usability and access to key information
More ways to contact us and get updates via our social channels.
Join our Discord for Q&As, discuss ideas, and meet others building AI Agents
Any doubts? Questions? Send us an email.
Are you building an AI agent? Compete in the AI Agents Global Challenge funded with a $1 Million pool prize. It’s easy to apply, free and there is no registration fee. Click here to learn about the challenge and how to apply.
If you are interested in a summer internship, please submit an email here to apply.
Thank you for reading Agentplex newsletter. Have a great day.