Open in app

Sign in

Write

Sign in

Hello World to Autogen

Multiple agent LLM application development framework

Xin Cheng
9 min readOct 28, 2023

After the super-popular AutoGPT (you just ask a question and AI does the job), there is a new kid on the block, Autogen. In this article, we will use Azure OpenAI service as the LLM. How it works, use local LLM, customize agent behavior, we will leave to future articles.

TLDR

OpenAI GPT-4 is needed. OpenAI GPT-3.5 usually is stuck at parsing web page to retrieve top 10 companies or correct code to compute YTD gain.

There are lots of articles, videos to get you started, but the most important first step is to get OpenAI or LLM config right.

Azure OpenAI config

    {
        "model": "<deployment name>",
        "api_key": "<your Azure OpenAI API key here>",
        "api_base": "<your Azure OpenAI API base here>",
        "api_type": "azure",
        "api_version": "2023-07-01-preview"
    }

Question 1

assistant = autogen.AssistantAgent("assistant", llm_config={"seed": 42, "config_list": config_list, "temperature": 0})
user_proxy = autogen.UserProxyAgent("user_proxy", llm_config={"config_list": config_list},     human_input_mode="NEVER",max_consecutive_auto_reply=5,is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
    code_execution_config={
        "work_dir": "coding",
        "use_docker": False,  # set to True or image name like "python:3" to use docker
    },)

# need Alpha Vantage API key, gpt-3.5 not able to complete fully
user_proxy.initiate_chat(assistant, message="Show me the YTD gain of 10 largest technology companies as of today.")

Result

user_proxy (to assistant):

Show me the YTD gain of 10 largest technology companies as of today.

--------------------------------------------------------------------------------
assistant (to user_proxy):

To get the YTD gain of the 10 largest technology companies, we will first need to fetch the list of these companies and their stock symbols. Then, we will use a financial API to get the YTD gain for each company.

Step 1: Fetch the list of the 10 largest technology companies and their stock symbols.
Step 2: Use a financial API to get the YTD gain for each company.
Step 3: Display the results.

Let's start with Step 1. We will use BeautifulSoup and requests libraries to scrape the list of the 10 largest technology companies and their stock symbols from a webpage. Please execute the following code to install the required libraries:

```python
!pip install beautifulsoup4 requests
```

After installing the libraries, execute the following code to fetch the list of the 10 largest technology companies and their stock symbols:

```python
import requests
from bs4 import BeautifulSoup

url = "https://www.investopedia.com/top-10-technology-stocks-4582443"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

company_list = []
for item in soup.find_all("strong"):
    if item.text.endswith(")"):
        company_list.append((item.text.split(" (")[0], item.text.split(" (")[1].strip(")")))

top_10_tech_companies = company_list[:10]
print(top_10_tech_companies)
```

Once we have the list of the 10 largest technology companies and their stock symbols, we will proceed to Step 2.

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):

exitcode: 1 (execution failed)
Code output: 
  File "", line 1
    !pip install beautifulsoup4 requests
    ^
SyntaxError: invalid syntax


--------------------------------------------------------------------------------
assistant (to user_proxy):

I apologize for the confusion. I provided you with a command that should be executed in a Jupyter Notebook or Google Colab environment. Since you are not using those environments, please execute the following code to install the required libraries:

```sh
pip install beautifulsoup4 requests
```

After installing the libraries, execute the following Python code to fetch the list of the 10 largest technology companies and their stock symbols:

```python
import requests
from bs4 import BeautifulSoup

url = "https://www.investopedia.com/top-10-technology-stocks-4582443"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

company_list = []
for item in soup.find_all("strong"):
    if item.text.endswith(")"):
        company_list.append((item.text.split(" (")[0], item.text.split(" (")[1].strip(")")))

top_10_tech_companies = company_list[:10]
print(top_10_tech_companies)
```

Once we have the list of the 10 largest technology companies and their stock symbols, we will proceed to Step 2.

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is sh)...

>>>>>>>> EXECUTING CODE BLOCK 1 (inferred language is python)...
user_proxy (to assistant):

exitcode: 0 (execution succeeded)
Code output: 
Requirement already satisfied: beautifulsoup4 in /opt/anaconda3/envs/deeplearning/lib/python3.11/site-packages (4.12.2)
Requirement already satisfied: requests in /opt/anaconda3/envs/deeplearning/lib/python3.11/site-packages (2.31.0)
Requirement already satisfied: soupsieve>1.2 in /opt/anaconda3/envs/deeplearning/lib/python3.11/site-packages (from beautifulsoup4) (2.4.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/anaconda3/envs/deeplearning/lib/python3.11/site-packages (from requests) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in /opt/anaconda3/envs/deeplearning/lib/python3.11/site-packages (from requests) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/anaconda3/envs/deeplearning/lib/python3.11/site-packages (from requests) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /opt/anaconda3/envs/deeplearning/lib/python3.11/site-packages (from requests) (2023.5.7)

[]


--------------------------------------------------------------------------------
assistant (to user_proxy):

I apologize for the inconvenience. It seems that the webpage structure has changed, and the previous code is not able to extract the information correctly. Let's try a different approach.

We will use Yahoo Finance to get the list of the 10 largest technology companies and their stock symbols. Please execute the following Python code:

```python
import requests
from bs4 import BeautifulSoup

url = "https://finance.yahoo.com/sector/ms_technology"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

company_list = []
for row in soup.find_all("tr", class_="simpTblRow"):
    symbol = row.find("td", class_="data-col0").text
    name = row.find("td", class_="data-col1").text
    company_list.append((name, symbol))

top_10_tech_companies = company_list[:10]
print(top_10_tech_companies)
```

Once we have the list of the 10 largest technology companies and their stock symbols, we will proceed to Step 2.

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):

exitcode: 0 (execution succeeded)
Code output: 
[]


--------------------------------------------------------------------------------
assistant (to user_proxy):

I apologize for the inconvenience. It seems that the Yahoo Finance webpage is using JavaScript to load the data, which makes it difficult to scrape using the BeautifulSoup library. 

In this case, we will use a different approach to get the list of the 10 largest technology companies and their stock symbols. We will use the FRED (Federal Reserve Economic Data) API to get the stock symbols of the top 10 technology companies by market capitalization. Please execute the following Python code:

```python
import requests

url = "https://fred.stlouisfed.org/graph/api/series?ids=SP500IT&api_key=YOUR_API_KEY"
response = requests.get(url)
data = response.json()

top_10_tech_companies = data["seriess"][0]["tags"][0]["notes"].split("\n")[1:11]
top_10_tech_companies = [(company.split(" ")[0], company.split(" ")[1]) for company in top_10_tech_companies]
print(top_10_tech_companies)
```

Please replace `YOUR_API_KEY` with your FRED API key. You can obtain a free API key by registering at https://research.stlouisfed.org/docs/api/api_key.html.

Once we have the list of the 10 largest technology companies and their stock symbols, we will proceed to Step 2.

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):

exitcode: 1 (execution failed)
Code output: 
Traceback (most recent call last):
  File "/opt/anaconda3/envs/deeplearning/lib/python3.11/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/deeplearning/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/deeplearning/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/deeplearning/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 5, in <module>
    data = response.json()
           ^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/deeplearning/lib/python3.11/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)


--------------------------------------------------------------------------------
assistant (to user_proxy):

I apologize for the inconvenience. It seems that the FRED API is not providing the expected data. Let's try another approach using the Alpha Vantage API to get the stock symbols of the top 10 technology companies by market capitalization.

Please execute the following Python code:

```python
import requests

url = "https://www.alphavantage.co/query?function=SECTOR&apikey=YOUR_API_KEY"
response = requests.get(url)
data = response.json()

top_10_tech_companies = data["Rank A: Market Capitalization"]["Information Technology"][:10]
print(top_10_tech_companies)
```

Please replace `YOUR_API_KEY` with your Alpha Vantage API key. You can obtain a free API key by registering at https://www.alphavantage.co/support/#api-key.

Once we have the list of the 10 largest technology companies and their stock symbols, we will proceed to Step 2.

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):

exitcode: 1 (execution failed)
Code output: 
Traceback (most recent call last):
  File "", line 7, in <module>
    top_10_tech_companies = data["Rank A: Market Capitalization"]["Information Technology"][:10]
                            ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'Rank A: Market Capitalization'


--------------------------------------------------------------------------------
assistant (to user_proxy):

I apologize for the inconvenience. It seems that the Alpha Vantage API is not providing the expected data either. As a last resort, I will provide you with a manually curated list of the top 10 technology companies and their stock symbols. This list may not be up-to-date, but it should be a good starting point.

```python
top_10_tech_companies = [
    ("Apple Inc.", "AAPL"),
    ("Microsoft Corporation", "MSFT"),
    ("Alphabet Inc.", "GOOGL"),
    ("Amazon.com, Inc.", "AMZN"),
    ("Facebook, Inc.", "FB"),
    ("Tesla, Inc.", "TSLA"),
    ("NVIDIA Corporation", "NVDA"),
    ("Adobe Inc.", "ADBE"),
    ("PayPal Holdings, Inc.", "PYPL"),
    ("Cisco Systems, Inc.", "CSCO")
]

print(top_10_tech_companies)
```

Once we have the list of the 10 largest technology companies and their stock symbols, we will proceed to Step 2.

--------------------------------------------------------------------------------

It finds the 10 largest technology companies, but it stucks at using alphavantage API (althoug yfinance can also do it like below)

Question 2

user_proxy.send(
    recipient=assistant,
    message="""Plot a chart of META and TESLA stock price change YTD and save to stock_price_ytd.png.""",
)

Result

user_proxy (to assistant):

Plot a chart of META and TESLA stock price change YTD and save to stock_price_ytd.png.

--------------------------------------------------------------------------------
assistant (to user_proxy):

To plot a chart of the stock price change YTD for Meta (previously Facebook) and Tesla, and save it to a file, we will use the `yfinance` library to fetch the stock data and the `matplotlib` library to create the chart. Please make sure you have already installed the required libraries as mentioned in the previous response.

Execute the following Python code to fetch the stock data, plot the chart, and save it to a file:

```python
import yfinance as yf
import matplotlib.pyplot as plt
import datetime

selected_companies = [
    ("Meta Platforms, Inc.", "META"),
    ("Tesla, Inc.", "TSLA")
]

# Fetch stock data
start_date = datetime.datetime(datetime.date.today().year, 1, 1)
end_date = datetime.date.today()
stocks_data = yf.download([symbol for _, symbol in selected_companies], start=start_date, end=end_date)

# Calculate YTD percentage change
ytd_percentage_change = (stocks_data["Adj Close"].iloc[-1] - stocks_data["Adj Close"].iloc[0]) / stocks_data["Adj Close"].iloc[0] * 100

# Plot the chart
fig, ax = plt.subplots(figsize=(10, 6))
ytd_percentage_change.plot(kind="bar", ax=ax)
ax.set_ylabel("YTD Percentage Change")
ax.set_xlabel("Stock Symbol")
ax.set_title("YTD Stock Price Change for Meta and Tesla")
plt.xticks(range(len(selected_companies)), [symbol for _, symbol in selected_companies], rotation=45)

# Save the chart to a file
plt.savefig("stock_price_ytd.png", bbox_inches="tight")
print("Chart saved to stock_price_ytd.png")
```

This code will fetch the stock data for Meta and Tesla, calculate the YTD percentage change, plot the chart, and save it to a file named "stock_price_ytd.png".

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):

exitcode: 0 (execution succeeded)
Code output: 

[                       0%%                      ]
[*********************100%%**********************]  2 of 2 completed
Chart saved to stock_price_ytd.png


--------------------------------------------------------------------------------
assistant (to user_proxy):

Great! The code has successfully fetched the stock data for Meta and Tesla, calculated the YTD percentage change, plotted the chart, and saved it to a file named "stock_price_ytd.png". You can now view the chart in the saved file.

If you have any further questions or need assistance with another task, feel free to ask. Otherwise, if everything is done, please reply with "TERMINATE".

--------------------------------------------------------------------------------
user_proxy (to assistant):

You're welcome! I'm glad I could help. If you need assistance with any other tasks or have questions in the future, don't hesitate to ask. Have a great day!

--------------------------------------------------------------------------------
assistant (to user_proxy):

TERMINATE

Troubleshooting

Test if Azure OpenAI service is accessible (especially behind corpnet)

import os
import requests
import json
import openai

openai.api_key = os.getenv("AZURE_OPENAI_KEY")
openai.api_base = os.getenv("AZURE_OPENAI_ENDPOINT")
openai.api_type = 'azure'
openai.api_version = '2023-05-15' # this may change in the future

deployment_name='REPLACE_WITH_YOUR_DEPLOYMENT_NAME' #This will correspond to the custom name you chose for your deployment when you deployed a model. 

# Send a completion call to generate an answer
print('Sending a test completion job')
start_phrase = 'Write a tagline for an ice cream shop. '
# 2023-12-11, found for ChatCompletion, you need to specify role, {"messages":[{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},{"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},{"role": "user", "content": "Do other Azure AI services support this too?"}]}, but for completion model, plain text is fine
response = openai.ChatCompletion.create(engine=deployment_name, prompt=start_phrase, max_tokens=10)
text = response['choices'][0]['text'].replace('\n', '').replace(' .', '.').strip()
print(start_phrase+text)

Quickstart - Deploy a model and generate text using Azure OpenAI Service - Azure OpenAI

Walkthrough on how to get started with Azure OpenAI and make your first completions call.

learn.microsoft.com

How to switch between OpenAI and Azure OpenAI Service endpoints with Python - Azure OpenAI Service

Learn about the changes you need to make to your code to swap back and forth between OpenAI and Azure OpenAI endpoints.

learn.microsoft.com

Azure OpenAI Service REST API reference - Azure OpenAI

Learn how to use Azure OpenAI's REST API. In this article, you'll learn about authorization options, how to structure a…

learn.microsoft.com

curl example

chat completion

curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2023-05-15 \
  -H "Content-Type: application/json" \
  -H "api-key: YOUR_API_KEY" \
  -d '{"messages":[{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},{"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},{"role": "user", "content": "Do other Azure AI services support this too?"}]}'

completion

curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/completions?api-version=2023-05-15\
  -H "Content-Type: application/json" \
  -H "api-key: YOUR_API_KEY" \
  -d "{
  \"prompt\": \"Once upon a time\",
  \"max_tokens\": 5
}"

Appendix

Simpler tasks without requiring to many tools (e.g. web)

Microsoft's AutoGen - A guide to code-executing agents

Read the latest insights from our journey | E2B offers sandboxed cloud environments for AI agents & AI apps with a…

e2b.dev

Building AI Agent Applications Series - Using AutoGen to build your AI Agents

In the previous content, we learned about AI Agent. If you didn't read it, please read my previous content …

techcommunity.microsoft.com

Microsoft Semantic Kernel and AutoGen: Open Source Frameworks for AI Solutions

Explore Microsoft's open-source frameworks, Semantic Kernel and AutoGen. Semantic Kernel enables developers to create…

techcommunity.microsoft.com

Capabilities

cache, autogen creates .cache/<seed> to save conversation history, save time and cost (e.g. does not need to hit LLM endpoint again and again for same input)

enhanced inference: hyperparameter tuning for code generation

RAG RetrieveAssistantAgent, RetrieveUserProxyAgent, customizing the embedding function, the text split function and vector database.

Artificial Intelligence
Large Language Models
Machine Learning

--

--

Xin Cheng

Written by Xin Cheng

367 Followers

Multi/Hybrid-cloud, Kubernetes, cloud-native, big data, machine learning, IoT developer/architect, 3x Azure-certified, 3x AWS-certified, 2x GCP-certified

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams