Session 6: Chat App with the Foundry SDK

Connect Python to your Foundry project with keys or keyless sign-in, build a lightweight chat client with the Foundry SDK, keep conversation history across turns, handle errors, rate limits and token usage, and optionally put a Streamlit front end on it.

The pieces

Install the packages (Python 3.10+):

pip install "azure-ai-projects>=2.3.0" azure-identity openai python-dotenv

Option 1: keyless authentication (recommended)

The Foundry SDK (azure-ai-projects) uses Microsoft Entra ID through DefaultAzureCredential. Run az login first, and make sure your account has a role such as Azure AI User on the project. The project client gives you a ready OpenAI client:

import os
from dotenv import load_dotenv
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

load_dotenv()

project = AIProjectClient(
    endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential(),
)
client = project.get_openai_client()          # OpenAI-compatible client for this project

response = client.responses.create(
    model=os.environ["FOUNDRY_MODEL_NAME"],      # the DEPLOYMENT name
    instructions="You are a helpful assistant for Contoso Bank. Keep answers short.",
    input="What is a fixed deposit?",
)
print(response.output_text)

DefaultAzureCredential tries several sign-in methods in order: environment variables, a managed identity (when running in Azure), the Azure CLI login, VS Code and others. The same code works on your laptop and in production.

Option 2: API key

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(
    base_url=os.environ["FOUNDRY_OPENAI_ENDPOINT"],   # https://<resource>.openai.azure.com/openai/v1/
    api_key=os.environ["FOUNDRY_API_KEY"],
)
response = client.responses.create(model=os.environ["FOUNDRY_MODEL_NAME"], input="Hello!")
print(response.output_text)

Keys are quick for labs, but store them only in .env or Azure Key Vault, never in code or Git, and rotate them if they leak.

Responses API vs Chat Completions

Foundry supports both OpenAI-style APIs. The Responses API (client.responses.create) is the newer one: it takes instructions and input, returns output_text, can keep state with previous_response_id, and supports tools. The older Chat Completions API (client.chat.completions.create(messages=[...])) returns choices[0].message.content. Know both shapes for the exam.

Managing conversation history

A model is stateless: it remembers nothing between calls unless you send the history.

Approach A: send the full message list yourself

history = [{"role": "system", "content": "You are Asha, Contoso Bank's FAQ assistant. Answer in under 80 words."}]

while True:
    user = input("You: ").strip()
    if user.lower() in {"quit", "exit"}:
        break
    history.append({"role": "user", "content": user})
    response = client.responses.create(model=os.environ["FOUNDRY_MODEL_NAME"], input=history)
    answer = response.output_text
    history.append({"role": "assistant", "content": answer})
    print("Asha:", answer)

Approach B: let the service chain turns

first = client.responses.create(model=MODEL, input="What is the size of France in square miles?")
follow = client.responses.create(model=MODEL, input="And what is the capital city?",
                                 previous_response_id=first.id)     # the service keeps the context
print(follow.output_text)

Long conversations grow the token count (and cost) on every turn. Keep the last N turns, or summarise older turns into one message, so you stay inside the context window.

Errors, rate limits and token usage

Each deployment has a tokens-per-minute (TPM) and requests-per-minute (RPM) quota. Exceeding it returns HTTP 429 (Too Many Requests). Retry with exponential backoff:

import time
from openai import RateLimitError, APIConnectionError, APIStatusError

def ask(messages, retries=4):
    for attempt in range(retries):
        try:
            r = client.responses.create(model=MODEL, input=messages, max_output_tokens=400)
            u = r.usage
            print(f"[tokens] in={u.input_tokens} out={u.output_tokens} total={u.total_tokens}")
            return r.output_text
        except RateLimitError:
            wait = 2 ** attempt
            print(f"Rate limited, retrying in {wait}s...")
            time.sleep(wait)
        except APIConnectionError:
            print("Network problem, retrying...")
            time.sleep(2)
        except APIStatusError as e:          # 400 (bad request / content filter), 401, 404...
            print("Request failed:", e.status_code, e.message)
            return None
    return None

Common errors:

• 401 / 403: wrong key, or your identity lacks a role on the project.
• 404: wrong endpoint or deployment name.
• 400 with content filter: the prompt or answer was blocked by the deployment’s guardrails.
• 429: rate limit. Back off, or raise the deployment’s TPM quota.

Optional: a Streamlit front end

# app.py  ->  run with:  streamlit run app.py
import os, streamlit as st
from dotenv import load_dotenv
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

load_dotenv()
MODEL = os.environ["FOUNDRY_MODEL_NAME"]

@st.cache_resource
def get_client():
    project = AIProjectClient(endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], credential=DefaultAzureCredential())
    return project.get_openai_client()

st.title("🏦 Contoso Bank Assistant")
if "history" not in st.session_state:
    st.session_state.history = [{"role": "system", "content": "You are a concise, polite bank FAQ assistant."}]

for m in st.session_state.history[1:]:
    st.chat_message(m["role"]).write(m["content"])

if prompt := st.chat_input("Ask about accounts, cards or UPI"):
    st.session_state.history.append({"role": "user", "content": prompt})
    st.chat_message("user").write(prompt)
    answer = get_client().responses.create(model=MODEL, input=st.session_state.history).output_text
    st.session_state.history.append({"role": "assistant", "content": answer})
    st.chat_message("assistant").write(answer)

Check yourself