Session 5: Prompts & Model Deployment
Write prompts that work: clear instructions, few-shot examples and a defined output format. Understand system vs user prompts, compare models in the Foundry catalog on capability, cost and region, deploy and test a model, and get an overview of grounding answers in your own data.
Prompt engineering: five habits
- Be specific about the task and audience. “Summarise this” is weak. “Summarise this complaint in 3 bullet points for a branch manager, and include the amount and date” is strong.
- Give context. Paste the policy text, the product details or the customer history the model needs. It cannot know your internal data.
- Show examples (few-shot). One to five input → output examples teach format and tone far better than a description.
- Specify the output format. Ask for JSON with named keys, a table or numbered steps, and give a length limit.
- Break big tasks into steps and let the model think step by step for reasoning problems (or use a reasoning model).
Zero-shot vs few-shot
Zero-shot:
Classify the sentiment of: "Loan approved in 2 days, very happy!"
Few-shot:
Classify the sentiment as positive, negative or neutral.
Review: "App crashed during payment." -> negative
Review: "Branch timings are 10 to 4." -> neutral
Review: "Got my refund the same day, thanks!" -> positive
Review: "Loan approved in 2 days, very happy!" ->
Output format example
Extract the fields from the message below and reply with JSON only, using exactly these keys:
{"customer_name": string, "issue_type": "card"|"upi"|"loan"|"other", "amount": number|null}
Message: "Hi, this is Priya. ₹1,200 was debited twice for one UPI payment."
System prompts vs user prompts
A chat conversation is a list of messages, each with a role:
| Role | Who writes it | Purpose |
|---|---|---|
| system (also called instructions or developer message) | The app developer | Persona, rules, tone, scope, safety limits, output format: applies to the whole conversation |
| user | The end user | The actual question or task for this turn |
| assistant | The model | Previous answers, which you send back as history |
[
{"role": "system", "content": "You are a concise assistant for Contoso Bank. Answer only banking questions."},
{"role": "user", "content": "How do I reset my UPI PIN?"},
{"role": "assistant", "content": "Open the app > Profile > UPI settings > Reset PIN, then verify with your debit card."},
{"role": "user", "content": "And if I don't have my debit card?"}
]
🎯 Exam tip. Put rules and persona in the system prompt, not in every user message. The system prompt is the main lever for safe, on-brand behaviour, together with the content filters on the deployment.
A good system prompt contains: role (“You are…”), scope (what to answer, what to refuse), style (length, language, tone), safety rules (never ask for OTPs), format, and what to do when unsure (“say you don’t know and suggest the helpline”).
Choosing a model in the Foundry catalog
The model catalog lists models from Microsoft, OpenAI, Meta, Mistral, DeepSeek, Cohere, xAI and others. Compare them on:
| Factor | Where to look |
|---|---|
| Capability: chat, reasoning, vision input, audio, image generation, embeddings | The model card’s tasks, input/output modalities and context length |
| Quality | Model leaderboards and benchmark comparisons in the portal, plus your own test prompts |
| Cost | Price per million input and output tokens (or per image or minute) |
| Latency / throughput | Leaderboard speed metrics; smaller models are faster |
| Region and deployment types | The model card lists the regions and deployment types (Global Standard, Data Zone, Provisioned…) |
| Licence and hosting | Models sold directly by Azure vs partner/community models; serverless vs managed compute |
Deploy and test
- Models → pick a model → Deploy.
- Choose the deployment type and deployment name. Optionally set the tokens-per-minute (TPM) rate limit and the content filter (guardrail) configuration.
- Open in playground: set the system message, try user prompts, adjust parameters.
- Use View code in the playground to get a ready-made Python snippet with your endpoint and deployment name, which is the starting point for Session 6.
Grounding responses with your own data (overview)
Models only know their training data. To answer from your documents (policies, product sheets, FAQs) use retrieval-augmented generation (RAG):
User question → search your indexed documents (Azure AI Search / file search) → top relevant chunks
→ prompt = system rules + retrieved chunks + question → model → grounded answer (with citations)
Benefits: fresher, organisation-specific answers, fewer hallucinations, citations you can check. In Foundry you can ground a model by connecting Azure AI Search (“add your data”), or give an agent a file search tool with uploaded documents (Session 7).
Grounding vs fine-tuning: grounding adds knowledge at query time and is easy to update. Fine-tuning changes the model’s behaviour or style through extra training. For “answer from our latest policy PDF”, choose grounding.