Session 2: Responsible AI & How Generative Models Work
The six Microsoft responsible AI principles applied to real scenarios, then how large language models actually produce text: tokens, embeddings, context windows, model choice, deployment options and the parameters that change every answer.
The six responsible AI principles
Microsoft builds and judges AI solutions against six principles. The exam gives a scenario and asks which principle is involved, so learn the trigger words for each.
| Principle | The question it asks | Trigger words in a scenario |
|---|---|---|
| Fairness | Does it treat all people equally? | bias, discriminate, gender, age, caste, region, “approves fewer loans for…” |
| Reliability and safety | Does it work correctly and fail safely, even in unusual conditions? | testing, edge cases, errors, “must not give harmful medical advice”, monitoring |
| Privacy and security | Does it protect personal data and resist misuse? | personal data, consent, PII, encryption, data retention, attacks |
| Inclusiveness | Does it empower everyone, including people with disabilities? | accessibility, screen readers, captions, multiple languages, low literacy |
| Transparency | Can people understand how and why it works? | explain the decision, “users must know they are talking to AI”, documentation, limitations |
| Accountability | Who is responsible for it? | human oversight, governance, audit, “a person must be able to override” |
🎯 Exam tip. Transparency is about understanding the system (explanations, disclosure, documenting limitations). Accountability is about people being responsible (governance, oversight, ownership). These two are the most often confused.
Applying the principles
Hiring. A CV-screening model rejects more women because past hires were mostly men → fairness. Fix: test outcomes by group, remove proxy features, keep a human reviewer.
Lending. A customer is declined and asks why → transparency (give reason codes). The bank must name an owner for the model and review its decisions → accountability. Applicants’ income data must be encrypted and not reused without consent → privacy and security.
Healthcare chatbot. It must never suggest a dangerous dose and must hand over to a doctor when unsure → reliability and safety. It should support voice input and regional languages for elderly patients → inclusiveness. Patients must be told they are chatting with an AI → transparency.
In Microsoft Foundry these principles show up as concrete features: content filters (guardrails) on model deployments, evaluations to test quality and safety, and documentation such as transparency notes.
How generative language models work
- Tokens. Text is split into tokens: whole words, word pieces or punctuation. Roughly, 1 token ≈ 4 characters of English ≈ ¾ of a word. Pricing and limits are counted in tokens.
- Embeddings. Each token becomes a long vector of numbers. Words with similar meaning get vectors close together (“bank loan” is near “credit”, far from “river bank” in context).
- Transformer. Layers of attention let every token look at every other token to understand context.
- Next-token prediction. The model outputs a probability for every possible next token, picks one, appends it and repeats. A response is generated one token at a time.
- Context window. The maximum number of tokens the model can consider at once: the system prompt, conversation history, your documents and the answer. Anything beyond it is cut off.
A language model does not look facts up in a database. It predicts plausible text, so it can hallucinate: confidently state something false. Grounding with your own data, which you will see in Session 5, reduces this.
Embeddings power semantic search. Convert documents and questions to embeddings and find the closest vectors. That is the “retrieval” in retrieval-augmented generation (RAG).
Choosing a model by capability
| Need | Choose |
|---|---|
| General chat, writing, summarising | A language (chat) model, such as the GPT family, Llama, Mistral or Phi |
| Understand images and text in one prompt | A multimodal model that accepts image input, such as GPT-4o or the GPT-5 family |
| Speech in or out | A speech-capable model (such as the GPT-4o audio models) or Azure Speech in Foundry Tools |
| Create images | An image-generation model, such as GPT-Image-1 or FLUX |
| Semantic search | An embedding model, such as text-embedding-3 |
| Complex multi-step reasoning, maths, code | A reasoning model (o-series, GPT-5): slower and costlier, but stronger |
| Low cost, low latency, on a device or at the edge | A small language model (SLM) such as Phi |
Small vs large. Large models are more capable but cost more per token and respond more slowly. Small models are cheaper and faster, and often good enough for focused tasks like classification or extraction. Compare candidates on the model leaderboard and with your own test prompts in the Foundry portal.
Deployment options in Microsoft Foundry
You never call a model in the catalog directly: you first create a deployment, which gets a deployment name that your code uses.
| Option | What it means | When to use |
|---|---|---|
| Global Standard | Pay per token; Azure routes traffic to any region with capacity | Default choice: highest availability, lowest friction |
| Data Zone Standard | Pay per token; processing stays inside a geographic zone (for example the EU or the US) | Data-residency requirements |
| Standard (regional) | Pay per token; processed in your resource’s region | Strict single-region needs |
| Provisioned (PTU) | Reserve throughput capacity for a fixed price | High, steady, predictable traffic needing guaranteed latency |
| Batch | Submit large jobs that complete asynchronously at lower cost | Offline bulk processing (classify a million reviews overnight) |
| Serverless API / managed compute | For partner and open models: pay per token, or host on VMs you choose | Models outside the Azure OpenAI family |
Configuration parameters
| Parameter | Effect | Typical setting |
|---|---|---|
| System prompt (instructions) | Sets the role, tone, rules and format for the whole conversation | “You are a polite bank FAQ assistant. Answer only about our products.” |
| Temperature (0–2) | Randomness. Low = focused and repeatable, high = creative and varied | 0–0.3 for facts and extraction; 0.7–1.0 for creative writing |
| Top-p (0–1) | Samples only from the smallest set of tokens whose probabilities add up to p | Adjust temperature or top-p, not both |
| Max tokens (max output tokens) | Upper limit on response length; controls cost and verbosity | Enough for the expected answer |
| Stop sequences | Text that ends generation | "\n\n", "END" |
| Frequency / presence penalty | Discourage repeating the same words or topics | Small positive values for less repetition |
Reasoning models (o-series, GPT-5 family) do not use temperature; they offer a reasoning effort setting instead. To experiment with temperature, use a non-reasoning chat model.