Session 3: Generative, Agentic, Text & Speech Workloads
Learn to recognise AI workloads from a business description: when generative AI is enough and when you need an agent, the classic text-analysis techniques (key phrases, entities, sentiment, summarisation, language and PII detection, translation), and speech recognition and synthesis.
How the exam asks it. “A company wants to … Which AI workload should they use?” Read the verb in the scenario: create → generative, act / complete a task with tools → agentic, detect / extract / classify text → text analysis, transcribe / speak → speech.
Generative AI vs agentic AI
| Generative AI app | Agent | |
|---|---|---|
| What it does | Produces content in response to a prompt | Pursues a goal: reasons, chooses and calls tools, may take several steps |
| Building blocks | Model + prompt (+ optional grounding data) | Model + instructions + tools + knowledge |
| Typical tasks | Draft an email, summarise a report, answer FAQs, write code | Look up an order in a system, check policy, create a ticket, book a slot, run calculations |
| Risk | Wrong or made-up text | Wrong actions, so tool permissions and human approval matter |
An agent is the right answer when the task needs live data or actions (APIs, databases, files), multiple steps with decisions in between, or calculations the model should not do “in its head”. Plain generative AI is enough when the answer can come from the prompt and the model’s knowledge, or from documents you add to the prompt.
Examples:
- “Generate five subject lines for our Diwali campaign” → generative AI.
- “When a customer reports a lost card, verify identity, block the card in the core banking system and send a confirmation” → agent (tools + steps).
- “Answer employee questions from the HR policy PDF” → generative AI with grounding, or an agent with a file search tool.
Text analysis techniques
These are the capabilities of Azure Language in Foundry Tools. A general LLM can also do them through prompting; the exam expects you to know what each technique returns.
| Technique | What it returns | Example input → output |
|---|---|---|
| Key phrase extraction (keyword extraction) | The main talking points | “The UPI app crashes during payment and support never replies” → UPI app, payment, support |
| Named entity recognition (NER) | Entities with categories: Person, Location, Organization, DateTime, Quantity… | “Ravi flew to Pune on 3 May” → Ravi (Person), Pune (Location), 3 May (DateTime) |
| Entity linking | Entities matched to a knowledge base (for example a Wikipedia URL) | “Mercury” → the planet or the element, decided by context |
| Sentiment analysis | Positive / negative / neutral / mixed, with confidence scores per document and sentence | “Great rates but terrible app” → mixed |
| Opinion mining | Sentiment about specific aspects | rates → positive, app → negative |
| Summarisation | Extractive (picks key sentences) or abstractive (writes a new summary) | Long complaint → two-line summary |
| Language detection | The language and a confidence score | “Mera account block ho gaya” → Hindi (romanised) or a mixed result |
| PII detection | Personal data found, with a redacted copy of the text | “PAN ABCDE1234F” → PAN ********** |
| Translation (Azure Translator) | Text in another language | English → Tamil |
🎯 Exam tip. Sentiment returns a label and confidence scores, not a star rating. PII detection returns the entities and redacted text. Key phrases are phrases, not full sentences.
Speech workloads
Azure Speech in Foundry Tools covers:
| Capability | Direction | Use cases |
|---|---|---|
| Speech recognition (speech-to-text, STT) | Audio → text | Call-centre transcripts, captions, voice commands, meeting notes |
| Speech synthesis (text-to-speech, TTS) | Text → audio | IVR menus, reading replies aloud, accessibility, audiobooks |
| Speech translation | Audio in one language → text or audio in another | Live multilingual meetings, travel assistants |
| Speaker recognition / diarisation | Who spoke when | Separating agent and customer in a call recording |
Key ideas:
- Real-time recognition (streaming from a microphone) vs batch transcription (large recorded files).
- Neural voices sound natural. You choose a voice name such as
en-IN-NeerjaNeuralorhi-IN-SwaraNeural. - SSML (Speech Synthesis Markup Language) controls pronunciation, pauses, speed, pitch and emphasis.
- Custom speech improves recognition of domain words (product names, medical terms). Custom voice creates a branded voice, with strict responsible-AI approval.
Voice-enabled assistants chain the pieces: speech-to-text → language model → text-to-speech. Multimodal models that accept audio can also take the spoken prompt directly (Session 8).