Azure AI-901 · Module 2 · Session 4

Session 4: Vision & Information Extraction Workloads

⏱ 10 min read📘 Live-session notes☁️ Microsoft Foundry

Computer vision (image analysis, captioning, object detection, OCR), what image-generation models can and cannot do, and how to pull structured information out of text, images, audio and video. The session ends with a 15-scenario workload drill and a Domain 1 practice quiz.

Computer vision capabilities

Capability Output Example
Image analysis / tagging Tags describing content (“car”, “road”, “outdoor”) Organise a photo library
Image captioning A human-readable sentence “A woman holding a credit card in front of a laptop”
Dense captions Captions for regions of the image Describe each product on a shelf
Object detection Objects with bounding boxes (coordinates) Count vehicles in a parking lot
Image classification One label (or a few) for the whole image Damaged vs undamaged parcel
OCR (optical character recognition) Printed and handwritten text, with positions Read a cheque, a shop sign or a scanned form
Face detection Location of faces (and limited attributes) Blur faces in CCTV footage
Visual question answering An answer to a question about an image “How many people are wearing helmets?”

🎯 Exam tip. Classification says what the image is. Object detection says what and where (bounding boxes). OCR reads text. Captioning describes the scene in a sentence.

Two ways to do vision in Foundry:

  1. A multimodal generative model (one that accepts image input): send an image plus a question in the prompt and get captions, descriptions, VQA and reasoning in natural language. This is the AI-901 focus (Session 9).
  2. Specialised services such as Azure Vision in Foundry Tools, or Content Understanding for extraction, when you need structured outputs at scale.

Responsible use: facial recognition and identification features are restricted (Limited Access) under Microsoft’s responsible AI policy, and some attributes (emotion, gender, age inference) are not offered.

Image-generation models

Models such as GPT-Image-1 and FLUX create images from a text prompt, and some can edit an existing image (inpainting: change only a masked area).

They can: create original illustrations, product mock-ups, marketing visuals and concept art, and follow style instructions (“flat vector, pastel colours”).

They cannot (reliably): guarantee accurate text inside the image, exact counts and positions, real people’s likenesses or trademarked characters (blocked by safety policies), factual diagrams, or the same character across many images without care.

Safety: every request passes through content filters (violence, sexual content, hate, self-harm), and outputs can carry provenance metadata (content credentials) showing the image is AI-generated.

Extracting information from any content

Content What you extract Tool
Text Entities, key phrases, PII, custom fields Azure Language, or an LLM with a structured-output prompt
Documents and forms (PDF, scans) Fields (invoice number, total, dates), tables, key-value pairs, layout Azure Content Understanding (or Document Intelligence)
Images Descriptions, objects, text, custom fields (“is the meter reading visible?”) Content Understanding or a multimodal model
Audio Transcript, speakers, summary, custom fields (“did the agent mention the fee?”) Content Understanding or Azure Speech
Video Key frames, scene descriptions, transcript, chapters, custom fields Content Understanding

Content Understanding uses an analyzer: a prebuilt one (for example invoices) or a custom one where you define a field schema (the fields you want and their types). One service handles documents, images, audio and video and returns structured JSON with confidence scores. You build one in Session 10.

Scenario drill: choose the workload

Cover the answers and decide yourself first.

# Business problem Workload
1 Draft personalised follow-up emails after a sales call Generative AI
2 Find out whether app-store reviews are mostly negative Text analysis: sentiment
3 Read totals and dates from 10,000 supplier invoices Information extraction (documents)
4 Count people entering a mall from CCTV Computer vision: object detection
5 Turn customer phone calls into searchable text Speech recognition
6 Read SMS alerts aloud for visually impaired users Speech synthesis
7 Detect Aadhaar and PAN numbers in chat logs and mask them Text analysis: PII detection
8 Create a poster image for a festival offer Image generation
9 A bot that checks an order in the ERP and issues a refund Agentic AI
10 Pull the main topics from 5,000 survey comments Text analysis: key phrase extraction
11 Read the number plate in a photo Computer vision: OCR
12 Summarise each recorded sales video into chapters Information extraction (video)
13 Translate support chat between Hindi and English live Translation
14 Describe uploaded product photos for alt text Computer vision: captioning
15 Identify which language each incoming email is written in Text analysis: language detection

Domain 1 practice quiz

1. Which responsible AI principle requires that people can understand how a system reaches decisions? Transparency.
2. Which parameter makes output more random and creative? A higher temperature (or a higher top-p).
3. You need the location of each car in an image. Classification or object detection? Object detection: it returns bounding boxes.
4. A model must accept a photo of a damaged car and a question about it. What type of model? A multimodal model that supports image input.
5. Which technique returns “positive”, “neutral”, “negative” or “mixed” with confidence scores? Sentiment analysis.
6. What does an analyzer’s field schema define in Content Understanding? The fields to extract and their types (for example InvoiceNumber: string, Total: number).
7. Name one thing image-generation models cannot reliably do. Render exact text, guarantee precise counts and layouts, or depict real people or trademarked characters (blocked by policy).
📝 AI-901 exam practice
🎤 Practise in a mock interview🔥 Daily challenge🎓 Certificates
Created by Ajeetkumar