What is computer vision? Explained with describing a photo on the phone
Computer vision is the part of AI that lets computers look at photos and videos and turn them into useful information. Think of describing a photo to a friend on the phone.
- Author
Umair Mateen- Published
- Reading time
- 5 min
- Category
- AI
In one sentence: Computer vision is the part of AI that lets computers look at photos and videos and turn what they see into useful information, such as a label, the position of an object or the text in a picture.
The everyday picture
Imagine you are on the phone with a friend. You are holding a photo, and your friend cannot see it. So you describe it: "It's a kitchen. There are two cups on the counter, on the left. The jar label says 'sugar'."
Your friend never sees the photo. They only hear your words. But your words are enough for them to act: buy more cups, or fetch the sugar.
Computer vision does the describing. A camera or a file gives it a picture. It turns that picture into words, labels and positions that other software can use. That other software is your friend on the phone: it never looks at the photo itself, it just acts on the description.
The vision model looks at the picture and turns it into labels, boxes or text. Your other software acts on that description, just as a friend on the phone acts on your words.
How it works, step by step
- The image arrives as dots (the photo in your hand). To a computer, a photo is a grid of tiny coloured dots called pixels, and each dot is stored as numbers. A video is many such pictures in a row.
- A trained model looks at it (you, looking at the photo). The model learned earlier from many example images that people had labelled, such as "cup" or "not a cup". IBM and AWS both describe this as gathering labelled images, cleaning them, then training a model on them.
- It spots patterns (you noticing edges, shapes and colours). Common model types build up from small details, like edges, to whole objects. Newer "vision transformer" models cut the image into small patches and compare them with each other.
- It gives a description (your words down the phone). Depending on the job, this is a label for the whole picture, a box around each object, an outline of each object, or the text it can read. Services usually add a confidence score to each answer.
- Other software acts on it (your friend fetching the sugar). A stock system counts items, a form system files the text, or a person reviews anything flagged.
Try it · Put it in order
Put the phone call in order
Tap the steps in the order they happen.
Where the picture stops
- It does not understand the scene like you do. It works with numbers and patterns. It can name a cup without knowing what a cup is for.
- It only knows what it was shown. IBM puts it simply: a model is only as good as its training data. Photos that look unlike its examples, such as odd angles, poor light or new products, can trip it up.
- It never gets tired. It can describe far more pictures than you could, without a break. That speed is useful, but it also means one mistake can repeat many times before anyone notices.
Why it matters for your business
- Less typing. Reading text from scans, receipts and forms (called OCR) saves staff from copying it by hand.
- Fewer missed faults. IBM and AWS both list visual inspection and quality control, where cameras flag damaged or faulty items on a line.
- Safer workplaces. AWS gives examples such as checking that people wear protective equipment in risky areas.
- New risks to manage. Pictures of people are personal data in many countries. Check the rules, and tell people, before you point a camera at staff or customers.
An example
Imagine a small online shop that sells second-hand books. Each week, staff photograph a few hundred covers and type in the title and condition. With computer vision, the photo goes in first. The system reads the title from the cover and flags covers that look torn or stained. A member of staff checks every flagged book and any title the system was unsure about, then approves the listing.
Try it · Everyday ↔ real
Match what you say on the phone to the vision task
Switch between the two and see how each part lines up.
- "It's a kitchen"
- "Two cups, on the left"
- Tracing the exact shape of a cup
- Reading the jar label aloud
- Following the cat across a video
Describing a photo to a friend who cannot see it
| Everyday | Real |
|---|---|
| "It's a kitchen" | Image classification |
| "Two cups, on the left" | Object detection |
| Tracing the exact shape of a cup | Image segmentation |
| Reading the jar label aloud | Text reading (OCR) |
| Following the cat across a video | Object tracking |
Frequently asked questions
Is computer vision the same as multimodal AI?
No, but they overlap. Computer vision is the whole field of getting computers to understand images and video. Multimodal AI models, which take in text and images together, are one of the newer ways to do it.
What is the difference between image classification and object detection?
Classification gives one label to the whole picture, such as "kitchen". Object detection finds each item in the picture, draws a box around it and names it, such as "cup, left side".
Do I need to train my own model?
Often not. Cloud providers offer ready-made services that tag images, find objects and read text. You train or fine-tune your own model when you need to spot something specific to your business, like a fault in your own product.
Do these services stay the same over time?
No, they change. For example, Microsoft says version 4.0 of its Image Analysis service is deprecated and will be retired on 25 September 2028. Check your provider's current documentation before you build on a service.
Related explainers
The DevDrift builds AI automation that reads documents and photos, with a person approving what matters: see how.
Sources
Could you explain this to a colleague now?
Mark it and posts that mention it will show a tick. Saved in this browser only.