What is OCR? How your phone reads text in pictures

OCR, short for optical character recognition, is technology that turns a picture of words into real text that a computer can search, copy and edit. When your phone lets you copy words from a photo or find a screenshot by typing a word, OCR is doing the work. So, what is OCR in practice? It is a way of teaching a device to read.

What is OCR, in plain words?

To a phone, a photo of a sign is just coloured dots. It has no idea that some of those dots form the word "Exit". OCR looks at the shapes in the image, works out which ones are letters and numbers, and writes them down as text.

The result is the same kind of text you would get if you had typed it yourself. You can search it, copy it into a message, or have it read aloud.

OCR has existed for decades, first on large scanners in offices. Today it runs on ordinary phones, which is why reading text in pictures now feels like an everyday feature.

How does OCR work?

Modern OCR systems use machine learning, which means they have learned what letters look like from a huge number of examples. The exact methods vary, but most follow similar stages.

1. Clean up the image. The system adjusts brightness and contrast and may straighten a tilted photo so the text is easier to see.
2. Find the text. It locates the parts of the image that contain writing and ignores the rest, such as backgrounds and pictures.
3. Split it into lines and words. It works out where each line starts and ends, and where the gaps between words are.
4. Recognise the characters. It decides which letter, number or symbol each shape is most likely to be.
5. Check against language. Many systems use knowledge of real words to fix likely mistakes, for example choosing "receipt" over a jumble of similar letters.
6. Output the text. The final text is stored, often together with where each word appeared in the image.

Why does OCR make mistakes?

OCR is very good at clear printed text and less good when the picture is difficult. Common causes of errors include:

• Lookalike characters. The letter O and the number 0, or the letter l and the number 1, can be almost identical in some fonts.
• Small or blurry text. If you can barely read it yourself, OCR will struggle too.
• Busy backgrounds. Text placed over a photo or a pattern is harder to separate from what is behind it.
• Decorative fonts. Script and stylised fonts do not look like the examples the system learned from.
• Handwriting. Everyone writes differently, so handwritten text is much less predictable than print.
• Unfamiliar scripts. A system trained on one writing system may not read another at all.

Why screenshots are ideal for OCR

Screenshots are some of the easiest images for OCR to read. A screenshot is a perfect copy of the screen, so it has no blur, no shadows, no tilt and no glare. The text is usually sharp and high in contrast, with standard fonts.

Photos of paper are harder. Light can be uneven, the page may be curved and the camera might not be quite straight. That is why the same OCR engine can be near perfect on a screenshot and a little less reliable on a photo of a crumpled receipt.

Where you already use OCR on your phone

What is OCR used for day to day? You have probably used it already without thinking about it.

• Copying text from an image. Tools such as Google Lens can pick out text so you can copy it. See how to copy text from a screenshot.
• Scanning documents. Document scanners often add a text layer so the scan can be searched.
• Translating signs and menus. Camera translation reads the text first, then translates it.
• Searching your pictures. Some apps use OCR to let you search the text inside your screenshots.

On the phone or in the cloud?

OCR can run on your device or on a remote server. On-device OCR keeps your images on the phone. Cloud OCR sends the image away to be processed. Both can work well, but they differ a lot for privacy, which we explain in on-device OCR vs cloud OCR.

How Screenshot Brain uses OCR

Screenshot Brain uses on-device OCR to read the text in your screenshots, so they are not uploaded. That text is what lets you search by a word you remember, and it is also how the app recognises receipts, bookings and coupon codes, spots return and warranty dates, and finds card numbers to cover before you share.

Frequently asked questions

What does OCR stand for?

OCR stands for optical character recognition. "Optical" refers to reading from an image, and "character recognition" means identifying letters, numbers and symbols.

Is OCR the same as AI?

Modern OCR uses machine learning, which is a branch of AI. But OCR has a narrow job: it reads text. It does not understand or summarise what the text means on its own.

Can OCR read any language?

Only the languages and scripts it was built to read. Many systems handle common Latin-based languages well, and support for other scripts varies between tools.

Does OCR need an internet connection?

Not if it runs on the device. Cloud-based OCR does need a connection, because the image is sent to a server.