Docs · Capabilities

Vision

Models with the Vision capability understand images alongside text: receipts, charts, screenshots, photos and more.

import base64

# Option 1: a publicly reachable image URL
url_part = {"type": "image_url", "image_url": {"url": "https://example.com/receipt.jpg"}}

# Option 2: a local file as a base64 data URL
b64 = base64.b64encode(open("receipt.jpg", "rb").read()).decode()
data_part = {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}}

resp = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What is the total amount on this receipt?"},
            data_part,
        ],
    }],
)
print(resp.choices[0].message.content)

Ways to send images

Public URL: the image must be reachable from the internet. Private-network and login-protected links will not work.
Base64 data URL: recommended for local files and private images.
One message can contain several images mixed with text.

Anthropic format

msg = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{
        "role": "user",
        "content": [
            {"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": b64}},
            # or {"type": "image", "source": {"type": "url", "url": "https://example.com/receipt.jpg"}}
            {"type": "text", "text": "What is the total amount on this receipt?"},
        ],
    }],
)

Cost and limits

Images are counted as input tokens (some models list a separate image input price). Higher resolution costs more.
Downscale images to the resolution you actually need — it cuts both cost and latency.
JPEG, PNG, WebP and GIF are commonly supported; oversized images are rejected with a 400.

Structured output

Reasoning