Agentic Vision in Gemini

A standout feature of Agentic Vision is its ability to automatically zoom into fine-grained details when processing high-resolution images. The model is trained to implicitly detect when a closer look is needed and then formulate a plan to crop and analyze specific regions using Python code. A real-world example is PlanCheckSolver.com, an AI-powered building plan validation platform. By enabling code execution with Gemini 3 Flash, this startup improved its accuracy by 5%: the model iteratively crops patches such as roof edges or building sections, analyzes them as new images, and appends the crops back into its context window. This process visually grounds the model's reasoning, confirming compliance with complex building codes. Such capability is invaluable for any industry requiring detailed visual inspection, from architecture and engineering to medical imaging and satellite image analysis.

Agentic Vision also excels in image annotation and visual math, providing a 'visual scratchpad' for the model. In image annotation, instead of merely describing what it sees, Gemini 3 Flash executes Python code to draw directly on the canvas. For example, when asked to count the digits on a hand in the Gemini app, the model draws bounding boxes and numeric labels over each finger. This ensures that its final answer is based on pixel-perfect understanding, eliminating miscounts. For visual math, Agentic Vision can parse high-density tables and execute Python code to visualize the findings. Standard LLMs often hallucinate during multi-step visual arithmetic, but Gemini 3 Flash offloads computation to a deterministic Python environment. The model identifies raw data, writes code to normalize values, and generates professional Matplotlib bar charts, replacing probabilistic guessing with verifiable execution.

The overall approach of Agentic Vision is to integrate code execution as a native tool within the model's reasoning process. Unlike traditional vision models that process an image once, Agentic Vision continuously updates its context by appending modified images. The workflow begins with an initial image and a user query. The model analyzes both through the Think phase, then acts by generating executable Python code that manipulates the image or performs numerical analysis. The Observe phase cycles the results back, allowing the model to refine its understanding iteratively. This agentic loop is powered by the Gemini 3 Flash model and is accessible to developers via the Gemini API in Google AI Studio and Vertex AI. Users can enable code execution in the AI Studio Playground by turning on the 'Code Execution' tool under Tools, or try the demo app to see the capability in action.

Concrete use cases for Agentic Vision span multiple industries. The building plan validation platform PlanCheckSolver.com uses it to improve accuracy by 5% for compliance checks on roof edges and building sections. In the Gemini app, users can count digits on a hand with pixel-perfect accuracy, using bounding boxes and labels drawn by the model as a visual scratchpad. For researchers and analysts, Agentic Vision can parse complex tables from charts, normalize data, and generate professional plots using Matplotlib, all without hallucination. A developer can test these scenarios in Google AI Studio by enabling code execution. The outcome is a consistent improvement in visual reasoning quality—the blog explicitly states a 5-10% quality boost across vision benchmarks—and a reduction in probabilistic guessing, making AI outputs more trustworthy and actionable for real-world applications.

Agentic Vision is targeted at developers, AI engineers, and product teams building applications that require precise visual understanding. It is also suited for startups like PlanCheckSolver.com and large-scale products like the Gemini app. The capability is available via the Gemini API on Google AI Studio and Vertex AI, and it is rolling out to the Gemini app (access by selecting Thinking from the model drop-down). The underlying tech stack involves Python code execution within the Gemini 3 Flash model, with support for libraries like Matplotlib for plotting and OpenCV-like operations for image manipulation. Pricing details are not explicitly provided in the blog, but the API is accessible to developers with a Google Cloud account. In summary, Agentic Vision redefines how AI interacts with visual data by turning passive recognition into an active, verifiable investigation—delivering higher accuracy and ground-truthed results for any visual task.

Updated 2026-02-28