MiniCPM-o 4.5

Key Features

• •Leading visual capability with an average score of 77.6 on OpenCompass, surpassing GPT-4o and Gemini 2.0 Pro, and supporting high-resolution images up to 1.8 million pixels and high-FPS videos up to 10fps.
• •Strong speech capability supporting bilingual real-time conversation in English and Chinese with configurable voices, natural expression, and voice cloning that outperforms CosyVoice2 via simple audio reference.
• •Full-duplex multimodal live streaming processes real-time video and audio inputs while generating concurrent text and speech outputs without blocking, enabling see-listen-speak simultaneously.
• •Proactive interaction allows the model to initiate reminders or comments based on continuous understanding of live scenes, with a decision-making frequency of 1Hz for timely responses.
• •State-of-the-art OCR and document parsing outperforms proprietary models like Gemini-3 Flash and GPT-5 on OmniDocBench, handling complex layouts and multilingual content over 30 languages.
• •Efficient local deployment via llama.cpp and Ollama for CPU inference, with int4 and GGUF quantized models in 16 sizes, vLLM and SGLang support, and web demos for full-duplex streaming on devices.
• •Trustworthy behaviors matching Gemini 2.5 Flash on MMHal-Bench, with support for both instruct and thinking modes to balance efficiency and performance across user scenarios.
• •Easy usage with PyTorch inference on Nvidia GPUs for precision, FlagOS support for multi-chip backends, and open-sourced APIs for traditional and vision-language requests.

Use Cases

• •Real-time speech conversation with voice cloning for role-playing, such as mimicking Elon Musk's voice for interactive Q&A sessions in English or Chinese.
• •Live sports commentary where the model provides continuous analysis based on real-time video feeds, using full-duplex streaming to sync visual and audio inputs.
• •Document parsing and OCR for high-accuracy text extraction from complex layouts, outperforming specialized tools like DeepSeek-OCR 2 on OmniDocBench benchmarks.
• •Proactive interaction in live streaming environments, offering reminders or comments based on visual and audio context, such as alerting users to changes in a scene.
• •Multimodal educational assistants that process diagrams, math problems, and spoken questions simultaneously, leveraging high visual and speech capabilities for tutoring.
• •Local deployment on devices like MacBooks or PCs for offline AI applications, using quantized models via llama.cpp or Ollama for efficient CPU inference.

Who is this for?

Developers, researchers, and businesses seeking a powerful, open-source multimodal AI for integration into applications requiring real-time vision, speech, and text processing. Ideal for those needing local inference on devices via support for llama.cpp, Ollama, vLLM, SGLang, and FlagOS. Targets users in fields like live streaming, education, document analysis, and interactive assistants who value compact models with top-tier performance and full-duplex capabilities for natural, proactive interactions.