The first wave of large models handled text. The current wave handles text, images, audio, and increasingly video — all in one model. This shift to multimodality is more than a feature checklist; combining senses changes what a model can understand.
What multimodal means
A multimodal model processes more than one type of input (and sometimes output) in a unified way. It can look at an image and discuss it, listen to audio and respond, read a chart and reason about it, or watch a video and summarize it. Rather than separate systems bolted together, the strongest multimodal models represent different inputs in a shared space and reason across them together.
Understanding isn't confined to one sense. A model that can see the chart and read the text around it understands the document better than either alone.
Why combining senses helps
Real understanding is often multimodal. A meme needs image and text together; a lecture needs slides and speech; a medical case needs images and notes. A model that perceives multiple modalities at once can connect them — grounding language in what it sees, or explaining an image in words. This cross-modal grounding tends to make models more capable and less prone to certain errors than text-only reasoning.
Where it's going
Multimodality is becoming the default, not a specialty. The trajectory points toward models that seamlessly handle any mix of input and output — describe an image, generate one, transcribe audio, answer questions about a video — through a single interface. As this matures, the boundaries between "language model," "vision model," and "speech model" keep dissolving into simply "model." The senses are merging, and with them, a more complete kind of machine understanding.
