Elon Musk’s company xAI presented a new model of the chatbot Grok, capable of processing requests in various formats.

The presentation took place a few weeks after the release of the previous version.

“Grok-1.5V competes with existing multimodal models in several areas: from interdisciplinary reasoning to understanding scientific diagrams, graphs, and screenshots,” the blog says.

In the press release, the developers provided several examples demonstrating the new capabilities of the chatbot:

• converting a sketch of a block diagram into Python code;
• generating a bedtime story from a child’s drawing;
• explaining memes;
• converting a table into a CSV file format.

Having tested competitors such as GPT-4V, Claude 3Sonnet, Claude 3 Opus, and Gemini Pro 1.5, xAI claims that its multimodal model is leading in many parameters.

The company representatives emphasized that Grok-1.5V outperforms its competitors in the RealWorldQA benchmark – a new metric created to evaluate spatial understanding of the real world.

To pass the test, the AI model was trained on more than 700 images, each accompanied by a question and answer for each element. xAI released RealWorldQA under a Creative Commons license.

Grok-1.5V was released less than a month after xAI published the open-source code of the model.

According to the developers, “significant” updates will be made to the chatbot’s capabilities for understanding and generating multimodal signals in the coming months.

Early testers and current users will have access to Grok-1.5V soon.

Recall that in December 2023, xAI notified the SEC of plans to raise $1 billion through a private sale of equity securities.

Found a mistake in the text? Highlight it and press CTRL+ENTER