CES 2024: Mercedes-Benz Unveils MB.OS Operating System with AI Virtual Assistant

At the Consumer Electronics Show (CES) 2024, Mercedes-Benz showcased a lineup of digital advancements. A standout highlight is the debut of the MBUX Virtual Assistant. The new operating system will debut in the upcoming new-gen CLA and on future Mercedes models built on the new MMA architecture. It is also expected that the automaker will introduce the tech across its lineup in subsequent stages.

Developed with the company’s new in-house developed Mercedes-Benz Operating System (MB.OS) - claimed to be distinct from the current MBUX - Mercedes claims that the new AI-powered assistant “is the most human-like interface with a Mercedes-Benz yet.” The company says that the foundation of the new MBUX Virtual Assistant lies in four distinctive 'personality traits' – Natural, Predictive, Personal, and Empathetic – integrated to create a comprehensive entity.

New MB.OS system also provides on-demand access to movies, gaming and songs.

Mercedes says the ‘Natural’ element comes into play almost immediately with the system. The voice assistant has been engineered to make conversation as natural as possible with users also able to engage with the system without the need to say “Hey Mercedes.” The new voice assistant also utilises large language models (LLMs), “for natural dialogue in response to queries and commands.” Mercedes says that the aim is to offer users an experience similar to talking to a friend. The assistant not only responds to queries but is also able to intelligently pose questions to clarify intent and make informed conclusions, drawing on general knowledge.

With its Predictive trait, the assistant utilises the new generative AI to provide personalised suggestions based on learned behavior and situational context. This includes features like suggesting morning news playback or initiating a preferred massage program at the end of the day. The overall experience is heightened through the incorporation of the vehicle’s ambient lighting and sound system, allowing for a customized and comfortable driving atmosphere.

Additionally, Mercedes says that the new assistant has a more emotionally nuanced neural voice adding a touch of empathy to the mix while users are also able to tailor the assistant to their specific preferences through a variety of options available in a centralised hub.

The new MB.OS features a new 3D mapping tech that integrates driver assistance information, surrounding vehicle alerts, and real-time augmented-reality instructions.

Adding to the vocal aspects of the assistant are visual feedbacks via the vehicle’s screens. The system uses advanced 3D graphics created using Unity’s game engine to provide visual cues through varying animations for actions, such as speaking, listening, thinking, making suggestions, or providing warnings.

For entertainment MB.OS also integrates Audible and Amazon Music, synchronising audio experiences, ambient lighting, and in-seat exciters for an immersive multimedia encounter, especially with Dolby Atmos-enabled content. Moreover, a collaboration with Sony Pictures Entertainment, brings popular titles like Gran Turismo and Spider-Man: Into The Spider-Verse for in-car streaming, while a partnership with Antstream offers a selection of arcade-style video games, accessible during parked situations or Level 3 driver-assisted conditions. Users can conveniently rent and purchase Sony Pictures Entertainment's content on demand, with advanced controls and features specifically designed for car use.

The new MB.OS also features a new 3D navigation system that provides sharp graphics and detailed driving directions, incorporating driver assistance information, surrounding vehicle alerts, and real-time augmented-reality instructions. This feature aims to aid decision-making in urban traffic conditions, displaying contextual surroundings like buildings, time of day, and weather conditions. Further, this assistant keeps passengers updated with latest content and features through Over-the-Air (OTA) updates.

Mercedes says that the new E-Class, serving as a precursor to MB.OS technology already benefits from some of these updates. Commencing in the first quarter of 2024, the E-Class will receive a new MBUX integrated Meetings app. This will enable participation in Microsoft Teams conferences, complementing existing support for Zoom and WebEx. The comprehensive update extends further to include convenient apps such as Booking.com, getAbstract, Just Eat Takeaway.com, among others. Future plans also encompass the introduction of cloud-based retro gaming from Antstream Arcade and additional immersive experiences as claimed by the automaker

Written by: RONIT AGARWAL