bryana
work
playground
about

VUI In-Car Google Maps

Nominated to present at World IA Day LA 2024 Student Showcase
‍
Duration
7 Weeks
‍
Project Type
Voice User Interface Design

Team Member
Issac Teng
Yuna Kim
Bryana Lee
Jaemo Seong

My Role
Research
User Testing
User Interface Design
‍
Software
Figma
Protopie

World IA Day LA 2024

Our team presented our project at World IA Day LA 2024 hosted at ArtCenter College of Design.

Click here to watch the full presentation video!

Project Overview

Our project is a design study on an advanced, multi-command Voice User Interface (VUI) integrated into Google Maps on a car dashboard, serving as an extension of Google Assistant's existing framework to further enhance hands-free, in-car functionality.

Google Maps

already has the reservation function and a massive restaurant list with the ability to link to Google Account

Google Assistant

expanding on an existing assisting AI framework and the wide Google network is a benefit for the project

Reliable and Calm Voice AI Bot

Meet Giga

About

Greetings. My name is Giga, your on-board assistant from Google. I’m here to provide you quick and professional assistance and make sure you are well taken care of. Rest assured I will give you the best recommendations and easiest routes on your many journeys.

“Greetings, might I suggest taking local roads to avoid highway traffic?”

Personality

Prompts

  • Wake Up : Hey Giga, / Giga!

  • Food Order : Hey Giga, I want to order food to deliver right when I get home

  • Navigation : Hey Giga, I need gas right now! Where’s the closest Chevron?

Common Conversation

User:
Hey Giga! I want to order food to deliver right when I get home.

Giga:
Of course. Shall I order your usual from Zankou Chicken or something new?

User:
Yeah Zankou Chicken sounds good, but I want to try out their Chicken Tarna bowl with Hummus instead this time.

Giga:
I will get that ordered for you promptly. Expect the food to be delivered home upon our arrival.

User:
Thanks Giga.

Giga:
The pleasure is all mine.

Flow 1

Ordering Food to Deliver Upon Arrival Home or Pick Up

1

User starts driving home

2

Command Giga to order food

3

Share user's ETA with restaurant for pick up

4

Reroute navigation to restaurant

Flow 2

Making Reservations, Navigating to and Notifying Multiple People

1

User starts driving home

2

Command Giga to make a restaurant reservation at a specified time

3

Request Giga to recreate navigation route to pick up other passengers

4

Share ETA with other passengers with live location updates

5

Navigate to recreated route

Flow 3

Navigating to and Sharing ETA with Multiple Destinations

1

User starts driving to grocery store

2

Giga finds curbside pick up order placed at grocery store

3

Share ETA with grocery store, elementary school and babysitter

4

Request Giga to route to grocery store, elementary school and home

5

Navigate to created route

Flow 4

Notifying Obstacle on Road and Alternative Routes

1

User is driving

2

Giga notifies user of an accident ahead on current route

3

User decides to switch to new route or stay on current route

4

Navigate to selected route

Why Google Maps?

The tasks that an AI voice bot can perform in an in-car experience are still relatively simple, such as navigating to a destination, making phone calls, sending text messages, and alerting drivers to road conditions. However, there is significant potential to make these interactions more personalized for users and expand the range of tasks available, all while helping drivers stay fully focused on the road.

A major trend that aligns this growth is the rise of food ordering. With food ordering becoming an essential part of convenience-oriented lifestyles, there is also growing demand for seamless in-car systems to support this trend. People are increasingly looking to place orders, reserve tables, or schedule curbside pickups while on the go, making in-car voice AI an ideal solution for safely managing these interactions during commutes.

For our project, our team decided to focus mainly on tasks related to food ordering, including reservations, delivery, and curbside pickup, and expand this functionality by incorporating more AI-driven features optimized for the in-car experience. We aimed to design our Voice User Interface (VUI) on a platform with an existing ecosystem, ultimately choosing Google Maps. Google Maps provides the perfect foundation for our AI-driven features because of its large user base and, most importantly, its developed functionalities related to locations, navigation, restaurant reservations, reviews, and mobile food ordering.

How Might We

... offer food recommendations based on user preferences, past orders, and dietary needs?

... design VUI to answer multiple requests in one command rather than separate conversations?

... minimize distractions and ensure safe VUI operation experiences for the users?

... structure a conversation that inspires trust in a VUI to execute commands perfectly?

... integrate food ordering with in-car navigation and driving through VUI?

... build trust with the user in terms of order accuracy, timely delivery, and payment security?

User Personas

Leo 21

The Tech-Savvy Gen Z

About

Leo is a college student who commutes to school with his car. He loves to hang out with his friends, but they usually carpool due to the heavy traffic. He is a foodie and although he is frugal usually, he likes to spend mostly on quality food.

Frustrations

  • has high expectations for quality, efficiency and convenience
  • feels overwhelmed managing small support tasks and staying focused on the road at the same time
  • doesn't want to waste time by getting stuck in traffic

Personality

Goals and Motivations

  • wants to save time and avoid traffic
  • wants conveniency and efficiency for monotonous or boring tasks
  • likes to have quality time hanging out with friends

Desires

  • prefers to pick up his food to bring home
  • wants to optimize his routes to save time
  • wishes to communicate easily with his friends when carpooling
  • wants to feel secure while driving

Brandon 36

Multitasking Millennial Parent

About

Brandon is a single parent with two kids, a 4th grader and a toddler. He is a business consultant and needs help from a babysitter on weekdays and his parents on weekends to take care of his kids, so that he can work and catch up with house chores with no worries.

Frustrations

  • needs to take care of kids while completing house chores and balance his life and work time
  • needs to coordinate with his parents, his kid’s school and the babysitter frequently whenever he is running late from work
  • often has to add quick stops for last-minute errands

Personality

Goals and Motivations

  • wants to be productive at all times including car trips
  • needs an easy way to communicate with his children
  • needs the ability to quickly add stops to grocery stores or pharmacy while driving

Desires

  • likes grocery delivery and curb side pickup services
  • wants to optimize and find the most effective route to save time
  • wants a more effective way of sharing locations and ETA

Voice Diagrams

Ordering Food to Deliver Upon Arrival Home

Making Reservation, Navigating to and Notifying Multiple People

Navigating to Multiple Stops and Sharing ETA with Multiple People

Notifying Obstacle on Road and Alternative Routes

User Testing

Round 1 : Sound or Voice-Based Alarm Notification

Voice assistant notifies user that there is an accident ahead on the navigation route and asks if user wants to take an alternative route.

1. Original Route

2. Accident Ahead

3. New Route

Test A : “Ping” Sound Notification

  • A “ping” sound is quite successful to draw the attention of the driver that something has changed, or that something out of the ordinary has occurred.
  • The “ping” sound without context or prior knowledge has no meaning behind, which causes confusion as there is little to no comprehension
  • It is only once the driver has removed his eyes from the road to look in detail at the screen do they begin to comprehend what has occurred.
  • Without comprehension, there is also a failure in projection following this “ping” sound in terms of what the driver can expect to happen.

Test B : “Ping” Sound + Voice Notification

  • A “ping” sound combined with voice over is on average more successful than test A because it is enough to draw the attention of the driver that something has occurred along the road.
  • The voice over serves as an explanation as to what the “ping” sound refers to. The immediate announcement following the “ping” creates a tight association between the notice sound and the announcement of a road condition change, and so there is comprehension of what the notification means.
  • The clarity in comprehension brings out success in projection since the driver is able to predict that an alternate route will be provided.

Conclusion

A “ping” sound combined with voice over is useful for creating situation awareness, especially when it is the driver’s first time ever hearing this notification.

  • The “ping” is the necessary attention grabber to create successful perception
  • The voice-over explains what is going on to create successful comprehension
  • The voice-over’s question that follows the description of the situation informs the driver of what they can expect to be the next procedure to create successful projection

Round 2 : Color or Text Screen Feedback with VUI

Asking voice assistant for nearby restaurant for food delivery, menu recommendations and delivery upon arrival home.

1. Activate Giga

2. Giga asks what user wants to order

3. User asks for a nearby option

4. Giga recommends choices

5. User wants Mendocino Farms and orders a salad

6. Giga asks for any additional task

7. Giga adds another item to the order

8. Order is complete

Test A : VUI Screen Feedback with Text

  • Drivers liked the visual look of the design was how it integrated into Google’s visual look, but were not satisfied with the glanceability of the design. The change in logos to indicate VUI’s turn or driver’s turn to speak too subtle and required pulling the driver’s attention from the road to look in detail.
  • Even though the screen mentioned “...listening...” to prompt the driver to speak, it was not obvious enough to see and the driver ending up hesitating a lot before speaking.

1. Activate Giga

2. Giga asks what user wants to order

3. User asks for a nearby option

4. Giga recommends choices

5. User wants Mendocino Farms and orders a salad

6. Giga asks for any additional task

7. Giga adds another item to the order

8. Order is complete

Test B : VUI Screen Feedback with Color

  • Color was effective in grabbing attention without needing the driver to pull their eyes off the road.
  • Color codes needed to be learned during the first exposure to the system, but was easily picked up during the second conversation loop where blue meant the VUI’s turn and yellow meant the driver’s turn.
  • Prediction of when to speak the next time was easier as time went on because the driver just needed to wait for the yellow light to turn on at a glance, indicating that the VUI was listening.

Conclusion

Color UI was more effective than using text for an in-car voice user interface based on situation awareness. When asked about a potential combination of color and text, testers were hesitant because they felt it would be informational overload. Testers did mention that a verbal repeat of their total order would be helpful to double check that their order was correctly heard.

Round 3 : Different Mental Models for Emergency Alerts

Voice assistant switching to a different route due to an emergency ahead with or without the decision of the user.

1. User is driving
‍

2. VUI senses an accident ahead

3. Giga asks user to decide route

4. User answers yes or no for switching to a new route

Test A : “Switch to a Different Route?” Yes or No (Ignore to Stay on Current Path)

  • Users responded very well to this option, stating that a “yes” or “no” option was suitable for an emergency situation. The binary option provided a clear A-B option tree for users to make a split decision in a short amount of time.
  • Users were compelled to answer and did not consider ignoring the traffic alert. The question format of the alert was an effective prompt to elicit an answer from the user without the need for the yellow glow prompt
  • Duration of time to answer yes or no was not too clear, adding indication of time left would be nice.

1. User is driving
‍

2. VUI senses an accident ahead

3. Giga switches to a new route automatically

4. User is automatically routed to the new route

Test B : “Switching to a Different Route in 5 Seconds” Cancel (Ignore for New Path)

  • Users responded poorly to this option, stating that it was very unclear if they were even required to answer the VUI.
  • Using an affirmative statement as an alert rather than a question format discouraged the users from answering the VUI. There was an overall sense of rudeness and automated assumption that the users did not appreciate. The users questioned why the VUI would make a decision on their behalf without consulting them first.
  • The option to cancel was not apparent, so many users did not even realize that it was a possibility to cancel the new route.

Conclusion

Providing a “yes or no” choice is a much more preferred option to cancelling an impending change during driving. The choice provides a much clearer indication to the driver of what process is happening and when they should provide input to the VUI.

Project Takeaway

From this project, our team was able to learn a lot of about the process of designing a voice user interface AI bot and the important aspects to keep in mind.
‍
Some specific takeaways that we got were:

  • Clear verbal affirmation and reiteration is needed for lengthy orders, commands or reservations
  • Visual feedback can be kept simple and minimal, as long as the auditory feedback is sufficient in notifying the user if the VUI is listening, what the VUI has heard, and whether a command has successfully been executed
  • The visual feedback should serve to support the voice as a reassurance, and not distract the user’s eyes away from the road
  • The VUI should never assume the user’s intentions; whether in an emergency alert situation or a food ordering situation or a food ordering situation, the VUI should provide the user options rather than an automated decision so that the user still feels in control and can build trust