Background:
Google Assistant is an AI-powered voice assistant that can answer questions, respond to commands (start a timer), and more. It is available on mobile devices and on smart speakers and displays for the home. Children can have their own personalized Google Assistant experience when they log in with a Google account managed by their parent or guardian through Family Link.
When kids were sent home from school during the COVID-19 crisis, the Google Assistant team knew they may be idle, their learning may be stalled, and their parents may be frustrated. They wanted to launch an engaging, educational experience for them, so the team created “Animal of the Day.” Animal of the Day teaches kids about animals using fun, interactive conversation, which includes sounds, fun facts, images, and a daily creative challenge, taking the experience from the online to the offline. (Try it yourself! Just say, “Hey Google, what’s the animal of the day?”)
Research Questions:
- Do kids know how to enter and make their way through the experience (usability)?
- How do kids respond to and engage with Animal of the Day (engagement)?
- What types of animal facts were most interesting to kids (engagement)?
- Do kids retain the animal facts they have learned (comprehension)?
Process:
- This study was conducted early during the COVID-19 crisis, and therefore had to be done remotely, which posed quite a few challenges for a voice-forward experience built for kids!
- Children who took part had some type of voice assistant in the home.
- Children took part in the session with a parent or guardian, who signed informed consent forms on behalf of their children and themselves. Data collected was anonymized and only reported in aggregate for the purpose of improving the Animal of the Day feature.
- Since the feature was not yet live on our devices, the researchers could not observe kids using it in their home environments. Instead, researchers had to recreate a voice-forward experience while interviewing kids and parents over video chat.
- Researchers had slides with pictures of various Google Assistant devices. When the child indicated which device they had in their home, the researcher pulled this slide up on screen, so it was as if the child was speaking to the device in their home.
- Children were instructed to go through the experience as if this was a real voice assistant device in their home, while the researcher followed the script of the experience off-screen, and played the appropriate line at the right time, depending on the child’s responses to Assistant (“Wizard of Oz” style). It mimicked the real voice experience exactly!
- The researchers observed children’s behavior and reactions during three animal experiences, and asked them questions about their favorites and what they learned afterwards (to assess appeal and comprehension).
Checklist
Ahead of the session, parents were told that while we would be talking about and exploring Google Assistant experiences, they did not need to have their Google Assistant in the room. Parents would only need a laptop with access to Google Meet, a comfortable spot for themselves and their child to sit within frame, and strong internet access.
Session preview:
Given the remote nature of this session, it could be very easy for kids to get distracted or bored. Before we started the session, I gave them a brief overview of what we would be doing so they knew what to expect. This also served to get them excited, since we’d be talking about animals. They were also told that if they needed to take a break for whatever reason, that was not a problem.
Real environment:
In order to make the experience feel as close to real life as possible (even though they were using a prototype over Google Meet), researchers put a picture of the Google Assistant device located in their home on-screen, complete with flashing lights when Assistant was listening. This made children feel that they were simply talking to the device in their home as usual.
Notice the nonverbal:
As children heard about three animals before we discussed them, researchers observed in the moment nonverbal cues during each experience as one measure of engagement for each animal experience.
Findings:
- Kids had a variety of ways they would want to enter the experience, beyond the expected question of “Hey Google, what’s the animal of the day?”
- Kids could easily navigate through the experience, knew when to respond to Assistant, and knew that they were in control of the experience and could stop it at any time.
- Extreme or surprising facts (“dragonflies are the fastest insects”) were more likely to be remembered than the mundane (diet, location), and they were more interesting and exciting to kids, as was made obvious by non-verbal cues (gasping, wide eyes, etc.).
- Kids had a hard time remembering number-based facts.
Impact:
We revised the content within Animal of the Day to be more novel, relevant, and understandable. We increased our focus on “extreme” animals and facts (the fastest, the biggest, etc.), and we revised our number-based facts to be more relevant (instead of providing the size of an animal in pounds, which kids may not grasp, we compared the size of an animal to a known or familiar object, like a school bus).
We made sure that we did not just accept “yes” or “no” to move through the experience, but programmed it to respond to the natural utterances heard in this testing (i.e., not only “yes” or “no,” but also “sure, yeah, okay.”).