image: Sensed states available from Significant Otter
Even though people stayed in touch during the pandemic's stay-at-home orders and social distancing, it was easy to feel out of touch with loved ones.
Technology and the internet have expanded the way humans communicate and added much to that communication -- think emojis, GIFs and memes. But they can still fall short of being physically with someone.
"Our social cues are limited online," said Fannie Liu, a research scientist at Snap Inc who earned her Ph.D. from the Human-Computer Interaction Institute in Carnegie Mellon University's School of Computer Science. "We're exploring a new way to support digital connection through a deeper and more internal cue."
Liu was part of a team from CMU, Snap and the University of Washington that built Significant Otter, an app designed primarily for smart watches that allows couples to communicate with each other based on their sensed heart rate. The team presented their work this month at the Association for Computing Machinery (ACM) Computer-Human Interaction (CHI) Conference.
As the app's name suggests, it uses otters to communicate. The app allows couples to send animated otters to one another that represent emotions and activities. For example, otters can be sad, excited, calm or angry, or they can be working, exercising, eating or tired. The app senses a person's heartrate and then suggests otters with the emotion or activity that may correspond to it. A fast heartrate could prompt the app to suggest an excited or angry otter, or an otter that is exercising or eating.
The partner can then respond with preset reactions. The reactions aren't based on the person's heartrate but are instead designed to give support to the person communicating based on their heartrate. Example reactions include otters hugging, holding each other's hands or even giving an encouraging thumbs up.
The team tested the app in April and May 2020 with 20 couples separated by the pandemic and found that the use of biosignals -- in this case, heartrate -- made for easier and more authentic communication. Liu and the team didn't intend to test the app during the pandemic, but couples who participated in the test said that the app gave them a sense of their partner's physical state even when they couldn't be physically together.
"It's coming from your heart," Liu said. "It can be a very intimate gesture to see or feel someone's heartbeat. It's a signal that you're living."