EchoSpeech glasses. Image credit: Cornell University

Researchers at Cornell University in the United States have developed a pair of sonar glasses, which track lip and mouth movements through acoustic sensing and artificial intelligence, and can continuously recognize up to 31 silent commands. Vocals offer help. A related paper will be presented at the Association for Computing Machinery's Human Factors in Computing Systems conference this month in Hamburg, Germany.

The glasses are a silent speech recognition interface called EchoSpeech. Equipped with a pair of microphones and speakers smaller than a pencil eraser, the glasses become a wearable AI-powered sonar system that sends and receives sound waves on the face and senses mouth movements. A deep learning algorithm then analyzes these echo profiles in real time with about 95 percent accuracy. The low-power, wearable glasses require only a few minutes of user training data to recognize commands and run on smartphones.

EchoSpeech can be used to communicate with others via smartphone in places where speaking is inconvenient or inappropriate, such as in a noisy restaurant or a quiet library. The silent voice interface can also be paired with a stylus and used with design software such as CAD, requiring little to no keyboard and mouse.

Researchers say they are "moving" sonar technology to the human body. Its small size, low power consumption, and privacy sensitivity are all important features for deploying new wearable technologies in the real world. Most techniques in silent speech recognition are limited to a select set of predetermined commands and require the user to face or wear a camera, which is neither practical nor feasible. Wearable cameras also present significant privacy concerns, both for the user and the people they interact with. And acoustic sensing technologies like EchoSpeech eliminate the need for wearable cameras.

In addition, since audio data is much smaller than image or video data, EchoSpeech only needs a smaller bandwidth and transmits it to the smartphone in real time via Bluetooth, and the data is local, not in the cloud, ensuring the security of sensitive private information.

[The editor-in-chief circles]

Since the beginning of this year, artificial intelligence has once again attracted great attention. A few years ago, experts were still discussing how to make artificial intelligence understand people's intentions more accurately and conduct multiple rounds of conversations with people. In the past, letting artificial intelligence understand lip language was just a bridge in science fiction movies. Today, these are rapidly becoming a reality. Faced with the unexpected changes brought about by artificial intelligence, we cannot stay at the stage of "watching the excitement". People need to really think about how to better dominate artificial intelligence, use it for the benefit of mankind, and try to avoid the possible harm it may bring. trouble.

(Originally titled "Artificial Intelligence Sonar Glasses Can Recognize Lips")