Silent Speech Recognition

  • A participant using the silent speech interface
  • Design of the silent speech interface
  • Researchers working on movement-driven synthesizer to help patients who have trouble with speech

Project Overview

People with laryngectomy (surgical removal of the larynx due to cancer) and speech motor impairment (e.g., due to neurological diseases) struggle to communicate with others because of their impaired oral communication. Limited treatment options are available to improve the quality of their speech.

The purpose of the Silent Speech Recognition project is to develop an articulatory movement-driven speech synthesizer that can enable those patients to speak using their tongue and lips (rather than by typing on an AAC device, which is limited by the slow manual input). The current focus of this project is to develop algorithms that can convert articulatory movement time-series data to text with high speed and accuracy. In the future, the algorithms will be integrated with a portable data collection device and text-to-speech synthesizer.

Stage of Development

This project is currently in the pilot stage of development. Nine core technical papers and three relevant papers have been published, based on data collected from healthy speakers. At the UT Dallas Communication Technology Center, data collection from speakers with impaired motor speech for the next stage of development will begin in the near future. This project is partly supported by a NIH R01 grant. A R03 grant application has been recently submitted to the National Institute on Deafness and Other Communication Disorders to fully support this project.

Development Team

The Silent Speech Recognition project is a collaboration among the Callier Center for Communication Disorders and Department of Computer Science, UT Dallas; Department of Computer Science and Engineering, University of Nebraska-Lincoln; and Massachusetts General Hospital in Boston. The team of researchers includes:

Jun Wang, PhD
The University of Texas at Dallas, Biomedical Engineering and Communication Sciences and Disorders

Jordan R. Green, PhD
Institute of Health Professions, Massachusetts General Hospital (MGH), Department of Communication Sciences and Disorders

Ashok Samal, PhD
University of Nebraska-Lincoln, Department of Computer Science and Engineering

Balakrishnan Prabhakaran,
The University of Texas at Dallas, Erik Jonsson School of Engineering and Computer Science