Silent Speech Recognition
People with laryngectomy (surgical removal of the larynx due to cancer) and speech motor impairment (e.g., due to neurological diseases) struggle to communicate with others because of their impaired oral communication. Limited treatment options are available to improve the quality of their speech.
The purpose of the Silent Speech Recognition project is to develop an articulatory movement-driven speech synthesizer that can enable those patients to speak using their tongue and lips (rather than by typing on an AAC device, which is limited by the slow manual input). The current focus of this project is to develop algorithms that can convert articulatory movement time-series data to text with high speed and accuracy. In the future, the algorithms will be integrated with a portable data collection device and text-to-speech synthesizer.
Stage of Development
This project is currently in the pilot stage of development. Four peer-reviewed technical papers have been published on this project at University of Nebraska-Lincoln, based on data collected from healthy speakers. At the UT Dallas Communication Technology Center, data collection from speakers with impaired motor speech for the next stage of development will begin in the near future. A grant has been submitted to the National Institute on Deafness and Other Communication Disorders in support of this project.
The Silent Speech Recognition project is a collaboration among the Callier Center for Communication Disorders and Department of Computer Science, UT Dallas; Department of Computer Science and Engineering, University of Nebraska-Lincoln; and Massachusetts General Hospital in Boston. The team of researchers includes:
Jun Wang, PhD firstname.lastname@example.org
Wang is a post-doctoral fellow and research scientist at UT Dallas’ Callier Center for Communication Disorders. He earned his PhD degree in computer science with a specialty on speech production from University of Nebraska-Lincoln. He was a post-doctoral fellow at the Neurogenic Communication Disorders Consortium, University of Nebraska-Lincoln and University of Nebraska Medical Center, before he joined UT Dallas. His research focuses on the core algorithm development of the Silent Speech Recognition project.
Jordan Green, PhD email@example.com
Green is a professor in the Department of Special Education and Communication Disorders, University of Nebraska-Lincoln. His research focuses on normal and disordered speech production, including motor speech impairment (e.g., due to amyotrophic lateral sclerosis), speech motor learning, speech development, early chewing development. He soon will join Massachusetts General Hospital Institute of Health Professions.
Ashok Samal, PhD firstname.lastname@example.org
Samal is a professor in the Department of Computer Science and Engineering, University of Nebraska-Lincoln. His research focuses on data mining, spatial data analysis, image analysis. He has been working on applying machine learning and data-mining techniques to different fields, including speech communication, geographic information system, crime analysis and biosciences.
Balakrishnan Prabhakaran, PhD – email@example.com
Prabhakaran is a professor of computer science at UT Dallas, specializing in multimedia systems. He is focusing on health-care data and video analytics; streaming of 3D video, animations and deformable 3D models; content protection and authentication of multimedia objects; and collaborative virtual environments. He previously has worked on multimedia databases, authoring and presentation, resource management and scalable web-based multimedia presentation servers.