A Visual Communication System for Hear-impaired and Talk-impaired Patients

CNU IR > College of Smart Living > Dept. of Information Management > CNU Project > Item 310902800/1231

Please use this identifier to cite or link to this item: https://ir.cnu.edu.tw/handle/310902800/1231

Title:	A Visual Communication System for Hear-impaired and Talk-impaired Patients 聽障者與不能發音者的電腦輔助溝通系統
Authors:	黃文楨;Wen-Chen Huang
Contributors:	資訊管理系
Keywords:	lip-reading spatial-temporal image difference pattern recognition 3D reconstruction
Date:	2003
Issue Date:	2008-06-30 10:33:22 (UTC+8)
Publisher:	台南縣：嘉南藥理科技大學資訊管理系
Abstract:	The most common communication disorders clinically are speech and hearing impairment. For the lengthening of life expectancy, hearing impairment becomes one of the most important issues in our society. After medical or surgical therapeutic processes, rehabilitation is the last resort to restore the capability of communication. A key component for such restoration depends on an alternative media through which the impaired function can be processed as well as possible. For speech and hearing disorders, communication procedure can usually be augmented by visual stimulation, for example, by signs or text. Owing to the development of world-wide-web system, not only our traditional oro-aural communication mode can be replaced by cyber-text and far-distance communication but also the communication mode of the speech or hearing impaired people can be. For the promising technique of data analysis, we can also build a visual lip-reading system for these patients, either for their post-operation communication or for their later speech rehabilitation period. The purpose of this research is to build a visual lip-reading system which recognizes the sentence from the image sequences of speakers. At the same time, many face images are taken from different angles through digital cameras to build a vivid 3D human head model. The recognized text drives the 3D human head model to talk as the real person. There are three parts in our framework: lip-reading recognizer, 3D head model generator, and talking face animation. The integrated system combines the web-based visual communication interface. Experimental results show that the recognition rate is about 97 percent of ten sentences for a specific person.
Relation:	計畫編號：CNMI9204
Appears in Collections:	[Dept. of Information Management] CNU Project

Files in This Item:

File	Description	Size	Format
92CNMI9204.pdf		107Kb	Adobe PDF	990	View/Open

Loading...