The most common communication disorders clinically are speech and hearing impairment. For the lengthening of life expectancy, hearing impairment becomes one of the most important issues in our society. After medical or surgical therapeutic processes, rehabilitation is the last resort to restore the capability of communication. A key component for such restoration depends on an alternative media through which the impaired function can be processed as well as possible. For speech and hearing disorders, communication procedure can usually be augmented by visual stimulation, for example, by signs or text. Owing to the development of world-wide-web system, not only our traditional oro-aural communication mode can be replaced by cyber-text and far-distance communication but also the communication mode of the speech or hearing impaired people can be. For the promising technique of data analysis, we can also build a visual lip-reading system for these patients, either for their post-operation communication or for their later speech rehabilitation period.
The purpose of this research is to build a visual lip-reading system which recognizes the sentence from the image sequences of speakers. At the same time, many face images are taken from different angles through digital cameras to build a vivid 3D human head model. The recognized text drives the 3D human head model to talk as the real person. There are three parts in our framework: lip-reading recognizer, 3D head model generator, and talking face animation. The integrated system combines the web-based visual communication interface. Experimental results show that the recognition rate is about 97 percent of ten sentences for a specific person.
