Analysis and Implementation of Lip Reading (Lip Movement) to Text Conversion for English Language Use

Abstract

In the rapidly evolving era of globalization, communication plays a crucial role. Understanding English, as a global language, is increasingly important for effective communication. Challenges arise when individuals need to comprehend the written form of English based on diverse pronunciations, especially for English language learners. The differences between pronunciation and written forms, along with various accents and dialects, add complexity. This research proposes an innovative method based on deep learning and Convolutional Neural Network (CNN) to translate lip movements into English text. The use of this technology is expected to assist individuals struggling to comprehend English text when spoken, enhancing communication skills, and enabling effective participation in critical situations that require oral English comprehension. Experimental results demonstrate the potential of this system in detecting lip movements in videos and translating them into text, although it still faces challenges related to variations in mouth and face positions, as well as pronunciation differences. The implementation of this technology represents a significant initial step in overcoming language barriers and facilitating improved communication in this era of globalization.