Baidu Launches AI Platform to Enable on-Device, Real-Time Translation from Speech to Hand Gestures
Baidu AI Cloud has launched an innovative AI sign language platform, XiLing, designed to create digital avatars for sign language translation and interpretation, addressing communication barriers for the deaf and hard-of-hearing community. This platform will debut during the Beijing 2022 Winter Paralympics. It includes all-in-one AI sign language translators, optimized for various public service environments, significantly improving accessibility.
With China hosting 27.8 million DHH individuals but only 10,000 qualified translators, the XiLing platform promises to enhance communication, particularly in medical and legal contexts.
- Launch of XiLing AI sign language platform enhances communication for 27.8 million DHH individuals in China.
- All-in-one AI sign language translators facilitate easier public service access.
- Significant reduction in production and operational costs for digital avatars, allowing for scalable service.
- Collaborative efforts with over 500 scholars to improve the sign language corpus.
- None.
BEIJING, March 3, 2022 /PRNewswire/ -- Baidu AI Cloud, a leading AI cloud provider, launched an AI sign language platform able to generate digital avatars for sign language translation and live interpretation within minutes. Released as a new offering of Baidu AI Cloud's digital avatar platform XiLing, this platform aims to help break down communication barriers for the deaf and hard-of-hearing (DHH) community by boosting the accessibility of automated sign language translation. An AI sign language interpreter developed using the platform will perform its duties during the upcoming Beijing 2022 Winter Paralympics Games.
Also released along with the platform are two all-in-one AI sign language translators, providing one-stop solutions with a streamlined set-up process and plug-and-use features. By enabling public service deployment in scale, the translators have been designed for a wide range of use scenarios such as hospitals, banks, airports, bus stations and other public areas.
With the technology enablement brought by AI, the production and operational costs of digital avatars have been reduced to a significant degree, making it possible for AI sign language to go scale and serve more deaf and hard-of-hearing individuals, said Tian Wu, Baidu Corporate Vice President.
Today, China is home to 27.8 million deaf and hard-of-hearing (DHH) individuals but is faced with a massive shortage of qualified professionals to serve their needs, with no more than 10,000 sign language translators, a gap especially felt in medical and legal settings.
The XiLing AI sign language platform and the all-in-one sign language translators are designed to fill this significant gap and address the communication difficulties facing the DHH community in both online and offline settings. For DHH individuals who want to study or socialize online without barriers, the platform can be quickly integrated into commonly used mobile applications, websites, and mini programs within a few hours, performing functions like sign language video synthesis and livestream synthesis, text-to-sign language translation, and audio-to-sign language translations.
The all-in-one translators are tailored for offline scenarios to improve the accessibility of public services. Baidu's translators come with two models, a full offline version V3, and a cloud-connected version P3. Both are embedded with core functions of the AI sign-language platform, able to realize ASR speech recognition, speech translation, and portrait rendering. This full range of functions offers incredible potential for empowering the DHH. For instance, DHH individuals will be able to visit the hospital and manage the complicated process of registration, consultation, payment, and medicine collection without further assistance. Additional applications hold the potential to allow the DHH community to travel, dine, and even work independently.
Technical Deep Dive
Compared to translations between spoken languages, the sign language translation is more complicated mainly because it is not translated word by word from verbal speech. Instead, the language refinement and word order must be adjusted in order to show the actual meaning of the sentence. As a relatively rarely-used language, a very limited amount of data on sign language is available for machine learning. It also requires lip language and facial expressions to assist understanding. In real-world settings, solutions are often faced with complex environmental factors making them difficult to deploy. All these practical barriers have posed numerous challenges to the development of AI sign language.
To make AI sign language comprehendible, Baidu scientists had to resolve three key challenges: the clarity of speech recognition, the accuracy of sign language translation, and the fluency of sign language movements.
To address speech recognition clarity, the XiLing AI sign language platform uses Baidu's home-grown SMLTA speech recognition model to achieve end-to-end modeling speech recognition through integrating acoustics and language. Based on Baidu's self-developed deep learning algorithm, targeted training can enable word accuracy in a wide range of fields such as tourism, medical care, and legal proceedings.
In terms of the accuracy and refinement of sign language translation, Baidu has built the first neural network-based sign language translation model with a controllable degree of refinement, which can automatically learn sign language translation knowledge from real data such as word order adjustment, word mapping and length control to generate natural sign language that conforms to the habits of hard-of-hearing people.
To ensure the accuracy of the sign language translation, Baidu has invited over 500 scholars and students with hearing loss in China to help enlarge and vet the sign language corpus, with many joining the project as volunteers. Tiantian Yuan, associate dean of Technical College for the Deaf, Tianjin University of Technology, said she and her students feel incredibly honored to have contributed their parts in collaborating with Baidu to fill in this gap for the community.
To ensure the fluency of sign language actions, the AI sign language platform has sorted nearly 11,000 actions based on the National Universal Sign Language Dictionary with its "action fusion algorithm", so that all digital sign language gestures have the degree of coherency and expression as human sign language. In addition, with the help of 4D scanning technology, the accuracy of mouth shape generation has been optimized up to
About Baidu:
Founded in 2000, Baidu's mission is to make the complicated world simpler through technology. Baidu is a leading AI company with strong Internet foundation, trading on the NASDAQ under "BIDU" and HKEX under "9888." One Baidu ADS represents eight Class A ordinary shares.
Media Contact
intlcomm@baidu.com
View original content to download multimedia:https://www.prnewswire.com/news-releases/baidu-launches-ai-platform-to-enable-on-device-real-time-translation-from-speech-to-hand-gestures-301495116.html
SOURCE Baidu, Inc.
FAQ
What is Baidu's new AI sign language platform called?
When was the XiLing platform launched?
How does the XiLing platform help the deaf and hard-of-hearing community?
What event will feature the AI sign language interpreter developed by Baidu?
How many deaf and hard-of-hearing individuals are there in China?
What is the significance of the all-in-one AI sign language translators?
