The Indian government, in its pursuit of enhancing digital service delivery, is constructing language datasets utilizing Bhashini—a language translation system driven by artificial intelligence.
Introduction
In the vibrant tapestry of India, linguistic diversity is not just a facet but a defining characteristic. With over 121 languages spoken by 10,000 people or more, the country poses a unique challenge and opportunity for the field of artificial intelligence (AI). In recent years, India has taken bold steps to harness the power of AI to address linguistic disparities, with groundbreaking projects paving the way for a more inclusive digital landscape. This article delves into the multifaceted initiatives, challenges, and economic implications of India’s foray into AI-driven language solutions.
1. The Karnataka Project: Building India’s First AI-based Chatbot for Tuberculosis
In the southwestern state of Karnataka, a novel endeavor is unfolding. Villagers, speaking the native Kannada language, are actively participating in a project aimed at constructing India’s inaugural AI-based chatbot for Tuberculosis. This ambitious initiative not only underscores the commitment to public health but also serves as a catalyst for language-centric AI innovations.
2. Linguistic Landscape of India: A Challenge for NLP
India’s linguistic diversity is staggering, with more than 40 million Kannada speakers, making it one of the 22 official languages. However, the majority of these languages lack comprehensive coverage in natural language processing (NLP), a crucial branch of AI enabling computers to understand text and spoken words. Kalika Bali, a principal researcher at Microsoft Research India, emphasizes the need for AI tools to cater to those who don’t speak widely recognized languages like English, French, or Spanish.
3. Bhashini and Karya: Crowdsourcing Language Datasets
Recognizing the monumental task of collecting extensive data in Indian languages, initiatives like Bhashini and Karya are pioneering solutions. Bhashini, an AI-led language translation system, is creating open-source datasets in local languages through a crowdsourcing initiative. Villagers in Karnataka and thousands of speakers of different Indian languages contribute speech data for tech firm Karya, which builds datasets for industry giants like Microsoft and Google.
4. Government’s Push: Bhashini and Language Datasets for Digital Services
The Indian government, in its quest to digitize services, is actively contributing to the creation of language datasets. Bhashini, with its AI-led language translation system, is not only a testament to technological advancement but also a strategic move to ensure that digital services cater to the linguistic diversity of the nation. This government-backed initiative aims to provide the foundation for AI tools in various sectors, including education, healthcare, and legal proceedings.
5. Challenges in Collecting Language Data
While the enthusiasm for AI-driven language solutions is palpable, challenges abound. Indian languages often have an oral tradition, and electronic records are not as abundant as in other parts of the world. Additionally, the phenomenon of code mixing, where two or more languages are used within the same context, poses a unique challenge. To overcome these hurdles, a special effort is required to collect data in less common languages.
6. Economic Impact of Speech Data: Karya’s Approach
One of the noteworthy aspects of Karya’s approach is its focus on economic empowerment. The tech firm collaborates with non-profit organizations to identify workers below the poverty line and pays them above the minimum wage in India for generating data. Contributors, who own a part of the data they generate, receive royalties, presenting a unique economic model that has the potential to benefit communities in areas like healthcare and farming.
7. AI for Social Enterprises: Gram Vaani and Grassroots Empowerment
Beyond economic value, AI is making a tangible impact on social enterprises. Gram Vaani, translating to “voice of the village,” utilizes AI-based chatbots to respond to questions on welfare benefits. These initiatives showcase the transformative power of automatic speech recognition technologies, mitigating language barriers and providing outreach at the grassroots level.
8. The Pioneering Initiatives Beyond Language: Google’s Project Vaani, EkStep Foundation, and AI4Bharat
India’s quest for linguistic inclusivity extends beyond the initiatives mentioned. Google’s Project Vaani, funded by Google, collects speech data from about 1 million Indians, open-sourcing it for use in automatic speech recognition and speech-to-speech translation. Bengaluru-based EkStep Foundation employs AI-based translation tools at the Supreme Court in India and Bangladesh. The government-backed AI4Bharat center has launched Jugalbandi, an AI-based chatbot that answers questions on welfare schemes in several Indian languages.
9. Future of AI in Preserving Languages: Academia, Preservation, and Demand
As AI continues its rapid growth, there is a demand for languages that were previously overlooked. Academics are exploring ways to preserve languages through AI, ensuring that the cultural and linguistic heritage is not lost. The demand for languages “we haven’t even heard of” presents an exciting frontier for AI applications, including those looking to preserve endangered languages.
10. Challenges and Considerations in AI Crowdsourcing: Ethics, Bias, and Scale
While crowdsourcing emerges as a powerful tool for collecting language data, ethical considerations are paramount. Ensuring awareness of gender, ethnic, and socio-economic biases is crucial. The process must be conducted ethically, with workers educated, fairly compensated, and efforts made to collect data for smaller languages. The scalability of these initiatives hinges on ethical practices.
Also read about BharatGPT
Conclusion
India’s venture into AI-driven language solutions not only represents a technological leap but also a commitment to inclusivity and economic empowerment. The confluence of linguistic diversity, crowdsourcing, and AI is transforming the digital landscape, making cutting-edge AI applications accessible to millions. As India continues to pioneer solutions for language-related challenges, it sets a precedent for global efforts in ensuring that the benefits of AI reach every corner of society, irrespective of language barriers.