India’s AI-powered human translation platform Devnagri is organising its data, essential for managing diverse linguistic content across languages to achieve scalability, facilitate the integration of expanding translation datasets and improve user interactions.

Devnagri’s co-founder and CEO, Himanshu Sharma told iTnews Asia that the company initially used basic databases including MySQL but faced limitations with the complexity and scale of content.

“As operations expanded to include translation tasks involving millions of sentences, there was a need for a scalable and responsive database solution,” he added.

Devnagri’s proprietary translation model covers 22 Indian languages and is tailored for business communication across industries including e-commerce, BFSI, publishing industry and EdTech.

It has trained the model with a dataset including 500 million sentences and was facing difficulties to efficiently manage and process the data.

The platform also experienced latency issues, with an average response time of one to two seconds per request for user queries or interactions.

Devnagri migrated to MongoDB for improving API responsiveness, scalability and to achieve real-time tracking of deliveries.

This resulted in bringing down the response time to an average of 400 to 600 microseconds per request.

The migration could decrease the translation time to an average of 400 microseconds per segment and by upto 60 percent.

Sharma said that before implementing MongoDB Atlas, the average translation saving time for Devnagri’s content was one second per segment.

Improving scalability

MongoDB Atlas offers a fully managed database service to ensure scalability, and security while minimising maintenance efforts, Sharma said.

Atlas Charts, the native data visualisation tool has equipped Devnagri with data visualisation capabilities, facilitating data driven decision-making and better performance monitoring.

The other features including indexing has helped the platform enhance data retrieval speed, replication to facilitate data distribution across machines and aggregation for pipeline-based data analysis and manipulation.

Sharma said the firm encountered challenges related to data migration, schema design, and performance optimisation during the implementation phase.

“Through collaboration with MongoDB’s support and leveraging resources like documentation and community forums, we could ensure a smooth transition and optimise database performance,” he added.

Devnagri is currently working on an In-Context Learning Engine (ICL) to improve translation accuracy and efficiency in real time, as the engine aims to adapt to the tone of specific brands and reduce human intervention in the translation process.

Unlike previous methods that required lengthy data collection and model rebuilding cycles, Sharma said ICL would operate in real time, leveraging feedback from translators to adjust and enhance translations instantly.

This approach will “break the traditional cycle of waiting” for periodic model updates, providing customers with updated and precise translations with each subsequent sentence, he added.

Source: https://www.itnews.asia/news/indias-devnagri-platform-achieves-scalability-with-database-migration-606189