AI-powered MedVoice: Enhancing medical documentation

AI-powered MedVoice: Enhancing medical documentation

MedVoice, an innovative AI-powered solution developed by RMIT’s students, holds immense potential to transform medical documents by automating transcription tasks. This advancement not only enhances efficiency but also allows healthcare professionals to devote more time to what truly matters – patient care.

Developed through the collaborative efforts of five students from the Bachelor of Information Technology program, School of Science, Engineering & Technology – Do Le Long An, Tran Minh Nhat, Vu Quoc Gia Quan, Nguyen Ha Kieu Anh, and Ho Buu Quoc Phong, MedVoice intersects artificial intelligence, healthcare efficiency, and student innovation.

Alt Text is not present for this image, Taking dc:title 'news-1.1-ai-powered-medvoice-enhancing-medical-documentation' [L-R] The RMIT students developing MedVoice: Vu Quoc Gia Quan, Tran Minh Nhat, Nguyen Ha Kieu Anh, Do Le Long An and Ho Buu Quoc Phong.

From live audio to structured medical documents

Healthcare professionals worldwide share a common struggle: time-consuming, error-prone documentation processes. Traditional methods consume valuable time that could be better spent on patient care, creating a critical gap that the MedVoice team has capably addressed.

At the heart of the tool lies a sophisticated technological ecosystem leveraging cutting-edge AI technologies. Medical staff sign in via their email and a one-time password (OTP). Once logged in, users can record patient conversations with a live transcript displayed in the app. The data is sent to a Medical Archive page, where a chatbot answers queries on patient information. Large Language Models (LLMs) process patient audio recordings that are sent to Google Cloud, retrieved by the FastAPI backend, and processed through the Whisper diarisation model to speakers and generates transcripts. The transcripts are fed into Meta Llama3-70B, formatted into JSON, and posted to the front end. The background process involves transcription and structuring of patient audio data, followed by embedding and storage using the ollama/nomic-embed-text model and Vector Database (pgvector). This ensures efficient data retrieval and compliance with security standards.

Alt Text is not present for this image, Taking dc:title 'news-2-ai-powered-medvoice-enhancing-medical-documentation' MedVoice, an innovative AI-powered solution developed by RMIT’s students, holds immense potential to transform medical documentation by automating transcription tasks.

“By utilising advanced ‘speech-to-text’ technology and ‘speaker diarisation’, MedVoice captures spoken words with precision and organises them into structured, accurate medical records,” explained team member Vu Quoc Gia Quan. “Tailored specifically for Vietnamese healthcare settings, the tool has been refined to handle complex medical terminology, noisy environments, and multi-speaker scenarios.”

Team member Tran Minh Nhat described building MedVoice as a challenging yet rewarding journey. “We tackled transcription accuracy by selecting optimised AI models to handle Vietnamese medical terms and noisy environments,” Nhat said. “For real-time efficiency, we fine-tuned separate models for Android and iOS to ensure seamless performance. We also prioritised data privacy, ensuring sensitive medical information is protected. Lastly, we improved AI-model performance on low-resource languages, making our tool accessible and reliable for healthcare professionals across Vietnam.”

This comprehensive approach underscores the team’s dedication to creating an effective and reliable solution.

Great learning experience and potential expansion

MedVoice's effectiveness has been demonstrated through both technical benchmarks and its recognition at major events, including AI4VN 2024, Vietnam’s largest AI-focused event, and RMIT Experience Day 2024, the University’s largest annual showcase, said team member Ho Buu Quoc Phong. 

Alt Text is not present for this image, Taking dc:title 'news-3-ai-powered-medvoice-enhancing-medical-documentation' RMIT IT Senior Lecturer Dr Arthur Tang presented MedVoice at AI4VN 2024.

At AI4VN 2024, MedVoice was featured during a keynote presentation by RMIT academics Dr Arthur Tang and Professor Fabio Zambetta, highlighting its innovation in applying AI to healthcare challenges. This recognition placed MedVoice at the forefront of cutting-edge AI solutions in Vietnam and underscored its potential impact on the healthcare industry.

MedVoice also stood out at RMIT Experience Day, which attracted more than 11,500 participants across Hanoi and Saigon South campuses. As one of over 100 workshops, it was chosen to showcase RMIT’s contributions to innovation and healthcare. Participants had the opportunity to experience hands-on demonstrations of MedVoice, which highlighted its ability to simplify medical documentation. Attendees, including students, parents, and healthcare professionals, praised the tool’s intuitive interface and its flexibility in allowing patient information to be edited directly within the documentation.

Team member Nguyen Ha Kieu Anh said that the true innovation happened when team members shared ideas and tackled challenges together. On behalf of the team, Kieu Anh expressed their gratitude for the guidance and feedback they received from their lecturers and supervisors, Dr Arthur Tang and Tom Huynh, which shaped their approach, teaching them to refine their ideas and remain focused on their goals.

“RMIT's RACE Hub, gave us hands-on experience with cutting-edge tools," Kieu Anh said. "This project also taught us perseverance. Late nights and countless iterations showed that progress requires patience and adaptability. Innovation is a shared effort rooted in mutual trust and a shared vision."

The team's vision extends beyond Vietnam. They aim to create a scalable solution that can address medical documentation challenges in low-resource healthcare settings globally.

“Future developments include improving support for regional Vietnamese dialects, enhancing AI model performance, and integrating with electronic health record (EHR) systems,” said team member Do Le Long An.

MedVoice represents more than a technological solution - it's a testament to how AI can improve healthcare. By reducing administrative burdens, the project promises to give healthcare professionals more time to do what they do best: provide compassionate, high-quality patient care. As Dr Arthur Tang noted, this is about leveraging research for impactful innovation and MedVoice is proving that innovation can indeed transform healthcare, one conversation at a time.

This project was funded by RMIT Vietnam Strategic Innovation Challenge and supported by computing resources from the RMIT AWS Cloud Supercomputing Hub (RACE Hub).

Story: Ha Hoang

Masthead image: Have a nice day - stock.adobe.com

Related news