RMIT student uses machine learning to decode doctors’ handwriting

RMIT student uses machine learning to decode doctors’ handwriting

An RMIT University student has helped solve a big challenge standing in the way of digitising massive Vietnamese medical records: deciphering doctors’ notoriously difficult-to-read handwriting.

Working closely with the Hospital of Tropical Disease (HTD) and Oxford University Clinical Research Unit (OUCRU), Bachelor of Software Engineering student Phung Minh Tuan successfully developed an end-to-end pipeline to recognise text on scanned Vietnamese medical records, potentially accelerating the digitisation of medical health records that the Government initiated in 2019. 

news-1-rmit-student-uses-machine-learning-to-decode-doctors-handwriting Bachelor of Software Engineering student Phung Minh Tuan successfully developed an end-to-end pipeline to perform text recognition on scanned Vietnamese medical records.

Overcoming the complexities of Vietnamese handwriting

“There have been great advances in handwritten text recognition, however, most existing methods have been developed for English language and there is little-to-no methods dedicated for Vietnamese language,” shared Tuan, now in his final year with the School of Science & Technology at RMIT.

“Vietnamese handwritten text recognition is fundamentally more challenging than that of English due to the presence of more character classes, complex vocals and tonal symbols.”

It took Tuan more than three months of trial and error to figure out the most effective way to turn an image of a hard-copy medical document into an electronic version. 

news-2-rmit-student-uses-machine-learning-to-decode-doctors-handwriting An illustration of perspective warping operation

“We tackled those challenges by enhancing various tasks in the text recognition pipeline,” he said.

“We applied a denoising process, performed text segmentation at a word level, and applied the Bigram language model to improve the probabilities of the possible correction given the neighbouring words.

“Most importantly, we combined and implemented a deep learning architecture which consists of a ResNet for feature extraction, a BiLSTM network for text sequence modelling, [both artificial neural networks], and CTC for final transcription task.

“At that point, the final output sequence aligned to a lexicon to further boost the accuracy.”

news-3-rmit-student-uses-machine-learning-to-decode-doctors-handwriting Phung Minh Tuan (pictured left) and RMIT School of Science & Technology lecturer and his supervisor Dr Dinh Ngoc Minh (pictured right).

A door has been opened to a better treatment process

RMIT School of Science & Technology lecturer and Tuan’s supervisor Dr Dinh Ngoc Minh emphasised the promising pipeline results, which can ultimately facilitate the digital transformation of medical centres and hospitals in Vietnam, to improve their readiness for adopting modern electronic health record management systems.

“The proposed work can accelerate the digitisation process of medical health record systems,” Dr Minh said.

“With the help of the machine in processing all the records, health facilities could gradually shift to an electronic system without any sudden change in protocol.

“Such systems would also allow remote medical centres or field health workers with limited computer access to continue with paper systems which could then be digitised easily.”

Dr Minh said that the patient records could be shared simply between departments, which could help reduce unnecessary tests and optimise treatment, to eventually improve health care quality.

“And most importantly, Tuan’s work can generate a digital medical-note dataset for a variety of potential medical machine-learning solutions,” Dr Minh said.

“In fact, our collaborators HTD and OUCRU are planning to use the generated data to develop expert diagnostic systems, improve the treatment process and minimalise errors in healthcare practices.

“Tuan has already proven himself as a professional software developer, and a quick-learner who has expanded his machine learning knowledge significantly during the six months of working on this project.”

Through this work, Tuan has been offered an internship at the OUCRU, and his work has been presented at the ACIS2020, an A-ranked international conference, the AHT conference, as well as the University’s recent student showcase.

Story: Hoang Ha

08 December 2020

Share

  • Research
  • Community
  • Engineering
  • Student projects
Engage

Related news

Teachers in Can Tho city practice using generative AI tools.

Over 1,400 educators attend free AI training delivered by RMIT
Icon / Small / Calendar Created with Sketch. 20 Dec 2024

RMIT Vietnam has delivered a free training program for teachers across Vietnam to enhance their capabilities in applying AI technologies in teaching and learning activities.

HCMC metro logo

How to transform the Ben Thanh - Suoi Tien metro into a city icon
Icon / Small / Calendar Created with Sketch. 19 Dec 2024

The Ben Thanh - Suoi Tien metro launch marks a milestone in Ho Chi Minh City's transport and is a chance to build a modern urban symbol, that shifts public habits from private vehicles to sustainable public transport.

Computer game designer

It's time for Vietnam's game industry to tell its own story
Icon / Small / Calendar Created with Sketch. 19 Dec 2024

Vietnam's game industry is booming, with revenue projected to reach US$2.7 billion by 2026. To lead in original content, it needs creative designers who blend art, science, and UX – not just skilled programmers.

Cyber security icons

How Vietnam's new Data Law transforms cyber security and economy
Icon / Small / Calendar Created with Sketch. 17 Dec 2024

Vietnam's new Data Law is a major step in strengthening cyber security and data protection. It addresses key challenges while promoting digital economic growth and enhancing international collaboration.

Load more +