Kyorin University and NTT DATA to Collaborate on AI Diagnosis of Facial Paralysis

Kyorin University and NTT DATA launched a joint research project in December 2020 to realize a quantitative evaluation of facial paralysis in clinical practice.

In this joint research, NTT DATA is developing an application to evaluate the clinical course of facial paralysis using AI facial expression recognition technology. Actual patient data collected from Kyorin University Hospital (hereafter referred to as "the hospital") will be used in the verification process.

Through this joint research, Kyorin University and NTT DATA will establish a new evaluation method for facial paralysis and promote the use of AI in medical diagnosis and treatment.

Background

The Yanagihara method is a method commonly used to evaluate facial paralysis. In this method, doctors visually evaluate the patient's face to assess the facial movements and diagnose paralysis. The Yanagihara method is superior in that it does not require special equipment and can be used easily in clinical settings. However, the evaluation using this method is subjective; the evaluation result therefore varies depending on the doctor who evaluates. This method is lack of reproducibility and consistency. A new objective and quantitative evaluation method have been required to diagnose facial paralysis.

Currently, a variety of objective evaluation methods are being studied, including methods that use electrophysiological techniques and methods that use image processing to capture video images of the patient's face. However, these methods face several challenges. First, these methods usually require using special equipment. Second, patients must keep their head as still to take multiple facial movement patterns, which stress them and makes it difficult to make natural facial expressions due to facial stiffness.

Overview

In this joint research, we are developing a new method to take patient's facial movements without adding stress for patients. Our method computes the left-right asymmetry based on the results of AI facial expression recognition technology that can recognize the position of each part of the face in video images, so that doctors can objectively assess and diagnose facial expression paralysis.

The roles of Kyorin University and NTT Data in this joint research are as follows:

Kyorin University:

• provides medical examination videos necessary for verification and evaluation of Clinical Progress Evaluation Application.
• evaluates the use of the Clinical Progress Evaluation Application in the actual diagnostic process in Kyorin University Hospital.

NTT DATA:

• develops the Clinical Progress Evaluation Application using AI facial expression recognition technology.
• verifies the application using the video images collected in the actual diagnostic process in Kyorin University Hospital.

In addition to a quantitative evaluation based on the calculated paralysis score, a doctor specializing in facial paralysis evaluates the application based on its usefulness in the actual diagnostic process, to realize an application that can support the doctor’s quantitative evaluation and diagnosis.

The Clinical Progress Evaluation Application uses AI technology to automatically detect landmarks in each part, such as the eyes and nose, in the videos of the patient's facial expressions. The left-right asymmetry of each facial region is computed and evaluated based on the detected landmarks to diagnose facial paralysis.

This application will deliver two benefits:

1. It improves patient experience with imaging processing technology that can reduce the burden of the examination on the patient by automatically correcting the face orientation to the front without fixing the head during video recording.
2. It is compatible with standard RGB cameras, so it is easy to adopt in clinical settings.

Future Goals

A variety of factors may cause facial paralysis. For example, central facial paralysis is caused by brain damage, and peripheral facial paralysis is caused by trauma to the facial nerve. Facial paralysis diagnosis requires consultation from multiple disciplines such as plastic surgery, otolaryngology, and neurology. Kyorin University and NTT DATA will develop this research first from the field of plastic surgery, and establish a new evaluation method directly related to actual clinical settings of facial paralysis in a wide range of fields including plastic surgery, otorhinolaryngology, and neurology. We plan to validate and verify new applications using AI facial expression recognition technology for quantitative medical evaluation of diseases other than facial paralysis.

Kyorin University and NTT DATA will continue to accelerate digitalization with AI in the medical field and support medical diagnosis utilizing information technology.