When patients with suspected influenza visited primary care clinics across Japan, staff obtained their consent for images of their pharynx.
A simple camera with an attached tongue depressor snapped a photo of the back of the throat. Researchers then an artificial intelligence model to analyze the images to accurately diagnose influenza infection.
Now, some of those same researchers devised a deep learning algorithm that analyzed more than 20,000 images of the pharynx and accurately identified patients’ sex assigned at birth. Their findings, published in a Nature Scientific Reports , highlighted anatomical differences between the sexes – a development that may widen understanding about disease progression and potential treatments.
The study’s senior author is , associate professor in the division of general internal medicine and health services research and director of the data core in the department of medicine statistics core at the .
He and co-authors Hiroshi Yoshihara and Memori Fukuda of Aillis, Inc., provided emailed responses to questions, which are lightly edited below.
What is the structure and function of the pharynx?
The pharynx is a muscular tube located behind the nasal and oral cavities, extending down to the larynx and esophagus. It serves as a pathway for both air and food, playing an important role in respiration and digestion by facilitating the passage of air to the lungs and food to the esophagus.
Additionally, the pharynx also serves a function in immune defense. The lymphoid tissue in the pharynx, which includes the tonsils and adenoids, helps protect against pathogens entering through the nose and mouth.
Why did you decide to study pharyngeal images to identify biological sex?
In 2018, researchers at Google found that AI can . This finding surprised doctors and researchers because such differences based on sex have not been previously known in medicine.
Our study is built upon this study, and shows, for the first time, that AI can predict sex from pharyngeal images too. The pharynx is unique in that it is one of the few areas in the body where blood vessels and immune tissues can readily be observed from outside the body non-invasively (similar to retinal images).
Identifying sex through methods beyond self-reporting, such as AI analysis of pharyngeal images, is important because it can uncover subtle anatomical differences that were not previously known in medicine and provide valuable insights into sex-specific health risks and disease manifestations.
Such an objective approach can inform precision medicine by deepening our understanding of how sex-related anatomical variations contribute to disease development and progression, potentially leading to more targeted and effective treatments.
In addition to devising the algorithm, you also had to create a “quantitative interpretation framework” to understand what aspects of the images were important in identifying sex. Why is deep learning so challenging to explain and how does the framework help?
Deep learning algorithms are often difficult to explain because they operate through complex architectures, such as neural networks with many layers and parameters. That makes it challenging to discern how specific inputs are transformed into outputs.
The complexity results in a "black box" nature where the decision-making process is not easily interpretable by humans, particularly in clinical applications where understanding the rationale behind decisions is important.
The quantitative interpretation framework allowed us to extract the areas of the pharynx that influenced the algorithm’s decisions, and better understand where the deep learning algorithm is focusing when predicting sex, leading to improved interpretability of the findings by humans.
The algorithm primarily focused on the posterior pharyngeal wall and the uvula. Why those areas of the pharynx?
The quantitative interpretation framework suggested differences in the size and redness of the uvula. This finding is also consistent with a previous study, which reported that in the context of sleep apnea, men are more likely than women . We thus considered this result reliable.
The approximately 20,000 images analyzed were part of the training data for the algorithm. Your study also “double-checked” the algorithm’s results through a validation method that used a whole other test dataset of nearly 5,000 pharyngeal images. What does validation mean and why was that important to do?
Training data may contain “noise” which can lead to reduced performance on new, unseen data (a phenomenon known as overfitting).
In the context of this study, for example, if there is a difference in the sex ratio of patients at some clinics, and if there are distinctive characteristics in the imaging skills of the clinic’s physicians, there is a possibility that the deep learning algorithm may discriminate based on image features derived from the physician’s imaging skills rather than actual sex differences.
Therefore, it is necessary to evaluate the model’s performance using images taken at clinics that were not included in the training data.
How could the results of your study help to assess systemic health?
As previous that predicted lifestyle-related diseases from retinal images have gained attention, it is believed that there may be functions of organs and links between organs that are still unknown anatomy and physiology.
Uncovering previously unknown pathways of relationships within the body can provide insights into the underlying physiological mechanisms and contribute to new medical discoveries.
How would you like to build on the learnings from this study in future research?
This study investigated the relationship between the pharynx and biological sex, but we believe that unknown relationships may also exist in other areas, such as lifestyle-related diseases. We will continue exploring the unknown potential of pharyngeal images in the future.
More generally, what are your hopes for the responsible development and use of deep learning algorithms in health care?
Deep learning has long been in the spotlight in predictive modelling due to its high accuracy. However, as demonstrated in our study, it can also be used as a tool to explore seeds for new medical discoveries when combined with a well-designed interpretation framework.
We believe deep learning-based exploratory research in health care, such as drug discovery, will become increasingly important.
