Can AI Identify Mpox Skin Lesions From Images?

🔍 Key Finding A deep-learning convolutional neural network (CNN) accurately classified mpox skin lesions from photographic images and was integrated into a personalized recommendation system app to facilitate patient guidance regarding testing and vaccination. The CNN demonstrated high sensitivity and specificity for mpox lesion detection across various skin tones and body regions, with potential for aiding in outbreak mitigation, though further real-world evaluation with larger datasets is needed.

🔬 Methodology Overview

• Data Sources: MPXV skin lesion images (n=676) from scientific literature, encyclopedias, news articles, social media, and a prospective cohort at Stanford University Medical Center; Non-MPXV skin lesion images (n=138,522) from five public dermatological repositories, two public datasets, and one institutional dataset.
• Image Selection and Annotation: Duplicate images removed; Images excluded based on absence of skin lesion, multiple lesions in one image, medical interventions, non-photographic images, or inaccessibility; MPXV dataset manually labeled for age, sex, skin tone, location, lesion number, duration, and association with the 2022 outbreak.
• Data Splitting: Datasets split into training (MPXV n=518, non-MPXV n=12,045) and testing (MPXV n=158, non-MPXV n=126,477) cohorts. Prospective cohort images used exclusively in the testing set.
• Image Processing and Training Algorithm: Images resized, data augmentation applied (cropping, flipping, rotation, zoom, warping, brightness/contrast adjustments); Transfer learning using a pre-trained ResNet34 CNN architecture, fine-tuned with skin lesion images; Weighted categorical cross-entropy loss function to address class imbalance.
• Algorithm Evaluation: Five-fold cross-validation on the training cohort; Sensitivity, specificity, and area under the curve (AUC) calculated for validation and testing cohorts; Subgroup analyses performed based on image characteristics, skin disease, body region, skin tone, age, and sex.
• Explainability: SHapley Additive exPlanations (SHAP) used to identify image regions contributing to MPXV classification.
• Personalized Recommendation System (PRS) Development: Web-based app (“PoxApp”) created, integrating MPXV-CNN with a survey for symptom and risk factor assessment; Personalized recommendations generated regarding testing, vaccination, and quarantine; Anonymous data donation feature included for future model development.

📊 Results Here are the key results and findings from the research paper, presented in bullet points:

• Dataset: A dataset of 139,198 skin lesion images was assembled, including 676 MPXV images and 138,522 non-MPXV images from various sources. The prospective cohort included 63 images from 12 male patients.
• Validation Cohort Performance: Sensitivity of 0.83, specificity of 0.965, and area under the curve (AUC) of 0.967.
• Testing Cohort Performance: Sensitivity of 0.91, specificity of 0.898, and AUC of 0.966. Sensitivity was 0.89 in the prospective cohort.
• Performance by Lesion Duration: High detection performance for lesions present <7 days (TPR = 95.7%) and for lesions present ≥7 days (TPR = 84.6%).
• Performance by Skin Tone: TPRs varied by skin tone, ranging from 85.7% to 100%. Higher false positive rates (FPRs) were observed for Fitzpatrick skin types V (12.1%) and VI (13.9%).
• Performance by Disease Type: When classifying MPXV vs. other acute skin diseases, specificity was 0.886 and AUC was 0.962. For MPXV vs. chronic skin diseases, specificity was 0.900 and AUC was 0.967.
• Web-Based App (PoxApp): A web-based app was developed to provide personalized risk assessment and guidance based on user-provided images and survey responses.

💡 Clinical Impact This deep-learning algorithm can identify mpox skin lesions from photographic images, potentially enabling earlier detection and isolation of infected individuals to mitigate outbreaks. Integrating this algorithm into a personalized recommendation system could accelerate appropriate care-seeking and improve adoption of behaviors to reduce onward transmission.

🤔 Limitations

• The MPXV-CNN makes predictions even if the image quality is low (e.g., poor lighting, blurriness).
• The MPXV-CNN has not been optimized for mobile devices due to model complexity and a high number of parameters.
• The current scarcity of MPXV photographic images may introduce biases.
• The reliance on case reports and media articles for MPXV images may overrepresent atypical or extraordinary cases.
• The overrepresentation of darkly pigmented skin in the MPXV dataset may limit generalizability.
• The Stanford University Medical Center patient population may not be representative of the broader US population.
• A mobile app using the MPXV-CNN should not be a substitute for medical tests or professional evaluation.
• The performance of the MPXV-CNN for specific differential diagnoses (e.g., orf, varicella) needs further improvement.
• The reliance on user-reported symptoms and risk factors introduces potential for error in the personalized recommendation system.
• The MPXV infection status of images used in the personalized recommendation system is unknown at the time of use.

✨ What It Means For You This deep-learning algorithm can identify mpox skin lesions from photographic images, potentially aiding in earlier diagnosis and outbreak mitigation. Integrating this algorithm into a patient-facing app, combined with a personalized recommendation system, could facilitate timely testing and postexposure vaccination, but requires careful consideration of ethical implications and limitations to avoid replacing formal medical evaluation.

Reference Thieme AH, Zheng Y, Machiraju G, Sadee C, Mittermaier M, Gertler M, Salinas JL, Srinivasan K, Gyawali P, Carrillo-Perez F, Capodici A, Uhlig M, Habenicht D, Löser A, Kohler M, Schuessler M, Kaul D, Gollrad J, Ma J, Lippert C, Billick K, Bogoch I, Hernandez-Boussard T, Geldsetzer P, Gevaert O. A deep-learning algorithm to classify skin lesions from mpox virus infection. Nat Med. 2023;29:738-747. https://doi.org/10.1038/s41591-023-02225-7