Article Details

Abstract

Ocular surface fluorescein staining images hold significant clinical value in the diagnosis and assessment of dry eye. However, manual evaluation is time-consuming and labor-intensive, with scores among the different physicians affecting consistency. To enhance diagnostic efficiency and accuracy, this paper proposes OFGD-Net (Ocular Fluorescence Grading and

Description Network), an automated model based on Deep Learning, which is used for grading and describing ocular surface fluorescein staining images. OFGD-Net comprises an image encoder, two decoders, and an Attention Mechanism. The encoder is responsible for extracting features, the decoders generate image description and lesion severity scores separately, and the Attention Mechanism enhances the attention to key regions. Comparative analysis with current advanced models demonstrates that the performance of OFGD-Net is superior in primary evaluation metrics, and it exhibits significant advantages in the accuracy and similarity of generated text.

Keywords

Deep Learning; multi-mode; dry eye; ocular surface fluorescein staining

References

1. [1] ERIN C O，MATTHEW H，MINA M-G，et al. Advances in Dry Eye Disease Treatment [J].Current Opinion in Ophthalmology，2019，30（3）：166-178.
2. [2] PELLEGRINI M，BERNABEI F，MOSCARDELLI F，et al. Assessment of Corneal Fluorescein Staining in Different Dry Eye Subtypes Using Digital Image Analysis [J].Translational Vision Science & Technology，2019，8（6）：34.
3. [3] CHAN H-P，HADJIISKI L M，SAMALA R K. Computer-aided Diagnosis in the Era of Deep Learning [J].Medical Physics，2020，47（5）：e218-e227.
4. [4] HÈDE P，MOËLLIC P-A，BOURGEOYS J，et al. Automatic Generation of Natural Language Description for Images [C]//RIAO'04.Paris：ACM，2004：306-313.
5. [5] ZHOU J W，WANG X，HAN J Z，et al. Spatial-temporal Attention for Image Captioning [C]//2018 IEEE Fourth International Conference on Multimedia Big Data （BigMM）.Xi'an：IEEE，2018：1-5.
6. [6] YANG X，TANG K H，ZHANG H W，et al. Auto-encoding Scene Graphs for Image Captioning [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach：IEEE，2019：10685-10694.
7. [7] FANG H，GUPTA S，IANDOLA F，et al. From Captions to Visual Concepts and Back [C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Boston：IEEE，2015：1473-1482.
8. [8] LU J S，XIONG C M，PARIKH D，et al. Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu：IEEE，2017：3242-3250.
9. [9] SHIN H-C，ROBERTS K，LU L，et al. Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Las Vegas：IEEE，2016：2497-2506.
10. [10] WANG X S，PENG Y F，LU L，et al. TieNet: Text-image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-Rays [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：9049-9058.
11. [11] LIU G X，HSU T-M H，MCDERMOTT M，et al. Clinically Accurate Chest X-Ray Report Generation [J/OL].arXiv:1904.02633 [cs.CV].（2019-04-04）.https://arxiv.org/abs/1904.02633v1.
12. [12] JING B Y，XIE P T，XING E. On the Automatic Generation of Medical Imaging Reports [J/OL].arXiv:1711.08195 [cs.CL].（2017-11-22）.https://arxiv.org/abs/1711.08195.
13. [13] XUE Y，XU T，LONG L R，et al. Multimodal Recurrent Model with Attention for Automated Radiology Report Generation [C]//Medical Image Computing and Computer Assisted Intervention-MICCAI 2018：21st International Conference.Granada：Springer Nature，2018：457-466.
14. [14] ZHANG Y X，WANG X S，XU Z Y，et al. When Radiology Report Generation Meets Knowledge Graph [J].Proceedings of the AAAI Conference on Artificial Intelligence，2020，34（7）：12910-12917.
15. [15] AMPARO F，WANG H B，YIN J，et al. Evaluating Corneal Fluorescein Staining Using a Novel Automated Method [J].Investigative Ophthalmology & Visual Science，2017，58（6）：BIO168-BIO173.

Funding

Research and Practice Project on Higher Education Teaching Reform in Henan Province in 2023 (Graduate Education Category) (2023SJGLX112Y); Research and Practice Project on Higher Education Teaching Reform in Henan Province in 2024 (2024SJGLX0332)

About the authors

Zhang Wanyu (1999.08-), female, Han ethnicity, from Nanyang, Henan Province, is currently pursuing a master's degree in artificial intelligence; Corresponding author: Li Xiuli (1967.12-), female, Han ethnicity, from Zhengzhou, Henan Province, professor, undergraduate, research direction: computer technology and applications.