At 4 p.m. on November 7, Secretary Gao Yang of the school invited Professor Yu Jun of Hangzhou Dianzi University to give an academic lecture entitled "Exploring Medical-Engineering Cross Multimodal AI Technology in the Context of Large Models".
Abstract:
In the era of large models, multimodal artificial intelligence technology has made rapid progress in medical interdisciplinary scenarios. The research on the basic model of multimodal medicine has gradually evolved from the direction of divide and conquer to universal unification of various clinical medical tasks, that is, the use of a single framework to understand medical imaging, and the combination of clinical diagnosis and electronic medical records to achieve automatic report generation and diagnosis and treatment plan recommendation. This report first briefly introduces the research background and scientific problems faced in the process of multimodal artificial intelligence for medical-engineering interdisciplinary scenarios. Then, starting from the diagnosis and treatment data of medical scenarios, the active learning method for multi-source heterogeneous clinical data is introduced, and the interpretability of medical multimodal autonomous cognitive processes and the representative work related to diagnosis and treatment preference alignment strategies are introduced in detail. Finally, the future development of deep learning at the intersection of multimodal medicine is prospected and reflected.
Reported by:
Professor and doctoral supervisor of Hangzhou Dianzi University, winner of the National Fund for Distinguished Young Scholars. His main research interests are transmedia analysis techniques. His work has been published in more than 100 SCI source journal papers, including more than 40 papers in ACM Transactions and IEEE Transactions and CCF Class A conferences. The paper has been cited more than 19,000 times by Google Scholar. More than 10 papers have been selected as ESI Highly Cited/Hot Papers; In recent years, he has presided over key projects of the National Natural Science Foundation of China and general projects of the National Natural Science Foundation of China, and has won the IEEE TMM, TIP, TCYB best paper awards in 2015, 2016 and 2017, the second prize of natural science of the Ministry of Education in 2018, and the first prize of natural science of Zhejiang Province in 2021. He has served as an associate editor for several international journals.
