چکیده مقاله
Accurately measuring the similarity between texts is crucial for numerous natural language processing tasks, from plagiarism detection to information retrieval This paper delves into various approaches to calculating text similarity, exploring their strengths and limitations We begin by analyzing character based methods, including the Jaro and N gram algorithms, suitable for detecting typos and minor edits Semantic and corpus based approaches are then addressed, offering deeper insights into meaning and context This includes techniques like Dice coefficient, Euclidean distance, and Cosine distance, which compare texts based on vector representations and set intersections Finally, we introduce the statistically robust Lin Wong Similarity measure, which quantifies the commonality between probability distributions of words, providing a powerful tool for capturing semantic similarity By comparing and contrasting these diverse methods, we highlight the importance of choosing the right measure for the specific task and dataset Moving forward, the paper identifies promising avenues for future research, suggesting the potential of knowledge graphs and deep learning techniques to further refine and advance the field of text similarity measurement This comprehensive exploration equips researchers and practitioners with valuable knowledge and insights for analyzing and comparing textual data
کلیدواژهها
نویسندگان
Alireza Pakgohar
Department of statistics, Payame Noor University PNU ;
Mehdi Fazli Aghdaei
Department of Mathematics, Payame Noor University, Tehran, Iran;
شیوه ارجاع
Pakgohar, Alireza and Fazli Aghdaei, Mehdi,1402,An Analysis of Text Similarity Measures: Introducing a Lin Wang similarity measure,5th International Conference on Software Computing,Rudsar
ارائهشده در
مجموعه مقالات پنجمین کنفرانس بین المللی محاسبات نرم16 اسفند 1402 · رودسر