Relevance is more significant than correlation: Information filtering on sparse data

Shang, Ming-Sheng; Lü, Linyuan; Zeng, Wei; Zhang, Yi-Cheng; Zhou, Tao

doi:10.1209/0295-5075/88/68008

Back

Journal article

Relevance is more significant than correlation: Information filtering on sparse data

Shang, Ming-Sheng Web Sciences Center, School of Computer Science and Engineering, University of Electronic Science and Technology of China - 610054 Chengdu, China
Lü, Linyuan Department of Physics, University of Fribourg, Switzerland
Zeng, Wei Web Sciences Center, School of Computer Science and Engineering, University of Electronic Science and Technology of China - 610054 Chengdu, China - Lab of Information Economy and Internet Research, University of Electronic Science and Technology of China, Chengdu, China
Zhang, Yi-Cheng Department of Physics, University of Fribourg, Switzerland - Lab of Information Economy and Internet Research, University of Electronic Science and Technology of China, Chengdu, China
Zhou, Tao Department of Physics, University of Fribourg, Switzerland - Department of Modern Physics, University of Science and Technology of China, Hefei, China

07.01.2010

Published in:

Europhysics Letters. - 2009, vol. 88, no. 6, p. 68008

English In some recommender systems where users can vote objects by ratings, the similarity between users can be quantified by a benchmark index, namely the Pearson correlation coefficient, which reflects the rating correlations. Another alternative way is to calculate the similarity based solely on the relevance information, namely whether a user has voted an object. The former one uses more information than the latter, and is intuitively expected to give more accurate rating predictions under the standard collaborative filtering framework. However, according to the extensive experimental analysis, this letter reports the opposite results that the latter method, making use of only the relevance information, can outperform the former method, especially when the data set is sparse. Our finding challenges the routine knowledge on information filtering, and suggests some alternatives to address the sparsity problem.

Faculty

Faculté des sciences et de médecine

Department

Département de Physique

Language