managementscienceУправленческие науки / Management SciencesManagement Sciences2304-022X2618-9941Financial University under The Government of Russian Federation10.26794/2304-022X-2026-16-2-124-135managementscience-868Research ArticleИНФОРМАЦИОННЫЕ И ЦИФРОВЫЕ ТЕХНОЛОГИИ В УПРАВЛЕНИИINFORMATION AND DIGITAL TECHNOLOGIES IN MANAGEMENTРекомендательные системы: методологические вызовы и проблемыRecommender Systems: Methodological Challenges and Issueshttps://orcid.org/0000-0002-6891-1869КульшинР. С.KulshinR. S.

Роман Сергеевич Кульшин – cтарший преподаватель кафедры автоматизации обработки информации, аспирант кафедры автоматизации обработки информации

Томск

Roman S. Kulshin – Senior lecturer of the Department of Data Processing Automation, Postgraduate student of the Department of Data Processing Automation

Tomsk

roman.s.kulshin@tusur.ruhttps://orcid.org/0000-0002-9236-3639СидоровА. А.SidorovA. A.

Анатолий Анатольевич Сидоров – кандидат экономических наук, доцент, заведующий кафедрой автоматизации обработки информации

Томск

Anatoly A. Sidorov – Cand. Sci. (Econ.), Assoc. Prof.; Head of the Department of Data Processing Automation

Tomsk

anatolii.a.sidorov@tusur.ruТомский государственный университет систем управления и радиоэлектроникиРоссияTomsk State University of Control Systems and RadioelectronicsRussian Federation202625062026162124135Copyright © Кульшин Р.С., Сидоров А.А., 20262026Кульшин Р.С., Сидоров А.А.Kulshin R.S., Sidorov A.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://managementscience.fa.ru/jour/article/view/868

Целью исследования явился анализ эволюции методов построения рекомендательных систем и подходов к оценке их качества в условиях многокритериальности и изменяющихся пользовательских предпочтений. В ходе работы рассмотрены основные этапы развития рекомендательных технологий, систематизированы современные алгоритмы и архитектуры, включая гибридные, нейросетевые и графовые модели. Особое внимание уделено методам оценки качества рекомендаций и классификации метрик, отражающих точность, уровень ранжирования, разнообразие и ресурсную эффективность по группам. В статье также показано, что ориентации исключительно на метрики точности для адекватной оценки результативности рекомендательных систем недостаточно. Авторы исследования рассмотрели ключевые методологические проблемы, включая холодный старт, разреженность данных и необходимость баланса между точностью и разнообразием рекомендаций, и сделали вывод о целесообразности комплексного многокритериального подхода к оценке рекомендательных систем. Результаты исследования могут быть использованы научными работниками, разработчиками рекомендательных систем, специалистами в области машинного обучения и анализа данных, а также представителями бизнеса при выборе и оценке эффективности рекомендательных алгоритмов.

The aim of this study is to analyse the evolution of methods for building recommender systems and approaches to evaluating their performance under conditions of multi-criteria decision-making and dynamically changing user preferences. The paper examines the key stages in the development of recommender technologies and systematises modern algorithms and architectures, including hybrid, neural network-based, and graph-based models. Particular attention is paid to methods for evaluating recommendation quality and to the classification of metrics reflecting accuracy, ranking quality, diversity, and computational efficiency. The study demonstrates that reliance solely on accuracy metrics is insufficient for an adequate assessment of recommender system performance. The authors address key methodological challenges, including the coldstart problem, data sparsity, and the need to balance accuracy and diversity in recommendations. The paper concludes that a comprehensive multi-criteria approach is essential for the effective evaluation of recommender systems. The findings may be useful for researchers, recommender system developers, and specialists in machine learning and data analysis, as well as for business practitioners involved in selecting and evaluating the effectiveness of recommendation algorithms.

рекомендательные системыоценка качества рекомендацийтенденции развитияперсонализациямоделиалгоритмыметрикиrecommender systemsrecommendation quality evaluationdevelopment trendspersonalizationmodelsalgorithmsmetricsРабота выполнена в рамках государственного задания Минобрнауки России; проект FEWM-2026-0011.The work was carried out within the framework of the state assignment of the Ministry of Education and Science of the Russian Federation; project FEWM-2026-0011.ReferencesGoldberg D., Nichols D., Oki B. M., Terry D. Using collaborative filtering to weave an information tapestry. Communications of the ACM. 1992;35(12):61-70. DOI: 10.1145/138859.138867Goldberg D., Nichols D., Oki B. M., Terry D. Using collaborative filtering to weave an information tapestry. Communications of the ACM. 1992;35(12):61-70. DOI: 10.1145/138859.138867Resnick P., Iacovou N., Suchak M., Bergstrom P., Riedl J. Grouplens: An open architecture for collaborative filtering of netnews. In: Proc. 1994 ACM conf. on computer supported cooperative work. New York, NY: Association for Computing Machinery; 1994:175-186. DOI: 10.1145/192844.192905Resnick P., Iacovou N., Suchak M., Bergstrom P., Riedl J. Grouplens: An open architecture for collaborative filtering of netnews. In: Proc. 1994 ACM conf. on computer supported cooperative work. New York, NY: Association for Computing Machinery; 1994:175-186. DOI: 10.1145/192844.192905Schafer J.B., Konstan J., Riedl J. Recommender systems in e-commerce. In: Proc. 1st ACM conf. on electronic commerce. New York, NY: Association for Computing Machinery; 1999:158-166. DOI: 10.1145/336992.337035Schafer J.B., Konstan J., Riedl J. Recommender systems in e-commerce. In: Proc. 1st ACM conf. on electronic commerce. New York, NY: Association for Computing Machinery; 1999:158-166. DOI: 10.1145/336992.337035Bennett J., Elkan C., Liu B., Smyth P., Tikk D. Kdd Cup and workshop 2007. ACM SIGKDD Explorations Newsletter. 2007;9(2):51-52. DOI: 10.1145/1345448.1345459Bennett J., Elkan C., Liu B., Smyth P., Tikk D. Kdd Cup and workshop 2007. ACM SIGKDD Explorations Newsletter. 2007;9(2):51-52. DOI: 10.1145/1345448.1345459Koren Y., Bell R., Volinsky C. Matrix factorization techniques for recommender systems. Computer. 2009;42(8):30-37. DOI: 10.1109/MC.2009.263Koren Y., Bell R., Volinsky C. Matrix factorization techniques for recommender systems. Computer. 2009;42(8):30-37. DOI: 10.1109/MC.2009.263Burke R. Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interaction. 2002;12(4):331-370. DOI: 10.1023/A:1021240730564Burke R. Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interaction. 2002;12(4):331-370. DOI: 10.1023/A:1021240730564Herlocker J.L., Konstan J.A., Terveen L.G., Riedl J.T. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS). 2004;22(1):5-53. DOI: 10.1145/963770.963772Herlocker J.L., Konstan J.A., Terveen L.G., Riedl J.T. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS). 2004;22(1):5-53. DOI: 10.1145/963770.963772Linden G., Smith B., York J. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Computing. 2003;7(1):76-80. DOI: 10.1109/MIC.2003.1167344Linden G., Smith B., York J. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Computing. 2003;7(1):76-80. DOI: 10.1109/MIC.2003.1167344Lops P., De Gemmis M., Semeraro G. Content-based recommender systems: State of the art and trends. In: Ricci F., Rokach L., Shapira B., Kantor P., eds. Recommender systems handbook. Boston, MA: Springer; 2011:73-105. DOI: 10.1007/978-0-387-85820-3_3Lops P., De Gemmis M., Semeraro G. Content-based recommender systems: State of the art and trends. In: Ricci F., Rokach L., Shapira B., Kantor P., eds. Recommender systems handbook. Boston, MA: Springer; 2011:73-105. DOI: 10.1007/978-0-387-85820-3_3Celma Ò. Music recommendation. In: Music recommendation and discovery: The long tail, long fail, and long play in the digital music space. Berlin, Heidelberg: Springer; 2010:43-85. DOI: 10.1007/978-3-642-13287-2_3Celma Ò. Music recommendation. In: Music recommendation and discovery: The long tail, long fail, and long play in the digital music space. Berlin, Heidelberg: Springer; 2010:43-85. DOI: 10.1007/978-3-642-13287-2_3Lang K. Newsweeder: Learning to filter netnews. In: Prieditis A., Russell S.J., eds. Proc. 12th Int. conf. on machine learning (ICML’95). (Tahoe City, CA, July 9-12, 1995). San Francisco, CA; Morgan Kaufmann Publishers Inc.; 1995:331-339. DOI: 10.1016/B978-1-55860-377-6.50048-7Lang K. Newsweeder: Learning to filter netnews. In: Prieditis A., Russell S.J., eds. Proc. 12th Int. conf. on machine learning (ICML’95). (Tahoe City, CA, July 9-12, 1995). San Francisco, CA; Morgan Kaufmann Publishers Inc.; 1995:331-339. DOI: 10.1016/B978-1-55860-377-6.50048-7Burke R. Hybrid web recommender systems. In: Brusilovsky P., Kobsa A., Nejdl W., eds. The adaptive web: Methods and strategies of web personalization. Berlin, Heidelberg: Springer; 2007:377-408. (Lecture notes in computer science. Vol. 4321). DOI: 10.1007/978-3-540-72079-9_12Burke R. Hybrid web recommender systems. In: Brusilovsky P., Kobsa A., Nejdl W., eds. The adaptive web: Methods and strategies of web personalization. Berlin, Heidelberg: Springer; 2007:377-408. (Lecture notes in computer science. Vol. 4321). DOI: 10.1007/978-3-540-72079-9_12Gomez-Uribe C.A., Hunt N. The Netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems (TMIS). 2015;6(4):1-19. DOI: 10.1145/2843948Gomez-Uribe C.A., Hunt N. The Netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems (TMIS). 2015;6(4):1-19. DOI: 10.1145/2843948Kanakia A., Shen Z., Eide D., Wang K. A scalable hybrid research paper recommender system for Microsoft Academic. In: The World Wide Web conf. (WWW’19). New York, NY: Association for Computing Machinery; 2019:2893-2899. DOI: 10.1145/3308558.3313700Kanakia A., Shen Z., Eide D., Wang K. A scalable hybrid research paper recommender system for Microsoft Academic. In: The World Wide Web conf. (WWW’19). New York, NY: Association for Computing Machinery; 2019:2893-2899. DOI: 10.1145/3308558.3313700He X., Liao L., Zhang H., et al. Neural collaborative filtering. In: Proc. 26th Int. conf. on World Wide Web (WWW’17). New York, NY: Association for Computing Machinery; 2017:173-182. DOI: 10.1145/3038912.3052569He X., Liao L., Zhang H., et al. Neural collaborative filtering. In: Proc. 26th Int. conf. on World Wide Web (WWW’17). New York, NY: Association for Computing Machinery; 2017:173-182. DOI: 10.1145/3038912.3052569Sedhain S., Menon A.K., Sanner S., Xie L. AutoRec: Autoencoders meet collaborative filtering. In: Proc. 24th Int. conf. on World Wide Web (WWW’15). New York, NY: Association for Computing Machinery; 2015:111-112. DOI: 10.1145/2740908.2742726Sedhain S., Menon A.K., Sanner S., Xie L. AutoRec: Autoencoders meet collaborative filtering. In: Proc. 24th Int. conf. on World Wide Web (WWW’15). New York, NY: Association for Computing Machinery; 2015:111-112. DOI: 10.1145/2740908.2742726Covington P., Adams J., Sargin E. Deep neural networks for YouTube recommendations. In: Proc. 10th ACM conf. on recommender systems (RecSys’16). New York, NY: Association for Computing Machinery; 2016:191-198. DOI: 10.1145/2959100.2959190Covington P., Adams J., Sargin E. Deep neural networks for YouTube recommendations. In: Proc. 10th ACM conf. on recommender systems (RecSys’16). New York, NY: Association for Computing Machinery; 2016:191-198. DOI: 10.1145/2959100.2959190Hidasi B., Karatzoglou A., Baltrunas L., Tikk D. Session-based recommendations with recurrent neural networks. In: 4th Int. conf. on learning representations (ICLR-2016). (San Juan, May 2–4, 2016). Appleton, WI: ICLR; 2016. URL: https://hidasi.eu/assets/pdf/gru4rec_iclr16.pdfHidasi B., Karatzoglou A., Baltrunas L., Tikk D. Session-based recommendations with recurrent neural networks. In: 4th Int. conf. on learning representations (ICLR-2016). (San Juan, May 2–4, 2016). Appleton, WI: ICLR; 2016. URL: https://hidasi.eu/assets/pdf/gru4rec_iclr16.pdfKang W.-C., McAuley J. Self-attentive sequential recommendation. In: 2018 IEEE int. conf. on data mining (ICDM 2018). (Singapore, November 17–20, 2018). New York, NY: IEEE; 2018:197-206. DOI: 10.1109/ICDM.2018.00035Kang W.-C., McAuley J. Self-attentive sequential recommendation. In: 2018 IEEE int. conf. on data mining (ICDM 2018). (Singapore, November 17–20, 2018). New York, NY: IEEE; 2018:197-206. DOI: 10.1109/ICDM.2018.00035Chen Q., Zhao H., Li W., Huang P., Ou W. Behavior sequence transformer for e-commerce recommendation in Alibaba. In: Proc. 1st Int. workshop on deep learning practice for high-dimensional sparse data (DLP-KDD’19). New York, NY: Association for Computing Machinery; 2019:1-4. DOI: 10.1145/3326937.3341261Chen Q., Zhao H., Li W., Huang P., Ou W. Behavior sequence transformer for e-commerce recommendation in Alibaba. In: Proc. 1st Int. workshop on deep learning practice for high-dimensional sparse data (DLP-KDD’19). New York, NY: Association for Computing Machinery; 2019:1-4. DOI: 10.1145/3326937.3341261Ying R., He R., Chen K., et al. Graph convolutional neural networks for web-scale recommender systems. In: Proc. 24th ACM SIGKDD Int. conf. on knowledge discovery & data mining (KDD’18). New York, NY: Association for Computing Machinery; 2018:974-983. DOI: 10.1145/3219819.3219890Ying R., He R., Chen K., et al. Graph convolutional neural networks for web-scale recommender systems. In: Proc. 24th ACM SIGKDD Int. conf. on knowledge discovery & data mining (KDD’18). New York, NY: Association for Computing Machinery; 2018:974-983. DOI: 10.1145/3219819.3219890Wang H., Zhang F., Wang J., et al. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In: Proc. 27th ACM Int. conf. on information and knowledge management (CIKM’18). New York, NY: Association for Computing Machinery; 2018:417-426. DOI: 10.1145/3269206.3271739Wang H., Zhang F., Wang J., et al. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In: Proc. 27th ACM Int. conf. on information and knowledge management (CIKM’18). New York, NY: Association for Computing Machinery; 2018:417-426. DOI: 10.1145/3269206.3271739Ricci F., Rokach L., Shapira B., eds. Recommender systems handbook. New York, NY: Springer; 2023. 1060 p. DOI: 10.1007/978-1-0716-2197-4Ricci F., Rokach L., Shapira B., eds. Recommender systems handbook. New York, NY: Springer; 2023. 1060 p. DOI: 10.1007/978-1-0716-2197-4Ge M., Delgado-Battenfeld C., Jannach D. Beyond accuracy: Evaluating recommender systems by coverage and serendipity. In: Proc. 4th ACM conf. on recommender systems (RecSys’10). New York, NY: Association for Computing Machinery; 2010:257-260. DOI: 10.1145/1864708.1864761Ge M., Delgado-Battenfeld C., Jannach D. Beyond accuracy: Evaluating recommender systems by coverage and serendipity. In: Proc. 4th ACM conf. on recommender systems (RecSys’10). New York, NY: Association for Computing Machinery; 2010:257-260. DOI: 10.1145/1864708.1864761Abdollahpouri H., Adomavicius G., Burke R., et al. Beyond personalization: Research directions in multistakeholder recommendation. 2019. URL: https://www.researchgate.net/publication/332896126_Beyond_Personalization_Research_Directions_in_Multistakeholder_RecommendationAbdollahpouri H., Adomavicius G., Burke R., et al. Beyond personalization: Research directions in multistakeholder recommendation. 2019. URL: https://www.researchgate.net/publication/332896126_Beyond_Personalization_Research_Directions_in_Multistakeholder_RecommendationZhang S., Yao L., Sun A., Tay Y. Deep learning based recommender system: A survey and new perspectives. ACM computing surveys (CSUR). 2019;52(1):1-38. DOI: 10.1145/3285029Zhang S., Yao L., Sun A., Tay Y. Deep learning based recommender system: A survey and new perspectives. ACM computing surveys (CSUR). 2019;52(1):1-38. DOI: 10.1145/3285029McNee S.M., Riedl J., Konstan J.A. Being accurate is not enough: How accuracy metrics have hurt recommender systems. In: CHI’06 Extended abstracts on human factors in computing systems (CHI EA’06). New York, NY: Association for Computing Machinery; 2006:1097-1101. DOI: 10.1145/1125451.1125659McNee S.M., Riedl J., Konstan J.A. Being accurate is not enough: How accuracy metrics have hurt recommender systems. In: CHI’06 Extended abstracts on human factors in computing systems (CHI EA’06). New York, NY: Association for Computing Machinery; 2006:1097-1101. DOI: 10.1145/1125451.1125659Amatriain X., Castells P., de Vries A., Posse C., Steck H. Workshop on recommendation utility evaluation: Beyond RMSE — RUE 2012. In: Proc. 6th ACM conf. on recommender systems (RecSys’12). New York, NY: Association for Computing Machinery; 2012:351-352. DOI: 10.1145/2365952.2366042Amatriain X., Castells P., de Vries A., Posse C., Steck H. Workshop on recommendation utility evaluation: Beyond RMSE — RUE 2012. In: Proc. 6th ACM conf. on recommender systems (RecSys’12). New York, NY: Association for Computing Machinery; 2012:351-352. DOI: 10.1145/2365952.2366042Jannach D., Jugovac M. Measuring the business value of recommender systems. ACM Transactions on Management Information Systems (TMIS). 2019;10(4):1-23. DOI: 10.1145/3370082Jannach D., Jugovac M. Measuring the business value of recommender systems. ACM Transactions on Management Information Systems (TMIS). 2019;10(4):1-23. DOI: 10.1145/3370082Pu P., Chen L., Hu R. A user-centric evaluation framework for recommender systems. In: Proc. 5th ACM conf. on recommender systems (RecSys’11). New York, NY: Association for Computing Machinery; 2011:157-164. DOI: 10.1145/2043932.2043962Pu P., Chen L., Hu R. A user-centric evaluation framework for recommender systems. In: Proc. 5th ACM conf. on recommender systems (RecSys’11). New York, NY: Association for Computing Machinery; 2011:157-164. DOI: 10.1145/2043932.2043962

The authors declare that there are no conflicts of interest present.