Systematic mapping of automated reviewer recommendation solutions

Guillermo Omar Decoppet; Patricia Silvana San Martin

doi:10.24215/16666038.24.e16

Authors

Guillermo Omar Decoppet Universidad Abierta Interamericana, Buenos Aires, Argentina https://orcid.org/0000-0001-8824-0786
Patricia Silvana San Martin Universidad Abierta Interamericana, Buenos Aires, Argentina https://orcid.org/0000-0001-7543-045X

DOI:

https://doi.org/10.24215/16666038.24.e16

Keywords:

Natural language processing, Peer Review, Recovery models, Selection process

Abstract

The increase in scientific production has generated a recurring problem on a global scale in the recommendation of reviewers for scientific journals and academic events, incentivizing the emergence of a significant diversity of automated solutions. This
article presents a systematic review of these reviewer recommendation solutions published in scientific journals and academic events in the period 2018-2023. Methodologically, the final selection focused on the analysis of twenty-five articles. It covered the domain of reviewer recommendation solutions, their methods, factors and the data sets utilized. The results achieved systematize the diverse types of proposed solutions allowing to observe the similarities between the different methods. It is estimated that the present mapping provides an original survey on this problem that provides well-founded comparative information to support future research on reviewer recommendations.

Downloads

Download data is not yet available.

References

G. S. Martínez, «La revisión por pares y la selección de artículos para publicación», Rev. Colomb. Psicol., vol. 21, n.o 1, pp. 27-35, 2012.

M. F. Abad, «El proceso de revisión por pares», en Revistas científicas: situación actual y retos de futuro, E. Abadal, Ed., Universitat de Barcelona, 2017, pp. 53-71. http://eprints.rclis.org/32131/

T. Riggs & R. Wilensky, «An algorithm for automated rating of reviewers | Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries», JCDL ’01: Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, 2001, p. Pages 381-387. doi: doi.org/10.1145/379437.379731.

P. Rigaux, «An iterative rating method: application to web-based conference management|Request PDF», ResearchGate. https://www.researchgate.net/publication/47800513 _An_iterative_rating_method_application_to_web-based_conference_management

M. Rodriguez, J. Bollen, & H. Sompel, «The Convergence of Digital-Libraries and the Peer- Review Process», J. Inf. Sci., vol. 32, may 2005, https://doi.org/10.1177/0165551506062327

M. A. Rodriguez & J. Bollen, «An Algorithm to Determine Peer-Reviewers», Proceeding 17th ACM Conf. Inf. Knowl. Min. - CIKM 08, p. 319, 2008, doi: 10.1145/1458082.1458127.

K. Balog, L. Azzopardi, & M. de Rijke, «Formal models for expert finding in enterprise corpora», en Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, en SIGIR ’06. Seattle, Washington, USA: Association for Computing Machinery, aug. 2006, pp. 43-50. doi: 10.1145/1148170.1148181.

D. Mimno & A. McCallum, «Expertise modeling for matching papers with reviewers», en Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, en KDD ’07. San Jose, California, USA: Association for Computing Machinery, aug. 2007, pp. 500-509. doi: 10.1145/1281192.1281247.

K.-H. Yang, T.-L. Kuo, H.-M. Lee, & J.-M. Ho, «A Reviewer Recommendation System Based on Collaborative Intelligence», en 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, sep. 2009, pp. 564-567. doi: 10.1109/WI- IAT.2009.94.

B. Kitchenham, O. Pearl Brereton, D. Budgen, M. Turner, J. Bailey, & S. Linkman, «Systematic literature reviews in software engineering – A systematic literature review», Inf. Softw. Technol., vol. 51, n.o 1, pp. 7-15, jan. 2009, doi: 10.1016/j.infsof.2008.09.009.

M. Aksoy, S. Yanik, & M. F. Amasyali, «Reviewer Assignment Problem: A Systematic Review of the Literature», J. Artif. Intell. Res., vol. 76, pp. 761- 827, mar. 2023, doi: 10.1613/jair.1.14318.

Z. Dong, Z. Shu, D. Zhen, C. Jie, Z. Yanping, & T. Jie, «A Multi-Label Classification Method Using a Hierarchical and Transparent Representation for Paper-Reviewer Recommendation | ACM Transactions on Information Systems», 2020, doi: doi.org/10.1145/3361719.

T. Pradhan, S. Sahoo, U. Singh, & S. Pal, «A proactive decision support system for reviewer recommendation in academia», Expert Syst. Appl., vol. 169, p. 114331, may 2021, doi: 10.1016/j.eswa.2020.114331.

J. Jin, Q. Geng, H. Mou, & C. Chen, «Author– Subject–Topic model for reviewer recommendation», J. Inf. Sci., vol. 45, n.o 4, pp. 554- 570, aug. 2019, doi: 10.1177/0165551518806116.

Y. Yong, Z. Yao, & Y. Zhao, «A Framework for Reviewer Recommendation Based on Knowledge Graph and Rules Matching», en 2021 IEEE International Conference on Information Communication and Software Engineering (ICICSE), mar. 2021, pp. 199-203. doi: 10.1109/ICICSE52190.2021.9404099.

C. K. Kreutz & R. Schenkel, «RevASIDE: Assignment of Suitable Reviewer Sets for Publications from Fixed Candidate Pools», en The 23rd International Conference on Information Integration and Web Intelligence, en iiWAS2021. New York, NY, USA: Association for Computing Machinery, dec. 2022, pp. 57-68. doi: 10.1145/3487664.3487673.

Y. Kalmukov, «Automatic Assignment of Reviewers to Papers Based on Vector Space Text Analysis Model», en Proceedings of the 21st International Conference on Computer Systems and Technologies, en CompSysTech ’20. New York, NY, USA: Association for Computing Machinery, aug. 2020, pp. 229-235. doi: 10.1145/3407982.3408026.

A. N. Medakene, K. Bouanane, & M. A. Eddoud, «A New Approach for Computing the Matching Degree in the Paper-to-Reviewer Assignment Problem», en 2019 International Conference on Theoretical and Applicative Aspects of Computer Science (ICTAACS), dec. 2019, pp. 1-8. doi: 10.1109/ICTAACS48474.2019.8988127.

K. Mittal, A. Jain, K. S. Vaisla, & D. K. Tayal, «A Novel Method for Reviewer Assignment Problem based on Reviewers’ Profile and Fuzzy Graph Connectivity Measure», en 2020 International Conference on Intelligent Engineering and Management (ICIEM), jun. 2020, pp. 386-391. doi: 10.1109/ICIEM48762.2020.9160042.

A. H. Patil & P. N. Mahalle, «Fair and Accurate Review in Publication Process: A learning-based Proactive Approach for Assigning Reviewers to Manuscripts», Turk. J. Comput. Math. Educ. TURCOMAT, vol. 12, n.o 4, Art. n.o 4, abr. 2021, doi: 10.17762/turcomat.v12i4.455.

V. Latypova, «Reviewer Assignment Decision Support in an Academic Journal based on Multicriteria Assessment and Text Mining», en 2023 IX International Conference on Information Technology and Nanotechnology (ITNT), apr. 2023, pp. 1-4. doi: 10.1109/ITNT57377.2023.10139187.

R. P. Kusumawardani & S. O. Khairunnisa, «Author-Topic Modelling for Reviewer Assignment of Scientific Papers in Bahasa Indonesia», en 2018 International Conference on Asian Language Processing (IALP), nov. 2018, pp. 351-356. doi: 10.1109/IALP.2018.8629124.

M.I.M.Ishag,K.H.Park,J.Y.Lee,&K.H.Ryu, «A Pattern-Based Academic Reviewer Recommendation Combining Author-Paper and Diversity Metrics», IEEE Access, vol. 7, pp. 16460- 16475, 2019, doi: 10.1109/ACCESS.2019.2894680.

B. Y. Alkazemi, «PRATO: An Automated Taxonomy-Based Reviewer-Proposal Assignment System», Interdiscip. J. Inf. Knowl. Manag., vol. 13, pp. 383-396, oct. 2018.

A. H. Patil & P. N. Mahalle, «Multi-label Reviewer Profile Building and Ranking based on Expertise, Recency, Authority and h-index: Vital Module of Reviewer Paper Assignment», Turk. J. Comput. Math. Educ. TURCOMAT, vol. 12, n.o 6, Art. n.o 6, apr. 2021.

Y. Kalmukov, «A Software Tool for Accuracy Evaluation of Calculated Similarity Factors between Papers and Reviewers», in 2020 7th International Conference on Energy Efficiency and Agricultural Engineering (EE&AE), nov. 2020, pp. 1-5. doi: 10.1109/EEAE49144.2020.9279032.

S. A. Al-Zboon, S. K. Tawalbeh, H. Ai-Jarrah, M. Al-Asa’d, M. Hammad, & M. Al-Smadi, «Resolving Conflict of Interests in Recommending Reviewers for Academic Publications Using Link Prediction Techniques», en 2019 2nd International Conference on new Trends in Computing Sciences (ICTCS), oct. 2019, pp. 1-6. doi: 10.1109/ICTCS.2019.8923033.

Z. Duan, S. Tan, S. Zhao, Q. Wang, J. Chen, & Y. Zhang, «Reviewer assignment based on sentence pair modeling», Neurocomputing, vol. 366, pp. 97- 108, nov. 2019, doi: 10.1016/j.neucom.2019.06.074.

J. Nguyen, G. Sánchez-Hernández, N. Agell, X. Rovira, & C. Angulo, «A decision support tool using Order Weighted Averaging for conference review assignment», Pattern Recognit. Lett., vol. 105, pp. 114-120, apr. 2018, doi: 10.1016/j.patrec.2017.09.020.

A. Q. Abduljaleel & E. Al, «Reviewer Assignment Using Weighted Matching and Hungarian Algorithm», Turk. J. Comput. Math. Educ. TURCOMAT, vol. 12, n.o 7, Art. n.o 7, abr. 2021, doi: 10.17762/turcomat.v12i7.2629.

C. K. Kreutz, K. Balog, & R. Schenkel, «Diverse Reviewer Suggestion for Extending Conference Program Committees», en IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, en WI-IAT ’21. New York, NY, USA: Association for Computing Machinery, apr. 2022, pp. 79-86. doi: 10.1145/3486622.3493931.

D. K. Pradhan, J. Chakraborty, P. Choudhary, & S. Nandi, «An automated conflict of interest based greedy approach for conference paper assignment system», J. Informetr., vol. 14, n.o 2, p. 101022, may 2020, doi: 10.1016/j.joi.2020.101022.

C. Yang, T. Liu, W. Yi, X. Chen, & B. Niu, «Identifying expertise through semantic modeling: A modified BBPSO algorithm for the reviewer assignment problem», Appl. Soft Comput., vol. 94, p. 106483, sep. 2020, doi: 10.1016/j.asoc.2020.106483.

A. H. Patil, P. N. Mahalle, V. H. Patil, & S. A. Bhavsar, «PROACTIVE SYSTEM FOR REVIEWER PAPER ASSIGNMENT», J. Data Acquis. Process., vol. 38, n.o 2, pp. 1569-1609, abr. 2023, doi: 10.5281/zenodo.776779.

L. Xu, D. Zeng, J. Dai, & L. Gui, «Combining Coverage with TMPS for Reviewer Assignment», en 2022 International Conference on Intelligent Education and Intelligent Research (IEIR), dec. 2022, pp. 107-113. doi: 10.1109/IEIR56323.2022.10050060.

M. Mirzaei, J. Sander, & E. Stroulia, «Multi-Aspect Review-Team Assignment using Latent Research Areas», Inf. Process. Manag., vol. 56, n.o 3, pp. 858- 878, may 2019, doi: 10.1016/j.ipm.2019.01.007.

D. M. Blei, «Latent Dirichlet Allocation», p. 30.

S. Robertson, S. Walker, S. Jones, M. Hancock- Beaulieu, & M. Gatford, «Okapi at TREC-3.», jan. 1994, p. 0.

J. Devlin, M.-W. Chang, K. Lee, & K. Toutanova,

«BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding». arXiv, may 2019. doi: 10.48550/arXiv.1810.04805.

A. Krizhevsky, I. Sutskever, & G. E. Hinton, «ImageNet Classification with Deep Convolutional Neural Networks», en Advances in Neural Information Processing Systems, Curran Associates, Inc., 2012. https://proceedings.neurips.cc/paper/2012/hash/c39

d3b9d6b76c8436e924a68c45b-Abstract.html

A. Sherstinsky, «Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network», Phys. Nonlinear Phenom., vol. 404, p. 132306, mar. 2020, doi: 10.1016/j.physd.2019.132306.

M. Rosen-Zvi, T. Griffiths, M. Steyvers, & P. Smyth, «The author-topic model for authors and documents», en Proceedings of the 20th conference on Uncertainty in artificial intelligence, en UAI ’04. Arlington, Virginia, USA: AUAI Press, jul. 2004, pp. 487-494.

H. Mou, Q. Geng, J. Jin, & C. Chen, «An Author Subject Topic Model for Expert Recommendation», en Information Retrieval Technology, G. Zuccon, S. Geva, H. Joho, F. Scholer, A. Sun, & P. Zhang, Eds., in Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015, pp. 83-95. doi: 10.1007/978-3-319-28940-3_7.

T. Mikolov, I. Sutskever, K. Chen, G. Corrado, & J. Dean, «Distributed Representations of Words and Phrases and their Compositionality». arXiv, oct. 2013. doi: 10.48550/arXiv.1310.4546.

Q. V. Le & T. Mikolov, «Distributed Representations of Sentences and Documents». arXiv, may. 2014. doi: 10.48550/arXiv.1405.4053.

K. Cho et al., «Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation». arXiv, sep. 2014. http://arxiv.org/abs/1406.1078

T. Hofmann, «Probabilistic Latent Semantic Analysis». arXiv, jan. 2013. doi: 10.48550/arXiv.1301.6705.

Y. Yao et al., «Prompt Tuning for Discriminative Pre-trained Language Models». arXiv, may. 2022. doi: 10.48550/arXiv.2205.11166.