Semantic Embedding and Profile-Based Ranking for Automated Reviewer Recommendation

Abstract

Manual reviewer assignment in peer review is difficult to scale because submission volumes grow faster than editors can inspect reviewer expertise, and reviewer profiles shift across topics and time. Existing automated approaches often rely on keyword or lexical matching, which cannot capture semantic similarity, and few combine dense retrieval with interpretable reviewer evidence. This study develops and evaluates an explainable reviewer recommendation system using a BERT-first Reciprocal Rank Fusion semantic-profile backend. The system retrieves candidate evidence using BERT and SPECTER2 semantic representations, extracts candidate reviewers from retrieved paper authors, and ranks them using fused retrieval evidence supported by frequency, h-index, and recency signals. The expertise-scoring component was evaluated using the Stelmakh/OpenReview benchmark, while end-to-end recommendation was evaluated on an OpenAlex citation-based proxy dataset using a validation split for configuration selection and a held-out test split for final reporting. SPECTER2 max pooling achieved a weighted Kendall tau loss of 0.22 on the Stelmakh/OpenReview benchmark, consistent with the public SPECTER2 baseline. On the held-out test split, the selected BERT-first RRF semantic-profile backend achieved the highest NDCG@10 of 0.2621, significantly outperforming BERT, SPECTER2-only, BM25, TF-IDF, and the previous profile-heavy backend. These findings indicate that rank-level fusion of complementary dense retrieval signals can improve reviewer candidate ranking while retaining interpretable profile evidence for editorial workflows. The local evaluation uses citation-based proxy relevance rather than true editorial assignments, so further validation using human-annotated reviewer data is needed.

GS Cited Analysis