Evaluation is a central aspect in the research and development of information retrieval (IR) and recommender (RecSys) systems. Offline evaluations using expert opinions often fail to meet the actual user needs, as the experts who carry out the evaluations do not correspond to the normal users of a platform. Online evaluations examine IR and recommender systems directly in productive operation, but such evaluations can only be carried out if there is direct access to a sufficiently large platform with many users.

The STELLA (Infrastructures for Living Labs) project aims to create an evaluation infrastructure that allows the academic environment to evaluate search and recommender systems within productive web-based platforms with real users. Two different target groups are relevant for the project: On the one hand, scientific infrastructure institutions and platform operators (e.g. GESIS, ZB MED, ZBW, DIPF or TIB), and on the other hand, researchers from the field of information science and computer science with a focus on IR and recommender systems. The goals of the project are to set up an e-research research infrastructure and to use the e-research research infrastructure as part of a living labs evaluation campaign for the IR and RecSys community.