The STELLA project aims to create an evaluation infrastructure allowing retrieval and recommender systems to be run within academic search systems with real users.Thus, STELLA provides an integrated e-research environment, a so-called living lab. The experimental setup differs significantly from classical offline TREC studies as it allows academic researchers to use an evaluation method previously reserved for industrial research or operators of large online platforms. The STELLA framework is designed to support academic search solutions, dealing with heterogeneity and mass of documents (and datasets) that still introduce new challenges to the disciplines of information retrieval, recommender systems, digital libraries, and related fields.

In STELLA I, the three project partners, TH Köln, ZB MED, and GESIS, aimed to strengthen the concept of user-centric living labs for the domain of academic search. By allowing researchers to evaluate their retrieval approaches in two real-world search portals, the methodological gap between real-world and lab-based evaluations was effectively reduced. As a result, STELLA was the cornerstone of the CLEF 2021 evaluation lab LiLAS and proved to be reliable and extensible.In STELLA II, the three partners will extend the existing framework to include the additional external partners DIPF, TIB, and ZBW and their corresponding search platforms and evaluation use-cases. Furthermore, STELLA's current ad-hoc retrieval and recommendation setup will be extended as a continuous evaluation platform.