Search as Learning - Investigating, Enhancing, and Predicting Learning during Multimodal (Web) Search (SALIENT)

Team: Ran Yu, Dr. Daniel Hienert
Leader: Prof. Dr. Stefan Dietze
Scientific unit: Dep. Knowledge Technologies for the Social Sciences (WTS)


The interdisciplinary project SALIENT researches methods to detect learning intents and aims at improving retrieval performance accordingly during multimodal (learning-related) Web search.

Established information retrieval approaches address the relevance of search results to an information need, whereas the actual learning scope of a user is usually disregarded. Recent research in the search as learning area has recognized the importance of learning scopes and focused on observing and detecting learning needs. However, it has often been restricted to limited and isolated feature sets or specific learning tasks. High-dimensional feature spaces, (audio)visual information, or the generalizability of previous work to support various learning needs by retrieval, ranking, and recommendations have not been investigated yet. 

The interdisciplinary project SALIENT – joining expertise from computer science and psychology – aims at closing this gap by researching methods to improve retrieval performance and individuals’ learning through (a) an accurate detection and prediction of learning needs and knowledge gains during search by means of query logs, navigation logs, eye-tracking and thinking-aloud data, which serve as a basis for (b) supporting users in their learning tasks through an enhanced retrieval and ranking process and recommendations, as well as for (c) suggesting appropriate and personalized recommendations including multimodal information (diagrams, slides, videos, etc.). Next to providing contributions to information retrieval and Web search, the project goes beyond previous work by also addressing semi-informal learning scenarios that involve search for scholarly and scientific (multimodal) information, literature, and videos in digital library portals, e.g., as offered by TIB’s main web portal and the TIB AV-Portal for scientific videos. Since different kinds of learning tasks are associated with significantly different user behaviors and preferences, a thorough understanding of learning needs and cognitive and behavioral learning patterns from a psychological perspective is required. Results will be evaluated in a variety of scenarios and will lead to generalizable models and methods.


Mai 2018 - 31.10.21

Sponsored by


  • German National Library of Science and Technology (TIB, Hannover)
  • L3S Research Center (Hannover)
  • Leibniz-Institut für Wissensmedien (IWM, Tübingen)


  • Yu, R., Gadiraju, U., Holtz, P., Rokicki, M., Kemkes, P., Dietze, S., Predicting User Knowledge Gain in Informational Search Sessions, full research track paper at 41st International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR2018), Ann Arbor Michigan, US, July 8-12, 2018, ACM.
  • Hoppe, A., Holtz, P., Kammerer, Y., Yu, R., Dietze, S., Ewerth, R., Current Challenges for Studying Search as Learning Processes, 7th Workshop on Learning & Education with Web Data (LILE2018), in conjunction with ACM Web Science 2018 (WebSci18), Amsterdam, NL, 2018.
  • Yu, R., Gadiraju, U., Dietze, S., Detecting, Understanding and Supporting Everyday Learning in Web Search, 7th Workshop on Learning & Education with Web Data (LILE2018), in conjunction with ACM Web Science 2018 (WebSci18), Amsterdam, NL, 2018.
  • Gadiraju, U., Yu, R., Dietze, S., Holtz, P., Analyzing Knowledge Gain of Users in Informational Search Sessions on the Web, full research track paper at ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR2018), New Brunswick, NJ, US, 11-15 March 2018, ACM.