Leif Azzopardi
University of Strathclyde, U.K.
Simulated Users, Agents and Humans
Simulation provides a powerful tool for the evaluation and analysis of IR systems, user behaviour and their strategies. To run a simulation, a model of the user is formalised, and then used, for example, as the basis of a metric, to create a test collection, or generate interaction data. The assumed hypothesis is that more realistic user models will lead to better measures, collections and interaction data. And thus, lead to more realistic evaluations. So to add more realism, user models have become increasingly sophisticated, encoding interaction probabilities, interaction costs, query and stopping strategies, models of attractiveness, relevance and memory. Suddenly, we are not just simulating, in some stochastic way, the interactions of users, but instead we are creating search agents - simulated users with (limited) agency - that can go off and perform the search tasks. This creates a new evaluation challenge, how do we evaluate the search performance of the user+system, whether that user be simulated, agent-like or human? And with search being embedded into other services, like virtual personal assistants and chat bots, increasingly more end-users of search technology will be other agents, and not humans. And so, how will this affect the way we model, simulate and evaluate in the future?
Bio
Dr. Leif Azzopardi is a Chancellor’s Fellow in Data Science and Associate Professor at the University of Strathclyde, Glasgow within Department of Computer and Information Science. He leads the Interactive Information Retrieval group within Strathclyde’s iSchool. His research focuses on examining the influence and impact of search technology on people and society and is heavily underpinned by theory. He has made numerous contributions in: (i) the development of statistical language models for document, sentence, expert retrieval, (ii) the simulation and evaluation of users and their interactions, (iii) the analysis of systems and retrieval bias using retrievability theory and the (iv) the formalisation of search and search behaviour using economic theory. He has given numerous keynotes, invited talks and tutorials through out the world on retrievability, search economics, and simulation. He is co-author of the Tango with Django (www.tangowithdjango.com
He is an honorary lecturer at the University of Glasgow (where he was previously a Senior Lecturer) and an honorary Adjunct Associate Professor at Queensland University of Technology. He received his Ph.D. in Computing Science from the University of Paisley in 2006, under the supervision of Prof. Mark Girolami and Prof. Keith van Rijsbergen. Prior to that he received a First Class Honours Degree in Information Science from the University of Newcastle, Australia, 2001.
