University of Edinburgh Tests AI-Driven Search with Drupal AI
The University of Edinburgh has conducted a series of experiments testing AI-enhanced search capabilities on its websites, exploring whether artificial intelligence can improve content retrieval for users. Led by Emma Horrell, User Experience (UX) Manager at the university, the study compared three search approaches: traditional keyword-based search, AI-boosted search using vector databases, and chatbot-driven AI Assistant search.
The research, conducted in collaboration with FreelyGive Ltd and Drupal AI specialists James Abrahams, Marcus Johansson, and Charlie Hull, revealed that AI-powered search tools offer advantages in precision and interactivity but require careful configuration to handle university-specific terminology effectively.
AI-Boosted Search vs. AI Assistant Search
The team used a case-study School website to simulate real-world queries. The AI-boosted search, which employs vector databases for semantic matching, performed well with precise queries but struggled when handling terms like “courses” versus “degrees.” Meanwhile, the AI Assistant chatbot provided more interactive responses, dynamically adapting to user questions, but also needed fine-tuning to improve contextual accuracy.
For example, when a query about “history courses” was tested, none of the three search methods returned the ideal result. AI-boosted search interpreted "courses" too broadly, returning links to individual course pages rather than a consolidated list of degree programs. However, after adjusting the AI Assistant’s prompt to recognize different user personas—such as prospective students—it successfully generated a structured response with relevant links.
Fine-Tuning AI for University Search Needs
The study highlighted that search effectiveness improves significantly when AI is trained with structured metadata and contextual instructions. The team experimented with AI Automators to index content more effectively, allowing AI-powered searches to return more relevant results.
Additionally, small refinements to AI prompts—such as instructing the chatbot to recognize university-specific jargon—improved search accuracy. One adjustment helped the AI Assistant distinguish between "Library," "Student Research Room," and "Study Space," leading to better search responses when users asked about borrowing books or library hours.
AI Search Requires Continuous Refinement
Horrell emphasized that AI-driven search is not a one-size-fits-all solution and requires iterative adjustments.
“Providing AI with structured data and refining prompts based on user needs significantly enhances search accuracy,”
she noted.
The findings underscore the need for a user-focused design approach when integrating AI into university search systems. AI can improve search experiences, but success depends on semantic understanding, query refinement, and continuous UX research.
The full study and its insights into AI-enhanced search on university websites are detailed in Emma Horrell’s blog post on the University of Edinburgh’s UX Service blog.
Read the full report here: Can AI help or hinder search?
Source Reference
Image Attribution Disclaimer: At The Drop Times (TDT), we are committed to properly crediting photographers whose images appear in our content. Many of the images we use come from event organizers, interviewees, or publicly shared galleries under CC BY-SA licenses. However, some images may come from personal collections where metadata is lost, making proper attribution challenging.
Our purpose in using these images is to highlight Drupal, its events, and its contributors—not for commercial gain. If you recognize an image on our platform that is uncredited or incorrectly attributed, we encourage you to reach out to us at #thedroptimes channel on Drupal Slack.
We value the work of visual storytellers and appreciate your help in ensuring fair attribution. Thank you for supporting open-source collaboration!
ASK: Drupal Community Needs Your Support
View all →
Note: The vision of this web portal is to help promote news and stories around the Drupal community and promote and celebrate the people and organizations in the community. We strive to create and distribute our content based on these content policy. If you see any omission/variation on this please reach out to us at #thedroptimes channel on Drupal Slack and we will try to address the issue as best we can.
Related Drupal Initiatives
Related Organizations
Related People
You may also like
