[Software] Design meets artificial intelligence to create new visual search engine

[Mohamed Nasser]

The Deep Discoveries project was launched to explore ways of creating a computer vision search platform that can identify and match images across digitised collections on a national scale. Credit: The National Archives, V&A, and RGBE
Novel methods of searching the nation's gallery, library and museum collections could soon be revolutionized by a visual search platform designed in collaboration with Northumbria University.

As the sector worldwide moves towards presenting collections online, the Deep Discoveries project was launched to explore ways of creating a computer vision search platform that can identify and match images across digitized collections on a national scale.

The expertise of Dr. Jo Briggs and Associate Professor Jamie Steane, from Northumbria School of Design, were enlisted to help deliver the collaboration between The National Archives, the University of Surrey and the V&A Museum.

Rather than typing a keyword into an empty search box, visual search uses a query image and computer vision artificial intelligence (AI), to match similar images from across digitized collections based on properties such as color, pattern and shape.

The Northumbria design team—made up of Jo, Jamie and talented graduate Andy Cain—joined the project at a later stage to help with information sharing and developing the user experience.

Their challenge was to find ways to visually demonstrate the AI reasoning and explain the search criteria of the platform.

Bernard Ogden, Research Software Engineer at The National Archives, built the prototype platform.

He said: "The design process allowed us to reach a shared understanding that permitted us to create a live prototype, accommodating the different points of view between in teams involved in this multi-disciplinary project."

Jo and Bernard recently attended the Digital Research Infrastructure for the Arts and Humanities (DARIAH) annual conference, this year on the theme of "Interfaces," to present their findings.

Project lead, Dr. Lora Angelova, Head of Conservation Research and Audience Development at The National Archives, added: "The energy and innovative design vision brought to the project by the design team, sparked new ideas and possibilities around visually communicating a progressive way of deploying AI for the benefit of our audiences."