We’ve just released an update to Talpa Search, our groundbreaking new way to search for books and other media in libraries. In other words, Talpa is better, so patrons will come away happier!
The update involves a decent improvement when looking for older books, and a big improvement when searching for newer books. The changes essentially eliminate Talpa’s “recency problem”—being better at finding older books than newer books. Newer books now get nearly the same benchmark scores as older books.
Here’s a graph of Talpa’s performance on the “What’s That Book?” benchmark, a set of 200 test searches we released last week. In this set each search has a single answer, which should (ideally) rank first in the results too. The new results are compared to the system in place yesterday, which dates mostly to August 2024. The full data—all 200 questions, answers and scores are at the end of this blog post.
Scoring
For all books, Talpa scores have improved from 75.8% to 91%. For older books, the score rose from 85.1% to 94.4%, and for newer books (published 2023–2024) from 66.5% to 87.5%—a 21% jump!
Our scoring is simple. Fully 162 of the 200 of the searches got a score of 100%—the desired book was first in the results. For another 30, the desired book was among the results, but not in first place. These were scored with 80% for second place, 70% for third place, and on down to 40% for 6th place. Positions 7–20 were scored at 30%.
The Hardest Ones
At present only eight searches are complete failures. See if you can get them. The answers are at the end of this post; each search is linked to its answer below.
- “girl finds necklace and meets pink bunny robot”
- “recent cartoon demon perspective book”
- “World war 1 historical fantasy taking place in flanders”
- “Magical realism, fiction, two boys who disappear for 6 months and can’t recall what happened”
- “Fantasy fiction about books with different powers/magic”
- “Sequel to YA fantasy about home for magical misfit children”
- “Book where a girl has to walk or else she will die”
- “thriller with cat and el morgan”
The “Non-Zero Scores” graph below shows the scores if any position is counted as a win. Talpa Search used to miss completely on 14% of benchmark searches. That number is now 4%.
What’s Next?
We’re always looking to improve Talpa Search. Some of the projects we are working on include:
- An update to the Talpa Cover Search, covering recent books and adding over 10 million covers.
- Direct integration with popular library OPACs, making Talpa as seamless as any other library search.
Other improvements are about how we test and improve:
- We will be publishing other “benchmark searches,” including one for movies and another for questions with 2+ good answers.
- Now that we’ve hit 90% success, we need harder questions! We’re going to be asking LibraryThing members and libraries to come up with searches they think should have worked, but didn’t.
Thank you for reading, and stay tuned for further updates!
The Scores
The following are the 200 “What’s That Book?” searches. You can also fine this information in a text file.
Show all answers
Question 32
Q: children’s book about boy throwing things into a tree
A: Show answerStuck by Oliver Jeffers
August 2024: 70% – Position: 3
January 2025: 0% – Not in results list
Question 34
Q: Marvel universe novel about retired superhero turned private investigator
A: Show answerBreaking the Dark by Lisa Jewell
August 2024: 100% – Position: 4
January 2025: 100% – Position: 1
Question 39
Q: Historical fantasy about servant with magical powers, set during Spanish Golden Age
A: Show answerThe Familiar by Leigh Bardugo
August 2024: 0% – Not in results list
January 2025: 100% – Position: 1
Question 41
Q: Fiction romance about a stand up comedian trying to figure out his breakup
A: Show answerGood Material by Dolly Alderton
August 2024: 100% – Position: 1
January 2025: 100% – Position: 1
Question 51
Q: Literary fiction about queer iranian immigrant guy
A: Show answerMartyr! by Kaveh Akbar
August 2024: 50% – Position: 5
January 2025: 100% – Position: 1
Question 54
Q: Fiction about teen girl boxers in Reno, nevada
A: Show answerHeadshot by Rita Bullwinkel
August 2024: 0% – Not in results list
January 2025: 100% – Position: 1
Question 77
Q: that dystopian book with jonas and gabriel
A: Show answerThe Giver by Lois Lowry
August 2024: 80% – Position: 2
January 2025: 100% – Position: 1
Question 83
Q: Popular 2024 romance book about children’s librarian
A: Show answerFunny Story by Emily Henry
August 2024: 100% – Position: 1
January 2025: 100% – Position: 1
Question 103
Q: ya comedic novel about beauty queens stranded on desert island
A: Show answerBeauty Queens by Libba Bray
August 2024: 100% – Position: 1
January 2025: 100% – Position: 1
Question 105
Q: recent popular historical fiction about women in vietnam war
A: Show answerThe Women by Kristin Hannah
August 2024: 100% – Position: 1
January 2025: 100% – Position: 1
Question 116
Q: The 22nd book in the Mitch Rapp series
A: Show answerCode Red by Vince Flynn
August 2024: 100% – Position: 1
January 2025: 100% – Position: 1
Question 133
Q: Retelling of Huck Finn from Jim’s point of view
A: Show answerJames by Percival Everett
August 2024: 100% – Position: 1
January 2025: 100% – Position: 1
Question 147
Q: Female friends at D.C. boarding house duringheight of McCarthyism
A: Show answerThe Briar Club by Kate Quinn
August 2024: 0% – Not in results list
January 2025: 100% – Position: 1
Question 148
Q: Magical realism, fiction, two boys who disappear for 6 months and can’t recall what happened
A: Show answerThe Lost Story by Meg Shaffer
August 2024: 0% – Not in results list
January 2025: 0% – Not in results list
Question 170
Q: Murder mystery about three foster sisters and a body found in their foster home
A: Show answerDarling Girls by Sally Hepworth
August 2024: 100% – Position: 1
January 2025: 70% – Position: 3
Question 177
Q: Romance where she gets the expiration date of the relationships she starts
A: Show answerExpiration Dates by Rebecca Serle
August 2024: 100% – Position: 1
January 2025: 100% – Position: 1
Question 196
Q: dystopian novel with big brother
A: Show answer1984 by George Orwell
August 2024: 80% – Position: 2
January 2025: 100% – Position: 1
Question 197
Q: historical murder mystery set in san francisco with a gay ex-cop
A: Show answerLavender House by Lev AC Rosen
August 2024: 0% – Not in results list
January 2025: 80% – Position: 2
