Wednesday, November 15th, 2006

UnSuggester fallout

Our new anti-recommendation engine UnSuggester (blogged about below), has taken off in a weird way. Today was covered in everything from top-shelf library bloggers Tom Roper, Steven Abram and Karen Schneider (via ALA TechSource), to the bouncy, zippy, web-culture vlog MobuzzTV.

Here Mobuzzer Karina Stenquist explains the disconnect between Middlesex and The Passion of Jesus Christ: Fifty reasons he came to die. Karina did something on us before and—I’m sorry—she’s great.

I love everything people are saying about it, with big discussions here and on various blogs, mosty people strive to find the oddest, funniest unsuggestions. My favorite blog discussion came off a short note by “Fontana Labs” (a philosophy professor?) on Unfogged, and is now past 120 comments. Much of it came from the proposal:

“There’s got to be some ultimately evil book out that that will generate the ideal library. Not The Bridges of Madison County, but like that.”

Speculation ensues, with the unsuggestions for Who moved my Cheese? much praised. I enjoyed this exchange:

“Sadly for Unfogged (but perhaps good for Western civilization) Who Moved My Cheese? For Kids, as perfectly soul-deadening a title as I’ve ever encountered, isn’t owned by enough LibraryThing users to produce results. …”

Who Moved My Cheese? For Kids. Oh my God, my soul just broke into three thousand tiny wretched pieces.”

Hey, I didn’t say it. De gustibus…

Labels: 1

Sunday, November 12th, 2006

BookSuggester and UnSuggester

People do not generally like BOTH Shopaholic and Critique of Pure Reason.

The “real” news today is the debut of BookSuggester, a new feature designed to expose LibraryThing’s excellent and varied recommendations to members and non-members alike. We put them alongside Amazon’s, which are also quite good. We are proud of our recommendations, but haven’t perfected the perfect algorithm yet. When we’ve made things as good as we can, we’re going to start offering recommended book data to libraries.**

But to heck with that! Let’s talk about bad recommendations. Today we introduce UnSuggester, “the worst recommendation system ever devised™.”

UnSuggester is a brand new idea in recommender technology. Recommender systems usually work by similarities. Amazon’s “Customers who bought this item also bought” and LibraryThing’s “People with this book also have” are typical of the type—What books do people buy together? What books occur often in the same member libraries?

UnSuggester flips this logic: What books DON’T occur in the same libraries? We took our “similars” algorithm and changed “sort ascending” to “sort descending” and—hey presto!—instead of similar books, we get opposite ones. You bet we’re going to patent it!

How does it work?

UnSuggester starts by finding every copy of the book in question and all of its owners. So, taking Thucydides’ History of the Peloponnesian War as an example, LibraryThing finds the 600-odd people who have entered this ancient classic in their account. Then it makes a big pile of all their other books, a pile of some 623,000 books in all. Then it does a little math. If LibraryThing has seven million books, then a pool of 623,000 book is about 8% of the total. If this pool were average, it would also contain 8% of the Harry Potters, 8% of the Derridas and 8% of the Danielle Steels. But this isn’t so. People who own Thucydides aren’t a random cross-section of the book-loving public. For example that 8% also contains almost half the Caesar and Plutarch in LibraryThing. At the other end of the scale, Thucydides-fanciers are particularly immune to the novels of Marian Keyes and Dean Koontz. The greatest disconnect occurs with Louise Rennison’s popular, teeny-bopper chick-lit novel Angus, thongs and full-frontal snogging : confessions of Georgia Nicolson—the top UnSuggestion.

What patterns emerge?

The Mists of Avalon and Desiring God are very uncommon shelf-mates.

Play with it a few minutes, and patterns emerge. Philosophy and postmodern literary criticism oppose chick lit, popular thrillers and the young adult section. Programming does not truck with classic literature. Memoirs of depression, like Prozac Nation, meet their match in the cheery The Night Before Christmas. Ann Coulter and David Sedaris do not see eye-to-eye. There is a strong disconnect between readers of much recent Protestant, mostly evangelical, non-fiction, and large swaths of contemporary literary fiction. For example, LibraryThing includes 2,300 readers who’ve logged Jeffrey Eugenides’ epic gender-bender novel Middlesex, and 222 readers of John Piper’s The Passion of the Christ: 50 Reasons He Came to Die. But the groups don’t overlap. No reader has both. Similar instances occur again and again.

These disconnects sadden me. Of course readers have tastes, and nearly everyone has books they’d never read. But, as serious readers, books make our world. A shared book is a sort of shared space between two people. As far as I’m concerned, the more of these the better.

So, in the spirit of unity and understanding, why not enter your favorite book, then read its opposite?

By the way, how about putting this up on Del.icio.us or Digg? Wait, we have a Digg now.*

*LibraryThing never been Dugg. But recently a pale immitation of LibraryThing was lofted to the heavens as the first social network for books. For one day Digg gave them twice our traffic. Fortunately, they fell like a stone after that and, this morning, Alexa has them at “too low to measure.”
**My friend Ben correctly points out to me that he suggested “find your book nemesis” almost a year ago.

Labels: 1

Thursday, November 9th, 2006

Seven million books

With spooky synchronicity we hit three major milestones today:

  • Seven million books cataloged
  • 100,000 registered members
  • 1,000 groups

The seven-millionth book was added by Liftedviolets, a new member with just over 150 books cataloged. If the profile picture (right) is anything to go by, there’s a lot more to come! The book was Fried Green Tomatoes at the Whistle Stop Cafe by Fannie Flagg. Liftedviolets wins a free paid account—congratulations.

The other two milestones are also big. The 100,000 members is, of course, a little inflated, including members who signed up and didn’t do much of anything. It even has a few sign-up do-overs and test accounts. Still the number has been creeping up, so it’s worth noticing a milestone. We’ve been running around 10% non-English sign-ups, to LibraryThing.de, LibraryThing.fr and the like.

1,000 groups is also impressive. Groups were only introduced in July. Use has increased of late. The 1,000th group is Ringers of Handbells, “A group for anyone who enjoys the art of English Handbell Ringing.” At present group has just one member, and is private. And that’s all I know!

Labels: 1

Tuesday, November 7th, 2006

An election-day tag cloud

For our US members:

LibraryThing is strictly non-partisan; I’m not trying to make any points with my selections! I tried to think of issues and topics that were in play this year, or that commonly motivate Americans’ votes. I left out education because there are so many books tagged education that it made all the other tags look like ants! And what this? Nine million tags and not one gubernatorial?.

I do have one semi-political comment to make. I went down to our new voting place, a freshly built school on Munjoy Hill in Portland. The school also houses the local branch of the Portland Public Library, and indeed the school’s library and the branch library are intermixed.

Well THAT lasted a few weeks! Now the branch library is closed to adults during the day so that patrons and school children can’t come into contact. How sad. How very sad.

Labels: 1

Tuesday, November 7th, 2006

TitleTrader joins the crew

Today we’re adding our seventh swap site.* The site is TitleTrader, and it’s a BIG addition. We can’t give site-by-site holdings statistics, but TitleTrader adds a lot. Finding a book to swap through LibraryThing is getting easier and easier. Check out these numbers:

  • Of the top 1,000 works on LibraryThing, 92% have a least one copy available for swapping.
  • Of the top 10,000 works, 73% have a least one copy available for swapping.

Here’s what the creator, Dan Abercrombie, has to say about making the TitleTrader and what sets it apart:

“The site was started in Sept of 2004. A few months earlier my sister came to me with the idea for a new website … She had a lot of books and a small house and thought it would be a good idea to get rid of them. I put it together in a few weeks and have been working on it since. Since then there have been a lot of entrants into the space, each with their own take on the idea. … From what I can tell, we’re the largest site that has not only books, but CDs, DVDs, VHS, etc.** It provides a nice flexibility being able to trade a CD for a DVD for example. We’re also one of the few sites that supports international trading, users can select which countries they will ship to. Our users also seem to like the ability to choose which item is sent to them and from whom, based on condition or rating.”

So give a warm welcome to TitleTrader, and to its members.

To the remaining sites that haven’t joined: What’s keeping you? Choice is good. Letting your users integrate their favorite websites is good. Aren’t you getting lonely in your walled gardens?

*Actually, we have eight in the system. I’m waiting on a technical tweak by number eight before I show the logo and blog about it. Does anyone know of any sites I’ve missed?
**You’ll notice that TitleTrader comes in fouth in the list. This is according to the original invitation, that gives book-only sites priority over non-book sites, and then sorts within those categories by Alexa rating.

PS: Minor technical note. The “Give yours” link for TitleTrader currently only works if you’re signed in. Dan tells me he can fix that.

Labels: 1

Tuesday, November 7th, 2006

CueCats, meet Abby cats

Back in stock and ready to go! They tried to deliver the latest shipment to me while I was away in Wisconsin, but this afternoon the UPS guy and I finally managed to be in the same place at the same time. It didn’t come, of course, in time for me to make it to the post office before it closed-but I have three paper grocery bags filled up with the envelopes, ready for an early morning trip. Now I understand why Tim was saying his hand hurt from writing out so many addresses…

So if you ordered a CueCat in the past few days, I’m sorry about the delay, but it’ll be en route to you as soon as I wake up in the morning. For everyone else, we still have some left, and even more coming by the end of the week – get your order in now and we promise a quick turn around. Only fifteen dollars, including shipping anywhere in the US.

(neither animals nor CueCats were harmed during this photoshoot.)

Labels: 1

Sunday, November 5th, 2006

Review pages

Bloggers rejoice! I’ve added dedicated pages for every review, so you can point directly to your review, not to the work page generally.

Basically, the link appears wherever you see a review. For example, you’ll see it on work pages and on your review page (here’s mine). I’m using the “permanent link” icon. I hope it’s not confusing.

Labels: 1

Saturday, November 4th, 2006

Let your reviews go!

Before now, it was never really clear what LibraryThing could or couldn’t do with your reviews. Unlike other such venues we’ve never had a “do anything” clause, although cynical members probably assumed we did. Now that we have some opportunities, we’d love to show some reviews outside of the site.

But they’re your reviews, so you decide where they can go. We’re giving you three options:

  • Keep reviews on LibraryThing only
  • Allow LibraryThing to give your reviews to non-commercial entities (libraries mostly)
  • Allow LibraryThing to give reviews to commercial entities (booksellers, publishers, authors, street gangs)

Change your setting by going to edit your profile.

If you don’t decide by December 1, we will default you to unrestricted use. In a day or two—once I do the code—all members who have posted a review will receive a message about this. New people will get a message when they post their first review. Whatever you choose, if your library itself is private, your reviews will be private too.

I’m in the process of writing protections into the TOS, eg., that reviews not be changed in any way. (I think I’m going to allow obscuring of swear words, like f**k.) I’d be interested to hear if anyone has any other concerns. LibraryThing won’t be DOING anything for some time, and we want to do this carefully, and listen to what people think, as we usualy do.

Why are we doing this? LibraryThing users have built up quite an impressive corpus of book reviews (almost 100,000). Our focus on books as books—not on selling books—together with the lack of any “review rules” has given rein to a lot of excellent writing.

In the last few months, Abby and I have been going to library conferences and meeting a lot of librarians. We’ve learned that libraries are eager to show more information in their catalog—particularly reviews and recommendations. The few already doing so are paying through the nose for this data. Often this means snippets from “professional reviews” assembled by the publisher for their ONIX feed and repackaged and resold to libraries by data companies. (Library patrons are, I think, generally unaware when reviews come from the book’s publishers!) Amazon of course provides user reviews, but only if a library is willing to make itself a sales channel in return. Commercial concerns, such as Abebooks, are also interested in showing reviews on their sites, and obviously can’t use Amazon’s.

So, we’d love to open up LibraryThing’s reviews—to sell them, probably, although we will be underselling the data companies by a mile. By allowing us to do it, you’ll be helping LibraryThing financially, and giving your opinions wider currency to boot.

In case it’s said, LibraryThing is not turning evil. Letting your review go is your choice, not ours. If you ever reviewed a book on an online bookseller you agreed to unlimited distribution and modification. One of LibraryThing’s competitors has a TOS that asserts COPYRIGHT over user reviews. (Prima noctis too, probably.) LibraryThing wants to give you control over your reviews. If you want to help us out, great. If not, and we’re very grateful you want them on LibraryThing. We’d kiss you if we could.

Instead, post here or come talk on Talk.

Labels: 1

Saturday, November 4th, 2006

MyPictr.com

A quick plug for MyPictr.com, which takes a pictr—I mean picture*—and scales and crops it for various social-networking sites. LibraryThing actually doesn’t have set sizes. (Many sites require small squares, which appear all over the site in lieu of names, which, as we all know, involve those tricky letters things.**) Even so an easy-to-use scale-and-crop service is welcome. (Hat tip: Steve Cohen)

*Creative site; lemming name.
**The chosen picture, smiling college student with beer about to spill is telling.

Labels: 1

Wednesday, November 1st, 2006

Recommendations refreshed, improved

I just completed a rather extensive regeneration of the book recommendations—the “people who own X also own Y” recommendations and the “similarly tagged” ones. (The “Special Sauce” is next.) Recommendations, in turn, affect the “Pssst!” system (recommendations based on your whole library) and involved recreating all the work-to-tag clouds. Quality has, I think, risen significantly. We’ve improved the algorithms and the non-stop growth of the LibraryThing data set—now at 6.8 million books—continually improve the results anyway.

Scope has also improved. The system only makes recommendations for works with more than ten copies. That comes to juse 93,000 out of 1.3 million works. But these works account for over 60% of the books in LibraryThing. Before the current re-do, only 50,000 works were covered.

The recommendations are better, but hardly perfect! We’ve made progress toward better algorithms, and have big plans for future improvements. On the UI side, we’re going to introduce users-feedback on recommendations, including both thumbs-up and thumbs-down buttons, and an “obscurity knob” for Pssst! (People love that on Last.fm.)

The system works best in the “middle,” on books with 25-500 copies, non-fiction, genre fiction, books with a well-defined readership, and books that are “about” something—books like Touching the Void, Prozac Nation, The Historical Figure of Jesus and An encyclopedia of fairies. It has the most trouble with bestsellers, “obligatory books” (think high-school classics), and literary fiction—books like the Da Vinci Code and Great Expectations. To some extent, the problem is almost philosophical. What would be good recommendations for the Da Vinci Code? Statistically-speaking not much stands out. At the high-obscurity end, there are of course books on the book and its themes—Cracking Da Vinci’s code or Baigen’s Holy blood, holy grail.—but most Da Vinci Code readers aren’t interested in that stuff. Ideally, we’d like a better mix of suggestions—some obscure stuff, and some of the quick, high-popularity reads it correlates with.

Burying the lead? We figure we’re a month or so away from getting the algorithms where we want them. Once they are, we’re going to start making them available to libraires, to spice up their online catalogs with top-notch readers’ advisory—and for much less than they’re currently paying for inferior services. We think it will make quite a splash!

NEWS: Tag-combining is back!

Labels: 1