Archive for the ‘tagging’ Category

Tuesday, June 12th, 2012

New feature: Tag translation

As many of you know, LibraryThing is available in more than a dozen languages like German (, French (, Dutch (, Finnish (, Polish ( and Slovak (

Basics: Today I introduced a new feature, called “tag translation,” to show many of LibraryThing’s 87 million tags in the language of the site. Translation has been seeded with translations drawn from one user-driven ecosystem, Wikipedia. LibraryThing users can help out by adding new translations, and voting on existing ones. Although words are rarely perfectly equivalent between languages, translation may prove useful to many of LibraryThing’s non-English members and, in time, to libraries that use LibraryThing’s data feeds and LibraryThing for Libraries.

The feature: Tags show up translated wherever tags appear(1). You can choose to see them that way, with color-coding (pink for translated) or you can opt to shut the feature off. Here’s a a version of Thucydides with current German-language tag translations.

The same can also be seen on tag pages, for example on this French page for “love.”

Tag translations can be examined, voted upon and edited at the bottom of tag pages. Here’s the expanded view of some of the tags for “Love.” This is the only part of tag translation that is seen on the English site

To turn off or to color the tag translations, use the little info button at the bottom of tag clouds (wording will vary according to language.) It pops up a little area to make the change.

Review translations: You can review recent translations, and vote on them here:

More information. For more information about how tag translation works and to comment come join us on Talk.

Labels: new feature, new features, tagging, tags

Thursday, August 25th, 2011

New tag-based recommendations algorithm

Short version. I’ve just finished up a new algorithm for calculating book recommendations based on tags. You can see them on the “Recommendations” sub-page for every work page, under a special “Tags” heading (for a limited time only!), as shown to the right. When a work doesn’t have a recommendation, it will make it. (Expect to wait 2-10 seconds.) Recommendations will be propagating through the system, and, combined with the four other recommendations algorithms we use, going into your personal recommendations over time.

Come discuss it on talk here.

Long version. Recommendation algorithms are always tricky, but doing it based on tags alone is particularly difficult. Do you consider total overlap? Overlaps by tag? How do you rate a book missing an important tag? How do you factor up meaty and meaningful tags, and downgrade meaningless, over obvious or ephemeral ones? LibraryThing has long had tag-based work-to-work recommendations, but they were of uneven quality. I haven’t been making new ones for a while now, and letting them die out of the system; all the old tag-based recommendations have been removed.

The new algorithm approaches the issue afresh, looking at all of a work’s tags, and taking into account various factors that can trip it up. It thinks about factors like tag salience on a work and generally, degree of agreement between tags and low-value tags. It also attends to similar levels of work popularity. After deciding on the basic algorithm, I toyed with various “knobs” for days, myself and with Jeremy, trying to get the best results for a set of sample and problem works. In my experience, you can’t do an algorithm like this without a sense of appropriateness, taste and proportion, and (I hope) that this is one reason LibraryThing recommendations are generally so good.

Once a tag-recommendation is generated, it takes a while for it to be incorporated into the “Combo recommendations” above. “Combo recommendations” incorporate tag recommendations to greater or lesser degrees, depending on its assessment of their quality and contribution. Your personal recommendations are based mostly on these “Combo recommendations.”

The tag recommendations are going to take a while to build for all works that need them. After that, we plan to do some sort of “Pepsi Challenge” test. We think LibraryThing recommendations are as good as any out there, and are eager to prove it.

Some examples. Tag recommendations work absolutely best on non-fiction titles about something very simple and clear cut:

Works really “about” two or more things are harder, as are books with a specific point of view, which might seem separate to some extent from the tags on it. Some examples:

  • PHP and MySQL for dummies by Janet Valade — A successful example, where most of the books deal with both PHP and MySQL, with an orientation toward “entry level” programmers (eg., the “Visual Quickstart” books)
  • Born Fundamentalist, Born Again Catholic by David B. Currie — A successful example that mostly recommends other Protestant-to-Catholic conversion stories (rather than Catholic-to-Protestant ones)
  • Goldwater by Barry Goldwater — A mixed but mostly decent result, surfacing some Goldwater-specific material, but also showing other biographies, especially from the period and involving senators. Ideally it wouldn’t have quite so many contender-bios, as I’m not sure the potential Goldwater reader is eager to dig into Edward’s bio.
  • Freakonomics by Steven D. Levitt and Stephen J. Dubner — Decent list, starting with a short-shelf of popular economics-in-life books, followed by popular introductions to economics and economics-oriented thinky-think books

Fiction, especially non-“genre” fiction, is the big problem. Literary fiction isn’t “about” what it’s about in quite the same way that non-fiction is. Creating some separation between adult and youth titles is also hard. But we’ve made significant progress.

  • Watership Down by Richard Adams — A decent list of animal-centered chapter books, with classics like Redwall and The Wind in the Willows and no The Runaway Bunny or Knuffle Bunny.
  • The Book of Three by Lloyd Alexander — A decent list of magical fantasy books, angled to youth and toward Welsh mythology.
  • The Hobbit by J. R. R. Tolkien — A good list, but obviously centered quite strongly on Tolkien. Works with significant secondary literatures (cf., Harry Potter, Narnia, etc.) tend to be dominated by guides, atlases and so forth.
  • Little Women by Louisa May Alcott — Starts out well, putting March second, but then goes off the rails somewhat after a dozen. The Mother-Daughter Book Club wins because it apparently takes place in Concord, Massachusetts and involves mothers and daughters. The Secret Life of Bees is also about mothers and daughters, coming of age, sisterhood, and was made into a movie. Meh.
  • The Book Thief by Markus Zusak — Decent list, with other books, mostly fiction, about children during the Holocaust.
  • The Shack by William P. Young — I haven’t read it, but I think have a sense of it. There are some winners here, like The Christmas List, Redeeming Love. The Year of Fog and The Deep End of The Ocean cover some of the same issues from a non-religious standpoint, and Where Is God When It Hurts covers them from a non-fiction, evangelical perspective, which might or might not be wanted. But C. S. Lewis and Paul Bunyan(!) aren’t winners, being rather different sorts of fictions. Tim LaHaye is winning almost solely on being “christian fiction,” and James Redfield for “religious fiction,” “inspirational,” etc.
  • Patient Zero by Jonathan Maberry — Zombies? We got zombies, focusing on plague-based zombie terror (no Pride and Prejudice and Zombies). Straight-up plauge titles, like The White Plague are also included. Preston’s The Demon in the Freezer probably shouldn’t be there, but it’s a great book.
  • Earth Abides by George R. Stewart — More apocalypse, this time with few zombies. The Brief History of the Dead is rather different, though I’m not sure how LibraryThing could know that. At least it doesn’t attempt to recommend other boring books, like Earth Abides.
  • Blueberry Girl by Neil Gaiman — Some good stuff. Sound lousy. The tags are dragging the recommendations all around—children, picture book, Gaiman, mothers and daughters, charles vess. Twinkle, Twinkle, Little Star is winning on “poetry,” “rhymes” and child-associated tags.
  • Love in the asylum : a novel by Lisa Carey — My wife’s book. A decent-list of insane-asylum fiction, with some memoirs.
  • Twilight by Stephenie Meyer — Largely okay, so far as I can tell, although with less secondary literature than I would have guessed.
  • Illyria by Elizabeth Hand is winning on tags like “forbidden love,” “teen romance,” “teen lit,” “romantic” and “contemporary fantasy.” I have no idea if the books are similar.

Come discuss it on talk here.

Labels: recommendations, tagging, tags

Friday, March 4th, 2011

“Tag Mirror” is back!

The much-loved, long-suspended “Tag Mirror” feature is back!

Your tag mirror is like your tag clouds, except that instead of seeing what you’ve tagged your books it shows what other members have tagged them. Sidelined because of speed problem, a series of database changes have made it viable again, without without extra caching. It’s not instant, but few users will find the speed insupportable.

Check it out:

And here’s the original blog post from back in 2007.

Labels: tag mirror, tagging, tags

Monday, February 15th, 2010

Dead horses to ponies…

Borrowing a joke from Brightcopy, I’ve turned some dead-horses into ponies, bringing some long-requested features to life, and even improving on them.

Books You Share Preferences. Some members have long campaigned for sorting the profile-page “Books You Share” box by author, not title. But I held off—that’s not the right choice for everyone. Instead I’ve added a preference for it, with a number of different sorting options.

Critically, I set the default to sorting by popularity from low to high, something nobody had ever requested. I thought members might pounce on me for it, but quite a few have said it was an unexpectedly good choice. It brings out the unusual books you share. And those are often the most interesting.

I also added a preference to change how many shared books are displayed.

See this topic for more about the feature.

Tag Combination. After a 16-month hiatus, new tag combinations and separations are back!

The idea is simple. LibraryThing allows members to combine tags that are highly similar in meaning and application. Classic examples are tags like “World War II” and “wwii” or “ww2.” We discourage combining terms that don’t entirely overlap, either in meaning or in usage. (If you’re interested in the ideas behind tagging, check out my What’s the Big Deal About Tagging? talk on YouTube.)

Tag combination only affects “global pages”; user tags are never changed.

So far as I know, we’re the only website to experiment with this idea, something noted in Gene Smith’s Tagging: People-Powered Metadata for the Social Web. Tag-combination combines a new idea—tagging—with an older idea—what librarians call “authority control.”

This time, however, we’ve given it a twist—democratic authority control. Any member can propose a combination or separation, but the matter is put up to a vote—with a supermajority needed for any action. We hope it will slow down the process and make it more deliberate.

It’ll also save our servers from having to recalculate tags. With more than 60 million tags, and “science fiction” now at three million uses, instant, any-user combinations were really putting a strain on our system.

See more about it, and some examples here.

Labels: new features, tagging

Monday, January 4th, 2010

The Great Group Revamp

I’ve revamped groups in ways small and large.

The result! The revamp is working. Since the change, daily group-joining rates have almost doubled for both old and new members. Nice.

New Groups page. There’s a new group tab (see here). The page is:

Group tags. Until now, there was no good way to find particular sorts of groups. Rather than designing some static and ultimately limited system of categories, we’ve asked members to tag groups. Of course, members went crazy at it. You can see the tags:

Local Groups. Groups can now have locations, and the group home and your groups page now show local groups. As members have pointed out, “local” is a relative term, but the results will improve as local groups are identified and added. (Go here to add a place to an existing group.)

At present, the largest groups are the Australians, Germans and Bostonians.

“Welcome to LibraryThing!” By popular request/agitation added a Welcome to LibraryThing! group, for introductions, questions and other conversation. As the description states:

“LibraryThing is a rich site, with a number of different communities and projects going on. It can also be a complex site—powerful but sometimes daunting to newcomers. This group is a friendly place for new members, and the experienced members who can help them make the most of it. Most questions and introductions are answered within minutes.

Members, new and old, are invited to check it out.

Dormant groups. The system now tracks groups for activity. If twelve months pass without a message—excepting private groups—the group becomes “dormant.” As befits a more than four year-old site, some 3,000 LibraryThing groups are currently dormant!

Groups “wake up” when a new message is posted to them. In many cases, however, it’s better to start a vibrant new group than revive a dormant one.

Other changes.

  • Better searching. Group searching is much improved, with activity graphics by every group, weighting by activity, tags figured in, dormant groups excluded by default and a better algorithm generally.
  • Better navigation. All group pages are now connected, with a common navigation.
  • Smaller pages. Pages are smaller and therefore faster. Caching is improved, so the results are both fast and updated frequently.

Talk about it These changes have been trickling out for more than a week, and conversation has been extensive—and very helpful. The more important topics are:

1. As explained elsewhere, tags are sized more according to the aggregate activity of the groups than the number of times they are tagged. This differs from how work tags work, but favors the goal of helping people find things.

Labels: group tags, groups, tagging

Tuesday, September 15th, 2009

Tagmash, redux: Tim’s favorite feature

Tagmash. I’ve redone, improved and expanded my favorite feature, tagmashing.

Introduced back in 2007, tagmashes, allow you to investigate what books satisfy two or more tags. It’s a great way to find books of a clear type, but for which no single tag really works.

For example, no one has yet used the tag “vegetarian Indian cooking” and there’s no Library of Congress Subject Heading for it either. But combine three tags, like vegetarian, India and cooking into the tagmash vegetarian, India, cooking and you get over 50 good matches.

Simple two-tag combination can work wonders:

Some of my favorites are off-beat: all those books about knitting for your dog and—shiver—knitting with dog hair can be found at knitting, pets. erotic, zombies is 80% Laurell K. Hamilton. And who can say no to humor, pirates? (Did you know that this Saturday is Talk like a Pirate Day? You will.)

On the serious end, fairly complex topics also work:

You can also use – (minus) or — (double minus) to mean “demote” or “remove” a tag. For example:

An important feature of tagmash is that it’s not just a “search.” Once created, tagmash pages stay there, and it enters the “swirl of relatedness.” Somtimes a tag page will suggest the perfect tagmash. Other times, a tagmash will suggest an unconsidered subject.

New Feature: Tagmash overlap. I’ve added a new feature that, I think, brings tagmash to a new level—the tagmash overlap.

It works something like tag mirrors. Instead of showing you how you tag things, it shows how others tag your stuff. Except instead of showing you Individual tags, it finds tagmashes.

The results is, I think, a good list of topics you’re interested in—topics more complex than a single tag can express. In my case, it surfaces topics like Macedonia, history, Greek, divination, Ottoman Empire, travel and erotic, poetry (!). Abby is apparently interested in adventure, surreal, English, death, love and—what a winner—evil, love.

You can find the feature from your profile statistics page. If you’re signed in, this link will take you to yours.

What do you think? Comment here or come over to the New Features Talk thread.

Labels: classification, tag mirror, tagging, tagmash

Tuesday, June 9th, 2009

Collections, at last

It’s arrived. Members can organize their books into “collections.”

The Motive. From the beginning, LibraryThing members have used the site for different things. Some used it to list only the books they own, others what they’ve read and a few even just the books they wanted. Meanwhile, people like me used it for everything—owned, read, lost, destroyed, wanted—using tagging as our sole way of keeping everything straight. But even tag-zealots like me had to admit there were times you wanted sharper distinctions—”buckets” or “sub-libraries”—and ways to tie those to how you connected with other members and with book recommendations. New members, whether familiar with tags or not, were regularly asking for some way to do wishlists and currently-reading lists.

The Feature. The feature, literally years in the making, gives members the ability to separate out categories of books, like “Wishlist” and “Currently reading” more definitely than could be accomplished with tags. Each collections works like a mini library and can be separately viewed, sorted and searched. Other members can see your collections, on your profile and elsewhere. Features like member-to-member connection and book recommendations react to the new system as well. (See below on integration progress.)

As we offer users new flexibility, we avoid forcing members into “our” way of thinking about books. We’ve provided a number of default collections—Your library, Wishlist, Currently reading, To read, Read but unowned and Favorites. Data from these collections can be aggregated across all users, and their names are even translated on LibraryThing’s non-English sites. But you can also create your own collections, and remove ours. And you can ignore collections entirely, keeping everything in “Your library.”

A Work in Progress. As members know, we play things pretty fast and lose here. Our motto is “beta, forevah!” But collections had to be different. Before public release we subjected it to a month of testing in our large (and non-exclusive) BETA Group. We cannot thank that group enough for all the work they did, and the passion they showed.

We hope we got most of the major bugs, but the feature is not “finished”—and this is hardly the last blog post you’ll see about the feature! Most significantly, collections is now mostly a “cataloging” feature, with only limited reach to other areas of the site. Although you can specify how collections affects member connections and recommendations—so you can stop having your Wishlist or for that matter your husband’s books running the social and recommendation parts of the site—implementation is basic and, in light of extraordinary collections-related load, there’s a lot of caching in place. We left a few features out in order to get it the main features out now.*

We also think “unfinished” (we prefer not prematurely specified) features are the best way to engage users, and get the best for everyone. Come and contribute on Recommended Site Improvements and Bug Collectors. We also have a Announcement post in New Features.

*We had spec’ed out a complex interaction between reading-dates and “Currently reading.” But the system was probably more than most members wanted. And it certainly was taking a long time to finish, so, for now at least “Currently reading” is just a collection.

Credits: Chris (conceptDawg) headed up the project, doing most of the user interface and a majority of the back-end code. Chris and I (timspalding) designed the feature together, and I did some core back-end code. Abby (ablachly) didn’t code, but she dogged us about it for years. (I’m not sure what she’s going to do with herself now.) But the most important factor was the members. Members, particularly the BETA group, contributed to the effort as I’ve never seen it—not in any website or project, ever. Chris and I owe members an enormous amount. (I’ll be blogging about this specifically soon. It needs telling.)

Top photo by radiant_guy” (Flickr, CC-SA).

Labels: cataloging, collections, new feature, new features, tagging

Thursday, August 23rd, 2007

Tag Mirror: See your books the way others do

UPDATE: I’m really enjoying the Talk discussion of this feature. Also, at this point it’s better to talk about the feature than to use it. Everyone using it at once has the server that handles it taxed rather seriously!

A major publisher recently asked us to show them a tag cloud of their books. It took a mental flip, but only a few lines of code to adapt this for individual use.

The result is Tag Mirror, available from your and everyone’s profile—here’s mine (and Abby‘s, Altay‘s, Giovanni‘s and Casey‘s*). If you’re signed in, here’s yours. (Please note: It takes serious processing power to analyze 22 million tags. Everyone is going to hit it at once, so be patient.)

Tag Mirror “holds a mirror” up to your books and to you. Instead of showing what you think about your books—what a regular tag cloud shows—it shows you what others think of them, in effect using LibraryThing’s twenty-two million tags to organize and surface interesting topics from within your own collection.** As with other tag clouds, size equals importance. When you click on a tag, you get a relevancy-ranked list of books tagged that way.

I can’t decide if it’s just the sort of cherry-on-top feature that makes LibraryThing unique or if it’s something genuinely new and interesting. I think it might be the latter. As Altay put it, it’s the sort of idea that seems obvious in retrospect.

I didn’t know I was interested in gender studies.

Here’s a for-example. I don’t use the tags gender studies, patristics or theory. They’re just not terms I use. To some extent, that reflects who I am. But I have a fair number of books that, to others, fall under those categories. It’s interesting to slice my books up in an alien way—to see them through other eyes. Maybe I’m more interested in gender studies than I thought.

More concretely, I do use the tag “alternate history,” but browsing my tag mirror page called up some alternate histories that I hadn’t tagged that way—useful stuff.***

Finally, Tag Mirror gives everyone a tag cloud, even those who don’t bother to tag anything. It seems almost unfair.

As our recent discussion of what tagging does to knowledge brought out so well, tagging is a complex mixture of private purpose and public good. I agree with those who say that we tag best when we tag for ourselves. But when everyone does that, a rich web of meaning is created.

I’ve done my best to push tagging in some new directions, trying subjects and tags together statistically, making book recommendations based on tag patterns, and with the tagmash feature. You can add Tag Mirror to that list. Little things. But they keep getting more interesting.

UPDATE: It’s 4:30am and, of course, I couldn’t finish blogging it before someone else started a thread about it (“Just noticed this on my profile”). Come talk about it.

*Casey has a surprising number of cookbooks! He’s coming up here in a few weeks—it’ll be the first time any of us have actually met him. We usually just order pizza. I think that plan’s changed.
**It doesn’t actually exclude your own tags. They still have an effect.
***It also brought up Howard Zinn’s People’s History of the United States. People tag unexpectedly, if humorously.

Labels: folksonomy, tag mirror, tagging, tagmash

Wednesday, August 22nd, 2007

International tags and more

We’ve had quite an upswing internationally, particularly among Dutch speakers. Dutch has surpassed French as our second-largest language community. (Next up: the Germans!) So I spent the evening adding some international features.

I’ve added special tag clouds to work pages on our non-English sites (,,, etc.) They show tags used by members of that site, or on books in that language.

It doesn’t always “work” that well. Perhaps half the tags on our non-English sites are still in English, the site tending to appeal to English-language speakers first. But I imagine that will change as the membership broadens, and tools like this make tagging in your own language more attractive.

The example above is from the Dutch site ( work page for Harry Mulisch‘s De ontdekking van de hemel (The Discovery of Heaven), the most popular work on the site. It’s more than half English tags. A more Dutch example would be De kanonnen van Navarone (The guns of Navarone), tagged avontuur (journeys by airplane) and spionage (spinach) alongside thriller and world war two.*

I also added an indication of how many of your linguistic compatriots have the books. Here is the French page for Amélie Nothomb‘s Stupeur et Tremblements (Fear and Trembling)—the fourth-most popular book among French members, but not in the first 10,000 among English-language members. The text is yellow and in English because I just added it, so no kind French user has yet volunteered a translation.

Lastly, I thought I’d announce and explain a feature just before killing it. (As Hegel said, “the owl of Minerva spreads its wings only with the falling of the dusk.”**) That feature is tag-coloring, an experiment that recently went site-wide (with the change in caching systems). The idea was to color personal tags lighter than subject tags, algorithmically at first, with some hand work from the LibraryThing for Libraries program, and then moving to let users weigh in on what was and what wasn’t personal.

I was never convinced either way, but I thought it worth a try. The reaction on Talk has, however, been pretty hostile, not helped by the fact I didn’t talk about it after it went live). I think I agree with the criticism now too. Anyway, chime in there if you like it. Otherwise, it’s going away… Sometimes beta means making mistakes.

*Hey, it’s 3:35am here.

Labels: dutch books, internationalization, tagging

Tuesday, July 24th, 2007


Tagmash: alcohol, history gets over the fact that almost nobody tags things history of alcohol

Short version: I’ve just gone live with a new feature called “tagmash,” pages for the intersections of tags. This is a fairly obvious thing to do, but it isn’t trivial in context. In getting past words or short phrases, tagmash closes some of the gap between tagging and professional subject classifications.

For example, there is no good tag for “France during WWII.” Most people just don’t tag that verbosely. Tagmash allows for a page combining the two: France, wwii. If you want to skip the novels, you can do france, wwii, -fiction. The results are remarkably good.

Tagmash pages are created when a user asks for the combination, but unlike a “search” they persist, and show up elsewhere. For example, the tagmash for France, Germany shows France, wwii as a partial overlap, alongside others. Related tagmashes now also show up on select tag and library subject pages, as a third system for browsing the limitless world of books.

Booooring? Go ahead and play a bit:

That’s the short version. But stop here and you’ll never know what Zombie Listmania is!

(full post over at Thingology, “Tagmash: Book tagging grows up”)

Labels: new feature, new features, tagging, tagmash