LibraryThing has a handful of competitors. I’m not too worried about them. LibraryThing wins on features, as many blog comparisons have shown (eg., the Powells blog). And LibraryThing certainly wins on size and traffic. The most frequent comparandum started the same week as LibraryThing but has 4% of the books. Another has 20% as many, but it’s been going for more than four years; it hasn’t broken the web’s top 100,000 sites in six months.

I’ve written to most of LibraryThing’s competitors, seeking a deal that would combine our sites, giving all their users lifetime accounts and bringing the developers on board as partners. So far, no dice. I guess it’s more fun to do your own thing.

A few days ago I came across a new site, Bokhyllan.com, a sort of Swedish LibraryThing. Between guessing-in-context (“LOGGA IN”), bad German and a second-degree connection with someone who knows Swedish, I managed to get the jist of it.

What a perfect situation! They’re just starting, and the site doesn’t have all the features it needs. LibraryThing has those features—even integration with the Swedish library system! But I have no desire to develop a Swedish LibraryThing, and no ability even if it did. It seems silly to do all this development multiple times. Why not combine forces and write the code once? Take the code, keep the domain, charge for it or make it free—I don’t care. But share your improvements and we all win.

I wrote. No answer so far. Maybe this plug will get their attention.

If anyone knows of any other non-English LibraryThings, let me know. If you want to start one, let me know. More generally, if you want to help LibraryThing grow, let me know.

There’s a more general conversation here about open-sourcing LibraryThing. I guess I’m a sceptic that it could work. At a minimum, a whole separate site would need to be set up, with fake data. Security would also be a big issue. If it didn’t work, it would be a big waste of energy. If it caught on, I feel like my job would become cat-herding programmers. In the worst case, LibraryThing would end up fragmented into a half-dozen forks.

Maybe we could discuss open sourcing on the LibraryThing Google Groups group. The group is growing in greatness—must keep using words starting with g. If possible, I’d like to keep blog comments on the blog.

All requests eventually come true. Searching has been improved, replacing the old “relevance” search with a more Google-ish search.

The search page now look like:

The main improvements are:

• Searching for “Greek history” now gives you all the books that have the phrase “greek history,” not some lame weighing of books that have just one or maybe just have a word close to “greek.”
• Book searches can use syntax like +greek -history
• You can now search LibraryThing for authors and tags. Instead of dropping you into a catalog with X-thousand books by J. K. Rowling or tagged “science fiction,” these take will take you to the LibraryThing page for J. K. Rowling and science fiction.

Still to be improved:

• Book searches are still ordered by relevance, which means they can’t be sorted by author, title, etc. I’ll fix this soon.
• You cannot currently search the entire 1.5 million books for a single book. I’m going to add this, but add it in a way that you get the book-page, not a catalog page loaded 10,000 copies of Harry Potter. The main problem right now is speed; I don’t want to introduce any more speed-hogging features.

Of course, you can search individual user’s libraries. To go their catalog and click the “search” link. But this doesn’t seem enough to me. Perhaps profiles should include a search box, or the search tab should have some way to select a user to search. I’ll tell you that I don’t want a menu with 21,000 people in it! Ideas taken.

Update: LibraryThing is still mostly up, but sometimes slow. I am working on the server issue on a number of levels and hope to show some progress soon. This is one issue I will lick, no matter how large it gets. That said, if you’re David Pogue and writing up an article on LibraryThing for the New York Times, please don’t publish it this week!

LibraryThing has hit 1.5 million books cataloged and 2 million tags, all since August 29.

What used to be a straight line of 9,000/day has become a bending curve—gulp! Even so the site pretty-much stayed up today. The database isn’t out of the woods yet, but it’s in better shape. You’ll see some stutters, but downtime should be minimal and I’ll be getting a second server for some of the “thinking.”

Some other updates:

• Profile pages now show ten random books from the user’s library. People were doing this with widgets, but that presented some security problems and also looked bad with the new design. I will probably add the ability to customize what people see (most recently-read books, and so forth).
• Acquisition, started and finished dates have been added. This feature hasn’t been extended everywhere—eg., you can’t use it in a widget. That will come soon. You can enter dates in North American or European format, but the catalog currently only shows them in the righ—um, North American—way.
• I added a statistic to the Zeigeist page listing the “50 ‘completist’ authors.” These are authors that, if you have one of their books, you have a lot. Think people who collect every single Agatha Christie, although the actual authors may surprise you. To qualify the author needs to be read by at least 100 users. Come to think about it, I could do a statistic for whose library is the most “completist.” I am quite sure that a science fiction fan would win.

It’s up again. Knock on wood, I think it will stay that way.

In general, the system is stronger and more disaster-resistant. I am working with not one but two competent database administrators. There will be some glitches, but also significant improvements. I am confident that there is a clear path to scaling this thing up very large.

There are some changes, some temporary, some permanent. Most of the heavy-duty processing has been moved away from the page-display. So it does not calculate the book suggestions when you enter a page, as it did before—although it then cached them. Now it gives you the cached suggestions if there are any, and no suggestions if there aren’t. The suggestions are being generated behind the scenes when nothing else is going on. Soon enough all books will have recommendations again. Since they don’t do much “thinking” the book pages are much faster.

You’ll also notice that the profile page looks different. If it’s your profile you get a lengthy list of who shares books with you. If another person’s it fronts the list of shared books, something that was pushed down the page before. It doesn’t give you a list of that person’s list of top sharers. I’d plan to add these other-profile statistics back in soon, but they will be subordinated to the shared books list, which is, I think, the most interesting piece of information.

The profile page currently lacks links to user reviews and the more intensive similar-libraries calculation. These will be back as soon as I decide where to put them. I plan to make the profile a multi-page affair, moving the comments off the main page. Those of you who have dozens of comments will be glad of that.

That’s the news. Thank you for all your support. It was so supportive that I plan to make downtime a weekly event. 😉

I’m sorry about the downtime. I have reached something of an impasse. Something is just different, and I haven’t been able to figure it out what, nor my various helpers. Suggestions to increase normalization or change to a different db engine are really beside the point. There is some reason it worked two weeks ago and won’t work now. The number of books is not much higher. No new feature has been added. In fact, they’ve been taken away.

If there is anyone out there who is confident they could help—it’s a PHP, MySQL site—give me a ring. I’m quite willing to pay $65/hour for competent help.

10:06 – Widget requests are now going through.

UPDATE: Okay. It’s up. Let’s see if it stays up. I’ll blog about various changes soon. You’ll notice some missing recommendations. The site will go down at 3am EST.

I’ve got good news to report on the database front. I’ve made something of a break-through. On the nastiest queries—for example, determining what 20 users share the most books with you and how many they share—it now performs 500 times faster than before. You probably won’t see that sort of increase, but it will be noticeable. If you have more than 1,000 books, visiting your profile will become fun again, not an invitation to wait. And the changes will allow LibraryThing to grow far beyond its current 1,498,000 books.

Although theoretically simple, the changes need to be made dozens of places. And tested. I’m behind on the sleep department, so I may not be able to finish it this evening. It will almost certainly be up by 5pm tomorrow, spiffier than ever. Within an hour or so I expect LibraryThing will surpass both 1.5 million books and 2 million tags.

Thanks to everyone who send letters of sympathy, encouragement and offers of assistance. Incidentally, I love the idea for the down page to sport a falllen pile repair-themed books.

PS: Is it just lack of sleep that makes the phrase downtime update seem linguistically funny?

I’m taking LibraryThing down for an hour or so this morning, when you should be asleep anyway—you’ve got work tomorrow. I’ve made some changes that improve the speed issues; I hope this proves another step in the right direction.

Thanks for all the comments on my last post. It’s thrilling to see so much passion and intelligence applied to these issues. I’d like to experiement with continuing this and other deeper conversation at the recently-created LibraryThing group on Google Groups ( http://groups.google.com/group/Librarything ).

I’m still making up my mind about this group. In the long term I’d like the conversations to happen on a LibraryThing forum, and until then I still enjoy the blog-comments model, despite its drawbacks. But I’m game to see how this might work in a forum. You can, of course, continue to post on the blog too.

Here’s my kickoff message there:

Many of them seem to have feared that LibraryThing was in fact going to veer into Friendster-style social software. Nothing could be further from the truth! I am of the opinion that LibraryThing must always be a great way to catalog books, whether a user wants to catalog everything or just keep track of their reading list. The social aspects must be rooted in this.

Just because they’re rooted does not mean that the social aspects are unimportant or even—I would argue—secondary. Lots of LibraryThing’s features are “social” without being Friendster-ish. Over 2,000 bloggers have mentioned the site, most inviting friends to come look at their collection. By contrast, the Mac based cataloging application and cheek-pinching darling of the technology press, Delicious Monster, has less than 40% as many mentions during a longer period. Why? Because it’s no fun talking about something nobody else can see! Among these bloggers over 650 have added a LibraryThing widget to their sidebar. And all you guys—even if you argued against social features, took the time to post a message on the LibraryThing blog. These are social uses.

Or take the author or tag pages, a very “rooted” social feature. Without LibraryThing’s social machinery, these would be pretty boring. For recommendations you’d get what Delicious Monster and every other cataloging ap provides—the holy Amazon five. The data people put into LibraryThing opens up all sorts of interesting prospects. Take author disambiguation. You can’t do that on any other service. (And yes, edition disambiguation is on its way.)

I think LibraryThing’s strength lies in the unique combination of social and non-social features. Different users will appreciate different aspects. I know there are users who want nothing more than a cataloging ap, keep their library private and probably never click to find out who else owns their books. On the other end there are impatient people who catalog their ten favorite books, throw them up on their blog and look around for people to discuss them with. To succeed, I think LibraryThing needs to continue to appeal to both types.

I think the solution involves these principles:

• The book catalog should always be the central metaphor of the site.
• The site is both useful and fun without any social interaction whatsoever.
• Users control their social exposure (eg., private libraries, turning comments off).
• Most social features should emerge from book catalog data, not from photos, locations or non-book interests.
• Social features should be developed with taste and restraint.*

Thanks once again for thinking through these issues with me.

*The NYT recently had an article about the “eww” factor when Friendster premiered a feature that let you see who had been looking at your profile and when.

I’ve done some tweaks to the design. I’ve wanted to do this for some time and never found the time. The nail in the coffin was an email that described the top bar on the old design as “dangerously close to greige.” Ouch.

There’s a lot more I need to tweak, but that’s true of every aspect of the site. I’m worried that tweaks like this make LibraryThing look too corporate and static. But the old design was ugly and corporate and static, so I think it’s an improvement.

Playing with the design got me thinking about large issues of graphic and social design. How “accessible” should LibraryThing be? By accessible I mean what people mean when they call a book accessible—easy for a large group of people to get into. I’m a little afraid of making LibraryThing too accessible, too appealing.

User photos. One way of doing this would be to put user photos front and center. Social sites like Friendster, Tribe or MySpace, even LibraryThing’s (nearly moribund) competitor Mediachest do this. Photos add impact and—let us be frank—sex appeal.

Social software home pages are particular locus of user-photo activity, with the meat-market aspect paramount. I have no proof, but I believe most social software site’s “random pictures” algorithms have a one-hottie minimum. I particularly appreciate Mediachest’s non-rotating homepage hottie, “Becca from Seattle” (at right), who attests that she used Mediachest and “met some cute guys that actually shared my interests.”* As Twain says, I am girding up my loins to doubt this. Not only does the photo not reside in the user photo directory, but Mediachest requires a state and zip code, and allows a geographic lookup—no Becca in Seattle.

I think there’s reason to believe that LibraryThing users don’t want that sort of atmosphere. LibraryThing makes it very easy to add an image. (The preset ones are one of my favorite features. It warms my heart every time someone discovers the software “easter egg” there.) But of even serious users only about half have chosen one. Among the ones who have are the two lasses at right. Their favorite tags are “democratic capitalism” and “almost but not quite a dictionary.” Their favorite books are… okay, that’s from Friendster and appears to be a magazine photo. (Real people don’t wear matching underwear.) As far as I’m concerned, this is the enemy. I think most LibraryThing users will agree.

Friends. There are a lot of other places where LibraryThing could broaden its appeal and play up the social aspect. Another example is “friends.” At present LibraryThing allows you to add users to a “watch list,” not a “buddy list.” The difference is that nobody knows who’s on your watch list—it’s a glorified bookmark. Buddies lists are very public. I did this because I hate the social dynamics of buddies lists. “I’m his buddy but he’s not my buddy.” “She’s got 200 buddies,” etc. Maybe it’s just because I’m old enough to remember Reagan’s first term, but I find this sort of this pretty irritating and pointless. Am I being a stick in the mud?

Incidentally, I will be adding “groups.” I’m not quite sure how they’ll work but the idea will be to allow people to search for books within a set of libraries. The point is more functional than social. Users have been begging for a way for their whole knitting or book club to get on LibraryThing and search a combined library.

Wrap up. So where should LibraryThing’s design go? Would more use of user photos make the site more fun and immediate or do they lead down a slippely slope? Should I turn the “watch list” into a “buddy list.” Should LibraryThing allow users to enter zip codes and search for them, a la Mediachest? In sum, should I bend the design more to the conventions and expectations of other social software, or should LibraryThing try hard NOT to do this?

Comments appreciated. Sorry for the long post.

PS: Holy smokes—21,000 books added yesterday! That’s double the ususal rate. I think LibraryThing will hit 1.5 million books tomorrow, and 2 million tags the day after.

*Note: I am doing criticism, which entitles me to use the photo as fair use.

Update: I changed something so you can now use Universal Import to import BookCrossing books more easily. See the BookCrossing forum.

Quite a few LibraryThing users are also users of the innovative and long-running site BookCrossing. As the site puts it:

“BookCrossing lets you share your books with the world, and track their individual journeys forever more. Our members have registered 2,657,603 books so far, and a good number of these are “in the wild” at any one time.”

To make the system work, people give their books a “BookCrossing ID” (BCID), composed of three digits, a dash and nine digits. The first digits are secret—available only to people with their hands on the book. You need this part to log where the book has gone and make journal entries for it. The second set are public. Some BookCrossers enter all their books, even if they haven’t given them away.
I’ve added the BCID to the edit screen (the pencil, ). And you can add the field to your catalog screen (use the “change fields” feature). In your catalog you can click on the BCID to go to the journaling page. Needless to say, it only shows the secret part when you’re looking at your own catalog.

So that’s what it does now. I’m all ears. I don’t use BookCrossing, so I don’t know what other features would be helpful. Oh, and does anyone want to post about it there. I feel a little chintzy doing it myself. 🙂

Note: If you see the BCID you’ll also bought, started and read dates. I’m still adding support for this, but feel free to leave comments about how you want it to work or what data should be presented.

I’m looking for book-pile photographs representing Martin Luther King Day and Black History Month. Do you have a lot of books on the topic?

I did a general book pile photo contest earlier (see winners and all submissions on Flickr). It was fun. I want themed bookpiles to become a regular thing on LibraryThing, much the way Google’s logo changes for special occasions.

The rules:

• Take a photo of a pile of books. I favor piles with easy-to-remove backgrounds and legible titles. See previous winners for what I favored.
• Post it to Flickr tagged “librarything” and “blackhistorymonth.” If you must send me files, send them to timspalding gmail.com.
• Winner gets a free one-year gift subscription, to use or give to someone else.
• I’d love to close it in time for MLK day, but that may be too early.
• By participating LibraryThing gets rights to use the photo for promotional purposes, with acknowledgment.

Incidentally, if anyone wants to become LibraryThing’s Dennis Hwang, coming up with holiday book piles, like Hwang’s Google holiday logos, on a semi-regular basis, I’d love to hear from you. I can’t really afford to pay you, but I can showcase your work and give you free memerships from time to time.

UPDATE: Some speed issues remain but the crashing issue is mostly solved. I’ve fixed a number of things and added a few features. I’ll blog about them soon.

I just discovered that Neil Gaiman, author of American Gods, Anansi Boys and other works (see his LibraryThing author page), dropped a mention of LibraryThing on his blog. So that’s why twenty-nine visitors signed up last hour! Gaiman writes:

“Which I find, to be honest, an extremely attractive sort of website, and if I only had a month or so with nothing to do, I’d input my own books.”

I’ll bet people would love to know what’s on Gaiman’s shelves, and those of a lot of authors. LibraryThing has a few authors already, such as Ann Douglas (profile/author page), a baby and parenting author. It needs more. Best selling authors are hard to incentivize. Would Mr. Gaiman take a free account?

Aside: I shouted out the news to my wife in the other room. Neil Diamond’s blog? The singer? Not yet. Not yet.

I know LibraryThing has been slow. I am working to get it working faster.

I have the following plea. If LibraryThing does not respond quickly, please do not click “refresh” trying to get it to speed up. This never works and has been behind the recent downtimes. Things are going smoothly, the database server gets a little behind, and people start hitting refresh until the server is so behind it crashes.

Thank you for your patience as I work to scale LibraryThing to satisfy its current popularity.

1. Performance update. Performance has been spotty recently. Accelerating growth—Sunday saw 16,000 books cataloged!—has strained current resources. Improving speed and reliability is now my number one goal. Occasional slowness and even downtime will happen, but things are heading in the right direction.

I’m getting some part-time assistance. (But I’m still looking for a good LAMP hacker; email or send resumes!) So far I’ve re-jiggered settings and optimized requests. I am looking very carefully at the path up. I will probably be setting up a dedicated database machine and/or a dedicated “thinking” machine. I have a new book-suggestion algorithm that is to book suggestions what Deep Blue was to chess—extremely good and extremely processor-intensive. Bringing it live would crash the present site. It might also take over NORAD computers and threaten nuclear war until forced to learn the meaning of checkers.

I’ve temporarily disabled all-catalog searching while I work on database speed. These searches were taking up a large percentage of database power.

2. Two new features. Before I turned my attention to speed I did complete two new author suggestion features. Author pages now include a “Similarly tagged” box; tag pages sport include a “Related authors” box.

They are hit-or-miss. A decent example would Ilaria Gozzini Giacosa, author of the cookbook A Taste of Ancient Rome. With only a few tags the system managed to match her up the author of a Medieval cookbook and with Apicius, author of De re cocquinaria, the only extant example of a Roman cookbook. Apicius is in the unique position of being “similarly tagged” with both Marcus Tullius Cicero and Betty Crocker—both fusty and overrated? At the other end, take Nuala O’Faolain, author of Are You Somebody: The Accidental Memoir of a Dublin Woman. She is paired with Barack Obama. Both wrote memoirs, I guess. As more tags enter the system, this sort of single-tag effect should lessen. Here is the “similarly tagged” for David McCullough and the related authors for World War II.

Earlier on this blog I called for a Flash programmer. A blog reader found me one. (The new Flash-based widget will be coming out soon.) Now I’m looking for a LAMP guy—Linux, Apache, MySQL, PHP, particularly the MySQL part of that—to help me with some scaling issues. Is that you? Do you know her?

I have some limited and particular problems now (eg., the slow query log). But I could see this evolving into something more steady. I can pay a low-decent hourly, and you’d have the pleasure of working on something cool.

Bonus points for living in Portland, ME or Boston, MA. But I suppose this doesn’t matter these days.

Update: The widgets are back up again. They were responsible for a lot of the speed problems. I had no idea how often they were getting requested! Fortunately, the code had a lot of room for speed optimization.

I’ve improved the tag searching. You can now ask for things like:

• birds, beasts — all books tagged both birds and beasts
• +birds +beasts — another way to say the same thing
• +birds -beasts — all books tagged birds but not tagged beasts
• greek* — all book with tags starting with “greek” (eg., “greek”, “greek history,” etc.)
• * history — all books ending with “history” (eg., “greek history,” “roman history”)
• *dog*, -fiction — all books with the word “dog” somewhere in the tags, excluding fiction
  

Announcing a decent tag search for LibraryThing is a bit like announcing that I’m finally using deoderant under one arm. “What about the other arm?” Well, the regular search function is still quite suboptimal, but I’m working on it. I’ve got to fiddle with some internals to do it.

I’d love to hear people on the topic. It’s clear to me that there should be a link in the catalog view to search that catalog. That functionality is already there. But should the search tab do the same thing, displaying every matching book and user in the catalog view? This seems an admirable way of looking at your books, but pretty poor when looking at others. I’m thinking the search should go instead to a page like the tag or author pages, that “rolls up” all the copies under a single book title. Clicking on the book title on that page would take you to the book-detail pages, but there would also be a link to see the books in the catalog. Hmm…

LibraryThing has added 151,440 XML/RSS feeds, and an equal number of HTML versions of the feeds. Someone asked for “feeds for everything.” This is pretty close.

LibraryThing now offers five types of feeds:

• Recent books added by a member. Follow what a user is reading.
• Recent reviews by a member. Follow what a user is saying about books.
• Recent reviews of books a member owns. Find out what other people are saying about your books.
• Recent books tagged something. Follow new items for any tag in LibraryThing.
• Recent reviews of books tagged something. Follow reviews for any tag in LibraryThing.

As you probably know, RSS feeds look like . To work with them you need some sort of feed reader, such as Bloglines. You can also plug them into your Google homepage and many other “start pages.”

If you don’t use a feed reader—I don’t!—I’ve provided an HTML version for every RSS feed. I’ve given these links the icon .

At present feeds are updated every twenty minutes. If usage is high I may need to cut that back a bit.

How you can help. I know bloggers love RSS feeds, so I’m asking for your help spreading the word. (Some people never signed up because of this issue!) And, as usual, I’d love help debugging and expanding the feature. Is there anything I’m missing?

Also, the HTML version is something of a hack; I’m having a deuce of a time getting PHP5’s XSLT support to work. Charitable PHP hackers are invited to email and find out just how clueless I am about this sort of thing.

Tag-based recommendations.

I have added a second set of book recommendations, this time based on the tags people apply to books. So if a book is tagged mostly “constitution” and “american history” it will match up with other books about the Constitution, favoring historical ones. The algorithm is quite complex, weighing the relative frequency of tags by book and overall, book popularity and more. I think you’ll find it’s quite solid.

Here are some examples:

• Ellis, Founding Brothers: The Revolutionary Generation
• Plath, The Bell Jar
• Al Franken, Lies and the Lying Liars Who Tell Them

The new system joins the existing system “people who own X also own Y.” More than solid this has the potential to provide truly inspired recommendations, crossing genres to get at what really binds people together around a book. For example, Brooks’ The Mythical Man Month triggers other software project-management books. But it also triggers a Dilbert book. People who do software project management have a lot of Dilbert books, and for good reason. Dilbert is, of course, a software engineer and half the jokes are about the stupidies of cubicles and “pointy headed bosses.” Or take the first Harry Potter. Amazon suggest five other books—all Harry Potters! LibraryThing also suggests Philip Pullman, Tolkien and—my favorite—Madeline L’Engle.

Connection uncertain.

Connection apparent.

Unfortunately, the system can backfire. Take Thomas a Kempis’ The Imitation of Christ, the spiritual reflections of a 15th century monk. This currently triggers Clifford the Big Red Dog. In this case duplicate copies are the problem, something that will shortly be fixed. But there’s a general problem when a book is held by only a few people. The results are statistically true, but not necessarily meaningful.

So there are now two systems, providing 25 recommendations in all (to Amazon’s five). They will certainly get better as users add more tags and books, but I would argue LibraryThing’s recommendations are already better than Amazon’s—richer, more inspired and based on what’s good to read NOT what’s selling this month. Community is, it seems, more powerful than commerce, even for commerce.

Finally, I have a third way of calculating recommendations, different from the other two. I don’t think I can talk how it works now, but it’s somewhere in between them in terms of noise vs. inspiration, and it works well when there are only a few books. The trick is that it’s very expensive—it takes forever to calculate! I think it will have to wait until I have a dedicated “thinking” machine. (If LibraryThing thing keeps growing, I’m forsee a stack of Mac Minis thinking all day.) I also plan to allow users to cheer and boo recommendations—Clifford boo!—and perhaps add their own.

And finally finally, I added another major new feature you may spot—RSS and HTML feeds all over the place. But I don’t want to step on my own toes and I have some quirks to work out before I announce it. Intrepid explorers are invited to let me know what you think in advance.

PS: Do you think I can get away with calling tag-based connections between books “contaguinity”?

UPDATE: Between yesterday and today Clifford slid off the Thomas a Kempis recommendations list, like a toaster that isn’t broken when you take it to the repair shop. [Yes, I know, repair shops vanished 30 years ago.] So I added them both to my library to force the connection. I’m NOT going to go looking for another example!

Sorry for the slow-down in the last 36 hours. There was a database issue. Recommendations weren’t being cached right, so it had to remake them every time. That took a lot of processing power. For example, to produce a recommendation for one of the Harry Potter books it had to retrieve and do math on the libraries of nearly half of of all LibraryThing! And when a page doesn’t come up immediately many users hit refresh over and over—the spiral of death.

I fixed the immediate problem, but it will be a little while before the cache is full again. I’m going to fix the larger problem by adding a second, “thinking” server, that will get a new copy of the book data ever night and sit around all day thinking about recommended books, related tags and so forth. LibraryThing’s slowness—when it’s slow—is all about these tasks. Looking at your catalog, adding books and so forth don’t tax it much. I also like the idea of a server that sits around all day thinking about books. It might even develop opinions.

Finally, I’m going to make some more tweaks at 3am EST tonight. This means that people in California will not be able to add books while watching Dick Clark’s New Year’s Rockin’ Eve. For this I apologize.

I’ve made a new feature to make tagging your books easier. You can now edit the tags within your catalog without having to “go into” each book individually.

The tag list now includes a link to “edit.”

Click “edit” and type your changes into the box that pops up. When you submit it both changes the screen and the database without leaving or refreshing the page.

“Ajax-tastic”? Programming and “Web 2.0” junkies will recognize this as “Ajax,” a term that is supposed to mean “Asynchronous JavaScript and XML” but has come to mean “changing the page without refreshing it” or “acting more like a desktop ap.” Expect to see more of this sort of thing.

Bugs? As with most new features, my ears are open for problems. I tested it, but not on every possible browser/OS combination (memo to self: buy ten computers). And people end up doing things I never anticipated. So, for the next day or so, treat it as a beta feature. Don’t change 100 tags without checking whether the edits are “sticking.”

Ideas? In theory, I could make every field editable this way. But I’m concerned about a cluttered user interface. What do you think?

I’m taking the site down for an hour at 3am EST (9:00 GMT). With luck, I’ll have some new features to announce shortly!

If you want to share your holiday loot, consider using tags like Christmas 2005, xmas05, Chanukah 2005, etc. We’ll tag combine and compare notes!

Christmas presents from the Mainstream Media! PCWorld’s February issue (on the web, if not the newsstands) has a feature article Scott Spanbauer “New, Improved Web,” on “Web 2.0” applications. The “Collaboration & Community” section covers companies like Flickr, Del.icio.us, Facebook—and LibraryThing!

“For book lovers, it’s a LibraryThing: This site is similar to Del.icio.us, but for the tweed set. If you love books, and love people who love books, LibraryThing is for you. Start by using the service to catalog your book collection: Tag your books by topic, share your catalog with others, and then endlessly browse the titles that they have on their shelves. The utterly book obsessed can add the LibraryThing widget to a blog to show visitors what they have been reading lately. Listing up to 200 books is free; listing any number of books beyond that costs either $10 per year or a one-time $25 fee.”

Except for the part about the “tweed set”—are readers as marginal as that?—that’s pretty sweet.

In related news, LibraryThing has been picked up by a third Welsh-language blog (see the buzz page). It makes more sense than you might think; the National Library of Wales is one of the 30+ libraries LibraryThing accesses. And the Welsh are big readers—Hay-on-Wye, the world’s greatest bookstore-town is located there. And they probably wear tweed a lot too.

(Hat tip to Steve Cohen’s LibraryStuff for the link.)

I’m looking to make a new, better LibraryThing blog widget. Have you seen those Flickr badges? I’ve got one over on the right, although it may not work*. If it’s not working, here’s a verbal description: The badge presents a grid of small photographs that come and go dynamically and randomly. From time to time a square of four photos fades out and is replaced by a larger photo, which then shrinks back to full one of the small squares. You can click on a photo at any time to jump to Flickr’s page for that photo.

Anyway, I’d like to do something similar for LibraryThing books—a Flash-based dynamic widget. It need not act like the Flickr badge at all, but it should be dynamic and fun—a little moving window into someone’s library. I would handle the server end.

I’m open to different ways of doing this. If there was enough interest I’d do a contest or sponsor an open-source project. But I lean toward a straight contract job at market rates. Rather than post the job somewhere, I thought I’d start here. That way the money can go to someone who loves books as much as I do.

What do people think of my plan? How do you think a dynamic widget should work? Are you interested in the job? Do you know anyone who might be interested?

* It’s trying to show all the public photos tagged “librarything,” but it just shows “no such photos.” For the life of me I can’t figure out why it’s not working. The process is simplicity itself—I CAN’T be doing it wrong. Does anyone know what’s wrong here? Maybe it’s just a server issue and it’ll be working soon.

Blogger Julie Meloni (No Fancy Name) has written a detailed guide to customizing your LibraryThing blog widget. As she explains, LibraryThing provides three preset styles and a “don’t style it” setting. She explains how to use the “don’t style it” setting to customize the widget to your taste, in the process providing a smart but low-impact introduction to Cascading Style Sheets (CSS). Ms. Meloni is a graduate student in English who also writes computer guides—a great combination!

Check it out at http://nofancyname.blogspot.com/2005/12/styling-librarything-blog-widget.html .

Use your own book covers. By popular request, I’ve added a feature to let users add their own book covers. You can upload covers from your hard drive or by providing a URL.

The feature is available from a book’s card, edit or social view. Underneath the present cover there’s a link for “change cover image.” At present, you can only upload covers to your own books, and the covers remain yours and yours alone. I’m working on the “big project”—user-determined editioning. When I get that working I’ll also provide the ability to snag and share covers when appropriate.

Here are some covers I found for books that didn’t have any. If you don’t have scans, Google is a good way to find them. Often Barnes and Noble or the publisher has a cover when Amazon does not. For now, I’ve left the feature open for all to use. If people abuse it with non-cover images, I’ll have to restrict it to paid members.

Profile improvements. You’ll also note that the image-uploading in your profile has been improved. Huge pictures are now shrunk, and a handy “view larger” link. A few photos got lost in the process—my apologies. If this happened to you, please be so kind as to upload the image again.

Review improvements. Reviews are now dated and sorted by date. I’m working on an RSS feed for reviews of books you have.

“Returning to catalog.” I changed the way the system handles it when you jump from your catalog into a book. The point is to “keep your place” in the catalog, no matter how many jumps you make between books. The previous system had various bugs. The new one is conceptually much cleaner, but may have some bugs too. Let me know if you find any.

I reversed an attempt to combine the tags glbt and lgbt (“gay lesbian bisexual transgender” vs. “lesbian gay bisexual transgender”). In fact, although neither has gotten much use (queer and gay have done better) this is a perfect example of seeming synonyms having a very different nuance as tags.

Here’s the top 20 lgbt books. Only 3 (in bold) also make the glbt list.

• Stone Butch Blues by Leslie Feinberg (5)
• Dykes To Watch Out For, The Sequel by Alison Bechdel (3)
• The Snow Garden by Christopher Rice (3)
• Beyond the Closet: The Transformation of Gay and Lesbian Lif… by Steven Seidman (2)
• Invasion of the dykes to watch out for by Alison Bechdel (2)
• Complete Hothead Paisan by Diane Dimassa (2)
• Annie on My Mind by Nancy Garden (2)
• The Original Coming Out Stories by Julia Penelope (2)
• Dykes and Sundry Other Carbon-Based Life-forms To Watch Out … by Alison Bechdel (2)
• The Indelible Alison Bechdel : Confessions, Comix, and Misce… by Alison Bechdel (2)
• More Dykes To Watch Out For by Alison Bechdel (2)
• The Essential Guide to Lesbian and Gay Weddings by Tess Ayers (2)
• Spawn of Dykes To Watch Out For by Alison Bechdel (2)
• Rubyfruit Jungle by Rita Mae Brown (2)
• Split-Level Dykes To Watch Out For by Alison Bechdel (2)
• Unnatural Dykes To Watch Out For by Alison Bechdel (2)
• Courting Justice : Gay Men and Lesbians v. the Supreme Court… by Joyce Murdoch (2)
• A Density of Souls by Christopher Rice (2)
• Fingersmith by Sarah Waters (2)
• Tipping the Velvet by Sarah Waters (2)

This is exactly where tags shine. Could this sort of social nuance be teased out of a Library of Congress subject heading?

I’ve added a “combine tag” feature, allowing users to combine VERY similar tags to be merged on the global level. (No users’ tags are actually changed.) As with author disambiguation, LibraryThing users make the decision. The choice isn’t pushed very hard; most users won’t see it, even if they benefit from it.

You can combine when you see this below the list of related tags:

As blog readers are familiar, I take a hard, idealistic line on tagging. Tags are about memory—your memory. Automated or suggested tags (other than your own) interfere with that process. If you’re gonna use someone else’s mental categories, use an expert’s, like say, the Library of Congress’. I buy Clay Shirky’s essay/talk extolling the “signal in the noise” between tags like cinema and movies.

As the saying goes, “I believe. Help my unbelief.” Reworking the related tags feature got me thinking about “tag synonyms.” Is there any difference between wwii and ww2? What about world war two, world war ii and world war 2? Is some trivial nuance really worth the social loss—World War II buffs thinking they’re alone, worse recommendations, and so forth? After all, the top World War II tag (wwii) is used only 1,300 times, but all the tags together hit 3,100!

So, I came up with a “combine tags” feature. It works like the “combine author” feature, except that the combine page has half a page of “philosophy” on it, begging users not to combine merely similar tags. There is also a tag combination log, allowing finicky LibraryThing-arians to follow the action, and separate tags at need. Like a wiki, it’s easier to correct damage than to do it. The combination log records users who combine tags, but not those who separate them. Go ahead and separate a tag; nobody will know you did it!

I’ve already separated some. In my book Farsi is not the same as Persian. Although Persian is a term for Farsi (perhaps more commonly applied to “old” and “middle” Persian than the modern language), Persian is also a general adjectival form of Persia (which, incidentally, has a totally different flavor than Iran). I also split to be read and unread. To be read implies intent to read. Unread does not.

Well, that was fun. Now back to the book-cover issue…

Algorithmic tangent: There are various ways of thinking of “relatedness” between tags. For the tag pages, I key it to “works” (Platonic books, as opposed to individual books). Tags are related to the extent they are applied to the same works. Using this model, one might think of synonymous tags as tags that often occur together by work but rarely by user or individual book. A little play found this to works okay, but not well enough to be definitive. So I’ve resorted to user control. In essence, I’m using one user-driven process to correct for occasional mistakes of another.

Has any other tagging site ever done this?

Perhaps someone can direct me to where people talk about this stuff; I certainly haven’t found it. LibraryThing’s tag algorithms have all been ex nihilo. This is scary. I mean, if it were up to me, sorting would probably have never gone past the “bubble sort.” Hello? I studied Greek and Latin in college!

Searching someone else’s library. You can once again search someone else’s library or tags from their catalog. You can do this with your own too. The search itself is still sub-optimal (eg., not indexing 3-letter words). That’s next up.

See more now the default. Some users discovered that if you clicked “see more” you’d get more information and buttons within your catalog, including the number of users per book and a delete button. But most never users never played with it, and missed out. I decided to remove the button and turn the feature on all the time. I think it’s better. It’s simpler and seeing at a glance how many user have one of your books is just plain fun. As for the delete, that’ll cut an hour off my weekly email load…

To make an omlette, you have to break some eggs. I got rid of the icon, which indicated when you had reviewed a book. It added clutter without much funtionality. To check what books you’ve reviewed, go to the profile and see your reviews page.

Widget now shows random books for tags. A user asked for it, and I like to please. You can now make a widget that shows a random book, but only within a given tag.

Related tags improved. I changed how related tags are computed and ordered. The old way surfaced too many popular tags, like fiction and read at the expense of less common but more juicy ones. Take the tag vampires. The old system kicked off with the unhelpful fiction and fantasy tags, and includes others like romance and read high up. The new way exposes the pale white neck of tagging: vampire, horror, anita blake and buffy. Check out the difference between the old and new science tag.

I love the way the blogosphere goes in strange directions. So, the LibraryThing book pile contest ended. The top-winner (there were six in all), lucytartan, won two lifetime memberships. As she just posted, she’s decided she’s going to hold her own contest to give them away, one at each one of the group-blogs she writes for. Quoting her:

At The Valve the contest is “tell us your best library anecdote.”
At Larvatus Prodeo, the contest is “tell us about your favourite book.”

So there’s a plug for her blogs. I swear, I didn’t know any of this until she told me.

Lucytartan got hers for artful photography, but I’m open to giving free memberships if other popular blogs want to run contests. Send me an email with your idea. I could use the exposure and the site gets more valuable with every book added. (No it’s not going to become completely free; I got needs, needs!)