Chris and I managed to bury the lead, too quickly. Check it out, there is a Book Pile Contest going on.

This weekend we made a number of important structural changes to how LibraryThing’s 34.8 million tags are stored in the database. (For database heads, tags are now “fully normalized.”)

The immediate upside is that tags can now be up to 255 characters long. It will also allow us to improve some features, such tag editing.

The downside is that the changes have hurt performance. Certainly the site is running very “hot,” forcing us to choose between running slow and pulling back service. Right now we’re doing the latter, redirecting most non-member traffic to the home page to sign in or sign up. This cuts back the large proportion of our traffic that is bot-related, but we can’t run this way for too long.

It’s unclear if the change is itself to blame or the loss of various tag-related caches, which need to be built up afresh with the new structures. There also appear to be some places that are hurting more than others, which code can perhaps be re-written. We’re going to be looking carefully at what we can do, and deciding whether we need to make additional changes, or pull back the ones we made.

Thanks for your patience. The next few days may see occasional slowness or downtimes. With the four of us on it, however, we hope to minimize problems and solve this to everyone’s satisfaction.

Lindsey Nichols (LibraryThing: lindseynichols) just joined the LibraryThing team as our “assistant librarian”—bringing the total librarians on staff count up to two! She’s going to be helping us out part time from the “Boston office”,* helping with emails, answering questions, dealing with account issues, mailing out CueCats, and whatever else we decide to throw at her.

Lindsey in a nutshell: She got her library degree from Simmons, and also has an MFA in creative writing. Clearly, she should be contributing to Help in Haiku endeavor on the new WikiThing. Lindsey’s favorite authors include Shelley Jackson, George Saunders, Peter Matthiessen, and Carson McCullers. She enjoys baking, reading commodity histories, and viewing educational museum taxidermy. Also long walks on the beach.**

You can email Lindsey at lindseylibrarything.com

I also wanted to make sure that we thanked Clifford Vickrey (LibraryThing: dinner_bell), our excellent and tireless intern over the summer. Cliff was amazingly likeable*** guy, who refused to sit anywhere except on the floor, and was an excellent chauffeur in the pouring rain. Thanks to him. We’ve also decided to give his entire school (Colby) free LibraryThing accounts, so watch for news on that.****

---

*aka, Abby’s apartment
**ok, fine, I added that last part.
***so sayeth Tim
****We’re thinking of giving free accounts to students at a handful of schools (starting with our own alma maters, of course).

We’re going to make some database changes tonight, starting at 2am Eastern US and continuing for 1-3 hours.

I’ve added two features to the Cover view in Your Library. You can resize your covers with handy plus and minus buttons. And you can sort your books within Cover view (before this you had to do it in List view).

The back story is that I’m rewriting* Your Library rather extensively, both because it needed to be done and to prepare the way for collections (which are finally going to happen!). Your library needs a good scrub before collections arrive. The changes are minor on the surface, but drastic underneath. If you see problems, let us know.**

*What programmers call “refactoring” for no good reason.
**Since there’s been some speculation, we wil NOT be releasing collections today—although it’s our second birthday—or, probably, this week.

We just hired Casey Durfee (LibraryThing: caseydurfee), a crackerjack “Library 2.0″ hacker. Casey’s first project—faster, better search—debuts today (see below). It’s a big win. We have a long queue of similar projects. Casey will also be heading up LibraryThing for Libraries, our project to get our data into library catalogs.

Casey will be working from Seattle, where he was recently left a job at the Seattle Public Library, managing their library system. Before that Casey worked at SirsiDynix*, so he has a lot of background in the arcane world of library systems—just what we needed for LibraryThing for Libraries. Casey is also responsible for L2, a handy Greasemonkey plug-in that adds Amazon content to library catalogs.

Perhaps the coolest thing Casey has done is a project called Helios, a “faceted” search on the Library of Congress records, done very simply and entirely with open-source software. It was a personal project, and although it’s not a complete solution, it searches the LC better than the LC’s million-dollar catalog. You can see Casey in action, talking about Helios, in this Code4Lib talk on Google Video.

Search. Check out the new work and author searches. They’re based on Solr, a simple but powerful search engine, and the same one Casey used on the LC data. Until now, we were relying on MySQLs fulltext capabilities. We had outgrown it, and slow performance was causing frequent database glitches.

It’s fast, accurate and searches all titles, not just the “leading” (mostly English) ones. But, as with everything we do, it’s not “perfect.” Casey has set up a Talk post about it. He has a variety of knobs he can turn, and is looking for feedback. I’m convinced it’s overzealous on “stemming”—picking up “loves” and “lovely” for “love.” That it even does stemming is quite an improvement from our previous solution. Once we’ve got it working the way we like, we’ll also be adding it to touchstones and elsewhere on the site.

Other projects. Casey is a certified library programmer. (I just play one on TV.) He knows his MARC21 from his UNIMARC, and his “glyphs from his diacritics”.** As time goes by we hope he can work on things like:

• Rewriting our library-data import, to get all the diacriticals right and squeeze more out of the MARC records.
• Adding more libraries. We’ve been avoiding UNIMARC libraries (eg., Italy libraries) and, until recently, most SRU/SRW-based ones. We can do better. We have also finagled access to British Library data, so look for us to add that too.
  

LibraryThing for Libraries. Right now, we have more than 350 libraries asking us about LibraryThing for Libraries. Altay and I have been going through at a snail’s pace. Casey should be able to crank that up a bit. We also think his SirsiDynix experience will come in handy. He’s already written a handy LTFL export script for HIP. He’s well-known in the HIP ILS community, and should move us past our current success among Innovative Interfaces catalogs.

That’s it. Welcome on board, Casey!

*When it was Dynix.
**Whatever that means; suggested by Casey.

For about three hours this afternoon some of our blog widgets went crazy, inserting very large covers where small ones were supposed to be. I was fixing a long-standing problem, in which some books never showed up in the widget.* In the course of fixing that problem I created another. Some users were amused, some dumbfounded and some pissed off. All good reations, I think.

Anyway, a few users emailed us, and Christopher stepped in to fix the problem. The widgets are okay now.

To the bloggers affected, my sincere apologies. I should have checked the code more before it went live. The change I made didn’t affect widgets of the type I have on the blog or one of the users with the problem I was fixing, so I missed it. It was a dumb mistake, and I wish I had caught it in ten minutes, not three hours.

By way of making it up, if you had one of these blogs and felt inconvenienced, please write me a note with your blog URL and your address, and I’ll ship you off a CueCat barode scanner. Please take me up on this. It’s the least we an do.

---

*Basically, LibraryThing had imperfect knowledge of what covers Amazon had available, so we weren’t showing everything we could.

We just inked a deal with AquaBrowser, an Amsterdam-based library catalog company. See the Thingology Post.

I’ve added four book sources for Dutch readers: the Dutch Royal Library (KB), the Royal Library of Belgium (KBR) and booksellers Bol.com and Bruna.nl. With luck, they’ll be a few more soon.*

This should be big boost for our Dutch site, LibraryThing.nl. Until now, there was only one Dutch-language source, TU Delft**. Between the four new sources LibraryThing now embraces virtually all in-print Dutch books, 70,000 cover images and millions of older books.

I hope this will make LibraryThing a much more attrative place for Dutch readers to catalog their books and do the other things LibraryThing is about.

Adding Dutch books required extensive work. Giovanni, LibraryThing’s newest (fractional) employee, helped us track down Bol and Bruna data, our first non-Amazon retailer. On my side it involved dealing with five new formats. Fortunately, much of the work will contribute to adding other sources.

There were some wrinkles:

• Bol and Bruna are a single search on combined data. Neither feed was intended for personal cataloging, and I found holes in both sets that the other could improve. Even so, the data is thin by LibraryThing standards, lacking publication years and other important fields.*** The covers are great.
• The Bol/Bruna mash-up was an innovation. LibraryThing trusts that both retailers want to let their customers catalog the books they bought at their stores. But if either tells us to stop, we’ll politely give up our affiliate accounts and withdraw their data. In appreciation for their service to Dutch-language readers, we have put Bol/Bruna links on all LibraryThing.nl work pages, not considering which contributed the data.
• KB was a tough addition, requiring us to parse two new formats, SRU and Dublin Core. Unfortunately, I have yet to figure out how to execute an ISBN search. The KB data is high quality, but slow to retrieve.
• KBR required the least attractive sort of data parsing, so-called “screen scraping.” Only titles can be searched, and the result list is very basic. But, when a book is selected, the data is good, and retrieval is fast.

We have also been reaching out to other prominent Dutch-language book sites looking to forge relationships and pick up or link to interesting content. Your suggestions would be most welcome.

Lastly, as those who follow it know, LibraryThing.nl has seen some controversy. A lot of excellent work has gone into the translation, which is farther along than any other. But there have also been uncollegial disagreements. In response I have urged members to respect the collective nature of the endeavor and instituted a Three-Times Rule, similar to Wikipedia’s Three Revert Rule. Except in very special circumstances, members may not translate a given phrase the same way more than three times.

Thanks to all. Comments, questions and criticisms wanted—as always.

---

*We are waiting for approval to add Proxis.be and implementation and/or approval from a number of other libraries in the Netherlands and Belgium.
**For much of LibraryThing’s existence we were also tying into the Catholic University of Leuven, but their Z39.50 server eventually went black.
***We had hoped to further supplement them with KR or KBR data, but we lack a working ISBN search for both.

Everyone say hello to LibraryThing’s newest employee, Giovanni Soltoggio—LibraryThing member Gio. Giovanni will be helping us expand outside the United States, tracking down data sources, talking about LibraryThing to Europeans, making important deals over small cups of coffee and so forth.

Giovanni hails from the Italian Alps—a valley called Valtellina—and now lives in Dusseldorf. He reads Italian, German and English. He speaks some Czech. His favorite authors are Haruki Murakami, Niccolò Ammaniti and Lorenzo Licalzi. He is sharp as a tack, but much nicer.

Strictly speaking, Giovanni is only 20% ours. Four days out of five he’s the European Managing Director for BookFinder.com and its European sites JustBooks.co.uk, JustBooks.fr and JustBooks.de (for which he writes the blog). He helped BookFinder make the jump to Europe. We’re very grateful to Bookfinder’s Charlie and Anirvan for letting him moonlight a little.

In a few hours I’m going to blog one of Giovanni’s first projects, finding us a decent store of Dutch bibliographic data and covers. He is working on an Italian one right now. We expect big things from him.

If you want to wish him well, or know of good stores in other languages, he’d love to hear from you.* Drop a message on his profile, or email him at giovannilibrarything.com.

---

*Finding good data is strangely hard. There are some commercial providers although they largely overlap with what Amazon provides for free. Smaller languages are hard. A few weeks ago we had an Armenian-reader up for a week of Facebook work. I spent much of the last day with him, trying to get hold of a source—any source—of Armenian book data. We would have been happy to link to one bookseller on every page. From a search engine perspective alone, our links are gold to small sites. No luck, alas.