Archive for November, 2006

Wednesday, November 29th, 2006

A small plug for Wordie

Fellow Portland Mainer John McGrath, of Squirl, hacked together Wordie, social cataloging and social networking for words. Basically, you “catalog” words and arrange them in lists (eg., my products named after their place of origin). If two users share a word, that connects them. It’s a deeply silly idea, but I love that he did it.

PS: Wordie is now included among LibraryThing’s “also on” list, available from your profile.

Labels: 1

Wednesday, November 29th, 2006

The OPAC sucks, LibraryThing inspires tattoos

Lyrics here. (From the Laughing Librarian, who also created Zen Librarian Koans; hat-tip Jessamyn.)

By contrast, an enthusiastic Thingamabrarian sent me this photo of her new tattoo:

ImageChef.com - Create custom images

Okay, I faked that. (Hat-tip Steve Cohen.)

UPDATE: Comment on this post pointed out the homepage pic of the University of Wales, Newport. Gah!

Labels: Uncategorized

Monday, November 27th, 2006

Small features and bug-fixes

I hope everyone enjoyed their Thanksgiving as much as we did. With all the tryptophan in our systems, we didn’t make any major progress over the holiday. But we chipped away at some lesser features and bugs, in between courses:

  1. Groups now sport RSS feeds for recent messages. We plan to add feeds for your groups, your posts, etc.
  2. Tag pages now offer feeds for the most recent books tagged X in your library. They are also available when you look at a tag in your catalog too.
  3. LibraryThing widgets are now available in the Latin-1 character set (UTF-8 remains the default). If you have a blog in Latin-1, and a lot of non-English books, widgets now work.
  4. LibraryThing’s universal import feature now accepts raw, encrypted CueCat data, so you can scan your books away from an internet connection.
  5. The “all books” links on tag pages (eg., biography) is much faster now. (It’s not always fast, but it won’t take five minutes.)
  6. The same goes for “recently tagged X” RSS feed; it’s faster on high-frequency tags. I don’t think many of you were watching the most-recent “fiction” tags, but Google and Technorati were, and all the “database churning” was slowing the site down.
  7. Chris may have solved a major forum bug—the “Bermuda triangle” bug, where one message in 50 or 100 gets inexplicably lost. Cross your fingers and hope he’s right.

Labels: 1

Monday, November 20th, 2006

How to get your bookstore on LibraryThing

Today we launched simple integration with Shaman Drum Bookshop of Ann Arbor, MI. Basically, users can put themselves down as customers and get availability and pricing information on work pages (see right). There’s are no strings or costs to the program; we’re just trying to give give people a better service.

To integrate you need to have an inventory system that can write a file to the web. To make it work, we need a simple XML feed. For performance reasons we can’t be querying an API book-by-book.

The format of the feed is very simple. Here’s an example:

<isbnlist>
<isbn count="1" price="29.95">3598710364</isbn>
<isbn count="2" price="19.95">351911304x</isbn>
<isbn count="1" price="69.50">3519112892</isbn>
<isbn count="4" price="69.50">3519112906</isbn>
<isbn count="1" price="59.50">3519112884</isbn>
...
</isbnlist>

We are open to modificatons (eg., if you can post availability, but not prices). In addition to the feed, we’ll need to have URLs to link to the book pages, and a URL for searching. We willl grab the file between midnight and 2am every night.

LibraryThing isn’t going to double your sales—you’ve probably already have the loyalty of the Thingamabrarians—but it’s a nice service to give your customers.

In the near future, I’ll be producing some stats for bookstores, like holdings patterns against work popularity, that might be interesting or useful to them.

Labels: Uncategorized

Monday, November 20th, 2006

LibraryThing integrates with Shaman Drum

LibraryThing now integrates with Shaman Drum, the legendary independent bookshop in Ann Arbor, MI. Edit your profile, check a checkbox (down at the bottom) and your work page will sport availability and pricing information from Shaman Drum and a link to their site.

Right now, it’s just Shaman Drum. But the program is open to any bookstore. So long as you have a decent inventory system, it should be a snap. We’ve published participation details on our other blog. I’m going to approach a few, but feel free to let your local bookstore know about it.

If you’ve spent time in Ann Arbor, you know that Shaman Drum is the best bookstore in town, and one of the best independents in the country. It doesn’t exactly lack for competition, with the flagship Borders store across the street and the Dawn Treader one street farther. I went to grad school in Ann Arbor, and Shaman Drum was practically a second home. I’m so glad the fine folks who run the place were receptive to my idea.

Labels: 1

Saturday, November 18th, 2006

Germans: Swap English books

An eighth swap site has set themselves up to integrate with LibraryThing—this time with a twist: Bookswapper.de is for Germans—you need to have a German address—to swap books in English.

I think it’s a great idea. For someone like me, living near a good library and two or three good bookstores, swapping doesn’t necessarily make that much sense. When I weigh the mailing and bother costs against the costs of buying or borrowing a book, I usually choose the latter. But it would make a lot of sense for, say, French books, which can’t be found near me.* By catering to residents of Germany (Germans and a lot of expats) who read English, Bookswapper makes easy the satisfying of wants that otherwise would take a lot of effort to satisfy.

About the site, the creators, Kata and Resi write:

“The makers of bookswapper.de are two real people, not some corporation. And bookswapper.de wasn’t born in an office building but in our home, on a sofa, beneath overflowing bookshelves.”

Amen to that.

Note: The original rules didn’t anticipate a site like this, so I’ve decided to list Bookswapper.de second in LibraryThing.de and last in all other LibraryThings.

PS: I fixed a bug that was preventing LibraryThing from seeing everything our swap-site partners had. Numbers have jumped!

Labels: 1

Saturday, November 18th, 2006

8,388,608 books—sort of

Last night LibraryThing hit 8,388,608 books. That’s not books in LibraryThing—which stands at 7,268,540—but books ever in LibraryThing, including ones later deleted and some shadows. You might not think it, but 8,388,608 is a significant number. It’s half of 224, the largest number you can store in three bytes. It’s also the limit for MySQL’s “signed medium integer.” It’s 111111111111111111111111. The drawers are full of ones and there ain’t no twos.

Anyway, we hit the brick wall last night. I had previously expanded the book number field, but I forgot to change the databases that store some related metadata and reviews. So, last night, you couldn’t add a book, and this morning you couldn’t review one.

I’m really sorry about this. We’re good to go now. We won’t hit another wall until 8.4 billion books.

Interestingly, the same thing happend to Slashdot last week. Even Homer nods.

PS: I also fixed a bad problem with “search all fields.” Some queries ran quickly but some took ten or twenty minutes, by which time the user has generally gone on to better things (after re-running the query a dozen times which, let me tell you, doesn’t help much). It turns out MySQL was making periodic mis-guesses about which index to use. Somehow the index with eight million integers looked better than the one with a few hundred strings.

Labels: 1

Friday, November 17th, 2006

Arguing against tags

I just read the short “Beneath the Metadata: Some Philosophical Problems with Folksonomy” by Elaine Peterson (D-Lib, Nov. 2006), which demonstrates that “A traditional classification scheme will consistently provide better results to information seekers [than a folksonomy].”

I hardly know where to begin, but take this idea:

“[I]f users can continuously add tags to articles, at some point it is likely that the whole system will become unusable. A folksonomic system threatens to undermine its own usefulness.”

The reasoning is that, as more tags are added, the number of wrong tags will incease. More bad tags mean less usefulness, eventually sliding all the way to complete uselessness—our old friend, the map of China that is the size of China, gets a mention. But tags are deployed statistically where possible, not by the one-for-one correspondence of a card-catalog subject heading. All arguments in favor of tags and all significant efforts to find and order information with tags (eg., Del.icio.us, LibraryThing, Flickr, CiteULike) are predicated on the heavy use of algorithms and statistics. This is a key part of the argument for tags, but Peterson’s article doesn’t mention it.

Imagine an argument against subject headings with a similar deficiency of key information—”LCSHs won’t work because most of us live too far away to visit the Library of Congress regularly.” I don’t think this misunderstanding is any less basic. Once you factor in statistics you’ll understand that as tag density increases, it becomes easier to spot and discount noise, not harder. If the census visited just one house in Maine, it might decide state residents were all Aleutian Islanders. As they visit more, the chance of coming to that conclusion swiftly vanishes.

I need to decide how to approach this stuff. I do not have, and never will have an MLS. This is a real disadvantage. There are also political minefields to be negotiated. When you’re in a discipline you know whom you can safely argue with, and whom you can’t.

I was contacted by an academic publisher today, interested to find out if I had a book in me. A proposal to discuss user-contributed metadata, particularly tags, in the library catalog did not prove interesting. I had meant to bow out anyway—I have no time!—but being refused lit a fire under me. Someone needs to write a good book on the topic. If not me, who? All I need are a dozen more plane trips without wifi. Fortunately or unfortunately, it looks like I’ll get that.

PS: I’m going to see Abby (and John Blyberg) talk tomorrow at a NELINET event, “OPAC 2.0: Reinventing the Library Catalog.” I’m thinking I’ll tape her talk on my MacBook. I wish the iSight camera faced outward. As it is, I’ll have to film myself reacting—pensive! amused! shocked! itchy!

Update: The article is similarly (if more politely) panned on Dystmesis. The blogger also wrote a paper on LibraryThing’s tagging, which I’ll blog soon.

Labels: Uncategorized

Thursday, November 16th, 2006

Bonjour SUDOC

LibraryThing now connects to SUDOC, the Système universitaire de documentation, a French union catalog of university libraries (see Wikipedia). SUDOC sports some seven million records—a huge boost to LibraryThing.fr and Thingamabrarians with French books generally.

Three cheers to Nicomo for helping me on this. He found it, fiddled with YAZ and sent me the exact connection info. SUDOC is actually the first time I’ve managed to parse Unimarc. Solving that (mostly) opens the door to many other libraries. Nicomo, you’re a star!

Speaking of French, my family has a non-English-speaking Frenchman over to Thanksgiving. My French is extremely rusty, but I’m thinking I can jog my memory by listening to some simple conversations, news reports and especially vocabulary lists while I work. Any suggestions?

Labels: 1

Wednesday, November 15th, 2006

UnSuggester fallout

Our new anti-recommendation engine UnSuggester (blogged about below), has taken off in a weird way. Today was covered in everything from top-shelf library bloggers Tom Roper, Steven Abram and Karen Schneider (via ALA TechSource), to the bouncy, zippy, web-culture vlog MobuzzTV.

Here Mobuzzer Karina Stenquist explains the disconnect between Middlesex and The Passion of Jesus Christ: Fifty reasons he came to die. Karina did something on us before and—I’m sorry—she’s great.

I love everything people are saying about it, with big discussions here and on various blogs, mosty people strive to find the oddest, funniest unsuggestions. My favorite blog discussion came off a short note by “Fontana Labs” (a philosophy professor?) on Unfogged, and is now past 120 comments. Much of it came from the proposal:

“There’s got to be some ultimately evil book out that that will generate the ideal library. Not The Bridges of Madison County, but like that.”

Speculation ensues, with the unsuggestions for Who moved my Cheese? much praised. I enjoyed this exchange:

“Sadly for Unfogged (but perhaps good for Western civilization) Who Moved My Cheese? For Kids, as perfectly soul-deadening a title as I’ve ever encountered, isn’t owned by enough LibraryThing users to produce results. …”

Who Moved My Cheese? For Kids. Oh my God, my soul just broke into three thousand tiny wretched pieces.”

Hey, I didn’t say it. De gustibus…

Labels: 1

Monday, November 13th, 2006

Shaping the future of Maine’s economy?

Out of the whole LibraryThing thing, I am most proud of the buzz page, which collects some 750 positive quotes about LibraryThing, almost all from blogs. Second to that are the invitations Abby and I have had to speak at library conferences. (Invite us to more! We sleep on sofas and eat like birds.)

But this has to run a close third: being select by the Maine paper Mainebiz for its “Next List.” Apparently I’m one of the “ten people shaping the future of Maine’s economy.” (Here’s the story in Google’s cache.) Mainebiz threw an awards party at the Portland Museum of Art, with wine and canapees and people in suits.* None of the party pictures turned out, so here is my son reacting to my lucite trophy-award-paperweight.**

Great as the honor is, I can’t avoid thinking “If I’m moving it forward, Maine’s in big trouble!” LibraryThing has only two employees in the state (a third makes her home among our former collonial masters).

Fortunately, LibraryThing and I are proxies for something far larger and very real: you can launch a web startup in Maine. We attended with John McGrath and Kristy Dahl of Squirl, another good example. That’s not quite a trend, but could it become one? I think so.

Maine is supposed to be a VERY bad place for tech start-ups. Loose talk about the “anihilation of distance” notwithstanding, startups cluster very strongly, with places like Silicon Valley and Cambridge, MA far in the lead. As Paul Graham put it, startups happen where you find “rich people and nerds.” Having both in abundance is, as Graham writes, quite rare. New York City has rich people, but no nerds. Pittsburgh has nerds, but no rich people. Compared to either Portland is a desert. Forget rich and nerdly—Portland hardly has PEOPLE. We’re the largest city in the state, and have 63,000 residents (compared to more than 100,000 for Cambridge, MA alone).

So why and how is Portland a good place for tech startups? LibraryThing worked because we began as a one-nerd operation. (Later, after much effort, LibraryThing hired Chris Gann, a Silicon Valley “blow in.”) And it worked because we didn’t NEED a rich person. It cost almost nothing to set up, and made money from the start. Back in the dot-com years the servers alone would have required angel funding. Now they don’t. (There’s a good NYT story on this phenomenon.)

Once you dispense with hiring and funding, it’s all about where you want to live. Fortunately, my wife is an author and can live anywhere. Portland is cheap—rent is about half what we were paying in Brookline, MA—and was close enough to Boston for me to continue freelancing for companies there. Later, when LibraryThing started, cheap rent helped balance the checkbook.

We chose Portland over other options because it’s just such a great city. We live two minutes from the Eastern Prom, a gorgeous tableau overlooking Casco Bay, and a great place to walk when you need to get away from a computer to think about data structures. We’re five minutes from the center of Portland, which is small but “real,” with funky businesses, an independent theater and an excellent internet cafe, known as “Tim’s second office.” And actually Portland DOES have rich people. The New Yorkers and Bostonians stop by on the way to the summer house or skiing, and some very nice restaurants have sprung up to serve them.***

All-in-all, not a bad place for a start-up.

I didn’t meet all the other winners****, but congratulations to architect and Renaissance-man Mitchell Rasor (MRLD.com) and Jeffrey Wood, founder and web developer for the worthy non-profit eHope.

*Who knew Mainers had suits? You hardly see them, even downtown. Where are these people hiding?
**Right after this photo was taken, Liam pushed it across the flooboards and I discovered that lucite scratches easily. Oh, well. Sic transit gloria plastici.
***If there were a decent Chinese restaurant, it would be paradise.
****And I completely dropped the ball on meeting Tom Allen, our congressman. Librarians may now scold me for it. I want to button-hole him about his vote for DOPA, the bill forbidding social networking sites in libraries which would, inter alia, illegalize many of LibraryThing current uses and future plans.

Labels: Uncategorized

Sunday, November 12th, 2006

BookSuggester and UnSuggester

People do not generally like BOTH Shopaholic and Critique of Pure Reason.

The “real” news today is the debut of BookSuggester, a new feature designed to expose LibraryThing’s excellent and varied recommendations to members and non-members alike. We put them alongside Amazon’s, which are also quite good. We are proud of our recommendations, but haven’t perfected the perfect algorithm yet. When we’ve made things as good as we can, we’re going to start offering recommended book data to libraries.**

But to heck with that! Let’s talk about bad recommendations. Today we introduce UnSuggester, “the worst recommendation system ever devised™.”

UnSuggester is a brand new idea in recommender technology. Recommender systems usually work by similarities. Amazon’s “Customers who bought this item also bought” and LibraryThing’s “People with this book also have” are typical of the type—What books do people buy together? What books occur often in the same member libraries?

UnSuggester flips this logic: What books DON’T occur in the same libraries? We took our “similars” algorithm and changed “sort ascending” to “sort descending” and—hey presto!—instead of similar books, we get opposite ones. You bet we’re going to patent it!

How does it work?

UnSuggester starts by finding every copy of the book in question and all of its owners. So, taking Thucydides’ History of the Peloponnesian War as an example, LibraryThing finds the 600-odd people who have entered this ancient classic in their account. Then it makes a big pile of all their other books, a pile of some 623,000 books in all. Then it does a little math. If LibraryThing has seven million books, then a pool of 623,000 book is about 8% of the total. If this pool were average, it would also contain 8% of the Harry Potters, 8% of the Derridas and 8% of the Danielle Steels. But this isn’t so. People who own Thucydides aren’t a random cross-section of the book-loving public. For example that 8% also contains almost half the Caesar and Plutarch in LibraryThing. At the other end of the scale, Thucydides-fanciers are particularly immune to the novels of Marian Keyes and Dean Koontz. The greatest disconnect occurs with Louise Rennison’s popular, teeny-bopper chick-lit novel Angus, thongs and full-frontal snogging : confessions of Georgia Nicolson—the top UnSuggestion.

What patterns emerge?

The Mists of Avalon and Desiring God are very uncommon shelf-mates.

Play with it a few minutes, and patterns emerge. Philosophy and postmodern literary criticism oppose chick lit, popular thrillers and the young adult section. Programming does not truck with classic literature. Memoirs of depression, like Prozac Nation, meet their match in the cheery The Night Before Christmas. Ann Coulter and David Sedaris do not see eye-to-eye. There is a strong disconnect between readers of much recent Protestant, mostly evangelical, non-fiction, and large swaths of contemporary literary fiction. For example, LibraryThing includes 2,300 readers who’ve logged Jeffrey Eugenides’ epic gender-bender novel Middlesex, and 222 readers of John Piper’s The Passion of the Christ: 50 Reasons He Came to Die. But the groups don’t overlap. No reader has both. Similar instances occur again and again.

These disconnects sadden me. Of course readers have tastes, and nearly everyone has books they’d never read. But, as serious readers, books make our world. A shared book is a sort of shared space between two people. As far as I’m concerned, the more of these the better.

So, in the spirit of unity and understanding, why not enter your favorite book, then read its opposite?

By the way, how about putting this up on Del.icio.us or Digg? Wait, we have a Digg now.*

*LibraryThing never been Dugg. But recently a pale immitation of LibraryThing was lofted to the heavens as the first social network for books. For one day Digg gave them twice our traffic. Fortunately, they fell like a stone after that and, this morning, Alexa has them at “too low to measure.”
**My friend Ben correctly points out to me that he suggested “find your book nemesis” almost a year ago.

Labels: 1