Tuesday, December 11th, 2007

Open data and the Future of Bibliographic Control

We’ve got until December 15th to submit comments on the draft report produced by the Working Group on the Future of Bibliographic Control.

No—keep reading! This is important. People in the library profession need to be involved in this stuff. Further, people outside the profession need to be involved too. As the report notices, library data is used by many outside the library world, starting with library patrons, and extending even to Amazon.com. It shouldn’t go unnoticed, for example, that draft report mentions LibraryThing four times. For while LibraryThing uses library data, it was invented by and is mostly used by non-librarians.

Aaron Swartz, the dynamo behind Open Library, sent me a note about one important aspect of the draft report, namely what it’s missing: It doesn’t mention open data. There is serious discussion about sharing, but also the alarming proposal that the LC attempt to recoup more money from the sale of it’s data. That’s a shame. I’m not alone in believing that open access to library data is the future. A report about the future should confront the future.

The economy of library records is a complex one but not primarily a free one. By and large libraries pay the Dublin, Ohio-based OCLC for their records, even if the records were created at government expense. That model looks increasingly dated. And it is killing innovation.

It hasn’t killed LibraryThing yet, but the specter has always hung over our head. It’s why LibraryThing has—so far—not pitched itself to small libraries. OCLC doesn’t care about personal cataloging, and the libraries we use are—in every conversation I’ve had—enthusiastic about what we do. They want their data out there; they’re libraries for Pete’s sake! But if we offered data to public libraries we’d be cutting into the OCLC profit model. That could be dangerous.

Aaron invited me to sign onto a list of people interested in the issue. I did so. I invite you—any of you—to do so as well. The text says it perfectly:

“Bibliographic records are part of our shared cultural heritage and should be made available to the public for re-use without restriction. This will allow libraries to share records more efficiently, but will also make possible more advanced online sites for book-lovers, easier analysis by social scientists, interesting visualizations and summary statistics by journalists and others, as well as many other possibilities we cannot predict in advance.”

“Government agencies and public institutions are increasingly making data open. We strongly encourage the Library of Congress to join this movement by recommending that more bibliographic data is made available for access, re-use and re-distribution without restriction.”

The petition is here: http://www.okfn.org/wiki/OpenBibliographicData .

Labels: library of congress, open data, open library, Working Group on the Future of Bibliographic Control

Monday, December 10th, 2007

SantaThing: Social networking goes Santa

Shameless cross-post from the main blog, but I want all my Lib 2.0 chums to come and join the new Secret Santa system, SantaThing, I cooked up last night. Secret Santa for booklovers. Can you resist?

Labels: new feature, santathing, secret santas

Friday, December 7th, 2007

Someone poke Syria

News reports* indicate that Syria is now blocking Facebook, alleging Israeli spies were infiltrating Syrian social networks. One can doubt that given some of the other sites on Syria’s blocked list—YouTube, Blogspot, Hotmail, Skype and—wait for it—Amazon.

But not LibraryThing! Syria bans sites, China bans sites. Heck the UAE just banned Twitter!** But we never get banned, by those guys or anyone else; our competitors don’t get banned either. I’m almost sorry about it. YouTube might someday bring down a government, but people talking about books does it all the time.


*See: SeattlePI, Washington Post/Reuters, Mashable, Fox/AP, Jerusalem Post.
**As a certified member of the Web/Lib 2.0 set I’m supposed to think Twitter is a serious thing. I don’t. I don’t fall for the argument that only books matter, or that blogs are giving us the attention spans and intellectual perspicacity of squirrels. And I don’t think social networking is vain shadow of real-life connection. But there is some lower limit to the length of an idea and the depth of a connection—and Twitter is it!

Labels: censorship, syria

Wednesday, December 5th, 2007

Here comes another bubble!

It’s funny because it’s true.

Ultimately I think it’s all about one thing. The hype isn’t misplaced, but a lot more value is created than captured. Facebook touches more people than Ford does, and, I suspect brings them more joy. But Facebook can’t get at that value the way Ford does.

Google got at a tiny slice of the value they create with AdWords, and it’s making them insanely wealthy.

LibraryThing also makes lot of people insanely happy. While we capture some value—it helps that our engineers are few and pay for their own beer—we’re never going to capture most of the value we create.

Good, I think.

Labels: Uncategorized

Monday, December 3rd, 2007

MARCThing: A simple, self-contained MARC and Z39.50 application

Over the past couple of weeks, LibraryThing has been rolling out major improvements to our cataloging system—a new system for retrieving and parsing book information we’re calling “MARCThing.”

MARCThing is a major advance for LibraryThing. We’ve sunk months of development time into it, but we’re not going to keep it to ourselves. We will be releasing all the code for non-commercial use in libraries and elsewhere.

When the dust settles, LibraryThing members will be able to draw on nearly 700 data sources worldwide, with greatly improved foreign character support and better data manipulation behind the scenes. With MARCThing underneath we will be able to introduce many new features and to reach a truly global audience. But we are confident that developers outside of LibraryThing will find many other, equally compelling uses for MARCThing, and make useful changes and extensions.

What it is. When I was given the task of improving LibraryThing’s cataloging system and other involving library data, I immediately thought of Solr, one of the most influential pieces of software to come out in the past couple of years. The big idea behind Solr is that it provides a “magic box”—an easy, self-contained interface to some very powerful but complex technology, the Lucene search engine. Solr hides the messy details of Lucene from the developer and provides all sorts of extra goodies in a self-contained package. The net result is you can instantly stick an extremely powerful search engine into your project with almost no work. This combination of power and ease-of-use has quickly made it a developer favorite, and spawned all sorts of interesting projects that never would’ve come out without Solr.

I wanted my own magic box that would handle the two main protocols used by libraries to transfer cataloging data, MARC and Z39.50, without anyone having to go into the details of how they work. And since I didn’t want to have to find or build another magic box, ever, I wanted something that could be easily used from any programming language.

Writing it was pretty easy—I used Django for the web part, Pymarc for MARC, and PyZ3950 for the Z39.50 support. With a good software library, working with Z39.50 or MARC records isn’t hard. The hard (or at least time-consuming) part of MARCThing was tracking down servers and dealing with oddball cases. There are many lists of Z39.50 servers out there, but the data is often incomplete, incorrect, or out of date. When you do find a Z39.50 server, oftentimes it’s non-standard in some way, or only has limited functionality. So the process of connecting to libraries using Z39.50 is fraught with guesswork and manual fiddling. That’s bad. The whole point of a standard should be to free you from guesswork.

How to use it. Using MARCThing is simple. Either send it some MARC records or what Z39.50 server you want to search and what you want to search on, and get back XML (or a variety of other formats) that you can use in applications without having to know a lick about library cataloging. All the messy details (and there are a lot of them) are hidden from view. Everything just works. You don’t need to know what a nonfiling indicator or a use attribute is, or the difference between MARC8 and UTF-8. You just need to know how to make an HTTP request.

What I hope is that this inspires allows people not in the library world to do cool things with library data. It’s sad that working with library data is such a hassle — there are so many underused resources out there. I won’t go too much into the technical problems with Z39.50 and MARC, but I do have a recommendation for anybody involved in implementing a standard or protocol in the library world. Go down to your local bookstore and grab 3 random people browsing the programming books. If you can’t explain the basic idea in 10 minutes, or they can’t sit down and write some basic code to use it in an hour or two, you’ve failed. It doesn’t matter how perfect it is on paper — it’s not going to get used by anybody outside the library world, and even in the library world, it will only be implemented poorly.

Open source plans. LibraryThing was already the only major cataloging site that used any library data. (The rest use Amazon’s data exclusively, a severe hurdle to book lovers in the US and an absolute barrier to those in most other countries.) It took us a long time to develop, and we have limited resources. We are not eager to give our competitors such a valuable tool — they can get their own library geeks. At the same time, we are eager to encourage non-profit use and to license its non-competing commercial use for a token amount.

We’re thinking of releasing the code under the Creative Commons Attribution-Noncommercial-Share Alike license, but it will depend on what people want to do with it. If you were bitten by a radioactive librarian and suddenly gained the power to search 700 libraries worldwide, what would you do?

Stay tuned; code is coming soon!

Labels: django, librarything for libraries, marcthing

Tuesday, November 13th, 2007

Shelfari astroturfing: The evidence

UPDATE: CEO Josh Hug left a omment on a piece by Seattle bookseller Michael Lieberman (also in the spamming charge, I’ve come up with over 50 examples.

The offender I found has the ID “schaufferwaffer.” The ID is the give-away. It’s really easy to Google something like that, and turn up this same evidence.

The posts below make it clear this was no slip of the keyboard. Deceiving people was the whole point. It’s icky to don’t go on and on about how much you “love” Shelfari without mentioning you’re paid by them, and worse to call yourself a “user” and wax rhapsodic about how you’re going to win a site contest for which employees are ineligible. But the low point is surely the blog post with the woman who just found out her mother-in-law had brain and lung cancer. On the plus side, Danny wished her well. On the down side, he still took the opportunity to lie about who he was and shill for his employer.

Again, I want to make myself clear. We have lots of honorable competitors (listed here and here). Shelfari is a bad actor. Considering that they’re the only well-funded site, with a reported $1 million from Amazon, their spamming and astroturfing is particularly despicable.

In fairness, “Danny” now comments mentioning that he works for Shelfari. Meanwhile the ID has been deleted. Nobody at Shelfari has ever mentioned or apologized for this. I triple-dog dare you to post about it on the site and see what they say.

“I have been on Shelfari for a couple of months now and absolutely love it.
Any good books you could recommend to me?
-schaufferwaffer

P.S. Check out the blog widget 🙂 Its pretty cool”

http://adversityuniversity.blogspot.com/2007/08/shelfari-tell-your-friends-what-youre.html

“Shelfari is such a great site. I joined a couple of months ago and I have been hooked on it ever since. All readers are welcome and everyones opinion is listened to.
Add me as a friend 🙂
-Schaufferwaffer”

http://www.westportlibrary.org/teenblog/2007/08/shelfari.html

“Thats the funniest thing I have read all day. I am adding you as a friend right now on Shelfari. I am still trying to convince all my friends to join. Any tips?”

http://ontherockswithsalt.blogspot.com/2007/08/they-see-me-rollin-they-hatin-trying-to.html

“friend me
schaufferwaffer
I dont know if i can compete with 300 books…but hey its all about rediscovering right?”

http://www.sarahannf.com/wordpress/?p=385

“Its my favorite website! Add me as well.
schaufferwaffer”

http://sirmartin.wordpress.com/2007/08/09/a-social-networking-site-for-book-lovers/

“Cool blog. Another Shelfari user! Great to see you joined the site. Any books you could recommend me? My username is schaufferwaffer”

http://kakistocracy.wordpress.com/2007/08/12/oh-the-monotony/

“I am also a huge Shelfari fan! I am looking for a new good read. Can you recommend me a book?
schaufferwaffer”

http://cookieaisle.blogspot.com/2007/08/shelfaricom.html

“Cool widget! I have been on Shelfari for a couple months now and LOVE IT! I like the books on your shelf. Which ones could you recommend me?
-schaufferwaffer (username on Shelfari)”

http://atlfemme.typepad.com/made_it_happen/2007/09/shelfari—virt.html

“I am a big Shelfari user. What are you waiting for? Sign up and add me as a friend (schaufferwaffer). Recommend a good book to me!”

http://debrichardson.com/blog/?p=1018

“Ohhh also on Shelfari. Also a big fan. I love the mystery genre! Recommend me some good books….schaufferwaffer is my id.”

http://www.voiceoftech.com/swhitley/?p=310#comment-8188

“Shelfari is a great site. I have discovered so many new books there! Have you read my new favorite “A Thousand Splendid Suns”.
Add me as a friend. My username is schaufferwaffer.”

http://oemar.wordpress.com/2007/08/30/two-sites-to-see/

“Hey, I just stumbled upon your blog. I am also a Shelfari member and I am totally hooked. Could you recommend me any good mystery books?

-schaufferwaffer”

http://judywatt.livejournal.com/1007749.html

“I like Shelfari because of the social interaction. I met a lot of new “friends” who have recommended some great books. Add me “schaufferwaffer”. Also, check out the new Harry Potter contest http://www.shelfari.com/harrypotter/”

http://community.livejournal.com/bookzine/18020.html?thread=232292

“Hey, I just came across your blog. I am also a Shelfari member, and i must say…I am completely obsessed! Do you know of any good fiction novels you could recommend me?
Add me as a friend! My username is schaufferwaffer 🙂

-Danny”

http://marblecake.org.uk/index.php/2007/09/16/got-the-book-bug/

“Hey, I just stumbled upon your blog. I am also hooked on Shelfari. I just added “How I Live Now” to my reading list!
Add me as a friend. My username is schaufferwaffer. It looks like you have some great books you could recommend me.”

http://llcoolworld.blogspot.com/2007/09/shelfari.html

“I am a huge Shelfari fan! Add me as a friend. My username is schaufferwaffer. Any good book recommendations?”

http://joelsfive.blogspot.com/2007/07/shelfari.html

“Hey, I am on Shelfari, but have not heard about RIT. I do enjoy reading, but don’t know if I have enough time with the school year starting up to join a book challenge.
Any books you could recommend me? My username is schaufferwaffer.

-Danny
P.S. Cool Widget”

http://jennyburd.blogspot.com/2007/09/my-rit-weight-loss-challenge.html

“Hey, I just stumbled across your blog. I found it pretty interesting b/c I love Shelfari and the Great Gatsby is one of my favorite books. I was required to read it through high school and was blown away. Any books you could recommend me?
Add me as a friend on Shelfari. My username is schaufferwaffer.”

http://cherylpeeves.blogspot.com/2007/09/fall-reading-starts-now-for-me-anyway.html

“Shelfari is definitely my favorite book social networking site. I think its easy to use and has a nice looking interface.
Add me as a friend! My username is schaufferwaffer”

http://kylefitzgibbons.wordpress.com/2007/08/12/shelfari-catalog-your-library/

“Add me as a friend! schaufferwaffer
Any good reads to recommend?”

http://alexgarciainseattle.blogspot.com/2007/07/whats-on-your-book-shelf.html

“Add me as a friend! schaufferwaffer
Any good reads to recommend?”

http://commavictim.blogspot.com/2007/07/shelfari.html (yes, same text)

“The best part of Shelfari is the new facebook application. It links with Shelfari.com and now i can access my books on facebook. Recommend a good read to me at schaufferwaffer”

http://www.ghostwheel.net/2007/07/12/shelfari/

“Add me as a friend! schaufferwaffer
Any good books i should read?”

http://hesperuspress.wordpress.com/2007/07/16/shelfari/

“Your book sounds really interesting. I’ll check it out! I am also on Shelfari. Add me – schaufferwaffer”

http://shookfoil.blogspot.com/2007/07/road.html

“Wow what a turnaround! Add me as a friend. My username is schaufferwaffer.
Any good books you can recommend me?”

http://www.hornes.org/mark/2007/08/15/web-widget-wednesday-shelfari/

“I like your bookshelf. I’m also on Shelfari. Friend me “schaufferwaffer”.
Any good books you recommend?”

http://700milestohome.blogspot.com/2007/07/its-been-too-long.html

“I like the blog 🙂 I also like Shelfari! Add me as a friend. My username is schaufferwaffer. Any good books you can recommend me???? To respond to the comment above me, the best part of Shelfari is that its FREE”

http://www.bronwynjameson.com/blog/2007/07/shelfari.html

“Checkout my shelf…www.shelfari.com/schaufferwaffer I am looking for a good mystery book to read. Any suggestions?”

http://www.feedmyapp.com/p/a/social-share-knowledge-web-20-applications-sites-shelfari/801

“Thanks for mentioning Shelfari in your post. I am also a pretty big fan of fiction, specifically historical fiction. My [sic] Schaufferwaffer, maybe you can recommend me some good books.
All the best,
Danny”

http://www.yardley.ca/dash/2007/06/24/shelfari/

“I love Shelfari!
Add me as a friend “schaufferwaffer””

http://spclibraryblog.blogspot.com/2007/07/shelfari-read-share-explore.html

“I am also familiar with Shelfari. Add me as a friend! Do you know of any good books you an recommend me?

-schaufferwaffer”

http://ceceliadowdy.blogspot.com/2007/08/shelfari.html

“Great Blog! Good luck with the job.
I am also on Shelfari. Add me as a friend. My username is schaufferwaffer.
Have you seen the blog widget? Are you going to add it to your blog?”

This is followed by a post from Dave, and is proof the company knew what Danny was doing. He pretends that the Danny’s comment was from a real user.

“Thanks for mentioning Shelfari 🙂 It looks like I was already beaten to the punchline….but the blog widget mentioned 2 comments above has just been completely redesigned with some cool new features. I’d love to hear your opinion on it. Give it a try and tell me what you think.
Happy Reading,
Dave”

http://friedafoxworth.edublogs.org/2007/08/04/tools-are-just-tools/

“I hope your mom is alright. I found this blog through google blogs. I am also a user of Shelfari and love it. Checkout my shelf at schaufferwaffer.
DO u have a facebook account? They now have an APP on facebook and its pretty cool!”

http://bibliobiography.blogspot.com/2007/07/shelfari-finally-invents-widget-other.html#1858124091838702835

“I think Shelfari is also great! Checkout my bookshelf, my name is schaufferwaffer.
Also, did u know that there is also a facebook App that links to your shelfari account? Its pretty cool!”

http://alexgarciainseattle.blogspot.com/2007/07/whats-on-your-book-shelf.html

“I like the blog! I am also a Shelfari member. What do u think of the new contest http://www.shelfari.com/harrypotter/
I think i can win it! [No, he can’t. People who work or the company are not eligible.]
Add me as a friend “schaufferwaffer”

http://girlygeekdom.blogspot.com/2007/07/shelfari.html

“Shelfari is my obsession. I spend way too much time on there, which is not a bad thing i guess
Add me as a friend!
schaufferwaffer”

http://www.makeuseof.com/dir/shelfari/

“Shelfari is a great site. I have been hooked on it since the beginning of the summer. Have you see the blog widget yet?
Add me as a friend. My username is schaufferwaffer. Do you know of any good books you could recommend me?”

http://the-visitor.blogspot.com/2007/09/for-book-lovers.html

“I joined Shelfari a couple months ago. What a great site! Add me as a friend!

-schaufferwaffer”

http://paprikapink2.blogspot.com/2007/08/keeping-without-keeping.html

“I’m liking the blog. I am also a huge fan of Shelfari. Add me as a friend! My username is schaufferwaffer.
Any good books you could recommend me?

Also, have you seen the blog widget?”

http://magysty.blogspot.com/2007/07/new-time-waster-on-web.html

“Cool Blog Widget! I have been on Shelfari for a couple of months now and just absolutley love it.
Do you know of any good books you can recommend me? How about “Life on the Run”. I have heard some good things.
Add me as a friend!
-schaufferwaffer”

http://www.rynosoft.com/mick/blog/2007/08/shelfari_1.html

“Shelfari is a great site! I am completely addicted right now. Could you recommend me any good books? Add me as a friend! My username is schaufferwaffer.”

http://gargleblogger.blogspot.com/2007/09/take-look-its-in-book.html

“I am also a Shelfari member and was wondering if you are participating in the Harry Potter contest? http://www.shelfari.com/harrypotter/
Grand Prize looks really tempting. Add me as a friend “schaufferwaffer”. Any good book recommendations?”

http://mommimi.typepad.com/the_dempseys/2007/07/harry-potter-an.html

“Thanks for mentioning Shelfari. I work there, add me as a friend “schaufferwaffer”. Do you
have any good book recommendations?
Have you seen our Harry Potter contest? http://www.shelfari.com/harrypotter/

http://authorsoundrelations.blogspot.com/2007/07/reading-and-technology-susanna-carr.html

“I like your list of books. I am also working on a list of books to read, however i am still compiling it. It can be seen on my account at Shelfari.com……My username is schaufferwaffer
Apparently Shelfari is coming out with a new blog widget that is supposed to be released in the next couple weeks. [Apparently indeed—You work for the company!]

http://daemon-ink.net/blog/?p=457

“I am a Shelfarian! Add me “schaufferwaffer”

You should add the blog widget. Its a pretty cool way to show off your books”

http://historicalromancemel.blogspot.com/2007/07/new-website-i-just-joined-shelfari-to.html#c942705946105895786

I really like the widget in your blog! I have read Jane Eyre and really liked it. Any books you can recommend to me? My username on Shelfari is schaufferwaffer…Send me a note!

http://tarasnichols.blogspot.com/2007/07/shelfari.html

“I like your shelf! Add me on Shelfari

“schaufferwaffer””

http://bruisedfancy.blogspot.com/2007/07/shelfari-book-shelf.html

“Hey, I just stumbled upon your blog. Whats happening in San Paulo is really fascinating. I have a few friends down there and I have to ask them what life is like without advertisements everywhere. Also, I am on Shelfari too. Add me as a friend! Recommend me a book 🙂

-Schaufferwaffer”

http://www.social-marketing.com/blog/2007/08/tip-jar-82007.html

“Sounds like you are a true bookworm
I am also on Shelfari. Add me as a friend! My username is schaufferwaffer
Any good books you can recommend to me?”

http://hhhl.wordpress.com/2007/08/15/what-i-do-on-my-summer-vacation/

“Shelfari is a great site. I discovered it a at the beginning of the summer. It’s pretty easy to use and the widget looks great on blogs.

Add me as a friend! Any good books you can recommend me?

-schaufferwaffer”

http://craig.cmehil.com/2007/08/shelfari-and-zoho-creator.html

“I just stumbled across your blog and I really like it! I signed up on Shelfari a few months ago and love it .
Add me as a friend, my username is schaufferwaffer.
Any good books you can recommend me?”

http://www.indezine.com/blog/2007/08/my-book-shelf.html

“Jusy Saw “Knocked up”. I couldn’t stop laughing. [Nice touch.]
Also a huge Shelfari fan! Add me as a member. schaufferwaffer”

http://www.sarahannf.com/wordpress/?p=384

The prosecution rests.

Labels: astroturfing, schaufferwaffer, shelfari

Tuesday, November 13th, 2007

Want out? Click the small, gray, non-underlined text in the corner.

Here’s the denument of this affair, which I shall follow-up with my evidence that Shelfari also “astroturfed.”

Last week, one of our competitors, Shelfari, received a sound and very well-deserved drubbing for their deceptive invite-spamming. (Best title: Where’s my shelefant gun?)

Although a far more popular blog, Gagetopia, was the leader of the attack, we are proud to have supplied much of the ammo–more than fifty angry and embarrassed blog posts from people who learned that Shelfari had sent hundreds or thousands of emails in their name. The emails went out to anyone they had ever emailed, been emailed by or appeared alongside in a CC or on a listserv.

To repeat, we are NOT against competition, just against one site’s underhanded tactics. We often praise our competition, from BookJetty to GoodReads, Babelio to Reliwa (and some 40 others). Post about LibraryThing on some sites and you’ll find the message gone the next day. We don’t work like that. In fact, in one thread we even posted a long list of competitors (now outdated) and asked users to sign up for as many as they could, figure out what they liked, and come back with suggestions for improvements.

Anyway, after the drugging the site “relented,” and changed the sign-up process somewhat. The CEO explained that they never had any intention to deceive and that it was a side effect of growth.

I’m unmoved. As Gadgetopia put it, albeit before the final post:

“Just do us all a favor and admit it — admit that you were over-zealous and getting new members was more important to you than the personal pain of your current members. You trampled all over them to make your investors happy with your growth numbers.

“Come on, Josh, admit what you did. At least be that much of a man about it. Is there anything below the PR facade? Or are you unable to turn that off and just level with us?”

Then again, the newest post does say they “deeply apologize to all of our users who mistakenly emailed unwanted contacts.” It’s not everything, but it’s certainly something.

The new method is certainly a lot better, but it’s still icky in my book. It’s automatic on sign-up, and at every turn the invasive option is in the center of the screen, with a big blue-green “continue” button. The only way “out,” other than doing as the site demands, is to click the words “Skip making friends for now,” which is small, grey, out-of-the-way, and not underlined. It stands alongside other text of the same size, color and non-underlinedness, except that text isn’t a link. As one blogger put it:

“And voila, if … [you’d move] your mouse over each line, you’d find that one of these two pieces of text is actually a link! How sneaky is that!”

And they still pre-select every contact, something very few people want.

At LibraryThing our invites are unforced and voluntary. You don’t go through the page automatically. The interface is clear. And we not only don’t preselect all your contacts, we even removed the “select all” link. Want to send it to a thousand friends? Better warm up your clicking finger…


*I claim hundreds because I found 51 blog posts as of last week, many of which have comments by other non-bloggers. This week produced another dozen or so, and one can safely assume that most people do not have blogs.

Labels: shelfari, spam

Saturday, November 10th, 2007

An academic take on LibraryThing tags

I just discovered Tiffany Smith’s “Cataloging and You: Measuring the Efficacy of a Folksonomy for Subject Analysis“.* It’s the first detailed academic study of LibraryThing tagging—and a very sympathetic one.

The article focus on five books, comparing their tags with their Library of Congress Subject Headings (LCSH). The books are Harry Potter and the Half-Blood Prince, Susanna Clarke’s Jonathan Strange and Mr. Norrell, Ian McEwan’s Atonement, Marjane Satrapi’s Persepolis and John Hodgman’s The Areas of My Expertise.

LibraryThing doesn’t “win” every comparison, but it comes out pretty well. I’ve already coopted her observations on two titles into my talks, namely Persepolis and Areas of my Expertise, both of which rate a single, very general subject. On the latter:

“How do you identify the subject of a fictionalized almanac, which, according to the Library Journal blurb on the back cover, is ‘a handy desk reference for those needing a dose of nonsense’? If you’re the Library of Congress, you call it ‘American wit and humor’, and move on to the next item on your book cart. You’d be accurate, because Hodgman is American and the book is witty and humorous, but you wouldn’t have captured the specificity of this item.”

Smith contrasts this with the LibraryThing’s florid tag cloud, sporting such terms as almanac, hoboes, alchemy, cheese, cryptozoology, eels, omens, portents and absurdities. Record-by-record these tags may only serve to amuse, but if you can’t recall the title, Hodgman’s strange work can be easily retrieved by looking for books tagged both “eels” and “humor” or “hoboes” and “almanac”. By contrast, I would not recommend wading through the American Wit and Humor subject!

I was also gratified to see the author notice an effect I’ve mentioned periodically but which has found no echo in other examinations of the topic and in the whole tired expert-vs-amateur polemic. As she writes, LibraryThing members pick up on the Napoleonic Wars element in Jonathan Strange, which LCSH misses:

“This may speak to the problem of the physical impossibility of the library cataloger reading the entirety of this roughly 800 page book to get to all of the detail. The Napoleonic element is not evident for the first third of the book and is not represented in the chapter titles, although it plays a pivotal role in the plot development.”

Fundamentally, I’m willing to concede the virtues of expertise, but there’s a lot to be said for reading the book all the way through, and library catalogers are not often able to do that.

In this connection, I’ve previously noted how my wife’s third novel, Love in the Asylum, acquired an erroneous “Alcoholism” subject, derived ultimately from bad publisher flap copy. Clearly neither the librarian nor the publicist had read the book. (My wife caught the copy before it went to print, but not before it had acquired Cataloging in Print LCSHs.) And the LCSH team also missed the topic of American Indians (Abenakis), a major presence in the book, but not touched on in the first 1/3 or the flap copy.

Anyway, it’s an interesting read. Since Smith did her research LibraryThing has grown almost 100%, and there are few things I’d quibble with*, but it’s a very good outside examination of why LibraryThing member’s tags should be dismissed by librarians interested in cataloging quality.


*”in”—as they say in academia—Lussky, Joan, Eds. Proceedings 18th Workshop of the American Society for Information Science and Technology Special Interest Group in Classification Research, Milwaukee, Wisconsin.
**For example, Smith was confused why some LibraryThing works had subjects that were not present in the Library of Congress record, which she believes is our source. In fact, we get our Library of Congress Subject Headings (LCSH) from many librares. Libraries are free to augement the LC’s headings, and many do; we pick up anything in the 600s of all the MARC records that make up a work.

Labels: academics, LCSH, LIS, tagging, tags

Thursday, November 8th, 2007

Shelfari spam: “basically social networking rapists”

LibraryThing’s success—we recently hit three-hundred thousand members and twenty million books—has spawned competitors, more than forty at last count. We respect them. Some of the best include BookJetty (gorgeous), Anobii (simple and big in China), Socialogue (does CDs) and GoodReads (friendly and not small; they just hit 5 million books). We think we do it better—for example taking from more than 75 soures other than Amazon–but we also know we aren’t perfect. To stay on top we have to work hard, and to be true to ourselves—loving books and being more open and collaborative than anyone else.

We respect our competitors with one exception: the site “Shelfari.com.” We have always spoken our mind, so here’s a piece of it: Shelfari has gained traction by engaging in unethical practices, including astroturfing (posting on blogs pretending to be users, not employees*) and putting out press releases about how they invented the idea. But the worst has been their spamming campaign.

These are strong accusations, but provable. This post lays out fifty-one posts that show they are, in the words of the well-known blog Gawker, “basically social networking rapists.”

Yesterday the popular Gadgetopia went on the attack, after Shelfari sent out thousands of invites in one author’s name. After getting the usual cut-and-paste non-apology, he pressed the point. I’d had my fill too, so I posted a comment with some of the evidence. Things built from there. And to my delight, Otis, the Goodreads guy, joined me, adding:

“I have two points in writing this: number one is to encourage people to keep blogging about Shelfari’s trickyness as long as they keep doing it. So far, people are having no problems doing that. However the saying ‘all press is good press’ is mostly true, so my second point is to encourage people to also mention that there are a number of other sites for book lovers that are well designed: notably, Goodreads and LibraryThing.”

The Spamming Campaign

Jesse Wegman of the New York Observer wrote the best accounts so far. Shelfari spammed “every single person with whom I have exchanged an e-mail in the past three years, in addition to every single person who has ever been on the same cc list as I have, regardless of whether we have ever met, in addition to every single listserv I have ever joined and every single Web site from which I have ever ordered anything.”

The method is simple. When you sign up for Shelfari you are dumped into a screen that offers to send out check-out-my-books invitations to friends. The user interface is confusing and deceptive, and what seems like an attempt to continue into the site really sends out hundreds or thousands of letters to everyone you’ve ever known by email. Reminder-letters follow. Skipping this step requires clicking out-of-the-way, gray non-underlined text.

Bloggers want to hide under their desks, ponder “moving to Tahiti” and beg God to “make [me] a bird so I may fly far far away.” They spit anger and frustration. The blog posts started months ago, and Shelfari has done nothing because a small percentage of recipients end up signing up, and this has driven their growth. They know perfectly well what they’re doing.

Below are 51 posts of bloggers and journalists livid or shame-faced at what Shelfari did to them.

Silly Shelfari Spam Shit
http://prosesanonymitus.wordpress.com/2007/11/01/silly-shelfari-spam-shit/

Startling Banals: Shelfari – horrible!!! Stay away from it!
http://quitsmoke.blogspot.com/2007/10/shelfari-horrible-stay-away-from-it.html

Tess Harrison: An Apology…Please Don’t Hate Me!
http://tshjourney.blogspot.com/2007/07/apologyplease-dont-hate-me.html

Apologizing to my network for Shelfari Spam
http://www.fullcirc.com/weblog/2007/10/apologizing-to-my-network-for-shelfari.htm

Gadgetopia: Shelfari Has Pissed Me Off
http://gadgetopia.com/post/6136#CommentID_435065

Mohtashims IT Tazee: Avoid #@$#! Shelfari like the plague!
http://www.ittazee.com/2007/10/21/avoid-shelfari-like-the-plague/

“I dont like to use any service that breaches my trust in the first 5 minutes…”

nm_eviled: Freaking Shelfari
http://nm-eviled.livejournal.com/111647.html

“I don’t know about the rest of you out there, but I’m about ready to commission a voodoo doll directed at the people who run Shelfari”

Shelfari SUCKS, the evil spamming bastiches
http://table4five.net/?s=shelfari

“If you get an invitation to Shelfari, from me or anyone, DELETE IT immediately. … It’s a big, huge, scam. Torches and pitchforks, anyone?”

The Journey: Got your invitation?
http://www.rodneyolsen.net/2007/11/got-your-invitation.html

“After Shelfari sent out another round of emails, which I specifically asked them not to, and failed to respond to the email I sent them, I have deleted my account. I refuse to deal with a company that has no regard for its users.”

Insights into Air and shadow: PSA and Apologies….
http://rbowspryte.livejournal.com/327691.html

“It is evil, evil, evil. It should die, die die!”

InsideView of Shelfari
http://www.insideview.ie/irisheyes/2007/10/insideview-of-s.html

“One hour after I pressed the button, Shelfari invited two dead people, one prisoner (he should probably read books but his warden is reading his mail), the CNN news desk, four European editors–and potentially a boatload of others who I hope I never meet.”

ShelfSpam! A Story of Books, Trickery & Embarrassment
http://bittersweetblue.blogspot.com/2007/11/shelfspam-story-of-books-trickery.html

Prateek Dayal: Why Shelfari.com sucks so much!
http://prateekdayal.net/blog/?p=14

[rough draft]: O, fie upon thee, Shelfari
http://cubswn.blogspot.com/2007/10/o-fie-upon-thee-shelfari.html

Scarlet Letters from the Literary Underworld: I am spam!
http://literaryunderworld.blogspot.com/2007/11/i-am-spam.html

Sending Me Spam Makes Us Friends, Right?
http://www.louisgray.com/live/2007/11/sending-me-spam-makes-us-friends-right.html

What do Squid Read?
http://squid-knits.blogspot.com/2007/10/what-do-squid-read.html

“After it sent out a second email today, I dove right back under the rock of humiliation, sank into the moat of despair and basically thought about changing my name and moving to Tahiti.”

In Rare Form: Myrt’s Blurts. Embarrassing Moments: The Catalog
http://inrareform.wordpress.com/2007/09/13/myrt’s-blurts-embarrassing-moments-the-catalog/

“Shelfari sent a bloomin’ email to every person I have ever emailed. Ex-boyfriends, people I hoped would never contact me again, coworkers, my vice-president, my CFO, my old boss, potential employers, my bishop, the entire Sunday school and relief society. EVERYONE. All now invited by spam to join my on-line book club. “

Dogster/Catster: Auto-Spamming of Contacts Has to Stop
http://blog.dogster.com/2007/09/07/auto-spamming-of-contacts-has-to-stop/

“[A] coworker had a ‘sitedown’ email contact which redirects to our pagers which made us think there was a site issue at midnight only to read some slimy Shelfari marketing blather”

Class 1 Spam Alert – Shelfari
http://rvbhute.org/2007/09/02/class-1-spam-alert-shelfari/#comment-293

“Have fun creating new passwords for your GMail, Yahoo and other accounts.”

V. Raghunathan warns us about Shelfari’s shenanigans
http://nanopolitan.blogspot.com/2007/11/shelfari-shenanigans.html

“This is evil. Pure evil.”

knitgrrl: Mea culpa — actually, no: Shelfari culpa
http://www.knitgrrl.com/?p=517

Comment: “All my personal and business contacts (including a couple CEOs) just got the invite.”

Shelfari
http://geekgirlie.livejournal.com/17413.html

“Please don’t sign up for Shelfari since they try to gain members through spam!”

Caught in Shelfari’s Sticky Web: No More Friends, Please! (Jesse Wegman, New York Observer)
http://www.observer.com/2007/caught-shelfari-s-sticky-web-no-more-friends-please

“Rather than reply individually to the hundreds of e-mails I have received, and continue to receive, since I signed up—a process during which I accidentally failed to uncheck the approximately 1,500 names in my Gmail address book that Shelfari had helpfully pre-checked for me, thereby inviting to join Shelfari, under my name (and ostensibly from my e-mail account), every single person with whom I have exchanged an e-mail in the past three years, in addition to every single person who has ever been on the same cc list as I have, regardless of whether we have ever met, in addition to every single listserv I have ever joined and every single Web site from which I have ever ordered anything (Amazon, Circuit City, and Law Students Against the Death Penalty have all, sadly, declined my invitation)—I thought I would write to you all here instead.”

PlanetMike.com: Shelfari.com Spam
http://www.planetmike.com/journal/2007/10/23/shelfaricom-spam/

Dear Author: Suffering Shelfari Spam? Opt Out
http://dearauthor.com/wordpress/2007/07/23/suffering-shelfari-spam-opt-out/

Brianna’s Mommy: Freaking Shelfari
http://nicemommy-evileditor.com/blog/?p=1013

This Is My Truth: Major Ranting
http://sanjukta.wordpress.com/2007/09/07/major-ranting/

Galleycat: Do We Like The Same Books? Go Away!
http://www.mediabistro.com/galleycat/web_tech do_we_like_the_same_books_go_away_69683.asp?c=rss

Dogs Steal Yarn, Apparently: I am a spammer, apparently.
http://dogsstealyarn.com/archives/000698.html

“And so then I got the bright idea to email an apology for the spam to my entire address book and Gmail decided that I’m a spammer and shut me down for 24 hours. But I only got to send that email to the first 500 people in my 837 contact address book. So I’m shut out of sending email AND I’ve now spammed people twice in a one-hour period.”

Beth’s Blog: Don’t Register for Shelfari: It Spams Your Address Book
http://beth.typepad.com/beths_blog/2007/10/dont-register-f.html

“So, don’t install Shelfari or you will be apologizing to your friends.”

PerthNorg: Got Your Shelfari Invite Yet?
http://perth.norg.com.au/2007/11/01/got-your-shelfari-invite-yet/

Will Porter: OOC – to anyone who got a Shelfari invitation from mun
http://will-porter.livejournal.com/166475.html

“I wish I’d never even heard of Shelfari. I think I’m going to delete my entire fucking account.”

Autumn Crumbs
http://www.lostinthegrooves.com/node/3209

“At any rate, if you got one of these, and if you decide to join, please read that article before you start inviting folks to join you. A very poorly designed website.”

“In Spamalot: The Shelfari Chronicles” by Bethanne Patrick, Publisher’s Weekly
http://www.publishersweekly.com/blog/670000267/post/1170016317.html?nid=3339

Gawker: The Internet Sucks
http://gawker.com/news/the-internet-sucks/-314436.php

“Finally someone takes on the great evil of our time: SHELFARI, the stupid book-centered stupidly-named social networking site that auto-sends an invite to everyone in your address book when you’re dingbat enough to sign up. I hate them so much and I scream a little inside every time one of these damned “DO WE LIKE THE SAME BOOKS?” emails arrives. They are basically social networking rapists.”

Sniff What I’m Steppin’ In: help
http://sniffwhatistepin.com/2007/10/15/help/

“Dear God, make [me] a bird so I may fly far far away. I just accidentally sent a Shelfari invite to my entire Gmail address book, which includes business folks, like, oh, my HR rep at the 7,000-person organization for which I now work. Help.”

SENSE N NONSENSE: Shelfari spam
http://dushti.blogspot.com/2007/10/shelfari-spam.html

With great reply from Shelfari: “We have actually evaluated numerous designs for this process and have chosen one that we felt was extremely clear explaining the process and what is happening.”

Blog comment: “That email went out to ex-boyfriends from ten years ago, their parents, high profile business professionals that I sent my resume to five years ago”

Deb on the Web: So Sorry!!
http://debaccuardi.typepad.com/deb_on_the_web/2007/10/so-sorry.html

jessie’s lament, and then some: shelfari fiasco
http://dyezebel.blogspot.com/2007/10/shelfari-fiasco.html

deponti to the world: What someone said about Shelfari
http://deponti.livejournal.com/297115.html

superstarksa: Shelfari – A PSA
http://superstarksa.com/2007/10/07/shelfari-a-psa/

James Galvin: Mass Invitation Spam Becoming the Default
http://www.jamesgalvin.com/2007/10/31/mass-invitation-spam-becoming-the-default/

“Anyone who is thinking “it’s your fault for not reading the whole page before clicking ‘continue’”, has probably never worked in IT. You don’t log in to a server as root for the same reason the fabled “big red button” has a plastic cover over it. When you’re dealing with large amounts of contact data, you are required to take extra precautions to maintain privacy.”

coniecto: About privacy, user options and spam
http://coniecto.blogspot.com/2007_10_01_archive.html#2153656353676130681

The Boar’s Head Tavern: Shelfari = spam?
http://www.boarsheadtavern.com/archives/2007/08/30/1254533.html

The Credence Dot Com: Shelfari.com and Yarri.com Are Spammers too !
http://www.thecredence.com/shelfaricom-and-yarricom-are-spammers-too/

PAgent’s Progress: Shelfari — A Poor Start
http://pagentsprogress.com/?p=657

Snap Judgment: Shelfari Spam (Tamil?)
http://snapjudge.wordpress.com/2007/09/17/shelfari-spam/

Mammaloves: Shelfari Is Not My Friend–Yet
http://mammaloves.blogspot.com/2007/09/shelfari-is-not-my-friend.html

Daring Young Mom: Shelfari is LIKE a Virus
http://www.daringyoungmom.com/2007/10/03/shelfari-is-a-virus/

Erotic Romance: Shelfari My Arse…
http://www.erecsite.com/2007/07/shelfari-my-arse.html

We rest our case.


*I’ll post the URLs for this if anyone is interested.

Labels: shelfari, spam

Tuesday, October 30th, 2007

LibraryThing for Libraries: October

October was another good month for LibraryThing for Libraries, with 7 new libraries coming on board.

First up on the list is the Los Gatos Public Library in California. Although a very small library, they show yet again that you don’t need to be big to innovate. They’ve promoted LibraryThing for Libraries extensively on their blog; they’ve even made a cool little video on YouTube of the widgets in action.

Library number two is the East Brunswick Public Library in New Jersey. Much more than most libraries, EBPL has really positioned themselves as a part of their community. You can see this in their offering of notary and passport renewal services at their libraries and their involvement with the community TV station, EBTV. I like seeing libraries that try to integrate themselves into their patrons’ daily lives like that. For an LTFL action shot, here’s “Treasure Island” in their catalog.

Next up is the Institute of Technology Tallaght, Dublin, our second library in Ireland. Along with our first one, the Waterford Institute of Technology, they’re a part of our nefarious plan to get every Irish engineering major hooked on LibraryThing.

Number four is the Kingston Information and Library Service in Melbourne, Australia. They have the honor of being our first Australian library, but we’ve got a bunch more on the way, starting with number five, the Australian Tax Office. The ATO’s running LibraryThing for Libraries on their intranet only, so I don’t have a demo URL for them. I’d like to thank them for coming up with a righteous javascript hack to make our widgets work with SirsiDynix’s new EPS/Rooms system.

Arlington Heights Memorial Library in Illinois is next up. I’ve already had a couple of other prospective customers ask to have their installation “look like Arlington Heights.” You can see why — they’ve done a great job blending LibraryThing for Libraries into their III catalog.

Finally, GMILCS is a consortium of academic and public libraries in New Hampshire. GMILCS runs SirsiDynix’s Horizon Information Portal catalog. It’s been cool to work with so many of the same people I knew when I used to support Horizon Information Portal for Dynix. Tim will be giving a talk about LibraryThing for Libraries at the CODI, the annual SirsiDynix user conference tomorrow along with Colleen Medling of the Salt Lake County Public Library. It should be a good one, so if you’re at CODI, check it out.

Along with picking up pencils, spatulas, and other stuff with vendor names on them, and talking to a lot of people in denim shirts, annual user conferences are always a good place to learn about new ways to make the software you’re stuck with do new things. That’s really valuable when change happens so slowly in the library software world; I remember helping GMILCS out when they first brought up their current catalog back in 2002. 5 years is a lifetime on the internet, and the gap between the speed that enterprise library software moves and the speed the web moves only seems to be getting bigger and bigger. So it’s vital for software vendors to make catalogs that can be modified, extended and customized both internally and externally. Customers shouldn’t have to wait for years for the shiny next generation product to get new features. It’s not just up to the vendors, though; customers need to keep finding ways to improve their out of the box systems (like David Pattern’s interesting new HIPPie project), library managers need to create a culture where change is embraced, and services like LibraryThing for Libraries need to keep adding more new functionality to existing systems. Legacy library software is inescapable — major upgrades will always be a gigantic chore, and even minor changes to the core of the system will often have huge repercussions on dozens of staff and thousands of patrons. That should not keep libraries from constantly making improvements to their public interfaces.

Labels: ahml, arlington heights, ato, australian tax office, codi, east brunswick, ebpl, gmilcs, iii, itt tallaght, kils, kingston, librarything for libraries, los gatos, ltfl, sirsidynix, slco