Archive for the ‘apis’ Category

Thursday, August 7th, 2008

A million free covers from LibraryThing

A few days ago, just before hitting thirty million books, we hit one million user-uploaded covers. So, we’ve decided to give them away—to libraries, to bookstores, to everyone.

The basics. The process, patterned after the Amazon.com cover service, is simplicity itself:

  1. Take an ISBN, like 0545010225
  2. Put your Developer Key and the ISBN into a URL, like so:
    http://covers.librarything.com/devkey/KEY
    /medium/isbn/0545010225
  3. Put that in an image tag, like so:
    <img src="http://covers.librarything.com/devkey/KEY/medium/isbn/0545010225">
  4. And your website, library catalog or bookstore has a cover.

Easy details. Each cover comes in three sizes. Just replace “medium” with “small” or “large.”

As with Amazon, if we don’t have a cover for the book, we return a transparent 1×1 pixel GIF image. So you can put the cover-image on OPAC pages without knowing if we have the image. If we have it, it shows; if we don’t, it doesn’t.

The Catch? To get covers, you’ll need a LibraryThing Developer Key—any member can get one. This puts a top limit on the number of covers you can retrieve per day—currently 1,000 covers. In fact, we only count it when a cover is made from the original, o our actual limit will be much higher. We encourage you to cache the files locally.

You also agree to some very limited terms:

  • You do not make LibraryThing cover images available to others in bulk. But you may cache bulk quantities of covers.
  • Use does not involve or promote a LibraryThing competitor.
  • If covers are fetched through an automatic process (eg., not by people hitting a web page), you may not fetch more than one cover per second.

You will note that unlike the new API to our Common Knowledge data, you are not required to link back to LibraryThing. But we would certainly appreciate it.

Caveats. Some caveats:

  • At present only about 913,000 covers are accessible, the others being non-ISBN covers.
  • Accuracy isn’t guaranteed–this is user data–and coverage varies.
  • Some covers are blurrier than we’d like, particularly at the “large” size. This is sometimes about original files and sometimes about our resizing routines. We’re working on the latter.

Why are you doing this? The goal is half promotional and half humanitarian.

First, some background. This service “competes” with Amazons cover service, now part of Amazon Web Services. Amazon’s service is, quite simply, better. They have far more covers, and no limit on the number of requests. By changing the URL you can do amazing things to Amazon covers.

The catch is that Amazon’s Terms of Service require a link-back. If you’re trying to make money from Amazon Affiliates, this is a good thing. But libraries and small bookstores have been understandably wary about linking to Amazon. Recent changes in Amazon’s Terms of Service have deepened this worry.

Meanwhile, there are a number of commercial cover providers. They too are probably, on average, better. But they cost money. Not surprisingly many libraries and bookstores skip covers, or paste them in manually from publisher sites.

That’s too bad. Publishers and authors want libraries and bookstores to show their covers. Under U.S. law showing covers to show off books for sale, rental or commentary falls under Fair Use in most circumstances. (We are not lawyers and make no warrant that your use will be legal.) We’ve felt for years that selling covers was a fading business. Serving the files is cheap and getting cheaper. It was time for someone to step up.*

So we’re stepping up. We’re hoping that by encouraging caching and limiting requests, we can keep our bandwidth charges under control. (If it really spikes, we’ll limit new developer keys for a while; if you submit this to Slashdot, we will be Slashdotted for sure!) And it will be good for LibraryThing—another example of our open approach to data. Although none of our competitors do anything like this—indeed our Facebook competitors don’t even allow export although, of course, they import LibraryThing files!—we think LibraryThing has always grown, in part, because we were the good guys—more “Do occasional good” than “Do no evil.”

If we build it, they will come. If the service really pick up, we’re going to add a way for publishers, bookstores and authors to get in on it. We’d be happy to trade some bandwidth out for what publishers know—high-quality covers, author photos, release dates and so forth. We’ve already worked with some publisher data, but we’d love to do more with it.


*In the past, we had been talking to the Open Libary project about a joint effort. We even sent them all our covers and a key to the identifiers that linked them. But nothing came of it. To some extent that was our fault, and to some extent not. (I think them and us would differ on the blame here.) In any case, I was tired of the time and transactional friction, and wanted to try a different approach.

Labels: apis, book covers, covers, open data

Monday, August 4th, 2008

API to Common Knowledge

In case you don’t subscribe to the main blog, there’s a development there of interest to readers of this blog: We’ve unwrapped a free public API to all our Common Knowledge data—series, fictional places, characters, author educational histories, etc.

I’d love to see some of this data appear in library catalogs. The series coverage is really quite excellent.

At one point I made a series widget for LibraryThing for Libraries–listing other members of the series–but I didn’t deploy it. There was some concern that LT’s series data would fight with the libraries’ own series data. If an LTFL library wants to use it, however, let me know.

Labels: apis, common knowledge

Friday, August 1st, 2008

Free Web Services API to Common Knowledge

Introducing the LibraryThing Web Services API.

The API will eventually do many things.

For starters it includes all of the data in LibraryThing’s Common Knowledge project, our groundbreaking “fielded wiki” for interesting book information (see original blog post). It includes fields like series, important characters, important places, author dates, author burial places, agents, edits, etc. If you’re interested in building or enhancing book-data applications, this should be very interesting.

Common Knowledge is always in progress, but the results so far have been quite impressive. Members have made over 500,000 edits, and certain data types have become exceedingly useful and comprehensive. I’m particularly proud of our Series coverage (eg., Star Wars), better—we think—than any commercial series data. 

Oh, and it’s free! The data is made available under the highly permissive Creative Commons Attribution Share Alike license.

Architecturally, the Web Services API is a straightforward REST XML-based API.  The back-end is modular, allowing us to easily expand the available methods in the future. It’s request and response styles were modeled closely on Flickr’s API—Chris is a big fan—so it should make it easier to find similar sample code. The documentation resembles theirs too.

Kudos to Chris for his work on this and let us know what you think (here).

Update: The other big announcement—another data release—won’t be happening today. Too much to do!

Labels: apis, common knowledge, web services

Monday, July 7th, 2008

LibraryThing JSON-based books API

Over on the main blog I posted news about the new LibraryThing JSON-based books API (see here). The new API, which supplements our works API, comes with a small library of functions to manipulate it–all open source.

The API should be of interest to the libraries, as there are a couple of cool things they can do with the API. For example, with a few tweaks, it should be possible for libraries that use LibraryThing to showcase new or selected titles—a very popular thing—to create a widget that links into their OPAC, not to Amazon or whomever.

I’ll probably write some basic functions to change linking along these lines, if someone doesn’t do it for me first…

Labels: apis, json

Sunday, July 6th, 2008

Introducing the LibraryThing books API

See bottom of post for updates…

I’ve just finished a first draft of a JSON-based API for book data, created a test page and typed up some basic documentation.

What is this for? The API gives you Javascript access to your book data. The most obvious use of this would be to create new, much better widgets. At first, we expect this to interest programmers, but as new widgets are developed, non-programmers will get cool things. I started by redoing our traditional widgets in a new way here. That’s the base, not the ceiling!

How does it work? Every user can retrieve their data, in JSON format—basically as a ready-made JavaScript data structure. You control what is returned—books, tags, ratings, etc.—how it’s sorted and so forth. By default we give you a standard library of functions to parse and display the data. You can use it, build on it or start from scratch. Find out more here.

What’s great? All our code for processing the JSON API has been and will be released as open source—available for use, reuse and modification. Better—since we’re not the best programmers, particularly in JavaScript!—we are requiring any software that builds upon the API to be released under similar terms, so everyone can take advantage of improvements and advances. 

Does this make code look sexy?

What’s the catch? The API is not intended for making backups or exporting your data to other programs. For that, use our CSV and TSV export functions, from the Tools tab. We are licensing the JSON API for browser-use only. This is about our data licenses. In-browser widgets have never drawn ire from our data providers.

Where can this go? This is just getting started. Everything can be expanded and improved. As members want new or different data, I will be only too happy to add it to the API. But the most interesting development will probably come from members, not LibraryThing employees.

I have created a LibraryThing API Development group to discuss the API, work through code and come up with new ideas.

At a minimum, I can see:

  • New widget types, like widgets showing your most recent reviews.
  • Widgets that take you to libraries, and other places other than LibraryThing. (Libraries have been clamoring for this for ages. Many use LibraryThing to feature new books on the website, and want the links to go to their catalog, not LibraryThing.)
  • New result sets, for your tags or authors (separate from our books), your book’s works, series info, etc.
  • Integration with other JS-based APIs, like Google Book Search.

What if I’m not a programmer? No problem. Come and LibraryThing API Developmenttell us what you want. We’ll help you, or maybe someone else will.


UPDATE: I’ve made some changes to the programming, changing how the code is structured and adding result sets for reading dates. We also have the first outside use of the API, a very promising—if not perfect—cover flip test by MMcM (here). Follow what’s going on in the LibraryThing API Development group.

Labels: apis, books api, javascript, json, member projects