Archive for September, 2005

Friday, September 30th, 2005

LibraryThing adds 10 new libraries

LibraryThing has made another great leap forward. Until now LibraryThing drew exclusively on the Library of Congress and the five national “Amazons.”

Today ten more libraries have joined the mix:

  • United States: Boston University, The University of California system, The University of Chicago, Yale University
  • Britain: The National Library of Scotland, The London School of Economics, The National Library of Wales
  • Canada: The Canadian National Catalogue
  • Australia: The National Library of Australia
  • Denmark: Det Kongelige Bibliotek

Searching for books should also be faster, both because connections are shared between users and because you can now shift libraries when one doesn’t respond well.

It is a starter list. The US additions are a strong start. I hope Canadians, Australians and Danes will be happy seeing their national libraries included. Brits may feel the absence of Cambridge, Oxford and the British Library—the latter two are open and will be added. The French and Germans were, I confess, slighted, although the Canadian National Library has a lot of French literature. And what can I say to the Brazilians who have flocked in such numbers after LT was profiled in O Globo? I’ve looked and I will keep looking. I have an open Z39.50-based library in Portugal, but it is either down or on the blink. There are also some private universities in Brazil that are said to have open catalogs. I will find something for you!

There are a number of ramifications to the change that aren’t yet resolved:

  • Although I now have richer data and can populate additional fields, including series, language and edition, I have not yet exposed them to viewing and editing.
  • You will note that newly added books lack accents. Look on the bright side—they don’t have the wrong accents.
  • Fortunately, accents are coming. New books have a checkbox for “Update as cataloging data improves.” Keep this checked and accents will soon appear where once there was none. All I have to do is crack the obscure character set Marc records employ…

Labels: 1

Wednesday, September 28th, 2005

250,000 books

Cheers to all for another milestone!

Labels: 1

Tuesday, September 27th, 2005

How much library info is too much?

Library-expansion is another day or so off, but librarians and other interested parties are invited to weigh in on what new fields I should provide.

“MARC” records open up a world of data. I can finally parse all names correctly, add secondary authors, strip “a”s and “the”s from sorting, separate out publisher, place of publication and date of publication, and even wring automated sense from “xix, 230; ill., 25 cm.”

I don’t want to go overboard. Library records have more in them than most users need. Who but an institution needs to distinguish between “Uniform Title,” “Title Statement” and “Varying forms of Title”? Who scans their shelves confused between Dan Brown (1964–), author of the Da Vinci Code, and Dan Brown (n.d.) the publisher of the 1704 tract Novum Lumen Chirurgicum vindicatum?

For starters I can dump publisher and physical info into the “publication” field. And I can put everything into a static “card catalog” field, as I currently do with Library of Congress data. But what details should I “pull out” and allow to be edited, sorted by, and displayed in catalog view?

My candidates are:

  • Secondary authors, editors and illustrators (It is likely their role will be elided, except in the cataloging field)
  • Number of pages (Arabic numeral pages only)
  • Language

If you’re not a librarian, see Understanding MARC Bibliographic for Marc’s “commonly used” fields. Then thank you librarian for knowing this kind of thing AND dealing with people who come in and ask for “that red book, you know.” Incidentally, and taking that seriously, if a book’s “variant tile” includes such wishy-washy info as the spine-title (when different from the “real” title), why doesn’t it capture the dominant cover color? User hh219 appears to be doing that; perhaps it helps when looking for something.

Labels: 1

Tuesday, September 27th, 2005

Oxford coming up; Librarians suggest libraries

Stay tuned for Oxford University’s collection, coming on line later today or tomorrow. Then I’ll start bringing other Z39.50 libraries on fairly quickly, in order of user interest, connection speed and record parseability (I’m trying to do everything with Marc records now). Oh, and they need to be open to anyone and available all the time. The Z39.50 world is fairly new to me, but there are apparently thousands of people involved in it.

If you’re a librarian and know what the heck I’m talking about, feel free to suggest candidates. For starters I’d like to add some large US university libraries, to fall back on when the LC is slow. Oh, and I need a shoulder to cry on: Marc21 is blithering nonsense!

Labels: 1

Monday, September 26th, 2005

Amazon Associates!

Your complaints have been heard—Amazon gave it a thumbs up! You can now use your Amazon Associates id within a blog widget, and take back a small fraction of the $1/day all 200 or so blog widgets are making me. You don’t even need to have a paid account to run off with my haypence. That’s how generous I am.

(Frankly, I’d rather not to link to Amazon, but it’s a requirement if I use their cover image. Of course the text links go to your library or the book itself.)

Labels: 1

Sunday, September 25th, 2005

Better URLs for users

People were complaining that the LibraryThing URLs were too complex and it was hard to find the URL for a direct link to your collection. From now on you can use URLs like:

The catalog one will “redirect” to a more complex URL, the server rules being extremely difficult and hazardous to play with while dozens of people are on. Your URL is also listed on your profile page.

Labels: 1

Sunday, September 25th, 2005

LibraryThing makes the Mainstream Media, in Brazil

Special thanks to a reader of the Brazilian newspaper O Globo for forwarding me their blurb. I never managed to find it on the website, but I have the original. After spending about 2 seconds puzzling over it, rusty French and Latin at the ready, I threw it into Google:

Sunday toast For who has some familiarity with the English, access to the InterNet and thinks about fichar its library. A small farm makes success that receives the heading from a book, searchs it in the Library of the Congress of the United States, captures its fiche and plays it in the archive of the customer. Librarything.com is called and was created by Tim Spalding, an American pc hacker with the feet in the classic culture. It is in the version Beta (with the risks that this means) and leaves favour for who wants to catalogue up to 200 books. For bigger libraries, it charges USS 10 for the limitless use of the instrument. In less of one month, librarything joined four a thousand users who ficharam 177 a thousand books. It gained news article in the “The Guardian” and the forecast of that somebody goes to gain money with this business. If it will not be Spalding, will be another person. The Library of the Congress is a colossus. Its catalogue has 28 book million in 470 languages. For example: 18 headings of Fernando Gabeira. (who to want to sapear, an acknowledgment: the instrument of not accepted search accents nor cedilhas.)

This is great, if largely from Brown’s Guardian piece. My only fear is that ten Brazilians will try to start their own LibraryThing, looking for that money. (The money ain’t happening, people. It’s nice not to feel guilty about searching for freelance, but this is not making me rich. Then again, I might be rich if I were in Brazil.)

One question: At least for me the “instrument” seems to work with “search accents and cedillas” (that part ought to be “the search engine does not accept accents or cedillas,” instrumento de busca não aceita acentos nem cedilhas). Am I the only one?

Labels: 1

Saturday, September 24th, 2005

200,000 books!

Users have now cataloged 200,000 books, and the pace is increasing day-by-day. Three quarters were cataloged in the last two weeks.

I expect LibraryThing to keep growing for some time. Not only do most users have a lot of books to go, but only a fraction of the world’s book-lovers have ever heard of LibraryThing.

Bloggers—Thank you for blogging LT; it would never have gone anywhere without you. Bloggers who haven’t blogged about it—What are you waiting for?

On a related note, LibraryThing will go down for scheduled maintenance at 2am Sunday morning EST. It will probably be down until 5am. When it comes up it will be ready for the next 200,000.

Labels: 1

Saturday, September 24th, 2005

Shelf view

You can now switch to seeing your book covers displayed side-by-side together as if they were on a shelf. (On a shelf facing out, which no book-lover would do!) Check it out.

Some notes:

  • This post was originally a request for QA help. I think all the errors can be traced to something which is now fixed. Before reporting an error, quit out of your browser and come back in. The “stylesheets” in your cache may be old.
  • If you report an error, also tell me you browser and OS. If you’re feeling generous, also give me a screenshot.
  • I will look into ways users can contribute cover images. I’m worried about it from storage, security and spamming angles. I might add such a feature, but restrict it to external URLs at established booksellers and publishers.
  • The effect is something like what Delicious Library does, although Delicious Library improves Amazon’s images and puts them on a photo-realistic shelf. (Rectangles floating in space is about all I can handle right now!) For what it’s worth, I didn’t steal the feature from DL. I doubt he patented it, but I’ve got prior year prior art from an eBook project I worked on at my former employer, Houghton Mifflin.

Labels: 1

Friday, September 23rd, 2005

New Amazon bookmarklet

I’ve made an Amazon bookmarket, a little bookmark you put on your toolbar. When you’re on Amazon and want to add a book to your library, click on the bookmarklet and it will add it to your library. Get the Amazon bookmarklet out the extras page.

Labels: 1

Friday, September 23rd, 2005

Barcodes added

I’ve added support for 13-digit UPC/EAN barcodes. Of course there’s never any reason to type a barcode—every book with a barcode I’ve seen prints the (shorter) ISBN above the barcode.
But I’ve received emails from people with barcode scanners that won’t make the transition, so this will be a help.

Labels: 1

Thursday, September 22nd, 2005

175,000 books / calling all PHP and MySQL programmers!

We’ve hit 175,000 books, with no slowdown in sight. The time has come for me to consider a book database of a quarter of a million, half a million, a million or more.

It’s time to ask for advice. Programming PHP/MySQL for thousands of records is just different from doing it for millions. (Even if the books don’t hit a million, the tag database will.) I’ve been reading everything I can on enterprise-scale database optimization, but I lack hands-on experience. There’s a million things I can do, but I don’t have a relative sense of their costs and benefits. It’s also hard to “test” solutions—just building the search index took an hour, during which time LibraryThing was completely unavailable.

So, I’m not too proud to ask! Praise, free accounts and gift accounts are available.

Labels: 1

Thursday, September 22nd, 2005

Search added. Better or worse?

I’ve added a “true” search feature, or anyway a better one. Search is now available at all times, from a “search’ tab. You can search for books (pretty much all data is indexed), for just tags or for users.

For now it works well—searching is much faster—but I’m concerned it slows down adding records. Basically, every time it adds a book it needs to also update the index. As I’m currently indexing pretty much every field, this takes a little longer than I’d like it to. Let me know if you see the difference. And if you’re a MySQL guru, drop me a note and give me your take on all-RAM index and other options.

I’d like to add the standard bells and whistles (+/-/quotes, etc.), but I need to upgrade my database first. For now it just works work with what you give it.

Labels: 1

Wednesday, September 21st, 2005

Similar libraries!!!

(Excuse the exclamation points. I need to express my glee somehow.)

Check out similar libraries feature. It tells you whose libraries are most like yours, but not by adding up the number of shared books. That method is distorted by common books and by large libraries. For example, you don’t share all that much when someone has Harry Potter—everyone has Harry Potter! It’s much better to be the only two people sharing “The Joy of Scottish Lacework.” Similarly, it doesn’t mean much that you share 10 books with oakesspalding—he’s got 2,100 of them.

I didn’t want to just toss up a number, so it computes it four ways. First, it adjusts for book obscurity. Second it adjusts for library size. Third it combines the factors. And fouth it gives you everyone who makes the other three lists—your library doppelgängers, if you will.

Let me know what you think. It works wonders in my case. Zette‘s library is huge and very similar. Oakesspalding‘s library is much the same (although the fact that we give each other books for Christmas distorts it). Languagehat shares many of my interests, like Greek and even Hittite. CaveatLector is the best find—he has only 106 books so I would have otherwise overlooked him, but the nine books he shares with me are all obscure treasures.

Let me know if it works for you, and which of the measures is the most accurate.

Labels: 1

Wednesday, September 21st, 2005

Language support finished

LibraryThing now provides good support for the familiar—to many—non-English diacriticals such as é and ñ and gamely handles the exotic—to many—ones like č and å. The encoding is now “UTF-8” so, in theory, is should be able to handle a much wider variety of scripts, but neither Amazon nor the Library of Congress transmit material in non-Latin scripts, so you’ll have to type it yourself for now.

LibraryThing is now ready to start adding foreign libraries, and that is high on my list of goals. I shall also see if I can get Amazon Japan working.

Everything should be good going forward. The trick was getting all the old material into shape. LibraryThing-ers had employed all manner of tricks to get diacriticals to work, and some of that editing was hard to detect and correct for. Again, French, Spanish and German books should be fine, but I’m not so sure about the Georgian books languagehat entered. (In this case, my Mac simply won’t show Georgian, so I couldn’t tell anyway.)

The other gap is material entered between 8pm and 1am. I shut down book editing and tried to process everything coming in, but at almost 1,000 books/hr, a delay of thirty seconds between processes may have left some books “half cooked.”

Labels: 1

Tuesday, September 20th, 2005

Language support

I’m spending the day on alphabet-support issues—getting all those diacriticals to work. Doing this requires some database and programming changes, some of which need to go through various steps, so you may see all your diacriticals go bad one minute, then correct the next. Pay no attention to the man behind the curtain.

Labels: 1

Monday, September 19th, 2005

Avast me hearties!

Shiver me timbers, LibraryThing hit the Yahoo “new and notable” page (http://dir.yahoo.com/). In 1995 this would have been HUGE. I’m not sure anyone checks it anymore, but maybe they do. It’s below and to the right of the main entry, “Talk Like a Pirate Day.”

Labels: 1

Monday, September 19th, 2005

Library of Congress faster

I did something technical that should eliminate long waits for the Library of Congress “session” to begin. Basically everyone now shares the same “session.”

Let me know if problems increase or decrease.

UPDATE: Or if it explodes, as it did! I guess the LC limits requests by session id. Sorry about that and thanks for taking the hit.

Labels: 1

Monday, September 19th, 2005

PS: 117,000 books!

Aside from having thousands of copies of Harry Potter and no exhibitionists in the stacks, this is getting to be a decent sized community library!

Labels: 1

Monday, September 19th, 2005

New server, new look

The new server came and, after some rough hours, everything is running smoothly. Ongoing server and programming tweaks should win further gains. The site is unlikely to be much faster during off-peak hours, but I’m hoping not to have hourly crashes and twenty-second waits during peak times.

I’ve done something of a major overhaul of the design. If you’re on Firefox, Netscape or Safari you shouldn’t see much of a difference. Windows Internet Explorer users will suddenly see… what you’re supposed to see. Why didn’t anyone tell me it looked so terrible! Owing to the death of my Windows machine, I hadn’t done much cross-platform testing. Ouch! Those who weren’t turned off by strangely-large fonts and boxes extending 5% off the edge of the window, I salute you!

Comments encouraged. I’m frankly a bit snowed-under right now, so replies may take a few days. Enjoy!

Labels: 1

Sunday, September 18th, 2005

Tags: speeding up and slimming down

I’m changing how tags work to speed things up a little before the new server arrives.

Selecting tags had become a chore. The list of other people using the tag was taking 1-2 seconds to load. The new one usually takes less than .001 seconds. Tonight and tomorrow I’ll be speeding other tag-based functions up similarly.

To achieve these effects I made a decision: From now on each tag cannot exceed thirty characters in length. This is enough for most purposes, even “used modern history textbooks,” but not for “things that are almost but not quite dictionaries” (a real tag!). Most processes now already use the “short'” version; the long versions will remain visible and editable until next Sunday. Then the axe falls! Actually, since most of the long tags are really comments or reviews, I will move them to the users’ comments field.

Labels: 1

Saturday, September 17th, 2005

Change tags implemented

I’ve implemented a Del.icio.us-like “change tags” feature, available under the tags tab (also improved) and in the catalog under “power options.” You can also delete tags and change one tag to many. The latter is useful for disambiguation, ie., change “un” to “u.n.” and “united nations.”

A “mass edit” feature will come tomorrow, allowing you to add tags or make other changes by ticking off books from a basic but complete list.

QUESTION:

Currently the tags are unsorted. If you enter “zebras, apples” it stays that way, and you can sort by it, making the first tag a sort of “primary tag.”

Making it always alphabetical would be easier and in some way more elegant. What is the norm? What do people want?

Labels: 1

Friday, September 16th, 2005

85,000 books!

85,000 books. I’m haven’t checked how many uniques that is (which is also somewhat definitional), but 1% of them are by J. K. Rowling…

Labels: 1

Friday, September 16th, 2005

New server coming

LibraryThing was featured on Lifehacker and Metafilter, among others. My server is melting and service is a little slow. I’ll have a much faster one up tonight or tomorrow.

My email box is also filling with suggestions. For new-comers, here are some improvements I’m working on:

  • User-defined fields
  • British Library searches, then other library searches
  • “Power” editing, including applying tags to multiple books in one sweep
  • Bar-code/UPC/EAN numbers
  • RSS feeds
  • iPod export (so you can check if you have something)

At present I’m whispering soothing words to the server. Once that’s finished I’ll shift my attention back to features. Keep sending suggestions, comments and criticisms. The site has come a long way since it opened less than two weeks ago, and user suggestions have been absolutely essential.

Labels: 1

Thursday, September 15th, 2005

Reviews and power users

I’ve added a “power user” button into the catalog (next to “search library”). I’m going to hang a lot on it later—I want to keep the interface simple for new users, but give people with hundreds of books new options.

At present it just adds two things: the delete icon , which people have been clamoring for, an a icon, which tells you if you’ve reviewed something ( just tells you it’s reviewed). When you’re in power-user mode you can also sort by whether or not you’ve reviewed it. It sorts your reviews first, then books reviewed by others, then unreviewed books.

Now my question: Clearly I need a page listing what books a given user has reviewed and another listing recently-reviewed books, books with the most reviews, etc. But how much stress should reviews have? How interested are people in it? Should I, for example, have a “Reviews” tab like the “Tags” tab? Would this convey the impression that LibraryThing was all about reviewing and discussing? I want to keep some focus. This site is not a universal book portal. It’s a book-cataloging service with some diverting social extensions.

Opinions pro and con solicited.

PS: Next up—”Power tagging.” Vroosh!

Labels: 1