After years of CSV and TSV exports, we’ve added a “true” Excel export for your catalog. Find yours here: https://www.librarything.com/export.php?export_type=xls.
It’s a minimal, simple implementation. We made the headings bold, changed some column widths and defined some columns as text and some as numbers, but otherwise left the data as-is. We tested it out, but there are so many versions of Excel out there, that we’d appreciate feedback from members too.
Let us know what you think in Talk.
Labels: export, new features
We’ve just released a new and improved export feature. Check it out here.
Major improvements include:
Try it out. Go ahead and try out the new export.
Discuss. Come discuss the new export features on Talk.
Thanks. Export was re-engineered by Chris, Ammar* and me (Tim). It is based on the improvements Mike made to “Your Books” searching, and indeed the JSON format is effectively the format that the search system now indexes. (This will prove useful for troubleshooting problems with members.)
Screen shots
UPDATE: We’ve added an explicit Excel format.
* Who is Ammar? Stick around, we’ll tell you soon.
Labels: new feature, new features, small libraries
I’ve added a tab-delimited export, for use in Microsoft Excel and elsewhere. Go to the Extras tab to download the file to your desktop.
Tab-delimited supplants the former CSV export, which had some drawbacks. (I’ll be working to get it up to snuff in the near future.) Among other things, it handles special characters better, and includes all your fields. Problems remain with non-Latin characters, such as Chinese and Georgian.
So far, testers on the Google Group have reported no problems. If you find one, go ahead and post it there, or here.
In other news, also check out the Google Group for a cool new beta feature I’m developing, and asking for comments on. It’s not ready for blogging yet.
Labels: 1
I just added a CSV export feature, so you can throw the data into Excel, Access, Filemaker, etc. So, what now?
Some ideas:
What do you think?
Labels: 1
Railfans, rejoice: October’s Library of the Month features the Washington, D.C. Chapter of the National Railway Historical Society. I thoroughly enjoyed learning more about their work from Volunteer Librarian Ralph R. Bitzer. Thanks to Ralph for fielding my questions this month:
Who are you, and what is your mission—your “raison d’être”?
We are the Washington, DC Chapter of the National Railway Historical Society, Inc. Our chapter was founded in 1944. We currently are one of the largest chapters in the Society. The chapter is a volunteer not-for-profit 501(c)(3) organization. The Martin F. O’Rourke library was opened to the public on September 22, 2007. The library is named in memory of longtime member and former Chapter President Martin F. O’Rourke. The library is located in the base of the former Pennsylvania Railroad Bowie Tower (pictured below), which is now part of the Bowie Railroad Museum complex including the tower, a freight station, passenger shelter, railroad caboose and visitor center. The museum complex is located in the historic district of Bowie, MD adjacent to the Washington DC-New York Amtrak rail corridor (formerly Pennsylvania Railroad).
Our mission is to provide information about railroading both from historical and current perspectives. Our emphasis is on railroads originating from the eastern United States. However, with over 1200 volumes on railroading and many hundreds of railroading magazines we cover the railroad industry from many regions and perspectives both in the U.S and many foreign countries. We also have a selection of books for children to learn about and enjoy railroading.
Tell us some other interesting things about how your library supports the community.
The library is open to the public through the City of Bowie Museum group and can be accessed Friday, Saturday and Sunday from 12:00pm to 4:00pm. We are a research library only and do not have a book lending program. However, we have many members in our organization who can help with research on railroad topics.
We have open houses to our library several times a year for the public. One of the major events we have been doing for many years with the City of Bowie Museum group is Trainspotting Day on the Sunday in November after Thanksgiving. This is the busiest train operation day in the year on the AMTRAK Washington DC-New York City rail corridor. There are many areas around the museum complex where visitors can watch trains and then learn more about them in the library and museum exhibits.
We also participate in the Old Bowie Community Festival. The library is open and staffed by our volunteers. We have book sales to the public to enable learning about railroads and to help raise money for library projects.
What are some of your favorite items in your collection?
We have an extensive collection of Baltimore & Ohio Railroad employee magazines which give a very good perspective on railroading from the early 1900’s through the 1950’s, what railroad life was like for employees over a period of many years, and how things have changed.Extensive and in-depth histories on both the Pennsylvania Railroad and the Baltimore & Ohio Railroad. These were two of the primary railroads in Washington DC and Maryland.
What’s a particular challenge your library experiences?
Space limitations. Expansion is difficult because of the amount of room space in the lower section of the tower in our present location. We must limit books that can be added to the permanent collection. We are storing books for possible future inclusion offsite in some of the chapter’s railroad equipment. These are not accessible to the public.
What’s your favorite thing about TinyCat, and what’s something you’d love to see implemented/developed?
Overall, we find the program easy to use and provide information in a logical and usable reporting system. We would like better instructions for creating Excel or CSV worksheets from data files.
Thanks so much for the feedback. You can export your library catalog in various formats from LibraryThing’s More > Import/Export page (including Excel and .tsv format), but I can certainly help guide you with additional formatting you might want.
Want to learn more about DCNRHS?
Visit their website at https://dcnrhs.org/, follow them on YouTube and Facebook, and explore their full TinyCat collection here.
To read up on TinyCat’s previous Libraries of the Month, visit the TinyCat Post archive here.
Want to be considered for TinyCat’s Library of the Month? Send us a Tweet @TinyCat_lib or email Kristi at kristi@librarything.com.
Labels: libraries, Library of the Month, TinyCat
To read more about TinyCat’s Library of the Month feature, visit the TinyCat Post archive here.
We’re kicking the new year off with a feel-good story about a retired law librarian who’s been spending his days establishing a free library for a local community in the Philippines. He recently completed the library and Librarian Peter Mazzei was kind enough to answer my questions this month! Read on:
Who are you, and what is your mission—your “raison d’être”?
Tito Peter’s Free Public Library is a small, charitable library and education center located in Tambo, Batangas, Philippines. It has been a dream of mine for many years to build a library in my wife’s barangay (village) where books and educational resources are scarce. The mission and purpose of the library is to promote literacy, provide free educational resources (books, computer access, paper, school supplies, etc.) for students of all ages, and encourage reading, writing and education.
Tell us some interesting things about how you support your community.
As books and computer access to educational and research resources are relatively expensive and scarce, the library provides these resources in a safe, quiet environment that is conducive to learning, thinking and discovering. The barangay does not have a library of its own, so our library sort of acts as an ad hoc public or community library. In the near future, post COVID, I plan on having special activities such as poetry writing, science experiments, identifying plants, rocks and minerals, star-gazing, fun with mathematics, reading groups, and classical music appreciation.
What are some of your favorite items in your collection?
Children’s books of all kinds, especially picture books with those wonderful illustrations. The kids are enthralled with interactive books like pop-up books, lift-the-flap books and other books with movable parts.
What’s a particular challenge your library experiences?
The librarian’s perennial paradox: Not enough space, yet always looking for more books to add to the collection.
What is your favorite thing about TinyCat, and what’s something you’d love to see implemented/developed?
I have many favorite things about TinyCat, too numerous to mention here. Suffice it to say that it is a first-rate online cataloging/circulation system for small libraries. I think it is the best out there.
For circulation statistics, you currently have data for “Records, Checked out, On hold, Overdue, and Patrons”. Can you add “Total Checked Out” which would be the total number of books checked out since the inception of the library using TinyCat. It would be even better if you can also give the option to break down the total number of books checked out by month or year. That would really help the librarian quickly visualize how the library collection as a whole is being circulated.
Great idea! We do give you the option to generate your own checkout reports and/or export your lending Transactions, but adding a quick reference to total checkouts might be nice.
Want to learn more about Tito Peter’s Free Public Library? Check out their full TinyCat collection here.
To read up on TinyCat’s previous Libraries of the Month, visit the TinyCat Post archive here.
Calling all TinyCat libraries: become TinyCat’s next Library of the Month—just send us a Tweet @TinyCat_lib or email Kristi at kristi@librarything.com.
Labels: libraries, Library of the Month, TinyCat
Today we’re happy to announce the official launch of TinyCat, the online catalog solution for tiny libraries. In other words, YAAAAAAAAAAAAAY!
Check it out here! https://www.librarycat.org
What’s this all about? Religious institutions, community centers, small schools and other “tiny” libraries (up to 20,000 items) have used LibraryThing to catalog their collections for years. TinyCat gives them an attractive and powerful library catalog, with easy-to-use circulation and patron-account features.
See TinyCat in action. Folio, a member-supported library and cultural center in the heart of Seattle, uses TinyCat. Check it out here.
Try it out. Already have a LibraryThing account? You can see your LibraryThing collection on TinyCat by starting here.
Find out more, including Frequently Asked Questions, pricing, and how to get started at https://www.librarycat.org.
Let us know what you think on Talk, or email tinycat@librarything.com. Join the TinyCat discussion Group on LibraryThing here: http://www.librarything.com/groups/tinycat.
Visit us at PLA. Stop by booth #437 at PLA in Denver this week to meet Tim and get an in-person demo of TinyCat!
Labels: small libraries, TinyCat
We’ve long supported imports from Shelfari into LibraryThing, and with the news that Shelfari is being merged with Goodreads, some folks are looking for a new online home for their books. Now you can import your Shelfari library, and receive an automatic upgrade to a free lifetime LibraryThing account! Here’s how to do it:
UPDATE: We’ve seen a number of Shelfarians becoming more active on LT recently, and we’re happy to have you! You might find some of your old friends over in the Shelfarians on LibraryThing group, created and managed by your fellows.
Export Your Shelfari Data
You’ve got a couple options when it comes to exporting your Shelfari library, and can either directly download a file, or have one emailed to you. See Shelfari’s instructions here.
Sign Up for LibraryThing
It’s quick and free! Head over to LibraryThing.com and click “Join Now.” Enter your desired username and password (we also recommend including your email address, for password resetting purposes), and you’re good to go!
If you’ve already got a LibraryThing account, you can import your Shelfari library to your existing account.
Import Your Shelfari File to LibraryThing
Once you have your Shelfari export file, go to our Shelfari Import page and upload it there.
Enjoy
Welcome to your new library on LibraryThing! We’re happy to have you. Take a look around, and get to know the place a bit. We recommend the Short Introduction for a quick overview of features.
If you have any questions, Talk—our discussion forum—is a great place to start, and you can always email us at info@librarything.com.
We’ve done a lot of hinting about unveiling some big news at ALA Annual Conference in San Francisco this week, and now, we’re giving you a sneak peek… at TinyCat.
What is TinyCat? TinyCat turns LibraryThing into a powerful, simple, online catalog for so-called “tiny libraries” (less than 20,000 titles).
These smallest libraries—churches, synagogues, small schools, community centers, academic departments, etc.—have been using LibraryThing to catalog their collections for almost a decade. That continues with TinyCat. TinyCat offers a new way to view the catalog that’s cleaner and focused on helping patrons find and discover books in your library.(1)
TinyCat is:
Screenshots (click images for large size)
Method to our madness? It’s no coincidence that we’ve been rolling out lots of new features lately, many of which have been necessary to make TinyCat possible (think Barcode Support, Circulation, Advanced Search and now New Feature: MARC Import). There’s still more to come. We couldn’t be more excited.
TinyCat is coming soon. (We were aiming for ALA, so it’s very close.)
See us at ALA. Stop by booth 3634 at ALA Annual this weekend in San Francisco to talk to Tim and Abby and see TinyCat live. (We’ll show it with your collection, if you have one.)
If you need a free pass to the exhibit hall, details are in this blog post.
Interested? If you’re a librarian of a small library who might be interested in beta testing TinyCat, get in touch! Drop us an email at info@librarything.com.
1. In library terms, TinyCat is the OPAC (“Online Public Access Catalog”), and LibraryThing is the ILS (“Integrated Library System.”)
Labels: new feature, new features, small libraries, TinyCat
MARC is the library standard for bibliographic records. We’ve always parsed MARC records behind the scenes, when members searched one of our 700 library sources, or our Overcat collection. A few years ago, we introduced the ability export your LibraryThing collections as MARC records, even if your records didn’t start out in MARC.
Now, we’re adding the last piece: MARC importing, for all the small but professionally-cataloged libraries that use LibraryThing.
Try it Out. Check it out on Import or directly to MARC Import.
How it works. To use MARC import, you’ll need to have your library data in a .marc file format. Depending on how large a file you’ve got, the import process may take a few minutes. The good news is, you’ll receive a notification from LibraryThing once it’s ready. From there, you’ll be able to review your import options—just like you would with any other import—and select the collections, tags, etc. you’d like to apply to the items you’re importing.
What is MARC? MARC stands for Machine-Readable Cataloging. It represents a set of digital formats for describing items held by libraries: books, maps, CDs/DVDs, etc. You name it, if it’s in a library, MARC can handle it. Libraries the world over use MARC to standardize their item records in such a way that information about different types of items can all be fed into (and retrieved from) cataloging systems uniformly.
MARC fields are denoted by numerical tags, that indicate what type of information is contained in that field. For example, the title of a given work is always in field 245.
Don’t Upload The New York Public Library! This is for small—or, better the tiny—libraries that use MARC records and LibraryThing. Uploads are capped at 10,000 records total, so don’t try to upload 100,000 records. “Regular” libraries, big and small, should check out LibraryThing for Libraries, a remarkable suite of catalog enhancements.
Questions? Comments? Let us know what you think on Talk.
Labels: cataloging, new features, small libraries
We’re pleased to introduce Ammar Abu-Yasein (member LT_Ammar) to the LibraryThing gang!
Ammar will be working as a developer from across the pond (in Jordan), mostly on LibraryThing.com, developing new features and improving old ones. His first feature was improved export. Say hello to Ammar on his profile, or join us in the “Welcome Ammar!” Talk topic.
About Ammar
Born and raised in Toledo, Ohio, Ammar has loved computers and video games from an early age. He graduated with a degree in Computer Science from the University of Toledo in 2011, and a year later he joined a strong and vibrant IT community in Amman, Jordan. Ammar has spent countless hours building all types of software, from mobile to web apps. He’s excited to join the LibraryThing team and develop further skills.
When he’s not in a staring contest with his monitor, Ammar enjoys picnics with his family, swimming, making money, and of course, reading! His favorite authors include Stephen King, Steve McConnel, and J.K. Rowling, respectively. Ammar dreams of one day owning a helicopter (who doesn’t?)!
Labels: employees
Yesterday I added calendar “date selectors” to many of the places you can enter dates into LibraryThing. The selectors are optional—you can always just type instead. But they may come in handy.
Click here.
And get a date selector.
Along with this, and along with the recent export improvements, a number of important long-running date bugs have been closed. We look forward to help identifying and squashing what remains.
Come discuss on Talk.
Technical note: We used jQuery UI’s datepicker.
Labels: new feature, new features, small libraries
As a security precaution, we are requiring ALL members to change their passwords, here: http://www.librarything.com/changepassword.php
A security review and search of our records has determined that LibraryThing suffered a data breach in November of 2011. The breach was narrow. We have found no evidence that any catalog or other book data was accessed, changed or lost. The breach did not include member names, so it is unlikely the hacker(s) were after LibraryThing accounts.
Unfortunately, the hacker(s) did assemble and retrieve two key pieces of data–email addresses and encrypted passwords of members who had listed an email and joined before that date. Our passwords are stored as a one-way encyption (in technical terms, a salted hash). Such hashes are difficult, but not impossible, to break, especially for simple passwords.
Although a minority of accounts were affected, we are requiring all members to change their password to take advantage of increased account security features.
The breach. The hacker(s) gained partial access to our system through a flaw in our WordPress blogging software. Read more in “The Full Details” below.
All evidence points to this being an email-hacking attack. We have every reason to believe no other LibraryThing data was taken, not even user names. The intent was probably to grab the emails for spam, and break the password hashes, if possible. When broken, the passwords could be used against members who used the same password for their email, or email-based services, as they used on LibraryThing. Using the same password across many services is bad practice, but not uncommon. No financial data could have been taken. We do not get or store credit card numbers or any other financial information.
Our response. Security has been tightened significantly since late 2011, and has been further improved across the board since we discovered the event during a security review on January 21st, 2014. We have now moved our WordPress blog off our servers entirely, so a successful hack leads nowhere. Our password and account-recovery systems have been upgraded to meet the highest industry standards, and we have implemented a slate of additional security measures.
Email notices are being sent out to all members with email addresses. You can change your password any time; you don’t need to wait for the email.
Our apology. The hack may come as a shock; it certainly was to us. Although events of this sort–and far worse–have become numbingly frequent, they are failures indeed. I regret and apologize that any such event could happen on my watch, and the rest of the team feels the same way. We are all committed to ensuring that LibraryThing is as secure as possible going forward.
We hope this failure will not sour you on our service or community. LibraryThing members are a dedicated and passionate bunch, and a pleasure and honor to serve. After years of getting by, the company has significant profits to sink back into development of the main site; we will meet this event with renewed dedication and resources. (Please see, and spread, our recent job ad.)
Because the hack undermines a customer relationship, we have chosen to upgrade to “lifetime” accounts all members who joined before November 20th, 2011. We included those who did not have email addresses listed.
Come ask questions and discuss on Talk. You are also welcome to email tim@librarything.com.
Sincerely,
Tim Spalding
Founder and President
What happened:
What was taken:
How passwords work:
Security improvements:
We can’t go into detail about security improvements. (If we did, we’d be compromising security.) But we can say what you can see:
Free accounts:
Labels: security, sysadmin, systems adminitration
Did you know that you can now sync between Goodreads and LibraryThing? You don’t have to choose. Use both sites!
Getting started
To sync, go to Import books from Goodreads.
If you’re signed into Goodreads an export file should download for you automatically; if it doesn’t, there are some fallback instructions on that page. Select the downloaded file using the “Choose File” button and click “Upload” to import it into LibraryThing.
Import options
Once you’ve uploaded your file, you’ll see a breakdown of the books in the file, displaying the total number of books, books already in your library, books without ISBNs, and the number of valid ISBNs.
Sync options
Under “Handle duplicates,” you’ll see options to import duplicates again (i.e., create a whole bunch of duplicates), omit duplicates, or sync duplicates.
If you sync, you’ll see options depending on the differences between your Goodreads books and your LibraryThing catalog.
After this, click “Import books.” Old books sync immediately. New books are added to the import queue.
Labels: import
LivingSocial announced last week that they’ll be shuttering their Books section, formerly called “Visual Bookshelf,” as of August 10.
Living Social recommends migrating to another site, but if you do it you’ll lose all your reviews. Living Social splits your data up into multiple files and only the main one can be imported, the “collections” file. We’ve cooked up a better option that preserves your reviews as well.
As a bonus, anyone importing 25 books or more will get a lifetime account on LibraryThing!
Here’s how to do it:
To add your reviews:
Come talk about it on Talk. And if you have any problems email tim@librarything.com.
*Ratings: The Living Social ratings export option doesn’t seem to be working at the moment. If it starts working in the future, we can work through adding it too.
We have added a new source to every member’s Add Books page: OverCat, LibraryThing’s new index of 32 million library records, assembled from libraries around the world, and the first step in a major upgrade of LibraryThing’s cataloging functions.
Sources. OverCat was assembled from over 700 sources. The core consists of full datasets from the Library of Congress, Washington State, Boston College, Oregon State, and Talis Base (a collection of UK libraries).* To this we’ve added records from the hundreds of thousands of books members have searched for and added from the 690 libraries LibraryThing connects to.
The end result is arguably the second-largest searchable database of library records in existence, after OCLC.**
How to use it. To use OverCat, go to your Add books page. OverCat has been added to everyone’s source list. (It can be removed but not yet reordered.)
High-quality results. To make it easier to find the edition you need, OverCat combines results into edition-level clusters, so you get one result per edition (rather than pages and pages of the same edition of the same book from different libraries). By default, it will give you what it guesses the best available record is for that edition, but you can select from any one of the alternate records if you want to.
OverCat isn’t everything. The Library of Congress data dump is not current–although it’s been supplemented with user searches. Our relevancy ranking isn’t as good as Amazon’s. (We could use your feedback to make it better.) But most users will find it a useful source, and many will find it the best one.
The Big Issue. OverCat is available to LibraryThing members in the course of normal site activity—cataloging small collections of books.*** It will not be available for external access, including by libraries. It is not a back door to OCLC data.
This will come as a disappointment to many, including us. We have long argued for library-data openness and against OCLC’s bid to privatize and monopolize library data. But we also made it clear to the libraries we search that their data will not be made available outside of the context of personal cataloging without their permission. This will not change, now or in the future.
We would love to open OverCat up, to make it OpenLibrary as we originally hoped it would be it, or like Amazon Web Services, but with free, high quality data. We believe data openness is critical to the survival of libraries in our increasingly free and open world. But we depend upon open search portals, and will never open up a library’s data against its wishes. Some of these libraries may want to open up their data, but some clearly do not, and almost everyone is afraid of OCLC and its new data policy.*** Either way, we will abide by libraries’ wishes.
For the 690 libraries we search little has changed. We will still send member searches to your systems, but fewer—reducing your load—and the requests may not come at the time of searching. As before, found records will be stored on LibraryThing systems, but can now be used by more than one user and will appear in OverCat searches. Bulk or non-personal access will not be possible.
Thanks. OverCat has been a long-term project of Casey Durfee. The Board for Extreme Thing Advances helped us nail down bugs and decide on the name.
The future. LibraryThing’s greatest strength is its cataloging, but we don’t want to rest on that. There are a lot of improvements we can do now that we have a flexible, scaleable structure and repository for our data. OverCat is the first step here.
Come talk about your suggestions, and OverCat generally, on Talk here.
*Some OpenLibrary data was omitted for being mostly duplicative or of insufficient quality.
**For background on the OCLC issue, see here. We will also honor requests to remove libraries’ data from OverCat, excepting those libraries (like the LC), whose records are public by both law and public dumps.
There are larger collections. Harvard, for example, is said to have contributed 81 million records to OCLC, but most can’t have been book records, as the volume-count of Harvard is less than that of the Library of Congress, which we include.
We could make part of the data free, and part closed. But since the free data comes from OpenLibrary it would be duplicative of their efforts. We may explore this avenue in the future, as our primary complaint against OpenLibrary is the lack of exportable library-data formats.
***Exports of your library are included, obviously, but no larger dumps. “Personal” includes some small institutions, like church libraries, clubs and so forth.
Labels: cataloging, new features
A few days ago, just before hitting thirty million books, we hit one million user-uploaded covers. So, we’ve decided to give them away—to libraries, to bookstores, to everyone.
The basics. The process, patterned after the Amazon.com cover service, is simplicity itself:
<img src="http://covers.librarything.com/devkey/KEY/medium/isbn/0545010225">Easy details. Each cover comes in three sizes. Just replace “medium” with “small” or “large.”
As with Amazon, if we don’t have a cover for the book, we return a transparent 1×1 pixel GIF image. So you can put the cover-image on OPAC pages without knowing if we have the image. If we have it, it shows; if we don’t, it doesn’t.
The Catch? To get covers, you’ll need a LibraryThing Developer Key—any member can get one. This puts a top limit on the number of covers you can retrieve per day—currently 1,000 covers. In fact, we only count it when a cover is made from the original, o our actual limit will be much higher. We encourage you to cache the files locally.
You also agree to some very limited terms:
You will note that unlike the new API to our Common Knowledge data, you are not required to link back to LibraryThing. But we would certainly appreciate it.
Caveats. Some caveats:
Why are you doing this? The goal is half promotional and half humanitarian.
First, some background. This service “competes” with Amazons cover service, now part of Amazon Web Services. Amazon’s service is, quite simply, better. They have far more covers, and no limit on the number of requests. By changing the URL you can do amazing things to Amazon covers.
The catch is that Amazon’s Terms of Service require a link-back. If you’re trying to make money from Amazon Affiliates, this is a good thing. But libraries and small bookstores have been understandably wary about linking to Amazon. Recent changes in Amazon’s Terms of Service have deepened this worry.
Meanwhile, there are a number of commercial cover providers. They too are probably, on average, better. But they cost money. Not surprisingly many libraries and bookstores skip covers, or paste them in manually from publisher sites.
That’s too bad. Publishers and authors want libraries and bookstores to show their covers. Under U.S. law showing covers to show off books for sale, rental or commentary falls under Fair Use in most circumstances. (We are not lawyers and make no warrant that your use will be legal.) We’ve felt for years that selling covers was a fading business. Serving the files is cheap and getting cheaper. It was time for someone to step up.*
So we’re stepping up. We’re hoping that by encouraging caching and limiting requests, we can keep our bandwidth charges under control. (If it really spikes, we’ll limit new developer keys for a while; if you submit this to Slashdot, we will be Slashdotted for sure!) And it will be good for LibraryThing—another example of our open approach to data. Although none of our competitors do anything like this—indeed our Facebook competitors don’t even allow export although, of course, they import LibraryThing files!—we think LibraryThing has always grown, in part, because we were the good guys—more “Do occasional good” than “Do no evil.”
If we build it, they will come. If the service really pick up, we’re going to add a way for publishers, bookstores and authors to get in on it. We’d be happy to trade some bandwidth out for what publishers know—high-quality covers, author photos, release dates and so forth. We’ve already worked with some publisher data, but we’d love to do more with it.
*In the past, we had been talking to the Open Libary project about a joint effort. We even sent them all our covers and a key to the identifiers that linked them. But nothing came of it. To some extent that was our fault, and to some extent not. (I think them and us would differ on the blame here.) In any case, I was tired of the time and transactional friction, and wanted to try a different approach.
Labels: apis, book covers, covers, open data
See bottom of post for updates…
I’ve just finished a first draft of a JSON-based API for book data, created a test page and typed up some basic documentation.
What is this for? The API gives you Javascript access to your book data. The most obvious use of this would be to create new, much better widgets. At first, we expect this to interest programmers, but as new widgets are developed, non-programmers will get cool things. I started by redoing our traditional widgets in a new way here. That’s the base, not the ceiling!
How does it work? Every user can retrieve their data, in JSON format—basically as a ready-made JavaScript data structure. You control what is returned—books, tags, ratings, etc.—how it’s sorted and so forth. By default we give you a standard library of functions to parse and display the data. You can use it, build on it or start from scratch. Find out more here.
What’s great? All our code for processing the JSON API has been and will be released as open source—available for use, reuse and modification. Better—since we’re not the best programmers, particularly in JavaScript!—we are requiring any software that builds upon the API to be released under similar terms, so everyone can take advantage of improvements and advances.
Does this make code look sexy?
What’s the catch? The API is not intended for making backups or exporting your data to other programs. For that, use our CSV and TSV export functions, from the Tools tab. We are licensing the JSON API for browser-use only. This is about our data licenses. In-browser widgets have never drawn ire from our data providers.
Where can this go? This is just getting started. Everything can be expanded and improved. As members want new or different data, I will be only too happy to add it to the API. But the most interesting development will probably come from members, not LibraryThing employees.
I have created a LibraryThing API Development group to discuss the API, work through code and come up with new ideas.
At a minimum, I can see:
What if I’m not a programmer? No problem. Come and LibraryThing API Developmenttell us what you want. We’ll help you, or maybe someone else will.
UPDATE: I’ve made some changes to the programming, changing how the code is structured and adding result sets for reading dates. We also have the first outside use of the API, a very promising—if not perfect—cover flip test by MMcM (here). Follow what’s going on in the LibraryThing API Development group.
Labels: api, apis, books api, javascript, json, member projects
Mike—welcome Mike!*—has completed a major upgrade to the import system. The improvements are:
Import is still based on the idea that—when possible—LibraryThing re-fetches the bibliographic data. This adds another step, an “import queue.” But it also allows members to import full records, which no other site exports, and to get high-quality library data, if they want it.
Tell us what you think on Talk. It’s probably going to take a while to spell out what it does and doesn’t do and to update the old Adding and Importing FAQs.
*Mike (member: notmydadslibrary) is a new intern up here in Portland. This was a doozy of a first project!
Labels: import, new feature, new features
Introducing our new series: “29 Things You Didn’t Know you Could Do with LibraryThing” our attempt to introduce cheesy “style magazine” graphics and arbitrary numbers to the LibraryThing blog!*
Today’s topic is Bookstore Integration. Did you know that LibraryThing integrates with local bookstores? Basically, we’ll tell you if your local bookstore has a copy of a book in the store, and what it’s going for. It’s free for you and for the bookstore. It’s particularly easy for the bookstore to set up.
So far, we only have three bookstores in the system:
If you’re near any of these stores, you can add them to your work pages in two ways. Either go to your profile and select “edit profile,” or click the little pencil next to “Buy, borrow or swap” on a work page.
We need bookstores! We’d love to get more bookstores involved. In particular, any independent that uploads its inventory to BookSense can upload the same file to LibraryThing—no problem. If you aren’t a BookSense member, but can still export inventory data, let us know. Basically we need ISBN, price and quantity updated at least once per week.
At one point we explored some “high-level” discussion but, like so many “deals,” nothing came of it. Since the whole thing is basically free promotion for local bookstores, and a service to local customers, we hope that we can get more bookstores, and even bookstore chains involved without another conference call with a corporate marketing director!
As stated, this is a free servie. Like our Early Reviewers program—free buzz for publishers, free books for readers—we aim to do nothing more than make LibraryThing more useful and fun for everybody**. We succeed if LibraryThing gets better, which “aligns” our interests much better than if we negotiating complex deals.
Email tim@librarything.com for more info! Please note that this is a local bookstore offer. We’re not soliciting online bookstores to send their inventories. AbeBooks.com and BookFinder exists already!
*Why 29? Because it looked better than 27 or 28!
**This also applies to our recent movie-ticket give-away, although in exchange for the promotion New Line Cinema did agree to cast Abby as lead in a future production.
Labels: booksense, bookstore integration, bookstores
We just hired Casey Durfee (LibraryThing: caseydurfee), a crackerjack “Library 2.0” hacker. Casey’s first project—faster, better search—debuts today (see below). It’s a big win. We have a long queue of similar projects. Casey will also be heading up LibraryThing for Libraries, our project to get our data into library catalogs.
Casey will be working from Seattle, where he was recently left a job at the Seattle Public Library, managing their library system. Before that Casey worked at SirsiDynix*, so he has a lot of background in the arcane world of library systems—just what we needed for LibraryThing for Libraries. Casey is also responsible for L2, a handy Greasemonkey plug-in that adds Amazon content to library catalogs.
Perhaps the coolest thing Casey has done is a project called Helios, a “faceted” search on the Library of Congress records, done very simply and entirely with open-source software. It was a personal project, and although it’s not a complete solution, it searches the LC better than the LC’s million-dollar catalog. You can see Casey in action, talking about Helios, in this Code4Lib talk on Google Video.
Search. Check out the new work and author searches. They’re based on Solr, a simple but powerful search engine, and the same one Casey used on the LC data. Until now, we were relying on MySQLs fulltext capabilities. We had outgrown it, and slow performance was causing frequent database glitches.
It’s fast, accurate and searches all titles, not just the “leading” (mostly English) ones. But, as with everything we do, it’s not “perfect.” Casey has set up a Talk post about it. He has a variety of knobs he can turn, and is looking for feedback. I’m convinced it’s overzealous on “stemming”—picking up “loves” and “lovely” for “love.” That it even does stemming is quite an improvement from our previous solution. Once we’ve got it working the way we like, we’ll also be adding it to touchstones and elsewhere on the site.
Other projects. Casey is a certified library programmer. (I just play one on TV.) He knows his MARC21 from his UNIMARC, and his “glyphs from his diacritics”.** As time goes by we hope he can work on things like:
LibraryThing for Libraries. Right now, we have more than 350 libraries asking us about LibraryThing for Libraries. Altay and I have been going through at a snail’s pace. Casey should be able to crank that up a bit. We also think his SirsiDynix experience will come in handy. He’s already written a handy LTFL export script for HIP. He’s well-known in the HIP ILS community, and should move us past our current success among Innovative Interfaces catalogs.
That’s it. Welcome on board, Casey!
*When it was Dynix.
**Whatever that means; suggested by Casey.
Labels: 1
UPDATE: Many books found, but not all. Crash problem solved. See end of this post.
If you’ve been following the blog, you know that LibraryThing recently moved to a new, much better server. This move has made LibraryThing fast again, but the transition has been rocky. Yesterday, a glitch erased some recently-entered books, and the new server’s backup failed.
The losses are from books added in the last four days (ie., since the server switch). Some books added since then were recovered from “bin” files. And all books added via import have been recovered, but only as a re-import, so any subsequent edits to those books have been lost. The site will go live this evening, once the import queue has finished and I’ve sent messages to users who lost books (see below).
Here are the steps I am taking for users:
For the site, I and LibraryThing’s database administrator are doing the following:
Although a “beta” project is subject to occasional lapses, this sort of problem is unacceptable. Those who have been affected have every reason to be angry. Since I am at present synonymous with LibraryThing, this is my fault. I offer you my apology. I will do whatever I can to make it right for you.
The person who has been helping me with the database offered the pithy remark “this sort of thing never happens twice.” It’s not that lightning never strikes twice, but that once you’ve been hit you stop bringing stainless steel cutlery to picnics. LibraryThing’s new backup regime—nightly backups and an enterprise level “hot backup” at a different location (in fact, on opposite sites of the continent)—will go a long way to ensuring future data security.
Thank you for your understanding.
UPDATE: Comments posted to this entry noted that some missing books predated the server transfer. This was a good catch. At the same time, the server could hardly stay up five minutes. The problems were related. (And related to the previously-mentioned crc32 change.)
I found all these older books, substantially reducing the number of users affected. (Over 100 users can keep their free accounts although they have fewer than 25 books missing—most have none.) As for books entered Monday night, few users managed to enter any, but those that were entered were lost—the crashes were just too severe. All this did have one salutary effect—I figured out the big problem. I think it will stay up now.
Labels: 1
Universal Import is now truly universal. It accepts both files and web pages. I’ve successfully tested it with:
See the post below for more on how it works. Again, it won’t fetch your comments, the date you bought something or track down books without ISBNs, but it should do most of what you want most of the time. If you have problems, be specific about them. Go ahead and send me files and URLs.
Let me know if you end up drawing books from a site I haven’t mentioned. I’m keen to add it to the list.
Labels: 1
I’ve added a “Universal Import” feature. After wrangling with a dozen or so different formats, I chucked the nonsense and made a single Swiss-army-knife import. Universal Import works on:
For each one, it grabs the ISBNs and looks them up against the libraries you specify. The upside is the data is fresh, top-quality and drawn from wherever you want—from Amazon to libraries in Turkey. The downside is that it only grabs the ISBNs. It doesn’t try to wrangle all the other stuff.
This was not done lightly. Individual filters take a long time to build and require all sorts of compromises. LibraryThing users clamoring for imports are distributed among a half-dozen applications and various home solutions. So, instead of making 5% of my users 100% happy, I decided to make 100% of my users 95% happy.
I hope you like it.
Coming tomorrow: Imports from web sites like Amazon and AllConsuming!
Labels: 1
LibraryThing was featured on Lifehacker and Metafilter, among others. My server is melting and service is a little slow. I’ll have a much faster one up tonight or tomorrow.
My email box is also filling with suggestions. For new-comers, here are some improvements I’m working on:
At present I’m whispering soothing words to the server. Once that’s finished I’ll shift my attention back to features. Keep sending suggestions, comments and criticisms. The site has come a long way since it opened less than two weeks ago, and user suggestions have been absolutely essential.
Labels: 1
