Tuesday, May 10th, 2011

LibraryThing is Faster, part II

It’s not a new “feature,” but speed and reliability are a key component of the appeal of a site. A few weeks ago we reported on a new server configuration that cut page-generation times in half (see LibraryThing is Faster). Now we’re reporting on some database tweaks that have made the process of finding, adding and editing data faster.

Like all large database-driven sites, LibraryThing can’t rely on a single database. Instead, we have a single “master” database which replicates its changes to a number of “slave” databases. (See Wikipedia: Database Replication.) Because sites “read” a lot more than they write, scalability is achieved doing most “reads” from the slave machines, which can be multiplied almost indefinitely to deal with increased traffic. Unfortunately, writes still need to move from the master to the slave, which necessarily involves a slight lag. If the lag becomes too great you get stale data or processes that pause (and pile up!) waiting for fresh information to pass from the master to the slaves. You also get bugs. And annoyances, like Talk posts not appearing right away. Replication lag also degrades query speed and therefore site speed generally.

As a heavy database-driven site running on relatively cheap hardware we’ve sometimes struggled to keep replication delay down. The problem is particularly acute on our weaker slaves. Fortunately, our ongoing review of performance issues has disclosed series of code and “schema” changes that have substantially improved replication speed. Here’s a chart of the average replication delay on one of our database servers over the last ten days. As you can see, two changes have made a big difference!

We’re excited about the progress we’ve made so far. It exceeded our expectations. Our performance review is continuing. We won’t stop until LibraryThing is as fast and reliable as it is powerful, rich in data and fun to use!

Come talk about this.

Labels: servers, speed

Wednesday, May 4th, 2011

May Early Reviewers batch up!

The May 2011 batch of Early Reviewer books is up! We’ve got 106 books this month, and a grand total of 2,551 copies to give out.

First, make sure to sign up for Early Reviewers. If you’ve already signed up, please check your mailing address and make sure it’s correct.

Then request away! The list of available books is here:
http://www.librarything.com/er/list

The deadline to request a copy is Monday, May 30th at 6 p.m.  EST.

Eligiblity: Publishers do things country-by-country. This month we have publishers who can send books to the US, Canada, the UK, Israel, and many others. Make sure to check the flags by each book to see if it can be sent to your country.

Thanks to all the publishers participating this month!

Mulholland Books Quirk Books Henry Holt and Company
University of Iowa Press WaterBrook Press HighBridge
Doubleday Books Harper Paperbacks Ballantine Books
Greenleaf Book Group Bantam Dell Eirini Press
Demos Health Penguin Young Readers Group Bloomsbury
Nolo Petra Books Taylor Trade Publishing
Grand Central Publishing Crown Publishing Houghton Mifflin Harcourt
Faber and Faber Clerisy Press Wilderness Press
Menasha Ridge Press Coral Press Red Telephone Books
Charlesbridge Open Road Human Kinetics
McFarland Eerdmans Books for Young Readers Hyperion and Voice
Spectra Bell Bridge Books Del Rey
Random House Putnam Books Riverhead Books
BookViewCafe BooksForABuck.com Oslerwood Enterprises Inc.
Downstream Press Rovira i Virgili University Press St. Martin’s Minotaur
Blacksmith Books St. Martin’s Griffin Mirth Press
DK Publishing Wolf Trail Press

Labels: early reviewers, LTER

Monday, May 2nd, 2011

LibraryThing for Publishers adds ONIX and FTP

We’ve added the capability for publishers participating in our LibraryThing for Publishers program to transfer their data to LT using the ONIX metadata format, a bibliographic format virtually all publishers produce, and using FTP, the industry standard for ONIX transfer.

For existing LTFP publishers, you can add ONIX feeds to your publisher profile via the “Set up FTP Transfer” option on the Edit profile and settings page. We’ll grab the ONIX file on the schedule you set via FTP and process it.

New publishers, you can set up your LibraryThing for Publishers page by following the instructions here. Any questions, or if you’d like to submit your data by another method, email me (jeremy@librarything.com)

We’ve also added a whole bunch of great publishers since our last blog-update. These include:

See the full list of publishers participating in LibraryThing for Publishers.

Labels: LibraryThing for Publishers

Monday, April 25th, 2011

April State of the Thing

This month’s State of the Thing, LibraryThing’s monthly newsletter of features, author interviews and various forms of bookish delight, should have made its way to your inbox by now. You can also read it online.

For our author interviews this month, I talked to former prosecutor Marcia Clark about her debut novel Guilt by Association, recently published by Muholland Books. Read the full interview.

I also chatted with Jessica Speart about her new book Winged Obsession: The Pursuit of the World’s Most Notorious Butterfly Smuggler (published by W.W. Norton and up for requests in this month’s Early Reviewers batch). Speart talks about the lengthy research process the book required, and about how the smuggler tried to make her his “front man” in a butterfly transaction. Read the full interview.

And we have a great third interview this month: Lisa Carey talked to Susan Conley (pictured at left) about her new memoir The Foremost Good Fortune, published by Knopf in February. Conley discusses her writing style, offers some sound advice for memoirists, and gives us a sneak peek inside her forthcoming novel.

Read previous State of the Thing newsletters:

If you don’t get State of the Thing, you can add it in your email preferences. You also have to have an email address listed.

Labels: author interview, state of the thing

Thursday, April 21st, 2011

Physical description fields added

We’re currently rolling out to all members some brand-new fields for physical description:

  • pagination (both Roman- and Arabic-numeral)
  • height, length, and thickness*
  • weight
  • volumes

In addition to the six separate fields, available for display and sorting your books, there are also two summary fields. “Dimensions” summarizes height, length and thickness in a “8 x 10 x 1.5″ format, and “physical summary” replicates the standard library-data format, displaying volume count, pagination, and the height of the book. The latter is also user editable.

The data comes comes either from the library or bookseller record you used to add your book, or, when data is missing, from the ISBN level. As elsewhere, data from your book is shown in black text, and data from another level is shown in green. The green text will turn black if you edit it or tab through the fields to confirm it.

You can edit all these new fields on either the book edit page or by adding them to “List” view on the “Your books” tab. To do that, click the little “gear” symbol on the top bar.**

Once added, double-clicking on any of these fields will bring up an “Edit Physical Properties” lightbox and allow you to make changes. There’s also an option there to convert the data for that record between pounds/inches and centimeters/kilograms, if you’re so inclined.

Naturally all these neat goodies lend themselves very well to cool statistics and charts, so we’ve also added a statistics/memes page. You can find yours here. If you’re not signed in, check out Tim’s here.

Find our how your books stack up (literally) against a hobbit, a giraffe, Michelangelo’s David, the Statue of Liberty, Big Ben, the Eiffel Tower, the Great Pyramid of Giza and so forth. Discover how many U-Haul book boxes it would take to pack your collection, or the value of your books’ weight in gold. If all the pages in all your books were laid out end-to-end, how far would they stretch? All that and more on the new stats page.

We’ve also included a handy chart showing how many of your books don’t contain data in these fields, in case you want to run off to grab the ruler and scale.

If these fields aren’t yet showing for you, they will be soon; you’ll receive a profile comment when the fields are available. Many thanks to the members of the Board for Extreme Thing Advances for their assistance with getting this feature ready!

Come discuss the new fields and the stats page in Talk.


* height = head to foot of spine; length = spine to fore-edge; thickness = “width” of the book on the shelf

** There’s also an option here to “Show volumes, pagination, dimensions and weight fields.” If you choose to hide them, they simply won’t display anywhere.

Labels: features, new features, statistics

Wednesday, April 20th, 2011

LibraryThing is faster

I’m happy to report that LibraryThing’s servers have undergone a considerable improvement. LibraryThing’s server administrator, John Dalton (member Felius), carried through an ambitious restructuring of how LibraryThing’s considerable traffic was distributed across our web servers. And the restructuring worked out.

Across the site “pages” have been sped up by about 100%, dropping from a median of about 1 second to about 0.5 seconds. Catalog, or “Your books,” pages have dropped from a median of 1.6 seconds to 0.8 seconds. Response has also become more predictable, with much a lower standard deviation of response times.

Good, but not everything. While server-rendering speed is important, it isn’t the only factor in perceived speed–the other two being transfer and rendering by the web browser. Server improvements also hide the fact that rendering times also include database actions, which were not improved by this change. The truly bothersome pages on LibraryThing are hindered by this, not by web server load per se. So, this change hasn’t done much to improve response time on catalogs with thousands or tens of thousands of books, hit for the first time that day, or on work-combination requests that require recalculating thousands of items of data. Basically, the improvement speeds up every page by an average of 0.5 seconds, but a 10 second page still takes 9.5 seconds.

Here are some graphs of the effect on different page types. The white band is a period for which we don’t have numbers.

All pages (includes widgets)

Catalog. Savings have been dramatic but, as noted above, mostly on the vast majority of “normal” requests, not on the rare but painful ones.

Talk topic pages. These have gotten much faster, because the data is easy to get from the database but also very copious, so it took a lot of server work to render it. This improvement has a perverse side-effect, however–the faster the page is made the more the Talk page can get ahead of master-slave replication. This issue will be addressed in an upcoming improvement.

Work pages haven’t improved because they were already well-distributed across our servers.

Labels: servers, speed

Friday, April 15th, 2011

ReadaThing Planned for April 23!

To celebrate their first anniversary, the folks in the ReadaThing group have planned a 24-hour readathon for Saturday, April 23. You can join up here by signing up to read for an hour of your choice, and list what book(s) you’ll read on the participation wiki.

Or, if you’re up for a bigger challenge, The Green Dragon is hosting a “Do Nothing But Read” Day on the same day, Saturday April 23. They invite all to join in, so if your schedule allows and you’re in need of a nice long day of reading (and who among us isn’t!?), take the plunge!

Labels: Do Nothing but Read Day, readathon, reading

Thursday, April 7th, 2011

Melvil Decimal System View in Your books

We’ve just added some handy new browsing functions to our Melvil Decimal System (MDS).

What is MDS? MDS is the Dewey Decimal System, Melvil Dewey’s innovative classification system, as it has been applied to books in LibraryThing members’ books. The base system is the Free Decimal System, a public domain classification created by John Mark Ockerbloom. The wording comes from out-of-copyright sources.

Here are some examples:

What’s new? You can now easily examine the MDS classifications of the books in your library, using the Melvil Decimal System view (accessible via the “Your books” tab; click the little divot to the right of Tags to show the available views).

When you click on one of the ten top-tier MDS classifications, you’ll see the books in your library which have been assigned to that level, and the second tier of MDS classifications will also display. You can continue drilling down through all five tiers of MDS classification.

See Tim’s books by MDS view at http://www.librarything.com/membermds/timspalding, or yours at
http://www.librarything.com/membermds/MEMBERNAME.

We’ll probably be adding some bells and whistles to this feature as we go forward. We’re planning to add a similar Library of Congress view of your library soon, so watch for that as well!

Come discuss in Talk.


Dewey, Dewey Decimal, Dewey Decimal Classification, DDC and OCLC are registered trademarks of OCLC. Read more about OCLC and the DDC on their website. LibraryThing is not affiliated with OCLC, but we have the same hatter.

Labels: features, member projects, new features

Wednesday, April 6th, 2011

April Early Reviewers Batch is Up!

The April 2011 batch of Early Reviewer books is up! We’ve got 110 books this month, and a grand total of 2,758 copies to give out.

We’re also happy to note that we’ve now given away more than 100,000 through our Early Reviewers and Member Giveaways programs!

First, make sure to sign up for Early Reviewers. If you’ve already signed up, please check your mailing address and make sure it’s correct.

Then request away! The list of available books is here:
http://www.librarything.com/er/list

The deadline to request a copy is Thursday, April 28th at 6 p.m. EST.

Eligiblity: Publishers do things country-by-country. This month we have publishers who can send books to the US, Canada, the UK, Israel, and many other countries. Make sure to check the flags by each book to see if it can be sent to your country.

Thanks to all the publishers participating this month!

HarperCollins Childrens Books Mulholland Books Quirk Books
Kregel Publications Henry Holt and Company Gefen Publishing House
Taylor Trade Publishing Dystopia Press Ballantine Books
WaterBrook Press Hyperion and Voice HighBridge
McBooks Press Beacon Press Picador
Harlequin Teen Coffee House Press William Morrow
South Dakota State Historical Society Press Silenced Press Random House
McFarland Random House Trade Paperbacks Faber and Faber
Wakestone Press Eos Human Kinetics
Bell Bridge Books Galaxy Press St. Martin’s Griffin
Nolo Exterminating Angel Press St. Martin’s Press
Omnific Publishing Bellevue Literary Press Rovira i Virgili University Press
Greenleaf Book Group BookViewCafe Plume
Bantam Dell Doubleday Books Maupin House Publishing
St. Martin’s Minotaur Zed Books Nilgiri Press
Great Potential Press Putnam Books Riverhead Books
St. Martin’s Paperbacks Monstrosities Books

Labels: early reviewers, LTER, member giveaways

Tuesday, April 5th, 2011

Lexile Measures in LibraryThing

We’ve just added MetaMetrics’ The Lexile Framework® for Reading, commonly known as “Lexile measures,” to LibraryThing. These offer another way for members to view their books—this time by reading level.

The feature. You can look at pages for any Lexile, or for a range of Lexile measures.

We’ve also added a view of all your Lexile measures:

You can, if you’d like, add the Lexile® measures column in Your books for easy viewing or sorting.

The Lexile measures in your catalog are based on more than 115,000 ISBNs to which Lexile measures have been assigned by MetaMetrics.

Background. Since LibraryThing was created we’ve drawn interest from teachers and school librarians. Our ease of use and advanced features have led a number of small schools to use us as their primary catalog, along with numerous classroom libraries and other collections. Many have, however, asked us to add something provided by other school-library systems, like Follett and Alexandria, namely Lexile® measures.

Lexile measures are based on the comprehensibility of the text—the lower the Lexile measure, the easier the book’s text is to comprehend. The official Lexile scale ranges from 200L to 1700L (see the Lexile map [PDF] for example texts), though actual Lexile measurements in LibraryThing range from 0L to 2000L. Check out Lexile.com for more on Lexile measures.

If you don’t want it… We recognize that Lexile measures are neither comprehensive or universally appreciated. We want to make them available to people who will find them useful, but hope they’ll be unobtrusive to others.

Come talk about it in the New Features group.

[Note: This post has been updated]

Labels: new feature, new features