Saturday, July 15th, 2006

Server problems abating

Stewie and Seamus, Chris’ dogs

Short version: We now have a three-server cluster. Speed has picked up. We’re hoping to solve lingering character set issues. A new employee, Chris Gann, set everything up.

Long version: Introducing… Chris Gann (LT: stalepez, chrisgannlibrarything.com), LibraryThing’s newest employee. Chris will eventually be coding, but his first task was to set up a “cluster” of powerful new servers to deal with LibraryThing’s current traffic, with a path to handling an order-of-magnitude increase in the future. He designed the architecture, ordered the boxes, set them up and arranged for the transfer from California. Chris is an old hand at this stuff. Back in 1999 he co-founded LinuxBox, a pioneering hosting facility for open-source projects, later acquired by OpenAvenue.*

The new servers have been up for almost two days—long enough to see that, at least as far as speed, things are shaping up. We’ve moved from one fairly modest server in California—a second one broke right before the Wall Street Journal article hit!—to three servers, two moderately powerful and one a “monster.” They are set up as a “master” and two “slaves.”** The master handles “writes,” the slaves “reads.” Read load is balanced between the two slaves, and they “fail over” to the other if something goes wrong. The system also provides increased data security, with complete database copies stored on three computers. It is also possible to take one server offline for backups without causing interruptions. I can also finally run statistics pages—the Zeitgeist particularly—without hiccups.

We saw an immediate improvement and speed has picked up as “caches” grew and as scripts were modified to take advantage of load distribution. (Maybe 1/3 of scripts have been rewritten, but they are the heaviest ones.) Between midnight last night and this morning, the master, “Zeus” had only three “slow queries” (11, 13 and 13 seconds respectively); everything else took less than 10 seconds. The slaves, Apollo and Athena, had zero and 12 slow queries respectively. That looked odd, so we dug into the code and discovered that the randomizing function was giving Athena too much work. I expect the number to drop as the load balances better. As for the servers presently in California, one will be charged with the worst queries—recommendations and relatedness—doing them on a schedule and caching the results. The second will become a development server, so when I try to run a “six degrees of Jane Austen” it doesn’t crash the database.

Oh, best of all, because of an order mixup, we have three more servers sitting in the LibraryThing foyer. As far as Dell believes, they don’t exist. They won’t let us send them back—the charge has been refunded too. I suspect they’ll eventually come to their senses and let us send them back. If not—hey—sever error in our favor!

Character-set issues. Users have reported problems with character sets. As the data transfer was binary, this is probably a configuration issue. (I believe the same thing happend before, and was fixed with a configuration change.) Chris is on the problem, and will report back here or on the Google Group as soon as he can.

We thank you for your patience. I can’t promise problems are forever over, but a significant step has been made. With luck, we won’t be firefighting all the time, and be able to push forward the site more.

* In a strange twist of fate, the co-founder of OpenAvenue, Jayson Minard, is now the CTO of Abebooks.
** As an American History major, these terms still give me the creeps.

Labels: 1

Thursday, July 13th, 2006

Various small changes

I have made some various small changes to the catalog page and I wanted to give you a chance to offer some feedback on these and the new catalog in general (except color scheme, that is for another discussion). Leave your comments here or bring them over to the Google Group for more discussion.

  • You can now search subjects via the catalog search box. Just select ‘Subjects’ in the drop-down menu.
  • ISBNs are now included in ‘Book’ searches.
  • Titles are now links to the social info page.
  • Multiple fixes to lingering in-place editing problems have been applied. If you know of a problem that has not been fixed, please let us know. It’s better to have too many reports than to have none.
  • The list of pages at the bottom of the catalog now displays correctly when you select “show all.”
  • Subject pages are now displayed with correct links to the global subjects and the global pages have correct display of the subject path.
  • I’d like to know if people are still getting the stack overflow errors. I applied a work-around last night but I’m not sure if it is correcting the problem for everyone.

Labels: 1

Monday, July 10th, 2006

Add books improved

I made some improvements to the Add Books tab. Notably, it now saves the libraries you use on the left, and remembers them between sessions.

We hashed many of the features out on the Google Group. But I’m still considering whether the tag box should be “sticky” or not. Oh, if you’re pining for the old one, use it here.

In a day or two we’ll be adding the most requested feature—adding multiple books at a time.

Labels: 1

Monday, July 10th, 2006

Shilling for Portland’s Longfellow Books

LibraryThing hearts Longfellow Books in Portland, Maine. Tim’s wife Lisa reads there. Axel, Stewie and Seamus gets dog treats. They even compliment Abby on her book choices. So, in our continuing quest to get more Portland, ME members–to invite for burgers–we’ve given Longfellow a stack of free gift accounts. If you’re local, stop by and pick one up.

Link dump:

Labels: 1

Friday, July 7th, 2006

Firefox Extension: LibraryThingThing

LibraryThingThing is a complex three-API Greasemonkey mashup, querying the Holding Lookup Service along with LibraryThing’s thingISBN and OCLC’s xISBN service. It makes my head spin a bit. Three cheers for Richard. Too bad he works for Talis, or he might have won the Mashing Up the Library Contest.

LibraryThingThing can be found at: http://www.talis.com/tdn/greasemonkey/librarythingthing

This is an exceedingly cool mashup, and a very good demonstration of all the components. To my mind, it would be more useful if it did less, telling you only if the book was in your library. Do you agree? How should LibraryThing tie into libraries. As always, your thoughts are much appreciated.

We were, actually, planning on doing something like this, and even started the code. When we bring something live it will be a lot less technically elegant—good old server-side programming—but also not browser- and extension-dependent.

In other news, Chris just came by to grab the ginormous server box. Abby took a picture of it, but her camera uses some arcane memory stick format, and she forgot the cable is too modern for Tim. We have photographic proof of a new server—really!

Labels: 1

Friday, July 7th, 2006

Server update / Cape Elizabeth, goats, pigs do the LibraryThing


Neither Zabby’s Traveling Farm Animals nor this pot-bellied pig endorse LibraryThing.

Tomorrow from 9-1:00 at Cape Elizabeth, Maine’s Thomas Memorial Library there will ice cream, crafts, a tent, face painting, magic lessons, a professional balloon-twister, a “standup chameleon,” and animals from a traveling petting zoo—including this pot-bellied pig—AND if that were not enough, Abby and Tim giving out free LibraryThing accounts!

The Thomas Memorial Library Foundation is sponsoring our table at a celebration of the 20th anniversary of the library rennovation. And free accounts will be available at the circulation desk for the rest of the year. With some luck, we’ll pick up some local members. If you’re a local, but not a Cape Elizabethan, stay tuned; we have some other local-area plans in the works too.

Meanwhile, while we’re with the goats, Chris is busy working on racking and synching the new “monster” server. By Friday we should have three servers online, and four or five by the Friday after that.

Labels: 1

Wednesday, July 5th, 2006

Big catalog update / welcome Christopher

Please extend a warm welcome to Christopher Holland, LibraryThing’s newest employee!

To kick things off Chrisopher has given the catalog major facelift and upgrade.You can now edit book information right on the screen, without using the “pencil” icon. Just double-click the cell and an edit box appears where the content was. The new interface also has a search box visible at all times. (I realized we needed then when, to my surprise, the search function turned out to be one of the top-hit pages.) Oh, and a new, soothing color.

Double-click to edit
Edit and save

The new control bar:

About Christopher. Christopher (LibraryThing conceptDawg) is a technology consultant by trade but studied fine art, graphic design, and digital media. He is also heavily involved in digital collections research in the area of the humanities, specifically in the field of archaeology. He is currently working on another project that is similar to LibraryThing, only it consists of archaeological data and objects from numerous museums and research projects. Christopher is also an avid painter and photographer and is a regular on the Technique forum at Flickr (username conceptDawg). His family is very book-oriented and his grandparents are collectors (soon to have their collection on LT).

Christopher can be reached at christopherlibrarything.com.

Come give us your thoughts, on the comments here or on the Google Group.

To compare and contrast here’s a link to the old catalog. You’ll need to get back into the new catalog after that.

Note on editing.
The in-place editing works for all editable fields. Okay, that’s a tautology. Basically you can’t change the source library, entry date or LC Subject Headings. Deweys and LC Call Numbers, however, are fully editable, including the “green ones,” which represent LibraryThing’s “best guess,” based on work-level information. By editing green fields you move it from a guess to your own data.

Labels: 1

Tuesday, July 4th, 2006

July 4th Tag Cloud

Labels: 1

Friday, June 30th, 2006

Scheduled downtime: 12am EST

LibraryThing will be going down for some well-deserved rest at 12am EST. I expect the downtime to last between 30mins and 1 hour.

Labels: 1

Tuesday, June 27th, 2006

Slow Afternoon

The site is slow now – we know, and we’re working on it. We got an influx of users today, thanks to the Wall Street Journal article and a Boing Boing reference. Traffic is ridiculously high (almost 15 times normal) and we’re working on adding servers to compensate. Stay with us while we grow over the next few hours!

TIM UPDATE: Go away! Wait, that’s unfriendly. We love you. If you go away, we’ll love you even more when you come back–we’ll kill the fatted calf!

But seriously, things are calming down a bit, but it’s still running past capacity. Don’t despair. Server upgrades are coming. Unfortunately, I can’t just throw money at the problem. But I’m doing that anyway :)

Labels: 1