Tuesday, February 4th, 2014

Security Notice and LibraryThing Password Reset

As a security precaution, we are requiring ALL members to change their passwords, here: http://www.librarything.com/changepassword.php

A security review and search of our records has determined that LibraryThing suffered a data breach in November of 2011. The breach was narrow. We have found no evidence that any catalog or other book data was accessed, changed or lost. The breach did not include member names, so it is unlikely the hacker(s) were after LibraryThing accounts.

Unfortunately, the hacker(s) did assemble and retrieve two key pieces of data–email addresses and encrypted passwords of members who had listed an email and joined before that date. Our passwords are stored as a one-way encyption (in technical terms, a salted hash). Such hashes are difficult, but not impossible, to break, especially for simple passwords.

Although a minority of accounts were affected, we are requiring all members to change their password to take advantage of increased account security features.

The breach. The hacker(s) gained partial access to our system through a flaw in our WordPress blogging software. Read more in “The Full Details” below.

All evidence points to this being an email-hacking attack. We have every reason to believe no other LibraryThing data was taken, not even user names. The intent was probably to grab the emails for spam, and break the password hashes, if possible. When broken, the passwords could be used against members who used the same password for their email, or email-based services, as they used on LibraryThing. Using the same password across many services is bad practice, but not uncommon. No financial data could have been taken. We do not get or store credit card numbers or any other financial information.

Our response. Security has been tightened significantly since late 2011, and has been further improved across the board since we discovered the event during a security review on January 21st, 2014. We have now moved our WordPress blog off our servers entirely, so a successful hack leads nowhere. Our password and account-recovery systems have been upgraded to meet the highest industry standards, and we have implemented a slate of additional security measures.

Email notices are being sent out to all members with email addresses. You can change your password any time; you don’t need to wait for the email.

Our apology. The hack may come as a shock; it certainly was to us. Although events of this sort–and far worse–have become numbingly frequent, they are failures indeed. I regret and apologize that any such event could happen on my watch, and the rest of the team feels the same way. We are all committed to ensuring that LibraryThing is as secure as possible going forward.

We hope this failure will not sour you on our service or community. LibraryThing members are a dedicated and passionate bunch, and a pleasure and honor to serve. After years of getting by, the company has significant profits to sink back into development of the main site; we will meet this event with renewed dedication and resources. (Please see, and spread, our recent job ad.)

Because the hack undermines a customer relationship, we have chosen to upgrade to “lifetime” accounts all members who joined before November 20th, 2011. We included those who did not have email addresses listed.

Come ask questions and discuss on Talk. You are also welcome to email tim@librarything.com.

Sincerely,

Tim Spalding
Founder and President


The Full Details

What happened:

  • The hacker broke into the system through a flaw in WordPress, the blogging software that we use. This gave them only partial access to the system, but was sufficient to query the user database and save the results.
  • The breach occurred on November 19th, 2011.
  • We have no evidence of further data breaches. They are not impossible. We are confident no similar attacks could have taken place since at least January 2013, when we added some specific security features.
  • We discovered the breach on January 21st, 2014. As it happened so long ago, we believe whatever damage could be done, has already been done.
  • We waited two weeks in order to understand the attack and to implement a new password system and a series of other security steps before going public, and potentially drawing hacker interest.

What was taken:

  • The hacker(s) exported three fields: email address, password hash and the IP address at sign-up. (The IP would not be of much use to them.)
  • Only members with accounts opened before November 20th, 2011, with email addresses, were affected. In total, 685,259 emails were exported.
  • We have no indication that other LibraryThing data was accessed or taken. It is significant that the hacker didn’t even export LibraryThing user ids or user names. They were surely after emails and passwords, not book data.
  • LibraryThing does not receive or store credit card information or any other financial details. If you registered for a paid account via PayPal, PayPal has your credit card information, but they do not send the numbers to us.
  • We have reasonable suspicion that someone has used the data as a list of live email addresses, and sent spam to them. We have no evidence that any password hashes have been broken, or LibraryThing accounts compromised.

How passwords work:

  • Systems like LibraryThing do not store passwords per se. Rather, we store complicated cryptographic transformations, called hashes, which are “salted” for increased security.
  • In theory, you cannot get from the hash to the password. In practice, hackers with powerful computers can break hashed passwords, especially if the underlying password is simple (e.g., “book” rather than “mypencilbreaks71” or “xyA1!oG3g”).
  • Hacked passwords are dangerous when someone uses the same password across multiple online services, so failure at any one service opens up the rest.
  • Members should change their password at LibraryThing and any other service on which they used the same password. Here and elsewhere, members should also choose longer, hard-to-guess passwords. We encourage you to read safe-password advice from Google or Twitter.

Security improvements:

We can’t go into detail about security improvements. (If we did, we’d be compromising security.) But we can say what you can see:

  • We have moved our WordPress blogs off LibraryThing servers entirely, and onto a separate subdomain, blog.librarything.com. This insulates us from potential WordPress problems.
  • We have upgraded our password system to the highest industry standards. Users who joined in the last week or so, or changed their passwords, are already on the new system. But for simplicity’s sake, we’re requiring everyone to change their password.
  • We have a new system for password resets and changes, including password-strength indicators.
  • Our password recovery system has been changed from one involving sending out a temporary password to one employing quick-expiring tokens.
  • We are now sending out emails whenever a password has been changed. When a member changes their email address, change notices go out to both the new and old email addresses.
  • To discourage spamming of public emails—something that happened recently to some members—we have added an option to show your public email to friends, to signed-in members or everyone. By default, everyone who formerly chose to display their email publicly will now be set to friends-only.

Free accounts:

  • All members with accounts opened before November 20th, 2011 have been upgraded to lifetime accounts.

Labels: security, sysadmin, systems adminitration

87 Comments:

  1. Morphidae says:

    The password strength indicator wasn’t working when I changed my password.

  2. JonB says:

    The email notification says that the data breach occurred in June 2011 but the Blog entry says November 19, 2011. Were there two breaches?

    Thanks for taking the action you have taken.

  3. Ambroise7021 says:

    Thanks for the info.
    My password is stored in my PC, and I dont remember it, but I’m still actively using it!
    Can you please send me a recovering process.

    Thanks

  4. Laura Brown says:

    Do you know where the hackers were located? I remember getting a notification from Gmail at around that time that someone from Mexico had tried to log into my account. At least now I know better than to use the same password elsewhere!

    • Loranne says:

      Saw your question in Talk first, but for the record, I’ll post here, as well.

      Unfortunately, we don’t have any other information about the individual(s) who did this.

  5. Michael says:

    Like Ambroise (above) I have long since forgotten my password – and log in automatically … Hmmm

  6. Rob says:

    I’m trying to update my password from the email link but that requires my original, which I don’t remember. So I do a password reset but that says “Your token has expired.” even though it is only a couple of minutes old.

    So, how, exactly do I reset my password?

  7. Matt Gibson says:

    Out of curiosity, was it a security flaw in the WordPress core, or in a plugin? If it was in the core, was WordPress up to date at the time? Thanks!

    • Felius says:

      You can change the URL by adding https:// on the front. There are a couple of images which are being redirected from a secure to an insecure URL, but the actual password will be transmitted over an encrypted connection if you do it this way.

      They should force this page (and any other that handles authentication) to redirect to a secure URL by default, however.

  8. Ine says:

    You know, saying that you have significantly improved your security measures, and then asking people to enter their passwords on an UNENCRYPTED page, is kind of… let’s just not go there.

    Security certificates. HTTPS. It’s a thing. Look into it.

    • Felius says:

      You can change the URL by adding https:// on the front. There are a couple of images which are being redirected from a secure to an insecure URL, but the actual password will be transmitted over an encrypted connection if you do it this way.

      They should force this page (and any other that handles authentication) to redirect to a secure URL by default, however.

    • karina says:

      No, let’s DO go there. Https IS a thing and without question these ‘new improved security’ measures should utilize this technology by default.
      (Um, we lost your email/password, please trust us not to compromise your data again…oh by the way we want you to give us the new pw out in the open…???)

      Response to:
      “…Let’s just not go there.”

    • Yikes. Good catch. I checked to see if the HTTP URL redirected to HTTPS, but it doesn’t.

      This is fail on so many levels. Why did it take *over 2 years* to notice the breach? Why isn’t LibraryThing using HTTPS for a password change process (or indeed for the whole site)? Most importantly, given the combination of those two factors, why should I ever trust LibraryThing’a security again?

  9. Joy says:

    I am trying to reset my password and it tells me it can’t find my account. I know my correct password because I successfully used it to sign in. Then I logged back out and tried again to reset my password. It still says it can’t find my account. Help?

  10. Soepkipje says:

    Sorry to hear this and not really a present if you’re as I (an paying) customer from the very beginning! :-((

  11. Ryan Grove says:

    Two questions:

    1) What hash algorithm was used to hash stolen password data? How were these hashed passwords salted? This has a huge bearing on how easily those hashed passwords might be to recover.

    2) What exactly are the “highest industry standards” that are used in the new password system? bcrypt? scrypt? Something else? Without details, this statement is meaningless.

  12. Rudy says:

    Thanks for the warning and the honesty. It’s very much appreciated.

  13. Guy Rintoul says:

    Credit where credit’s due – obviously you don’t want to be hacked in the first place, but this feels like an open and honest response, whereas many other sites try to cover this stuff up. That’s appreciated!

  14. Stuart Jones says:

    I tried to change my password but it says that my current password is incorrect.

    I tried to reset my password, but when the reset form submits it says “no data submitted” – this is in Chrome and FireFox.

  15. Murphy's Lawyer says:

    Thanks for letting us know.

    This is why I use a different password for every. single. online. account I have – makes something like this a minor annoyance instead of having to spend the next two days changing everything.

    This is why I don’t log in via FarceBook either.

    Noted Ine’s comments above re. not using HTTPS to change passwords. How much is it to get an Extended Validation certificate from a reputable vendor? You may find it worth your while?

  16. too little too late says:

    Is there a way to delete an account without logging in?

  17. Karen Schulz says:

    don’t know my current password or member account. Any suggestions? I tried sending my email, but it says no data submitted…I didn’t use the site much, but I don’t care to be locked out because of that…

  18. Tim Spalding says:

    The “no data submitted” error has been fixed. Thank you.

  19. John says:

    Good response, y’all. Of many Websites I’ve used over the years, you are one of the most responsible and one of the few who never baffles its users with bull. Thanks for your wonderful service. I’m proud to be a lifetime member since the day I joined.

  20. Alex says:

    over 2 years to notify users that site was breached? Facepalm

    Asking users to change passwords on non https site? Double Facepalm

    • It sounds more like 2 years to *discover* that the site was breached, and 2 weeks to notify.

      That said, as a security programmer, I’d really like to hear what logging procedures allowed knowledge of the initial breach to be captured, and what security review procedures allowed discovery of the breach.

  21. brittany says:

    soooooo this happened in 2011 and you’re just now telling us?

  22. Karen Anderson Keith says:

    Thanks for your prompt action and notifying us. I know you must be devastated, but try not to kick yourselves. LibraryThing is one of the most precious sites on the net (IMHO). So keep up the good work. Karen

  23. flowdam says:

    I paid for lifetime membership back in 2008.

    What are you going to do to recompense me for this breach?

  24. Although at first I was a bit dismayed to see the 2+ year delay in reporting this (and wrote about it online) I still think LibraryThing is doing a great job being transparent and interacting with their customers on this issue.

    Hopefully the worst damage will be just a bit more (library or book?) spam for us members.

  25. Misti says:

    What about those of us affected by the breach who already had lifetime accounts?

  26. Tracey Stewart says:

    2011. And I thought Target’s failure to notify people of a breach in November /2013/ was bad.

  27. Chris Dotson says:

    I just wanted to add my thanks for the way you’ve handled this.

    Of course everyone agrees it would have been better if the breach had never happened, or if you’d uncovered what happened instantly, but given that you didn’t find out about it until very recently a full explanation plus pushing a password reset was very reasonable. I suspect many places wouldn’t have bothered telling anyone!

  28. Roger Hare says:

    I cannot reset my password as requested as my original password
    Is flagged as incorrect!

    This is very annoying as I have recently changed all my email
    addresses for other reasons, so this is all unnecessary as far as
    I am concerned.

    Please fix this so I can reset my password.

    RJH

  29. Abbey Sparrow says:

    What kind of hash? and were the passwords salted?

  30. Timothy Clough says:

    I’ve changed my password three times, but I can’t log in. It keeps saying “Name or password is wrong.” Yet if I try to “change” the password to the same thing as last time, it will give me an error message saying that the new password can’t be the same as the old one.

    • Loranne says:

      Sorry you’re having trouble, Timothy. It’s true that our system won’t allow you to change your password right back to your most recent one. Unfortunately, you’ll have to pick something different.

      • Timothy Clough says:

        I forgot to add the fact that trying to “change” to the same password, and having it fail, meant that the new password was there, but that it still wouldn’t let me log in.

        Fortunately, it works now–I can log in no problem.

  31. Lori says:

    Is it of any consequence that the password change form to which the email directed me is not “secure?” Can I proceed with confidence in using this form?

  32. Walter says:

    Sh** happens. Good response. All the best.

  33. Jean-Louis Taffarelli says:

    We know there are a lot of sharks prowling on the web.
    I appreciate the way you handle things.

  34. Note that you got loads of “mixed content” (http served while the main page is https) serving from the website. For example:

    “http://ecx.images-amazon.com/images/P/0060930187.01._SY60_SCLZZZZZZZ_.jpg”

    “http://www.librarything.com/i/flags/de.gif”

    I suggest you install HTTPS Everywhere https://www.eff.org/https-everywhere and make sure no errors occur.

  35. Martin says:

    This explains why I’m getting spam to my LibraryThing e-mail address, which I’ve never ever used on any other site.

  36. drasvola says:

    This might explain the surge of unsolicited emails that I’ve been receiving! Thank you for alerting us and fixing the problem.

  37. whizse says:

    The password reset does not seem to work very well at all.

    I try to reset to a safe passsword, something with a lot of random characters like ?]Rx/8[i5HC_X~gR^5g$ (not my real password obviously) it will happily accept this, but won’t let me log in, complaining about wrong password.

    If I use the password reset again and try the same password, it complains about not letting me reuse the same password so it is correct.

    If I use a simple alphanumeric password without special characters it seems to be working.

    A bug in the login code?

  38. Barry says:

    Thanks for explaining what happened and for your positive response to the incident. I’m actually more impressed by your company now than I was before.

  39. drasvola says:

    I’m getting a second request to change my password. I did that successfully yesterday.

  40. Tim says:

    @Flowdam, @Misti. Send me (timspalding) a private message, if you will. I am happy to upgrade another account for you, or send a gift.

    @Roger Hare and some others. If you can’t remember your password, go to http://www.librarything.com/lostsomething.php to get a timed password-reset link by email.

    @martin: That’s how we discovered this initially. But we’ve also had users report their LT-only email has never gotten a single spam email. If you have the time and inclination, can you email tim@librarything.com. I’d love to pin down whehter your problem is this, or the various other ways it can happen.

    @whizse: “A bug in the login code?” — Yes. It was wrongly interpreting a particular character—not one of the ones you listed, but one that’s used. The bug is fixed and should be okay for you. Let me know if you see more problems.

    @drasvola: If it’s a second email, our apologies. There were a few doubles sent out, although most are because someone had the same email on multiple accounts. If you follow the links, you’ll know if you need to change your password. You probably don’t.

    @Finn Årup Nielsen: Right. LT has some warnings when you use HTTPS. The Amazon links are the important one. We need to investigate options, but those images don’t work via HTTPS.

    We are, however, working to shift to all-HTTPs in the near future. Give the challenges, and the breach, we did not think we could delay.

  41. Hubert Geelen says:

    Dear,
    Today i received an email about : Security Notice: LibraryThing Password reset. My ‘problem’ is: i do not know this organisation, i never used it as far as i know and i am quite sure i do not pay for any membership. Maybe i made a contact very loooong time ago, that i totally forgot ? What shall i do ? Thank you. To LibraryThing: if necessary, please contact me about it on my emailadres. Thank you.

  42. Tim says:

    @karina et al. : As you noticed, the site is not currently using HTTPs. We are moving to it, and hope to have it live soon. But it presents a number of challenges, and we thought it would be better to inform members sooner. We are probably going to start with what you suggest—some pages use it.

    The time has come, I agree. But always-HTTPs is relatively recent. Of our four closet analogues in the book world, two do not employ it anywhere. That will come off as lame to you, perhaps. As I said, the time has come.

    • Tim says:

      Correction: 3/4 do not use it. If we count the top two swap sites, it’s 5/6. We should do better.

    • Always HTTPS is not “relatively recent”. I was using it for websites years ago. If the book-swap world is not using more HTTPS, then that says more about the ineptitude of book-swap site developers than anything else.

      I will grant, though, that HTTPS is probably not *fundamentally* necessary for anything but the login page. But HTTPS isn’t hard to implement; once you have it on one page, you might as well go out in the whole site.

    • Jay Heiser says:

      Relatively recent in geographic terms, human terms, or technology terms?

      As a security professional, I don’t see a huge impact to this breach–depending upon the form of password protection used–but it does raise questions about whether you just now noticed it, or just now decided to tell everyone.

      The biggest risk for a Librarything user is not that someone is going to hijack their account. I could envision some disgruntled lover wanting to mess with someone that way, but truly, the likelihood of account sabotage seems very, very low. The much bigger risk for most people is that they used the same simple password on Librarything that they also used on something more substantive. Hackers are constantly on the lookout for passwords and user names that they can then try on other sites. SSL will reduce the potential for one form of password theft, without eliminating risks. PAsswords are inherently flawed, and no amount of any form of encryption can make them fully resistant to attack.

      This would be a good opportunity to remind Librarything users to periodically backup their database. I keep a copy on my home PC.

  43. P says:

    Plz remove us from all lists.

  44. Cannot access the DPL.

  45. Judith van Oyen says:

    Please remove me from your lists

  46. Caroline says:

    It is sad to see it happen to this site. But I am a loyal customer and will stay. Thank you for being forthcoming when you discovered the problem.

  47. eva says:

    i barely remembered having an account here… i stopped using it when i got to 200 books and couldnt add anymore…
    i already changed my email password a long time ago, am i good?

    • Loranne says:

      Eva, regardless of how long ago you last changed your password, we are requiring all users to do so now. I’m sorry for the inconvenience.

      • eva says:

        fine, fine. do i have to change all my accounts again that had the same password (like i had to do when ravelry was hacked)? that will take forever. i deactivated my librarything account, wasn’t using it anyway.

  48. Andrew T. says:

    You might consider providing the SSL link to change password, instead of the non-SSL version. This opens users up to a second attack if they’re using an open wi-fi network..

  49. Tom says:

    Thank you for your communication about the incident. As far as I can tell no serious harm done.

    cheers…

  50. KinomiyaMichiru says:

    1. I wasn’t even a member of LT till June 2012.
    2. What do we do if we use FB connect or Twitter connect to log in? Just delete the connection, and re-enable it?

  51. Lyne Jones says:

    I have changed my password. The hacking could be the reason why I am receiving emails from a bank I hav never dealt with, advising me that my e statements were available. I noticed that the address of the sender was not the correct address for the bank and therefore sent the emails to their hoax department. There was a link in the email where I was supposed to click and advise if I thought that the email was a hoax, but I did not click on this link as I said the address in the ‘sender’ line was not the correct address for the bank. They said they have had several customers complain about receiving such emails and also they have had a rush of such complaints this morning, when I received my latest supposedly email from them. So it seems that this hacking of your site may be the reason why I am receiving emails from a bank with whom I have never dealt. By the way, I am 73 years old and have been banking with another Australian bank since 1957.

  52. Bart says:

    First of all, thank you for sharing the details and warning us about the hack.

    Second, as suggested by other posts, can you please use HTTPS:// in your URLs for the password reset?

  53. […] they discovered a breach of security after someone exploited a weakness in the WordPress that they […]

  54. Dave Graham says:

    I’m having a problem understanding why some of the membership is being so pathetic about this very minor inconvenience. For example somebody wants to know what ‘recompence’ will be offered ‘for this breach’. Breach of what? What’s actually been lost apart from a few minutes changing a password. Is it just an urge by some people to make a fuss about nothing because they have such empty lives? If you use the same password for multiple purposes then you’re probably too simple to be using the internet and bank accounts unsupervised. Please don’t pander to their silliness, just offer them their subscription back and suggest they go elsewhere. Librarything is outstanding value, great fun and a delight to use and you shouldn’t be bullied by people with inflated ideas of their own importance.

  55. Beatles1964 says:

    I was able to change my password without any difficulty.

    Beatles1964

  56. […] In addition to directly emailing all members who joined before the time of the breach, they wrote a lengthy blog post  about it, including when they discovered the unusual activity, the number of possibly-compromised […]

  57. Warren Post says:

    Many thanks for your transparency on this incident. Far too many web service providers either keep silent or report such sketchy and whitewashed details as to be useless. This transparency confirms yet again what I already knew, which is that you guys rock!

  58. Anthony B says:

    I hope you are doing more frequent security reviews going forward.

  59. Suzanne says:

    I changed my password this morning. But I keep getting the message to change it as if I haven’t changed it. Argh.

  60. effrenata says:

    Now there is pharmacy spam on the topic listings.

  61. […] I was recently gifted a lifetime membership by LibraryThing, I went and cataloged all my books, or at least the 633 of them on the premises […]

  62. The password strength indicator wasn’t working when I changed my password.

  63. itacal says:

    I am extremely pissed off by this! First, this happened in 2011 and we are just now being told about it. Secondly, now I have to change, and remember, yet another goddamn password. I, like most people, have too many to remember now. I can’t help that your security was breached. I’m not changing anything. Simply refund the money I gave you for a lifetime membership.

Leave a Reply to Tim McCormack