Matasano Chargen - Latest Comments in Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

joe — Thu, 04 Jun 2009 10:31:57 -0000

Sounds like you are a database "expert" and live in your tunnel vision of the database is the center of the universe. lol.
But in my experience dba's and security experts tend to over-engineer and be paranoid un-necessarily, and like to restrict other from doing useful work.

But i liked the article and it made sense to follow, except for the crypto details which went over my head.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Andre — Sun, 24 May 2009 14:48:08 -0000

@James,

Firstly, how is it the user's fault if database readers can see their password information? If your bank stored your bank password in plaintext, is it your fault? And are you actually suggesting that banks and companies like PayPal store their user credentials in plain text?

Secondly, how does a timer delay prevent delays in brute-forcing or using pre-imaging attacks offline? The assumption here is that password database has been read, and now the hashes exist on attackers machines. All the timers in the world won't help you with anything.

If you want to continue living in the fantasy land where no system every gets compromised, please have the decency to do give us your full name and the name of the company you work for, so none of us ever hire you or purchase something that may have been built from you.

And please, if you are an expert in database security, please point us to your blog so that we can share in your wisdom.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Brian Tung — Wed, 06 May 2009 17:10:53 -0000

In my experience, security experts are more likely than others to rely on the advice of other security experts. At a very high level, security really is a "weakest link" property, whereas other properties are typically more "average link." As a result, whereas you may only have to fix the top two or three bottlenecks to get acceptable performance, you typically have to fix *all* the security flaws to get acceptable security--speaking very broadly, of course.

And the array of places where security flaws can crop up is breathtaking in its variety and depth. The various cryptographic primitives may have flaws. Their implementation in a particular language may have flaws. Their compilation and deployment on a particular computer architecture may have flaws. The security protocols that use the primitives may have flaws. Their use of the primitives may have flaws. *Their* implementation and deployment may have flaws. The security system used to protect the computing resources may (will) have flaws. The browser may (will) have flaws. Etc., etc., etc. Even the list of places to look for flaws will have flaws. The notion that any individual person, or even a small team, could have enough expertise and unflagging attention to comb reliably for all of these possibilities is laughable. It doesn't matter how intelligent or informed or educated or experienced they are. They simply won't catch everything, and everything is what needs to be caught. That's really why security through obscurity won't work. It's not just that obscurity gives you a faulty impression (though that is true, too). It's that obscurity means unresearched, and unresearched means undiscovered flaws. Until they are, with your pants down.

BTW, hi, Thomas. It's been a long while, and I don't know if you remember me. Heck, I don't even remember when we met, specifically.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Clayton Hughes — Mon, 04 May 2009 19:57:55 -0000

I'm amused to see this thread still running after years. I wish I could tell James why he's so completely wrong, but all I can come up with is "but you're sending it in plaintext, and anyone could be listening."

It seems there is a lot of interest among everyone but Mr. Ptacek in designing their own system. I believe that's the tendency of smart people and they "didn't build it; can't trust it" fallacy. Sadly, that attitude is prominent among developers - perhaps the case is different in the field of "security experts?"

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

James — Sun, 03 May 2009 01:22:35 -0000

This is the stupidest article I've ever read.
Store the password AS IT IS, and SECURE YOUR DATABASE. It's that simple.
If the data in your database is stolen, the ADMINISTRATOR HAS already FUCKED UP.
If users care that employers see their passwords in the database, the USER HAS FUCKED UP.

You want your verification to run slower? USE FUCKING TIMER TO DELAY THE RESULT! You don't need bcrypt.

Users must use a password manager, and generate/use huge random passwords. Only retards memorize simple passwords and use them over and over again; such people deserve to have their shit stolen.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Derek — Wed, 22 Apr 2009 14:12:23 -0000

I like reading these kind of articles mostly because I'm curious about the theory behind encryption and hashing. (I've never actually implemented a password system before). One technique from this article that I didn't know about before is using a seperate salt (the nonce mentioned) for every password. Then to do a dictionary attack, every password must be attacked seperately.

I liked the fact that the author stressed using a proven password system instead of just throwing together your own. If you provide a service that you feeling a compelling reason to provide password secured user accounts you might as well do it right. OpenId may also a good option if you don't want to have to worry about implementing managing user passwords and authentication yourself. I think in general, far too many online services require passwords and user accounts. This page did it right in not requiring me to make an account to post a comment.

In general, good article. Educational and practical.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

guy from binrev — Tue, 14 Oct 2008 03:49:11 -0000

To Vaughn: the system seems alright in principle. Correct me if I'm wrong, but this is your algorithm: 1. take user password. 2. hash with user-specific salt using MD5. 3. hash that with another salt using SHA-256. 4. store into DB. I like your idea of having two salts, but it would ultimately be unnecessary. You just need a good random salt for each user, and u don't have to put salts in different places. Your not protecting them: ur protecting passwords. Also, avoid MD5. If u want to use two different hash functions, use RIPEMD-160 as other one. Better than MD5. Example for a document I hashed and protected w/ two:

sha2: 8Z1ZCnhVnFi/FnkLMUtk4DMytgGGGspGR0j0EAuBd1E=
salt (random.org): 1kfzD3z3txrp6qYs
ripemd-hmac (sha2value+salt+password): vOJaZnRsgyhJvMjgkJU/airz92U=

You don't really need this for a password system. If u want to use two, just hash password and big salt with RIPEMD-160, then hash that a bunch of times with fast one like SHA-1. Should defeat most attacks... esp brute force.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Kari Pätilä — Thu, 03 Jul 2008 14:54:30 -0000

Any opinions on http://www.openwall.com/phpass/ version of bcrypt?

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Jörn Zaefferer — Thu, 26 Jun 2008 09:05:13 -0000

I'm currently evaluating jBCrypt for my Java web application password storage. What I find confusing is that the checkpw-method uses the hashed password as the salt to hash the plaintext password.

I guess it stores the salt as part of the hash, which contradicts what I've read here (eg. store the salt in a different DB column).

Can someone explain that, maybe recommend a better alternative?

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

CJ — Wed, 21 May 2008 08:51:10 -0000

Sorry Vaughn, as you've put plenty of effort into that evidently. I have no authority to comment on it, but (as above in reply to others) even a pro can only make one quick (or alternatively, cost-effective) judgment: there's a better (more dependable) choice.
One of the number of points the author made on which there seems full agreement is: it doesn't matter how nifty one's newly invented algorithm seems, because you're not going to know for sure until it's too late. What matters is you have a strongly reviewed system using best-practices and (in case the password is valuable elsewhere) known very secure algorithms implemented with well-proven components.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

vaughan — Sat, 03 May 2008 06:32:49 -0000

so how would you rate this system for storing passwords?

example code below.

when installing the software (this is web software programmed in PHP, not a PC application). a unique 64 character random string is generated. we store this as $mainSalt, and it is then written to a file with a unique encrypted filename & stored outside of the document root.

$salt is also generated with a randomly generated 64 character string.

so yes we use 2 salt keys, 1 stored outside the web root in a file with a randomly generated filename.

the 2nd salt is stored alongside the hash in the db for that particular user.

the 2nd salt key '$salt' is unique to that particular user. and is regenerated on each change of password.

so then we have our encrypt function.

function icms_encryptPass($pass, $salt)
{
$mainSalt = ICMS_DB_SALT;

$pass_hash = hash('sha256', $salt.md5($pass).$mainSalt);

unset($mainSalt);
return $pass_hash;
}

$pass = plaintext password entered by user.

$salt = a random 64 character key uniquely generated for that user (stored in the users DB table, regenerated on each change password)

$mainSalt = a randomly generated 64 character seed that is stored in a file outside of the web root in a file that has a randomly generated filename.

ICMS_DB_SALT is defined as the 64 character seed that is generated and stored in file outside the web root.

by using 2 salts (1 which is stored in a file outside web root), the 2nd being unique to each user, makes it that no 2 users will ever have the same password hash even if they have exactly the same plaintext password.

the salt keys can be upto 255 characters each.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Tyler — Thu, 10 Apr 2008 22:34:06 -0000

Oh, and awesome article by the way. I'm not self-absorbed, I swear.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Tyler — Thu, 10 Apr 2008 22:33:15 -0000

I am doing some work on a login system. Could someone point me in the direction of some adaptive hashing resources? I have been doing a little research and I can't seem to find much. In fact, I made a thread on devShed and it's already the #2 in a google search for 'adaptive hashing.' Unfortunately, no one has even replied yet.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

davorin — Fri, 07 Mar 2008 18:20:59 -0000

"In other words , it is dangerous to store two differently generated hashes of the same password —"

if you are using the same algorithm to get the hashes, i can't see how you can get different hashes for the same password. Hashes would be useless then. However, you could get the same hash for two different passwords, but that is highly unlikely (even more when using salts).

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Stephan Wehner — Thu, 24 Jan 2008 01:53:54 -0000

I'm looking for references/comments about storing two hashes of the same password.

If f1, f2 are two one-way-functions in the sense that one is satisfied they cannot be inverted within reasonable time. Then it seems to me the function x->( f1(x), f2(x) ) will not necessarily have that property.

In other words , it is dangerous to store two differently generated hashes of the same password --

Is that "true"? Are there examples from real-life?

-- Stephan

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Patrick — Sat, 19 Jan 2008 13:05:16 -0000

The sole reason you store a hash instead of the password in clear-text in the database, is as everybody knows, because it shouldn't pose any danger even if the data were to leak. Why wouldn't md5 hashes do the job?

The only ways to crack these hashes are by bruteforce since dictionary attacks and rainbow tables are only used to get ridicously easy passwords. If you use good salt, there is no way to bruteforce the password. Consider the following:

MD5(MD5(PASSWORD)+PASSWORD)

This would make it impossible to bruteforce, since it will contain more than 20 different characters. And since you were mentioning that MD5 is too fast, well you can always do the following.

DO 50 Times:
PASSWORD += MD5(PASSWORD)

PASSWORD = MD5(PASSWORD)+PASSWORD

Please explain to me how that is possible..

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

sbell — Wed, 21 Nov 2007 20:40:49 -0000

> Thomas Ptacek September 11th, 2007 11:16 pm: I’ll post a correction on the rainbow table attack description.

But you haven't?

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

davorin — Tue, 06 Nov 2007 14:37:56 -0000

aran, of course, i've read the article. but all the comments left me a bit confused so i wanted to clarify ;-)

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Aran — Tue, 06 Nov 2007 11:41:54 -0000

davorin, the point of this article is that some popular hashes DO NOT protect you from someone stealing your DB and reading passwords. If your passwords are stored in a weaker encryption scheme, then someone who gains access to your DB table data could possibly figure out what your password is based on the hashed string. They could use a brute force attack, or use precomputed rainbow tables for a faster attack.

The point of salting, is to make it more inconvenient for a person trying to decrypt a long series of hashed passwords.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

davorin — Sun, 04 Nov 2007 16:13:29 -0000

well, from what i think i know, hashes only protect you from someone stealing your db and reading your passwords. if you store hashes instead of passwords you're safe because they can not invert hashes back to passwords. but hashes aren't protecting you from brute force or any other attacks, just from someone looking at your stored passwords should they gain access to your db.

salt makes passwords stronger (more variation) and also unique (eg a rand() hashed, concated to the password and everything hashed again, then finally salt is appended to the hash so you can extract it for comparing).

and if you don't want passwords to be seen in plaint text between client and server you must use https.

well, at least that's how i see it...

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Sam — Wed, 24 Oct 2007 03:16:45 -0000

Couldn't you just do a little constructive coding and make MD5 work for you.

1. Reverse the hash before storing.
2. Salt the hash using say 100 unique salts, then server brute-force the password with those salts when validating. (Wouldn't require storing the salt)
3. Hash the password, and loop on the result as many times as you wanted, though once would suffice.

I would think that doing any one of these things would negate the flaws of MD5.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Ben — Mon, 15 Oct 2007 15:35:16 -0000

Matt,

The reason character sets aren't discussed is because they aren't vitally different from a somewhat longer password. After all, you could always think "iluvCgraveaacute"... instead of using the fancy ASCII characters you typed.

In the end you discuss number of bits of information content, and a larger character set simply means those bits are stored more compactly.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Matthew — Sat, 29 Sep 2007 15:41:05 -0000

Okay guys, I know absolutely nothing about cryptography, I have only heard the words md5, blowfish, etc. No research done what so ever.

I am only here because I came across a rar 3.x file with a password on it, but gained interest in the thread and read well 9/10th's of it anyways.

But what got me thinking was when someone said "Think like the attacker".

I keep hearing iterations and hashes and rainbow tables for keysets, etc.

like 15 character passwords and the like but what I don't understand is this, expand the base character set. Umm, okay don't flame me because I admit i truly don't have any idea what i'm talking about, (really I don't).

But in my mind you could have a password of 1 character, and i have no idea how many characters are in like unicode or whatever. But say this.

Build yourself a Brute force program, and Dictionary program and 100 petabyte rainbow table dictionary "thing" and i guess use your standard keyboard as I saw someone put it, like 0123456789abcdefgh...

But if your password was iluvĈăŖŚ, they'd never get it cuz they dont' figure anyone uses ascii symbols BUT if they did, there rainbow tables would have to include all ascii symbols same with brute force attacks and the like and then things would be SEVERELY timely for them.

ok, ok, ok, I know no one is going to type ascii symbols into there password but I would (do when allowed)

Why I don't know, but a Suse Linux guru I used to work with did it and I thought it was a great idea.

Sorry for dumbing the current running IQ of your board down, but I didn't hear anyone talk about charsets enough so I thought I might say something, keep the thread going, you know :)

Have Fun everyone

Matt :-þ

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Scrod — Sat, 22 Sep 2007 13:57:27 -0000

I concur. What's wrong with PBKDF2 to compute an iterated, salted key with SHA1 as the message digest algorithm? This combination seems like the most obvious solution to me, as both are widely-used standards and have received the most attention from security experts. And if SHA1 is of concern, I would suggest Whirlpool, which is nice and slow.

Re: Enough With The Rainbow Tables: What You Need To Know About Secure Password Schemes

Luis Bruno — Wed, 19 Sep 2007 18:12:07 -0000

Care to expand on PBKDF2? I -- one of the idiots on reddit talking about cryptographic hashes like they were all the same -- just stumbled upon PBKDF2, via Wikipedia.

AFAICT, I can replace your advice to "use bcrypt" with "use PBKDF2" and it's still the same, right?

Or is PBKDF2 used in a different scenario?

Many thanks for a great article!