netnix.org
Networking and Unix


AES Encryption with HMAC Integrity in Java

 April 19th, 2015Apr 19th, 2015        20

UPDATED: 12th August 2016

I recently came across a requirement to provide password based encryption and decryption of data in a Java program. I initially assumed I just needed to pass my data through some internal Java “encrypt (plaintext, password)” type function and all would be fine. Unfortunately I found it isn’t quite as simple as this and there are quite a few pitfalls you need to overcome if you want to do this securely and properly.

I also wanted to work within the limitations of Java and only use native libraries (e.g. “javax.crypto”), which rules out the popular Bouncy Castle cryptographic library – rolling your own crypto functions is also a very bad idea (repeat “very bad idea“) as even the experts can get it wrong sometimes. I also wanted to ensure it worked with Java 7, which rules out some of the newer more modern modes of AES like GCM (Galois/Counter Mode).

After spending a considerable amount of time reading numerous posts related to this topic on StackOverflow, mostly on how not to perform encryption, I came to the following conclusions:

  1. Java only supports AES encryption with 128 bit keys out of the box – if you want 192 or 256 bit keys then you need to install the “Java JCE Unlimited Strength Jurisdiction Policy Files” for your version of Java. Every user of your tool will need to install them as well.
  2. You should be using PBKDF2 (“bcrypt” and “scrypt” might be better alternatives, but Java doesn’t support them) to generate your secret key which takes your password and a pseudorandom salt and passes it through lots of iterations of a hashing algorithm to produce the final key. This can be time consuming and is supposed to make it time consuming for an attacker to brute force it.
  3. AES encryption on its own doesn’t provide any integrity of the data (unless using GCM mode to provide Authenticated Encryption with Associated Data – AEAD) so it is recommended to use something like HMAC-SHA-256. The HMAC should be applied after encryption (Encrypt-then-MAC) to provide protection to Padding Oracle Attacks. When applying the HMAC it should be across the Salt, IV and Ciphertext – a common mistake is only performing it on the Ciphertext, which means any changes to the IV which alter the Ciphertext won’t be noticed.
  4. You should really be using separate keys for AES encryption and the HMAC – if one of the algorithms is compromised then both are. You can either generate a larger output from PBKDF2 and split it into multiple keys (HMAC-SHA-512 to produce a 256 bit AES key and a 256 bit HMAC key) or you can generate separate keys from a master key using something like HKDF (RFC5869), but Java doesn’t provide an implementation.
  5. The pseudorandom salt should be a cryptographically secure random number and should be at least the block size of your hashing algorithm (i.e. 160 bits for SHA-1). This requirement is referenced from RFC5869 (HKDF) which states “ideally, the salt value is a random (or pseudorandom) string of the length HashLen”.
  6. Your key length should match your encryption/HMAC algorithm (i.e. 128 bits for AES-128 and 256 bits for HMAC-256).
  7. If you are using the same secret key to encrypt multiple strings you must be using a random Initialisation Vector (IV) every time – this prevents the same key generating the same ciphertext from the same plaintext. You will also need to include the IV with the salt (they aren’t considered secret) when you store it as you need the same IV and salt for decryption. If you never use the same key twice then an IV is fairly redundant.
  8. In the same sense, you should avoid using AES in ECB mode (as identical plaintext, when encrypted with the same key always outputs the same ciphertext) and stick with CBC (with PKCS#5 Padding) or CTR mode, but it really depends on your application.

For the benefit of anyone who is interested, I have adopted the following encryption and decryption routines in my Java programs. This should help anyone who is struggling with implementing encryption as well as allow anyone to comment on what I have done for everyone to see.

Key Derivation Function

public byte[] deriveKey(String p, byte[] s, int i, int l) throws Exception {
  PBEKeySpec ks = new PBEKeySpec(p.toCharArray(), s, i, l);
  SecretKeyFactory skf = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
  return skf.generateSecret(ks).getEncoded();
}

The above function take a password (p), a salt (s) an iteration count (i) and a key output length (l) in bits. It then uses PBKDF2 using an SHA-1 HMAC to generate a key. Unfortunately Java 7 doesn’t support PBKDF2WithHmacSHA256 and it isn’t recommended to generate a larger output key than your hashing algorithm (i.e. 160 bits when using SHA-1) as it will result in multiple executions of the function at a computational cost to the defender. It will return a key of size (l) in bits.

Encryption Function

public String encrypt(String s, String p) throws Exception {
  SecureRandom r = SecureRandom.getInstance("SHA1PRNG");

  // Generate 160 bit Salt for Encryption Key
  byte[] esalt = new byte[20]; r.nextBytes(esalt);
  // Generate 128 bit Encryption Key
  byte[] dek = deriveKey(p, esalt, 100000, 128);

  // Perform Encryption
  SecretKeySpec eks = new SecretKeySpec(dek, "AES");
  Cipher c = Cipher.getInstance("AES/CTR/NoPadding");
  c.init(Cipher.ENCRYPT_MODE, eks, new IvParameterSpec(new byte[16]));
  byte[] es = c.doFinal(s.getBytes(StandardCharsets.UTF_8));

  // Generate 160 bit Salt for HMAC Key
  byte[] hsalt = new byte[20]; r.nextBytes(hsalt);
  // Generate 160 bit HMAC Key
  byte[] dhk = deriveKey(p, hsalt, 100000, 160);

  // Perform HMAC using SHA-256
  SecretKeySpec hks = new SecretKeySpec(dhk, "HmacSHA256");
  Mac m = Mac.getInstance("HmacSHA256");
  m.init(hks);
  byte[] hmac = m.doFinal(es);

  // Construct Output as "ESALT + HSALT + CIPHERTEXT + HMAC"
  byte[] os = new byte[40 + es.length + 32];
  System.arraycopy(esalt, 0, os, 0, 20);
  System.arraycopy(hsalt, 0, os, 20, 20);
  System.arraycopy(es, 0, os, 40, es.length);
  System.arraycopy(hmac, 0, os, 40 + es.length, 32);

  // Return a Base64 Encoded String
  return new String(encodeBase64(os));
}

WARNING: Please be aware that the above code doesn’t calculate the HMAC in the recommended way – it should include the “ESALT” otherwise changes to this value would change the decrypted Ciphertext even though it correctly authenticates!

We start off by creating a 160 bit salt to derive our encryption key using “PBKDF2WithHmacSHA1” with 100,000 iterations (160 bits because SHA-1 generates 160 bit hashes). This generates a 128 bit encryption key that we use for AES in CTR mode. We pass through an empty IV as we never use the same key twice but always generate a new salt and key before encrypting. Once we have our ciphertext (es) we then create a new 160 bit salt to derive our HMAC key using the same process – the only difference being we generate a 160 bit key (in theory HMAC-SHA-256 benefits from a 256 bit key, but Java is lacking in being able to produce that – Java 8 supports “PBKDF2WithHmacSHA256”). We then perform a HMAC operation over the ciphertext using our key. If anyone was to fiddle with our encrypted string or the HMAC then the validation would fail and we would know someone has tampered with it. The final step is to combine all of them together – the encryption salt (esalt), the HMAC salt (hsalt), the ciphertext (es) and the actual HMAC (hmac) into a single byte array that we encode using Base64 and return as a String.

Decryption Function

public String decrypt(String eos, String p) throws Exception {
  // Recover our Byte Array by Base64 Decoding
  byte[] os = decodeBase64(eos.toCharArray());

  // Check Minimum Length (ESALT (20) + HSALT (20) + HMAC (32))
  if (os.length > 72) {
    // Recover Elements from String
    byte[] esalt = Arrays.copyOfRange(os, 0, 20);
    byte[] hsalt = Arrays.copyOfRange(os, 20, 40);
    byte[] es = Arrays.copyOfRange(os, 40, os.length - 32);
    byte[] hmac = Arrays.copyOfRange(os, os.length - 32, os.length);

    // Regenerate HMAC key using Recovered Salt (hsalt)
    byte[] dhk = deriveKey(p, hsalt, 100000, 160);

    // Perform HMAC using SHA-256
    SecretKeySpec hks = new SecretKeySpec(dhk, "HmacSHA256");
    Mac m = Mac.getInstance("HmacSHA256");
    m.init(hks);
    byte[] chmac = m.doFinal(es);

    // Compare Computed HMAC vs Recovered HMAC
    if (MessageDigest.isEqual(hmac, chmac)) {
      // HMAC Verification Passed
      // Regenerate Encryption Key using Recovered Salt (esalt)
      byte[] dek = deriveKey(p, esalt, 100000, 128);

      // Perform Decryption
      SecretKeySpec eks = new SecretKeySpec(dek, "AES");
      Cipher c = Cipher.getInstance("AES/CTR/NoPadding");
      c.init(Cipher.DECRYPT_MODE, eks, new IvParameterSpec(new byte[16]));
      byte[] s = c.doFinal(es);

      // Return our Decrypted String
      return new String(s, StandardCharsets.UTF_8);
    }
  }
  throw new Exception();
}

The decryption routine is the opposite of the encryption routine – we start off by decoding the Base64 string and then recovering the encryption salt (esalt), the HMAC salt (hsalt), the ciphertext (es) and the actual HMAC (hmac) into separate byte arrays. We then regenerate the HMAC key using the HMAC salt (you need to ensure you match the number of PBKDF2 iterations on encryption and decryption). If the computed HMAC matches the recovered HMAC then no one has tampered with our data. We then regenerate our encryption key using PBKDF2 from the recovered encryption salt and finally decrypt the ciphertext and return the plaintext. If any errors occur (i.e. HMAC verification failed, invalid password or something wrong with the encoded string) then an Exception is thrown.

There is one improvement I might make, as and when Java supports it, which is using AES in GCM mode which mitigates the need to generate a HMAC of the encrypted string and thus multiple salts and secret keys. If this isn’t possible then using “PBKDF2WithHmacSHA256” to generate my HMAC key would allow a 256 bit input key instead of the 160 bits by using SHA-1. However, you could argue that AES-CTR with HMAC-SHA-256 provides better integrity than AES-GCM, as although it provides authentication using a GHASH function, it has a maximum Authentication Tag length of 128 bits – this is half the size that SHA-256 provides.

Elliptic-Curve Cryptography using AES-GCM in Java 8

So far we have dealt with symmetric encryption that works when a person wants to encrypt and decrypt data using a password, but what if two people (Alice and Bob) want to securely communicate over an insecure channel? With symmetric encryption we have the age old problem of how to exchange the shared secret in the beginning without someone (let’s call her Eve) intercepting it.

This is where Public Key Cryptography comes into play and using Diffie-Hellman to establish a shared secret between two parties, without ever sending the actual secret. Alice and Bob both generate a public/private key pair using either traditional Diffie-Hellman or the newer Elliptic-Curve Diffie-Hellman. As the name suggests, the public key is public and can be shared openly with anyone, but the private key must remain private to the person who created it. They then exchange their public key with each other – it doesn’t matter if this is over an insecure channel as they aren’t private. Alice and Bob are now able to compute the same shared secret by combining their private key with the received public key.

The mathematics behind this is based on an algorithm which is easy to do in one direction, but difficult in the other. In the case of traditional Diffie-Hellman, the easy algorithm multiplies two prime numbers (the public and private keys). If multiplication is the easy algorithm, its difficult pair algorithm is factoring the product of the multiplication into its two component primes. Elliptic-Curve Diffie-Hellman uses a different problem and is considered a stronger algorithm as the problem is significantly harder to solve than factoring – it is based on trying to solve a discrete logarithm problem based on points on an elliptic curve.

If Eve was listening to the communication she would have the public keys of both Alice and Bob, but would be unable to compute the same shared secret without the private keys. She should also not be able to solve the factoring problem in traditional Diffie-Hellman or the harder discrete logarithm problem in Elliptic-Curve Diffie-Hellman.

Now that Alice and Bob have the same shared secret they can use symmetric encryption to encrypt and decrypt data between themselves without Eve being able to read – they would also potentially use a HMAC algorithm for integrity to ensure Eve wasn’t able to modify it either.

However, how does Alice know she is actually talking to Bob? What if Eve intercepted Bob’s public key and swapped it for her own when Bob sent it? What if Eve also intercepted Alice’s public key and sent her own to Bob? This is probably the hardest part of Public Key Cryptography and usually relies on a web of trust with Certificate Authorities signing certificates that are ultimately trusted. After Bob generated his public key, he would get someone that Alice trusted to sign it, so when it is sent to Alice she knows it is actually Bob’s public key. The other approach would be for Alice and Bob to validate the hash of their public keys via an offline method to confirm they have the legitimate keys.

Now we have a basic understanding of how Public Key Cryptography works we are going to look at how we can use Elliptic Curve Cryptography (ECC) in Java 8 with AES in GCM mode. We could easily use AES in CTR mode with HMAC-SHA-256, but GCM mode is better suited as it doesn’t require additional keys for a HMAC. I also haven’t provided a working example so far, so it is also used for completeness. To keep it simpler I will cover authentication at the end using a separate example.

Generate ECDH Key Pair

SecureRandom r = SecureRandom.getInstance("SHA1PRNG");
KeyPairGenerator kpg = KeyPairGenerator.getInstance("EC", "SunEC");
ECGenParameterSpec ecsp = new ECGenParameterSpec("secp256r1");
kpg.initialize(ecsp, r);

// Generate Key Pair
KeyPair dhkp = kpg.genKeyPair();

// Extract Public Key as Byte Array
byte[] dhpkb = dhkp.getPublic().getEncoded();

The above piece of code would be executed by both Alice and Bob to generate their own key pair to be used with ECDH. We have specified to use the “secp256r1” (aka “NIST P-256”) elliptic curve to generate the keys from – there are different curves which could be used, but this generates a 256-bit output which is recommended by NIST for AES-128. Some people feel the NIST curves (P-XXX) have been backdoored by the NSA after recent revelations from Snowden – I am not sure if there is an ounce of evidence in this, but people are left to use whatever curves they feel gives them the right level of security. Finally we extract the public key into a byte array (dhpkb) ready to send to the other person.

Compute Shared Secret Key

KeyAgreement ka = KeyAgreement.getInstance("ECDH");

// Convert Received Byte Array into PublicKey
PublicKey dhpk = KeyFactory.getInstance("EC").generatePublic(new X509EncodedKeySpec(dhpkb));

// Generate Shared Secret Using Local Private and Received Public
ka.init(dhkp.getPrivate());
ka.doPhase(dhpk, true);

// Generate an SHA-256 Hash and Truncate to 128 Bits
MessageDigest sha256 = MessageDigest.getInstance("SHA-256");
byte[] sk = Arrays.copyOfRange(sha256.digest(ka.generateSecret()), 0, 16);

Once the public keys have been transmitted, the same shared secret key would be computed by both Alice and Bob. As the public key would be transmitted as a byte array we need to recover it back into the “PublicKey” class which we do via the “generatePublic()” function. We then use the “KeyAgreement” class with an “ECDH” instance to combine the local private key with the received public key to produce the shared secret. We should now pass the generated secret through a KDF (Key Derivation Function) like HKDF to produce an encryption key of the desired length – as Java doesn’t support HKDF we use SHA-256 and truncate the output to the desired length. In theory if everything has worked correctly, both Alice and Bob should now be in possession of the same 128-bit key.

Encryption using AES-GCM

private String encrypt(String s, byte[] k) throws Exception {
  SecureRandom r = SecureRandom.getInstance("SHA1PRNG");
  // Generate 128 bit IV for Encryption
  byte[] iv = new byte[12]; r.nextBytes(iv);

  SecretKeySpec eks = new SecretKeySpec(k, "AES");
  Cipher c = Cipher.getInstance("AES/GCM/NoPadding");

  // Generated Authentication Tag should be 128 bits
  c.init(Cipher.ENCRYPT_MODE, eks, new GCMParameterSpec(128, iv));
  byte[] es = c.doFinal(s.getBytes(StandardCharsets.UTF_8));

  // Construct Output as "IV + CIPHERTEXT"
  byte[] os = new byte[12 + es.length];
  System.arraycopy(iv, 0, os, 0, 12);
  System.arraycopy(es, 0, os, 12, es.length);

  // Return a Base64 Encoded String
  return Base64.getEncoder().encodeToString(os);
}

This is similar to AES-CTR mode that we have used previously – we provide the encryption key (k – this is our computed 128-bit key) and the data to be encrypted (s) and the function will return a Base64 encoded string (combining the IV with the ciphertext). Unlike our previous example, where we never use the same key twice, with Public Key Cryptography we would use the same key multiple times, which means it is absolutely critical that the Initialisation Vector (IV) must be unique every time. If you use the same key and IV twice then an attacker can potentially decrypt both messages and forge further message – don’t do it! The IV (or nonce in GCM terms) should be 96-bits as recommended by the spec – it should be generated in such a way that it is extremely unlikely that you re-use values. When we initiate the cipher we pass through a “GCMParameterSpec” which takes as the first parameter the Authentication Tag length – this is the size of the GHASH function which is computed over the encrypted message. The maximum length is 128-bits and it should be in multiples of 8 – I would recommend you use the maximum possible as this already provides less security than using a 256-bit HMAC function.

Decryption using AES-GCM

private String decrypt(String eos, byte[] k) throws Exception {
  // Recover our Byte Array by Base64 Decoding
  byte[] os = Base64.getDecoder().decode(eos);

  // Check Minimum Length (IV (12) + TAG (16))
  if (os.length > 28) {
    byte[] iv = Arrays.copyOfRange(os, 0, 12);
    byte[] es = Arrays.copyOfRange(os, 12, os.length);

    // Perform Decryption
    SecretKeySpec dks = new SecretKeySpec(k, "AES");
    Cipher c = Cipher.getInstance("AES/GCM/NoPadding");
    c.init(Cipher.DECRYPT_MODE, dks, new GCMParameterSpec(128, iv));

    // Return our Decrypted String
    return new String(c.doFinal(es), StandardCharsets.UTF_8);
  }
  throw new Exception();
}

For decryption we pass through the secret key (k) and the ciphertext to be decrypted (eos) and we start off by decoding the Base64 string and confirming it contains at least the IV (12 bytes) and the Authentication Tag (16 bytes) – 28 bytes in total. After extracting the IV we pass what is left through the AES-GCM cipher – we also specify the same Authentication Tag length (128-bits) – an exception would be thrown if someone has modified our message in transit. Finally we return our unencrypted data as a String.

Using ECDSA for Authentication

I mentioned that authentication was a challenge in Public Key Cryptography and that it relies on a web of trust for someone ultimately to be trusted by Alice and Bob. In the above example we could get Alice and Bob to generate an SHA-256 hash of their ECDH public key and ask each other to verify it offline for every session. If you are generating a new set of ECDH keys for every session (i.e. using ephemeral keys) then this isn’t manageable. The alternative is using ECDSA (Elliptic-Curve Digital Signature Algorithm) to generate a static pair of signing keys that you validate offline to confirm they are legitimate – these are then stored (the private key should be stored on disk using strong symmetric encryption) and used to sign your ECDH public ephemeral key for every session. This would then allow Alice and Bob to validate the public key belonged to the other person and hadn’t been swapped by Eve. If you require non-repudiation then you can use ECDSA to sign every message before encryption – this allows you to verify if it was actually Alice or Bob who sent the message.

To implement authentication in Java to sign your ECDH public key using a pre-verified ECDSA key, both Alice and Bob would start off by generating another key pair to be used by ECDSA.

SecureRandom r = SecureRandom.getInstance("SHA1PRNG");
KeyPairGenerator kpg = KeyPairGenerator.getInstance("EC", "SunEC");
ECGenParameterSpec ecsp = new ECGenParameterSpec("secp256r1");
kpg.initialize(ecsp, r);

// Generate EC Key Pair for ECDSA
KeyPair dsakp = kpg.genKeyPair();
byte[] dsapkb = dsakp.getPublic().getEncoded();

// Generate an SHA-256 Hash of Public Key
MessageDigest sha256 = MessageDigest.getInstance("SHA-256");
String dsapfp = String.format("%064x", new BigInteger(1, sha256.digest(dsapkb)));
System.out.println("SHA-256 Hash of Public ECDSA Key: " + dsapfp);

They would then transmit their public ECDSA key (dsapkb) to each other and compute the SHA-256 hash of the received key. Using an offline method they would verify the SHA-256 hash matches on both sides to confirm they are in possession of the legitimate key – they would then securely store this so they could use it to validate signed messages in the future.

Whenever Alice or Bob generates a new ECDH key pair they would sign their generated public key (dhpkb) with their ECDSA private key using the following.

Signature s = Signature.getInstance("SHA256withECDSA", "SunEC");
// Sign our ECDH Public Key with our ECDSA Private Key
s.initSign(dsakp.getPrivate());
s.update(dhpkb);

byte[] dhsb = s.sign();

The public ECDH key (dhpkb) would be transmitted to the other party with the ECDSA generated signature (dhsb). On receipt, before converting the byte array into a “PublicKey”, the signature would be verified against the pre-verified ECDSA public key (dsapkb) of the other party.

// Retrieve Recipients Verified ECDSA Public Key from Disk and Convert into PublicKey
PublicKey dsapk = KeyFactory.getInstance("EC").generatePublic(new X509EncodedKeySpec(dsapkb));

Signature s = Signature.getInstance("SHA256withECDSA", "SunEC");
// Verify their ECDH Public Key with their ECDSA Public Key
s.initVerify(dsapk);
s.update(dhpkb);

if (s.verify(dhsb)) {
  System.out.println("Signature Verified!");
}

Once this has been done on both sides you would have verified that the ECDH public key actually belongs to Alice and Bob respectively. You can then continue to encrypt and decrypt messages using this verified public key knowing only the other party would be able to decrypt it.

Secure Password Hashing

Although the above covers encryption and decryption, password hashing is something which is commonly used to store passwords (or should be if you don’t want your customer’s passwords being leaked all over the Internet, like some high profile companies have experienced recently). Like most things, there are recommended ways to do this, as well as the not so recommended ways (e.g. just storing the MD5 hash of the password).

If you decide to store just the MD5 (or SHA-512) hashes of a password then you leave yourself open to dictionary type attacks (if someone was to obtain your password file). This is where people can compare your stored hashes with pre-computed lookup tables (which are freely available on the Internet with a large number of passwords already hashed). You should be using “salted” passwords – this means that any same two passwords won’t result in the same hash being generated and avoids hackers being able to use lookup tables. Don’t reuse salts and ensure they are long enough and securely random using a Secure PRNG.

If you have followed the above advice, this should mean the only option available to hackers is to try and brute force your password by trying every different combination of password (or using downloadable lists of common passwords or previously leaked passwords – this is why it is so important to use different passwords for different services and something which isn’t made up of common words). The computational effort is going to be the time it takes to generate the hash from the candidate password and the number of different combinations they have to try (i.e. a 6 character password using alphanumerical characters (26 uppercase, 26 lowercase and 10 numerical) is going to result in 56 billion combinations – 62^6. In comparison an 8 character password is going to result in 218 trillion and a 10 character password is going to result in 839 quadrillion possible combinations). A hacker is never going to have to try them all, unless you are extremely lucky and have picked the last password they try.

Modern hashing and encryption algorithms have been designed to be fast and easy to implement in hardware. Although this may be an advantage, so that the use of encryption/decryption doesn’t create a noticeable delay, it has the disadvantage when it comes to trying to brute force passwords. Algorithms like SHA-256 are quick, which means hackers can try different password hashes very quickly (especially using GPU based hardware) – this is why it isn’t recommended to just use simple hashes for storing passwords. Functions like PBKDF2, bcrypt and scrypt have been designed to be slow and computationally expensive so that it takes hackers longer to generate hashes. As explained above, PBKDF2 is the only function supported natively by Java and the effort is controlled by the number of iterations – this should be set low enough so that it doesn’t cause an intolerable delay to the user, but high enough that it causes some delay to someone trying to brute force lots of hashes. Hardware is always getting quicker, so the number of iterations that might be acceptable in 2015 are probably nothing for the hardware we will be using in the future.

Taking all of this into consideration, this is based on what I have already talked about when generating keys for AES encryption. We start off by generating a random salt which matches the length of our HMAC algorithm (e.g. 160 bits for SHA-1). We then generate a 160 bit key from the salt and our password using PBKDF2 and 100,000 iterations. Finally we add the salt to the hash and encode using Base64.

public String generate(String p) throws Exception {
  // Generate 160 bit Salt using SHA-1 PRNG
  SecureRandom r = SecureRandom.getInstance("SHA1PRNG");
  byte[] salt = new byte[20]; r.nextBytes(salt);

  // Generate 160 bit Password Hash using PBKDF2 (100,000 Iterations)
  byte[] hash = deriveKey(p, salt, 100000, 160);

  // Construct Output as "SALT + HASH"
  byte[] os = new byte[20 + 20];
  System.arraycopy(salt, 0, os, 0, 20);
  System.arraycopy(hash, 0, os, 20, 20);

  // Return a Base64 Encoded String
  return new String(encodeBase64(os));
}

To authenticate a user we reverse the process – we obtain the hash from the database and pass it with the candidate password to a function. This obtains the previously generated salt and hash from the Base64 encoded string. It then re-generates the key using the candidate password and salt via PBKDF2 (the number of iterations have to match). Finally it compares the hash from the database with the hash it has generated from the candidate password – if they match then the user has provided the correct password. Please avoid using “Arrays.equals()” to compare cryptographic hashes as they open you up to timing based attacks – you should be using a time constant comparison function, which takes the same amount of time regardless of outcome. Comparison functions which return as soon as they have found a difference allow an attacker to guess hashes one character at a time. Java provides “MessageDigest.isEqual()” for this purpose, but be aware that it was vulnerable to the very same timing attacks before it was fixed in Java 6 (Update 17).

public boolean authenticate(String p, String h) throws Exception {
  // Recover our Byte Array by Base64 Decoding
  byte[] os = decodeBase64(h.toCharArray());

  // Check Length (SALT (20) + HASH (20))
  if (os.length == 40) {
    // Recover Elements from String
    byte[] salt = Arrays.copyOfRange(os, 0, 20);
    byte[] hash = Arrays.copyOfRange(os, 20, 40);

    // Regenerate Password Hash using Recovered Salt and Password
    byte[] phash = deriveKey(p, salt, 100000, 160);

    // Do they Match (using Time Constant Comparison)?
    return MessageDigest.isEqual(hash, phash);
  }
  return false;
}

Security minded people will know that SHA-1 is no longer recommended as a hashing algorithm as it is considered insecure against well-funded opponents and that SHA-2 or SHA-3 are recommended instead. According to NIST, SHA-1 is no longer recommended to be used as a hashing algorithm or for digital signatures. However, NIST still consider it perfectly valid and secure to be used beyond 2030 as a key derivation algorithm or as a PRNG. When using PBKDF2 the strength of the hashing algorithm isn’t the most important factor (unless it is fundamentally broken like MD5) – the most important aspect is the computational effort required to generate the hash and the inability to do this easily in hardware – this is why SHA-1, SHA-2 or especially SHA-3 are really bad choices as they are all engineered to be fast and hardware friendly – you can write a function which takes the same computational effort whether using SHA-1, SHA-2 or SHA-3 by just changing the iteration count.

If I had a requirement to store passwords and I was using Java 8 or later then I would probably use PBKDF2 with SHA-384 or SHA-512 – only because using the SHA-2 family of hashing algorithms is probably a safer idea than using SHA-1.

SecureRandom r = SecureRandom.getInstance("SHA1PRNG");
byte[] salt = new byte[48]; r.nextBytes(salt);

PBEKeySpec ks = new PBEKeySpec("secure_random_password".toCharArray(), salt, 100000, 384);
SecretKeyFactory skf = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA384");
byte[] hash = skf.generateSecret(ks).getEncoded();

However, being able to modify your iteration count and reviewing it on a regular basis is probably a good idea to ensure, as hardware advances, that your hashes still take a reasonable amount of time and effort to generate. That and enforcing a password policy that mandates random, long (12 or more characters) and secure passwords that utilise the whole ASCII character set.

To put this into perspective, if hackers were able to generate 1 trillion SHA-384 hashes per second (this would be very impressive, even on GPU based hardware in 2015) then we would still be looking at an unfeasibly long time to try all the combinations of a 10 or 12 character password – even if you manage to find the password after only trying half the combinations. Password complexity and length is probably the single most important factor in stopping hackers brute forcing your password!

hash

I have provided the full source (including Base64 routines) in the AES class and ECC class for Elliptic-Curve Cryptography which you can download from this website. If anyone has any comments, suggestions or questions then please leave them below.

General Cryptography Java


Thoughts on "AES Encryption with HMAC Integrity in Java"...

by Phil on November 8, 2015 at 22:26

Great article thanks!

Reply

by Muhammad on December 17, 2015 at 14:40

Thanks for the article, really help my project.
Hi Chris, do u have papers about your argument in this article? i’ll be glad if you share it to me 🙂
And .. if you have article about how to implement perfect forward secrecy in java, please share 🙂
Again .. thanks Chris.

Reply

    by Chris on December 19, 2015 at 17:30

    By using DH with ephemeral keys (e.g. a new public/private key pair per session) it provides PFS. With DH you are never actually encrypting data with a public key so it doesn’t matter if the private key was ever recovered. Both parties are generating a unique symmetric encryption key from the public/private keys – if they were discovered later on then you still couldn’t use them to decrypt a previous session as keys would be unique per session.

    Reply

      by Muhammad on December 21, 2015 at 16:52

      ow yeah … thanks for the explanation.
      I’ll be here again if i have a question about this article 😀 thanks for your attention Chris.

      Reply

by Sann-X! on January 14, 2016 at 05:20

Hi,
You wrote:
5. The pseudorandom salt should be a cryptographically secure random number and should be at least the block size of your hashing algorithm (i.e. 160 bits for SHA-1).

But you are wrong. The size of the salt is independant to your choice of hash. See here: http://security.stackexchange.com/a/18006/87096

Reply

    by Chris on January 14, 2016 at 06:34

    The referenced post also states: “According to the PBKDF2 standard, the minimum recommended size for the salt is 64 bits, though I’d personally recommend 128 bits or higher for a decent safety margin.”

    Recommending that you use at least the block size of your hashing algorithm, provides a decent safety margin and fulfills the recommendation above. Also, according to https://crackstation.net/hashing-security.htm: “To make it impossible for an attacker to create a lookup table for every possible salt, the salt must be long. A good rule of thumb is to use a salt that is the same size as the output of the hash function. For example, the output of SHA256 is 256 bits (32 bytes), so the salt should be at least 32 random bytes.”

    Reply

by Sann-X! on January 15, 2016 at 10:47

About https://crackstation.net/hashing-security.htm
I think you need to consider the context. If you need to hash a password you must choose the size of the salt properly. But with PBKDF2 the salt has other goals. Thus I think the size of the salt is independent to your choice of hash.

Also see PBKDF2 java Implementation “F” function (93th string here http://cryptofreek.org/2012/11/29/pbkdf2-pure-java-implementation/ ). And see wiki (https://en.wikipedia.org/wiki/PBKDF2):
“The function F is the xor (^) of c iterations of chained PRFs. The first iteration of PRF uses Password as the PRF key and Salt concatenated with i encoded as a big-endian 32-bit integer.”

Thus if PRF is HMAC-SHA1 then:
U1 = funcHMACSHA1(Password, some_bytes)
where some_byte must have a size atleast SHA1 size – 160 bit. But some_byte is really as (in pseudocode):
some_bytes = Salt || INT_32_BE(i))
where || is concatenation.
Thus salt size have 160 – 32=128 bits.

PS
Sorry for my bad english

Reply

    by Chris on January 17, 2016 at 09:38

    Thanks for the references, but I still can’t see why the salt should be limited to 128 bits. The salt is used to initialize PBKDF2 on the first iteration by being concatenated to a 32-bit integer (if dkLen is <= hLen then this integer will normally be 1). When using a 160 bit salt, this results in a 192 bit output (160 + 32) which is then passed to HMAC-SHA-1 as the message with the password. This results in an output of hLen (160 bits) that is then used for further PBKDF2 iterations.

    The length of the message passed to HMAC-SHA1 doesn't have to be limited to hLen (160 bits), so the salt doesn't have to be limited to 128 bits (160 – 32) for SHA-1. If an attacker is attempting to use a pre-computed Rainbow Table with a list of common passwords and salts, then surely having a longer salt reduces the feasibility of this – especially if the user insists on picking a small dictionary based word as their password?

    I think you are saying that there is no direct correlation between hashing function output length (hLen) and the size of the salt – the choice should be made independent of hashing function and would be based on the level of entropy you want to provide. For password hashing this might be the length of your hashing algorithm, but for encryption this might be the length of your encryption key (thus using a 160 bit salt for a 128 bit encryption key makes it easier for someone to attack the key as opposed to the password).

    Reply

by SUVRA GUPTA on November 10, 2016 at 11:46

HI how can we generate the same string after encryption.? I have implemented AES/GCM/NoPadding but i need to get the same string after encrypting.
Eg. if ‘Tom’ is encrypted as ‘hzzzzhgft’ . When i again encypt Tom i should recieve the same string. How can we implement that.

Reply

    by Chris on March 1, 2017 at 18:23

    The ciphertext should always be different and that is what the initialisation vector is for (encrypting the same plaintext using the same key and same IV will result in the same ciphertext). To quote the article – “If you use the same key and IV twice then an attacker can potentially decrypt both messages and forge further message – don’t do it! The IV (or nonce in GCM terms) should be 96-bits as recommended by the spec – it should be generated in such a way that it is extremely unlikely that you re-use values.”

    Reply

by Andri on January 14, 2017 at 03:29

Hai chrish thx for your good work 🙂 chris if alice and bob want send her ecdsa public key and dart intercept all public key and change with dart public key and send dart signature to alice and bob so dart can mitm?

Reply

    by Chris on March 1, 2017 at 18:28

    Correct – this is always the challenge in public key authentication. That is why you need some way to authenticate that you actually have Alice or Bob’s actual public key. In the article I gave an example of the offline method – both Alice and Bob would compute a SHA-256 hash of their public key and then find a way to exchange them out of band. They could then use those stored hashes to confirm they are actually in possession of the correct public keys.

    Reply

by Latha on March 16, 2017 at 10:44

Hi,
Need few more details to use the best technique for data encryption and decryption for passwords and other data.
Can I have a complete program for this requirement.

Thanks in advance.

Reply

by mpaul24 on June 22, 2017 at 01:12

Thanks, it is a great article.

Reply

by Suresh Babu on July 17, 2017 at 10:11

Can you please give the full implementation of PBKDF2WithHmacSHA384 with jdk 1..8?

Reply

    by Chris on July 20, 2018 at 09:35

    In Java 8 you can use the native implementation of PBKDF2WithHmacSHA384 as follows – this will generate a 256 bit key derived from your password (you will need to define PBKDF2Iterations with the number of iterations you require.

    private byte[] deriveKey(String password, byte[] salt, int iterations, int length) throws Exception {
      PBEKeySpec ks = new PBEKeySpec(password.toCharArray(), salt, iterations, length);
      SecretKeyFactory skf = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA384");
      return skf.generateSecret(ks).getEncoded();
    }
    
    SecureRandom r = SecureRandom.getInstance("SHA1PRNG");
    byte[] salt = new byte[64]; r.nextBytes(salt);
    
    byte[] key = deriveKey("this is my password", salt, PBKDF2Iterations, 256);
    
    Reply

by Arthur on November 7, 2017 at 10:57

Hello Chris, I have a question.
In your post you warn “Please be aware that the above code doesn’t calculate the HMAC in the recommended way – it should include the “ESALT” otherwise changes to this value would change the decrypted Ciphertext even though it correctly authenticates!”.

Can you please explain what a proper calculation of the HMAC would look like ?

Reply

    by Chris on July 20, 2018 at 09:28

    If you are calculating a HMAC then it should be across all your data – its purpose is to verify that you are seeing the true decrypted string and not something that has been manipulated. In my example I should be performing a HMAC operation across “kLEN || SALT || IV || CIPHERTEXT” – if I only performed the HMAC over the CIPHERTEXT and someone was to change the SALT or IV within the string, then the HMAC would still pass but I would get a completely different decrypted string – this is bad. See the post “Updated Encryption Routines in TemplateFx v2.54” for an updated HMAC routine which calculates it across all the data.

    Reply

by Michael on September 16, 2018 at 11:08

Hello Chris, as I can see in your sources you do not specify an individual IV for encryption/decryption but a “nulled one”: c.init(Cipher.ENCRYPT_MODE, eks, new IvParameterSpec(new byte[16])); gives an IV of “00000000000000000000000000000000”.
The created IV is used for PBKDF-functionality. As read many times the IV should be a secured random value so why you use a static IV ?
Kind regards, Michael

Reply

    by Chris on September 16, 2018 at 18:53

    In my example I use a null IV as I am only using the encryption key once – I generate a new key and only encrypt a single piece of data with it. If you plan to use your key more than once than it is absolutely critical then you generate a secured random IV every time. I have addressed this in the Updated Encryption Routines in TemplateFx v2.54 article, where I state: “We generate a 16 byte pseudo-random IV – this isn’t technically necessary as the routines never use the same key twice, so there is no risk of creating duplicate ciphertext. However, it adds extra protection for anyone who thinks it is a clever idea of re-using the SALT.”

    Reply

Leave a Reply

Comment:

Name:

e-mail Address: