Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upcrypto: add support for AES-CCM #18138
Conversation
tniessen
added
semver-major
notable-change
labels
Jan 14, 2018
nodejs-github-bot
added
C++
lib / src
labels
Jan 14, 2018
tniessen
changed the title
[WIP] crypto: add support for AES-CCM
crypto: add support for AES-CCM
Jan 15, 2018
tniessen
added
the
crypto
label
Jan 17, 2018
tniessen
force-pushed the
tniessen:ccm-is-still-weird
branch
from
991a94c
to
6d0a3e0
Jan 17, 2018
This comment has been minimized.
This comment has been minimized.
|
Rebased due to a conflict with #18017. @nodejs/crypto and @nodejs/tsc, is there anything I can do to aid in discussing and reviewing this change? |
jasnell
reviewed
Jan 17, 2018
| let plaintextLength = -1; | ||
| if (options && options.plaintextLength != null) { | ||
| plaintextLength = options.plaintextLength; | ||
| if (~~plaintextLength !== plaintextLength || plaintextLength < 0) |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
jasnell
reviewed
Jan 17, 2018
|
Generally LGTM but needs docs. |
This comment has been minimized.
This comment has been minimized.
|
@jasnell Thanks! Yes, documentation is still pending, I just want to discuss the general API before documenting its details :) |
tniessen
requested a review
from nodejs/tsc
Jan 21, 2018
tniessen
referenced this pull request
Jan 22, 2018
Closed
Additional validation of tag length in AES GCM decryption #17523
This comment has been minimized.
This comment has been minimized.
|
ping @nodejs/tsc |
jasnell
approved these changes
Jan 22, 2018
bnoordhuis
reviewed
Jan 22, 2018
| - When decrypting, the authentication tag must be set via `setAuthTag()` before | ||
| specifying additional authenticated data and / or calling `update()`. | ||
| Otherwise, the behavior of the cipher is currently undefined and might result | ||
| in an error being thrown. Note that decryption without setting the correct |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
tniessen
Jan 23, 2018
Author
Member
This was supposed to be kind of a loophole for us. Many crypto libs (e.g. bouncycastle, pycryptodome etc.) allow decryption without the authentication tag, OpenSSL currently does not, and I am not sure which behavior is desirable. I can change it to "decryption will fail" though.
| ```js | ||
| const crypto = require('crypto'); | ||
| const key = Buffer.from('266defe0911630ffac0e3633514e70d7', 'hex'); |
This comment has been minimized.
This comment has been minimized.
bnoordhuis
Jan 22, 2018
Member
Tiny take-it-or-leave-it suggestion: use e.g. const key = 'passwordpassword'; here, the Buffer.from(...) kind of detracts from the important logic. Likewise for aad...
This comment has been minimized.
This comment has been minimized.
tniessen
Jan 23, 2018
Author
Member
setAAD() only accepts buffers, so I don't think there is much I can do about it for now.
| }); | ||
| const plaintext = Buffer.from('Hello world', 'utf8'); | ||
| cipher.setAAD(aad, { | ||
| plaintextLength: plaintext.length |
This comment has been minimized.
This comment has been minimized.
bnoordhuis
Jan 22, 2018
Member
...and maybe add a comment here that plaintextLength should be in bytes.
Or use a string and { plaintextLength: Buffer.byteLength(plaintext) }, that's more self-explanatory.
| function getAuthTagLength(options) { | ||
| let authTagLength; | ||
| if (options && (authTagLength = options.authTagLength) != null) { | ||
| authTagLength = options.authTagLength; |
This comment has been minimized.
This comment has been minimized.
| if (plaintextLength >>> 0 !== plaintextLength) | ||
| throw new errors.TypeError('ERR_INVALID_OPT_VALUE', 'plaintextLength', | ||
| plaintextLength); | ||
| } |
This comment has been minimized.
This comment has been minimized.
bnoordhuis
Jan 22, 2018
Member
It's somewhat incongruous that getAuthTagLength() is a function while the logic for plaintextLength is inline. If you pass a key to the former and generalize it a bit, you can share the code. Or vice versa, inline the former for consistency.
| if (!cipher->SetAAD(Buffer::Data(args[0]), Buffer::Length(args[0]))) | ||
| CHECK_EQ(args.Length(), 2); | ||
| CHECK(args[1]->IsInt32()); | ||
| int pt_len = args[1]->Int32Value(); |
This comment has been minimized.
This comment has been minimized.
bnoordhuis
Jan 22, 2018
Member
plaintext_len? I get that you do it to make the line below fit in 80 columns but pt_len is not a great name.
| "Permitting authentication tag lengths of %u bytes is deprecated. " | ||
| "Valid GCM tag lengths are 4, 8, 12, 13, 14, 15, 16.", tag_len); | ||
| ProcessEmitDeprecationWarning(cipher->env(), msg, "DEP0090"); | ||
| int mode = EVP_CIPHER_mode(EVP_CIPHER_CTX_cipher(cipher->ctx_)); |
This comment has been minimized.
This comment has been minimized.
bnoordhuis
Jan 22, 2018
Member
EVP_CIPHER_CTX_mode(cipher->ctx_). There's a few more of them further down.
As well, consider using const int mode = ....
| int mode = EVP_CIPHER_mode(EVP_CIPHER_CTX_cipher(ctx_)); | ||
| if (!r && kind_ == kDecipher && mode == EVP_CIPH_CCM_MODE) { | ||
| pending_auth_failed_ = true; | ||
| return 1; |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
tniessen
Jan 23, 2018
Author
Member
This was actually intentional, as update() probably should not throw in this case, but final() should (that's what pending_auth_failed_ is there for). This is to keep consistency with GCM.
| pending_auth_failed_ = true; | ||
| return 1; | ||
| } | ||
| return r; |
This comment has been minimized.
This comment has been minimized.
|
|
||
| // In CCM mode, final() only checks whether authentication failed in update(). | ||
| // EVP_CipherFinal_ex must not be called and will fail. | ||
| int r = mode == EVP_CIPH_CCM_MODE ? !pending_auth_failed_ : |
This comment has been minimized.
This comment has been minimized.
bnoordhuis
Jan 22, 2018
Member
This mixing of ints and bools, even if technically correct, is something best avoided, IMO.
tniessen
force-pushed the
tniessen:ccm-is-still-weird
branch
2 times, most recently
from
be707a9
to
78d0441
Jan 23, 2018
This comment has been minimized.
This comment has been minimized.
|
@bnoordhuis Thank you for your thorough review! I tried to address all suggestions (except #18138 (comment)), PTAL if you can. |
bnoordhuis
approved these changes
Jan 25, 2018
| "OpenSSL constants differ between GCM and CCM"); | ||
| if (IsAuthenticatedMode() && | ||
| !EVP_CIPHER_CTX_ctrl(ctx_, EVP_CTRL_GCM_SET_IVLEN, iv_len, nullptr)) { | ||
| EVP_CIPHER_CTX_free(ctx_); |
This comment has been minimized.
This comment has been minimized.
bnoordhuis
Jan 25, 2018
Member
I would suggest moving the EVP_CIPHER_CTX_free() calls to the callers. For ease of comprehension, clean-up on error should take place in the function that allocated the resource.
You can automate that with std::unique_ptr, by the way:
#include <memory>
// ...
auto d = [this] (...) {
EVP_CIPHER_CTX_free(ctx_);
ctx_ = nullptr;
};
std::unique_ptr<void, decltype(d)> free_on_return(&d, d);
// ..
free_on_return.release(); // if the happy path was taken| len)) { | ||
| return false; | ||
| const EVP_CIPHER* const cipher = EVP_CIPHER_CTX_cipher(ctx_); | ||
| int mode = EVP_CIPHER_mode(cipher); |
This comment has been minimized.
This comment has been minimized.
bnoordhuis
Jan 25, 2018
Member
const int mode = EVP_CIPHER_CTX_mode(ctx_)? cipher does not seem to be used apart from this line.
tniessen
force-pushed the
tniessen:ccm-is-still-weird
branch
from
e2f87d4
to
4c4dad0
Jan 25, 2018
This comment has been minimized.
This comment has been minimized.
|
|
maclover7
force-pushed the
nodejs:master
branch
from
bb5575a
to
993b716
Jan 26, 2018
cjihrig
force-pushed the
nodejs:master
branch
from
993b716
to
082f952
Jan 26, 2018
This comment has been minimized.
This comment has been minimized.
|
Seems like I found the cause of the FIPS failure thanks to gdb. openssl/openssl@9cca7be introduced a slight change to the CCM API back in 2015, and this change never made it into the FIPS release. Without this commit, the authentication tag must be specified along with its length when decrypting, and I believe this makes it incompatible with our current design. The only option I can think of right now is to make CCM unsupported when in FIPS mode and to throw an error in This is the conflicting line: (gdb) f
#0 aes_ccm_ctrl (c=0x3d1b490, type=17, arg=8, ptr=0x0) at e_aes.c:1284
1284 if ((c->encrypt && ptr) || (!c->encrypt && !ptr))
(gdb) next
1285 return 0;There is one other thing I came across, OpenSSL 1.1.0 seems to have fixed the problem with |
This comment has been minimized.
This comment has been minimized.
I'm okay with that. |
This comment has been minimized.
This comment has been minimized.
|
I'm good with not supporting CCM in FIPS also. |
This comment has been minimized.
This comment has been minimized.
|
For now, I only disabled CCM decryption in FIPS mode, PTAL. |
This comment has been minimized.
This comment has been minimized.
bluetiger30
commented
Jan 31, 2018
|
great work guys , I was waiting for this |
This comment has been minimized.
This comment has been minimized.
|
@bluetiger30 This change will most likely be included in all releases beginning with node 10.0.0, but not in previous releases. |
Trott
removed
the
semver-major
label
Apr 4, 2018
This comment has been minimized.
This comment has been minimized.
|
@addaleax I suggested to make it semver-minor in #18138 (comment) without response, I am okay with that. The only reason I marked this semver-major is that it can break code which incorrectly used CCM before, because we did not account for that in previous releases, e.g., if you pass invalid CCM options to |
Trott
removed
the
tsc-agenda
label
Apr 4, 2018
This comment has been minimized.
This comment has been minimized.
|
Yes, I'm fine with this addition and the API and being conservative on semver-major is probably a good idea although I wouldn't say it's essential. However #19794 is more important than getting this landed for Node 10 and that PR is going to heavily impact this one. AES-CCM is going to have fairly minimal use due to the necessarily awkward API so this is pretty low priority (sorry @tniessen but I suspect you already understand this). Getting #19794 safely landed without holding it up even further is critical so I'd rather see this PR have to change to fit that PR rather than the other way around. @tniessen could you do a quick investigation to see what changes might be required to sit on top of the 1.1.0 support @shigeki is adding? I wouldn't discard the exact code in this PR btw, we could possibly relax the semver-major and land this on 8.x as it is now so it's not entirely useless as it is now. I think I'd prefer to wait until after 10.x goes live to make a call on 8.x support for CCM. |
This comment has been minimized.
This comment has been minimized.
Side note: On a par with OpenSSL.
It indeed seems to be, the lack of TSC attention was noticeable long before 1.1.0h was even released. There haven't been any changes to this PR in weeks.
It lands cleanly and lite CI passes: https://ci.nodejs.org/job/node-test-commit-lite/589/ |
This comment has been minimized.
This comment has been minimized.
|
R=@danbev perhaps? |
This comment has been minimized.
This comment has been minimized.
|
beautiful, if this lands cleanly with 1.1.0 then I see no other holdup, lgtm |
rvagg
approved these changes
Apr 5, 2018
This comment has been minimized.
This comment has been minimized.
|
Only CI that didn't finish last time was Linux, so here's a re-run of just that: https://ci.nodejs.org/job/node-test-commit-linux/17674/ |
tniessen
added a commit
to tniessen/node
that referenced
this pull request
Apr 6, 2018
tniessen
added a commit
to tniessen/node
that referenced
this pull request
Apr 6, 2018
This comment has been minimized.
This comment has been minimized.
|
I created tniessen@a191f6b by "landing" #19766, #18138 (this PR) and #19794 (in this order) on top of the current master. Again, there were no conflicts. Additional CI for that commit: https://ci.nodejs.org/job/node-test-commit/17477/ Unless someone objects, I intend to land this along with #19766 today. |
danbev
added a commit
that referenced
this pull request
Apr 6, 2018
danbev
approved these changes
Apr 6, 2018
This comment has been minimized.
This comment has been minimized.
|
CI is a mess, but not because of these changes. lgtm, let's get it merged, thanks for the hard work on this @tniessen. |
tniessen
added a commit
that referenced
this pull request
Apr 6, 2018
This comment has been minimized.
This comment has been minimized.
tniessen
closed this
Apr 6, 2018
tniessen
added
the
semver-major
label
Apr 6, 2018
This comment has been minimized.
This comment has been minimized.
|
@Trott I marked this PR as semver-major again given that multiple TSC members approved after nodejs/TSC#516 and that it landed in time for node 10. At the very least, this is semver-minor. |
tniessen commentedJan 14, 2018
•
edited
CCM ("Counter with Cipher Block Chaining-Message Authentication Code")
OpenSSL currently supports two AEAD algorithms, GCM and CCM. While node has supported GCM for years, CCM is a bit more difficult to implement and use.
Currently, this PR implements an API similar to what was proposed by @brycekahle in #2383:
createCipher,createDecipher,createCipherivandcreateDecipherivaccept an optionauthTagLength. While the tag length is not relevant for GCM, it is for CCM. (And yes, using CCM withcreateCipher/createDecipheris inherently insecure.)setAADaccepts additionaloptionsas an optional second argument. The only currently recognized option isplaintextLength, which specifies the length of the plaintext / ciphertext in bytes.There are some critical aspects:
authTagLengthoption when using CCM. Not providing this option causes an error to be thrown bycreate(De|C)ipher(iv)?.plaintextLengthoption when using CCM if and only if AAD is provided. The explanation is simple, the length of the plaintext is encoded into the first block of the cipher, and thus must be available before performing an update (and before encoding the AAD).update()can only be called once. Again, the reason is very simple, CCM operates in what is sometimes called a "packet" or "offline mode", meaning that all data is processed at once.Some of these points can be relaxed:
authTagLengthoption, we could simplify the migration to this "new" AEAD. We could stick with the default value used within OpenSSL (12 bytes) or upgrade to 16 bytes (as e.g. pycryptodome does).setAADuntilupdateis called, we take away the need to specify theplaintextLength. All data still needs to be passed toupdatein a single call.We should discuss both, but it should be possible to add these features later on if they turn out to be good ideas (as long as they don't break anything). Aside from that, I am always happy about feedback and suggestions!
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passesAffected core subsystem(s)
crypto