Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upEnsure async iteration works with destroyed streams #23785
Conversation
This comment has been minimized.
This comment has been minimized.
nodejs-github-bot
added
the
zlib
label
Oct 20, 2018
mcollina
referenced this pull request
Oct 20, 2018
Closed
Stream destruction breaks async-iteration and ends nodejs event loop #23730
This comment has been minimized.
This comment has been minimized.
|
@mcollina does this supersede my zlib destroy leak PR? |
This comment has been minimized.
This comment has been minimized.
|
node/lib/internal/streams/async_iterator.js Lines 90 to 91 in 0983ed8 Nit: |
This comment has been minimized.
This comment has been minimized.
|
@mafintosh which one? Very likely that should be merged first. |
devsnek
approved these changes
Oct 20, 2018
|
It seems this relies on a race condition? I won't pretend to know anything about our crazy streams but I want to just verify what's going on here is what is intended. |
| if (this[kStream].destroyed) { | ||
| // this is needed because if .destro(err) is called, the error | ||
| // will be emitted via nextTick | ||
| return new Promise((resolve, rejects) => { |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@devsnek this is normalizing a race condition that we cannot get rid of inside our stream machinery :/. |
devsnek
added
stream
experimental
labels
Oct 20, 2018
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Ah I didn’t know we were working on essentially the same issue! Can you pick up the change that I did inside processCallback? It might even be worth a test on its own. I’ll rebase this on top of yours. |
lpinca
reviewed
Oct 20, 2018
addaleax
reviewed
Oct 21, 2018
|
|
||
| Zlib.prototype._destroy = function(err, cb) { | ||
| const handle = this._handle; | ||
| if (handle && handle.buffer !== null) { |
This comment has been minimized.
This comment has been minimized.
| _close(this, callback); | ||
| this.destroy(); | ||
| Zlib.prototype.close = function(callback) { | ||
| this.destroy(null, callback); |
This comment has been minimized.
This comment has been minimized.
mcollina
force-pushed the
mcollina:fix-23730
branch
from
c13cc44
to
0d029f6
Oct 23, 2018
This comment has been minimized.
This comment has been minimized.
|
CI: https://ci.nodejs.org/job/node-test-pull-request/18090/ PTAL, this should be ready for review. |
mcollina
force-pushed the
mcollina:fix-23730
branch
from
0d029f6
to
80487d0
Oct 24, 2018
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
mafintosh
reviewed
Oct 24, 2018
| @@ -158,6 +174,9 @@ const createReadableStreamAsyncIterator = (stream) => { | |||
| stream.on('readable', onReadable.bind(null, iterator)); | |||
| stream.on('end', onEnd.bind(null, iterator)); | |||
| stream.on('error', onError.bind(null, iterator)); | |||
| // needed because some streams will not emit 'end', e.g. Zlib. | |||
| // https://github.com/nodejs/node/issues/23730 | |||
| stream.on('close', onEnd.bind(null, iterator)); | |||
This comment has been minimized.
This comment has been minimized.
mafintosh
approved these changes
Oct 24, 2018
lpinca
approved these changes
Oct 24, 2018
| const passthrough = new PassThrough(); | ||
| const err = new Error('kaboom'); | ||
| pipeline(readable, passthrough, common.mustCall((e) => { | ||
| assert.strictEqual(err, e); |
This comment has been minimized.
This comment has been minimized.
| try { | ||
| await readable[Symbol.asyncIterator]().next(); | ||
| } catch (e) { | ||
| assert.strictEqual(err, e); |
This comment has been minimized.
This comment has been minimized.
addaleax
removed
the
zlib
label
Oct 24, 2018
This comment has been minimized.
This comment has been minimized.
|
@mcollina There’s nothing speaking against landing this as far as I am concerned :) |
mcollina
force-pushed the
mcollina:fix-23730
branch
from
80487d0
to
14586b0
Oct 24, 2018
This comment has been minimized.
This comment has been minimized.
|
CI: https://ci.nodejs.org/job/node-test-pull-request/18110/ PTAL, I've switched to use |
This comment has been minimized.
This comment has been minimized.
|
Still LGTM. |
mmarchini
approved these changes
Oct 24, 2018
mmarchini
added
the
author ready
label
Oct 24, 2018
This comment has been minimized.
This comment has been minimized.
|
Landed in 3ec8cec |
mcollina
closed this
Oct 24, 2018
mcollina
deleted the
mcollina:fix-23730
branch
Oct 24, 2018
mcollina
added a commit
that referenced
this pull request
Oct 24, 2018
This comment has been minimized.
This comment has been minimized.
|
(Also still LGTM for the record) |
targos
added a commit
that referenced
this pull request
Oct 24, 2018
This was referenced Oct 26, 2018
This was referenced Nov 2, 2018
MylesBorins
added
dont-land-on-v8.x
land-on-v10.x
labels
Nov 26, 2018
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
go for it!
Il giorno lun 26 nov 2018 alle 22:41 Myles Borins <notifications@github.com>
ha scritto:
… Landed in 10.x with 398418d
<398418d>
and b1e1fe4
<b1e1fe4>
Please lmk if it should be backed out
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#23785 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AADL4-2fOTTylhaBVvajyK70fXegKpuvks5uzGAUgaJpZM4Xx6oH>
.
|
mcollina commentedOct 20, 2018
•
edited
This PR is twofold:
makes sure thatdone in #23734.destroy()works correctly on zlib streams (before it was not cleaning up the_handle)Fixes #23730 .
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passes