★ wanayoo — archive 1999 https://github.com/nodejs/node/pull/22921Nouvelle recherche | Portail wanayoo
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vm: update source, related basic & cached data tests for ArrayBuffer #22921

Closed
wants to merge 10 commits into from

Conversation

Projects
None yet
10 participants
@BeniCheni
Copy link
Contributor

commented Sep 18, 2018

Refs: #1826

Referring to the comment in "mentor-available" #1826, this PR tries out the vm item from the checklist of the comment:

  • accept ArrayBufferView type(s) in place of Uint8Array in the source

  • related test code in test-vm-basic.js & test-vm-cached-data.js.

(P.S.: if any potential test case(s) or vm doc update is missed, please advise. Thanks!)

  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines
@targos
Copy link
Member

left a comment

The native code must be changed to handle the new allowed types.

@targos

This comment has been minimized.

Copy link
Member

commented Sep 18, 2018

Example showing that the C++ part is not ready for it:

'use strict';

const vm = require('vm');

let script = new vm.Script('var x = 0;', { produceCachedData: true });
const cachedData = script.cachedData;

new vm.Script('var x = 0;', { cachedData: new Uint16Array(cachedData.buffer) } );
$ ./node test.js
./node[10954]: ../src/node_contextify.cc:685:static void node::contextify::ContextifyScript::New(const v8::FunctionCallbackInfo<v8::Value>&): Assertion `args[4]->IsUint8Array()' failed.
 1: 0x8dbd90 node::Abort() [./node]
 2: 0x8dbe65  [./node]
 3: 0x910fea node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo<v8::Value> const&) [./node]
 4: 0xb6e1f1  [./node]
 5: 0xb6e7bf  [./node]
 6: 0x1c3b281cfc3d 
[1]    10954 abort (core dumped)  ./node test.js

@BeniCheni BeniCheni force-pushed the BeniCheni:vm-array-buffer-view branch from 3f29942 to e136b76 Sep 26, 2018

BeniCheni added a commit to BeniCheni/node that referenced this pull request Sep 26, 2018

vm: update contextify source
Use ArrayBufferView/IsArrayBufferView in places of Uint8Array/IsUint8Array,
in the node_contextify.cc source code.

Refs: nodejs#22921 (comment)
@BeniCheni

This comment has been minimized.

Copy link
Contributor Author

commented Sep 26, 2018

Thank you, @ targos / ping reviewer(s).

Learned about the “vm” module & dug deeper in the node_contextify.cc code, based on the stack-trace error from your helpful test case, and able to use ArrayBufferView/IsArrayBufferView in places of Uint8Array/IsUint8Array. Please review at your convenience,

Tested w. the following test snippet:

‘use strict';

const vm = require('vm');

let script = new vm.Script('var x = 0;', { produceCachedData: true });
const cachedData = script.cachedData;

new vm.Script('var x = 0;', { cachedData: new Uint16Array(cachedData.buffer) } )
@BeniCheni

This comment has been minimized.

Copy link
Contributor Author

commented Oct 1, 2018

Ping/Dear reviewers,

Friendly reminder that - Made updates according per previous review comments, & was able to passed the suggested “vm” test w. cached data. Please review again at convenience.

@jasnell

jasnell approved these changes Oct 1, 2018

@lundibundi
Copy link
Member

left a comment

I think change to the doc is missing here https://github.com/nodejs/node/blob/master/doc/api/vm.md#new-vmscriptcode-options and here https://github.com/nodejs/node/blob/master/doc/api/vm.md#vmcompilefunctioncode-params-options.

Also, I don't see any tests for TypedArray or DataView as cachedData, is there any?

@BeniCheni BeniCheni force-pushed the BeniCheni:vm-array-buffer-view branch from e136b76 to 6f63ef5 Oct 3, 2018

BeniCheni added a commit to BeniCheni/node that referenced this pull request Oct 3, 2018

vm: update contextify source
Use ArrayBufferView/IsArrayBufferView in places of Uint8Array/IsUint8Array,
in the node_contextify.cc source code.

Refs: nodejs#22921 (comment)
@BeniCheni

This comment has been minimized.

Copy link
Contributor Author

commented Oct 3, 2018

@lundibundi,

I think change to the doc is missing here https://github.com/nodejs/node/blob/master/doc/api/vm.md#new-vmscriptcode-options and here https://github.com/nodejs/node/blob/master/doc/api/vm.md#vmcompilefunctioncode-params-options.

Updated the "cachedData" description for new-vmscriptcode-options & vmcompilefunctioncode-params-options docs. Thank you!

Also, I don't see any tests for TypedArray or DataView as cachedData, is there any?

The updated test in test/parallel/test-vm-cached-data.js for ERR_INVALID_ARG_TYPE should cover cachedData with the validation of TypedArray & DataView type error.

Please guide me other test(s) that'd add test for TypedArray or DataView as cachedData?

{
  code: 'ERR_INVALID_ARG_TYPE',
  type: TypeError,
-message: /must be one of type Buffer or Uint8Array/
+message: /must be one of type Buffer, TypedArray, or DataView/
}
@lundibundi

This comment has been minimized.

Copy link
Member

commented Oct 3, 2018

Thanks for the swift changes =). It seems there is no reasonable way to test for cachedData besides simple arg checks, so it should be fine.

Though, I cannot understand how can we actually get the cachedData in anything besides Buffer. IIRC https://nodejs.org/api/vm.html#vm_script_createcacheddata is the only way to get cachedData therefore is there any sense of having cachedData available as TypedArray or DataView?

Also, this line https://github.com/nodejs/node/pull/22921/files#diff-0cf206672499c2f86db4ffb0cc5b668bR683 and this https://github.com/nodejs/node/pull/22921/files#diff-0cf206672499c2f86db4ffb0cc5b668bR1019 extracts data as uint8_t will this still be correct if we pass i.e. Int32Array (it should work but would that be correct)?

@BeniCheni

This comment has been minimized.

Copy link
Contributor Author

commented Oct 4, 2018

@lundibundi and/or other dear reviewers,

Per these two topics, I'd need some guideline to know how to update the PR to move forward. Would you mind advising?

...sense of having cachedData available as TypedArray or DataView"

...extracts data as uint8_t will this still be correct if we pass i.e. Int32Array (it should work but would that be correct)?

@BeniCheni

This comment has been minimized.

Copy link
Contributor Author

commented Oct 6, 2018

Ping / dear reviewers,

Would you mind guiding this PR for next step based on the questions in last comment? (Hoping the PR would not become stale; thank you!)

@lundibundi

This comment has been minimized.

Copy link
Member

commented Oct 6, 2018

ping @addaleax @jasnell @targos as the ones who reviewed this could you provide some input on #22921 (comment)?

Just to be clear: I'm not blocking this, my concern is - maybe it's worth changing something here for it to become a bit more relevant.

CI: https://ci.nodejs.org/job/node-test-pull-request/17668/

@TimothyGu

This comment has been minimized.

Copy link
Member

commented Oct 6, 2018

I don’t quite see any tests for the newly supported types. You could use the common.getArrayBufferViews() function to make sure all TypedArray and DataView types are actually supported. Check out https://github.com/nodejs/node/blob/master/test/parallel/test-fs-write-file-typedarrays.js for an example

@BeniCheni BeniCheni force-pushed the BeniCheni:vm-array-buffer-view branch from 6f63ef5 to 262787d Oct 21, 2018

BeniCheni added a commit to BeniCheni/node that referenced this pull request Oct 21, 2018

vm: update contextify source
Use ArrayBufferView/IsArrayBufferView in places of Uint8Array/IsUint8Array,
in the node_contextify.cc source code.

Refs: nodejs#22921 (comment)
@BeniCheni

This comment has been minimized.

Copy link
Contributor Author

commented Oct 21, 2018

Ping / dear reviewers,

Per the suggestion in last comment, I've rebased & updated the tests in "test-vm-cached-data.js" to use common.getArrayBufferViews() function, to ensure all TypedArray and DataView types are actually supported. Please review again at convenience.

(P.S.: sorry about the delayed update. Was in Node+JS Interactive (glad to meet some of you in person), & a busy week at work after the conf. Looking forward to your review again to pick back up this PR, to work towards landing/closure. 🙏 )

@BeniCheni BeniCheni force-pushed the BeniCheni:vm-array-buffer-view branch from 262787d to 19eb056 Oct 22, 2018

BeniCheni added a commit to BeniCheni/node that referenced this pull request Oct 22, 2018

vm: update contextify source
Use ArrayBufferView/IsArrayBufferView in places of Uint8Array/IsUint8Array,
in the node_contextify.cc source code.

Refs: nodejs#22921 (comment)
for (const parsedData of common.getArrayBufferViews(data)) {
// It should consume code cache
const script = new vm.Script(source, {
cachedData: parsedData

This comment has been minimized.

Copy link
@lundibundi

lundibundi Oct 23, 2018

Member

Nit: I'd prefer to just rename parsedData -> cachedData and the this line will become just

Suggested change
cachedData: parsedData
cachedData
@lundibundi

This comment has been minimized.

Copy link
Member

commented Oct 23, 2018

CI: https://ci.nodejs.org/job/node-test-pull-request/18070/

Also, ping @targos, I believe your concerns were addressed.

@targos targos dismissed their stale review Oct 23, 2018

native layer was updated and tests added

@BeniCheni BeniCheni force-pushed the BeniCheni:vm-array-buffer-view branch from b4b26f5 to c47abe4 Oct 23, 2018

BeniCheni added a commit to BeniCheni/node that referenced this pull request Oct 23, 2018

vm: update contextify source
Use ArrayBufferView/IsArrayBufferView in places of Uint8Array/IsUint8Array,
in the node_contextify.cc source code.

Refs: nodejs#22921 (comment)
@BeniCheni

This comment has been minimized.

Copy link
Contributor Author

commented Oct 23, 2018

@lundibundi, updated to use cachedData, since the term is indeed more contextually aligned than parsedData. Please review again at convenience. Thank you!

(P.S: also thank you @targos for reviewing the "native layer" part!)

@BeniCheni BeniCheni force-pushed the BeniCheni:vm-array-buffer-view branch from b6b21b1 to 0f20aca Oct 31, 2018

@BeniCheni

This comment has been minimized.

Copy link
Contributor Author

commented Nov 1, 2018

Dear reviewers, (cc @refack & @lundibundi)

Thanks for validating for the "author ready" status & label & the CI run. Would you mind guiding to landing at your convenience?

@refack

This comment has been minimized.

Copy link
Member

commented Nov 1, 2018

If you are not familiar with our review and landing process, it's covered in CONTRIBUTING.md

P.S. If you have any question you can also feel free to contact me directly.

[that is my canned response to new contributors]
tl;dr A Collaborator (e.g. me) needs to makes sure all the items on the checklist are done, and land. I've triggered a new CI job https://ci.nodejs.org/job/node-test-pull-request/18280/. I will try to keep an eye on it, and land when it passes.

@refack refack self-assigned this Nov 1, 2018

@refack

This comment has been minimized.

Copy link
Member

commented Nov 1, 2018

@Trott

This comment has been minimized.

Copy link
Member

commented Nov 2, 2018

@refack

This comment has been minimized.

Copy link
Member

commented Nov 2, 2018

Landed in 65fe999 🎉

@refack refack closed this Nov 2, 2018

@refack refack added doc test and removed author ready labels Nov 2, 2018

refack added a commit that referenced this pull request Nov 2, 2018

vm: allow `cachedData` to also be TypedArray|DataView
PR-URL: #22921
Refs: #1826
Refs: #22921 (comment)
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>

targos added a commit that referenced this pull request Nov 2, 2018

vm: allow `cachedData` to also be TypedArray|DataView
PR-URL: #22921
Refs: #1826
Refs: #22921 (comment)
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>

@BeniCheni BeniCheni deleted the BeniCheni:vm-array-buffer-view branch Nov 2, 2018

@refack refack removed their assignment Nov 6, 2018

@BridgeAR BridgeAR referenced this pull request Nov 14, 2018

Merged

Release proposal: v11.2 #24350

BethGriggs added a commit that referenced this pull request Apr 8, 2019

vm: allow `cachedData` to also be TypedArray|DataView
PR-URL: #22921
Refs: #1826
Refs: #22921 (comment)
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.