★ wanayoo — archive 1999 https://github.com/nodejs/node/pull/26745Nouvelle recherche | Portail wanayoo
Skip to content
Please note that GitHub no longer supports Internet Explorer.

We recommend upgrading to the latest Microsoft Edge, Google Chrome, or Firefox.

Learn more
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new ESM implementation #26745

Open
wants to merge 10 commits into
base: master
from

Conversation

@MylesBorins
Copy link
Member

MylesBorins commented Mar 18, 2019

This PR updates the current --experimental-modules implementation based on the work of the modules team and reflects Phase 2 of our new modules plan.

A longer form description of these changes can be found in our draft blog post.

The largest differences from the current implementation include

  • packge.type which can be either module or commonjs
    • type: "commonjs":
      • .js is parsed as commonjs
      • default for entry point without an extension is commonjs
    • type: "module":
      • .js is parsed as esm
      • does not support loading JSON or Native Module by default
      • default for entry point without an extension is esm
  • --type=[mode] to let you set the type on entry point. Will override package.type for entry point.
  • A new file extension .cjs.
    • this is specifically to support importing commonjs in the module mode.
    • this is only in the esm loader, the commonjs loader remains untouched, but the extension will work in the old loader if you use the full file path.
  • --es-module-specifier-resolution=[type]
    • options are explicit (default) and node
    • by default our loader will not allow for optional extensions in the import, the path for a module must include the extension if there is one
    • by default our loader will not allow for importing directories that have an index file
    • developers can use --es-module-specifier-resolution=node to enable the commonjs specifier resolution algorithm
    • This is not a “feature” but rather an implementation for experimentation. It is expected to change before the flag is removed
  • --experimental-json-loader
    • the only way to import json when "type": "module"
    • when enable all import 'thing.json' will go through the experimental loader independent of mode
    • based on whatwg/html#4315
  • You can use package.main to set an entry point for a module
    • the file extensions used in main will be resolved based on the type of the module
@MylesBorins

This comment has been minimized.

Copy link
Member Author

MylesBorins commented Mar 18, 2019

@MylesBorins MylesBorins requested review from guybedford and bmeck Mar 18, 2019

return `Cannot use ${typeString} because nearest parent package.json ` +
((typeFlag === 'module') ?
'includes "type": "commonjs"' : 'includes "type": "module",') +
` which controls the type to use for ${filename}`;

This comment has been minimized.

@BridgeAR

BridgeAR Mar 18, 2019

Member

This seems to miss an else case in case conflicts is neither 'extension' nor 'scope'.

@devsnek
Copy link
Member

devsnek left a comment

I don't think the -m is a wise addition, being the only short flag that doesn't match the long one, and being the only short flag that can disagree with its long counterpart.

@antsmartian

This comment has been minimized.

Copy link
Contributor

antsmartian commented Mar 18, 2019

I was trying to build this locally on my mac machine to play around, but got back the following error:

/Users/anto/programs/node/node/out/Release/obj.target/v8_snapshot/geni/embedded.cc:5:10: fatal error: 'src/snapshot/macros.h' file not found
#include "src/snapshot/macros.h"
         ^~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[1]: *** [/Users/abelginrayen/programs/node/node/out/Release/obj.target/v8_snapshot/geni/embedded.o] Error 1
make[1]: *** Waiting for unfinished jobs....
rm f821e0c8fdac2538c3eb950594d8386c58c50c3d.intermediate b79cbb721bec5e5c6395b336bec4afcbcee7dbe9.intermediate
make: *** [node] Error 2

BTW, I'm on mac os sierra 10.12.6.

@MylesBorins

This comment has been minimized.

Copy link
Member Author

MylesBorins commented Mar 18, 2019

@antsmartian have you built node since the last V8 update? I had the same error as well and had to make clean and attempt to build again. This fixed it for me, (the issue was cache related)

@kristoferjoseph

This comment has been minimized.

Copy link

kristoferjoseph commented Mar 18, 2019

This PR doesn't mention .mjs what is the status of support for this extension?

@MylesBorins

This comment has been minimized.

Copy link
Member Author

MylesBorins commented Mar 18, 2019

@kristoferjoseph .mjs is unchanged. In the commonjs mode (which is deafult) .mjs is the only way to import esm. In the module mode .mjs can still be used.

@MylesBorins

This comment has been minimized.

Copy link
Member Author

MylesBorins commented Mar 18, 2019

@devsnek I've removed -m if you could remove your objection. We can bring the flag back to the modules team to review.

Show resolved Hide resolved doc/api/esm.md Outdated

@devsnek devsnek requested a review from vsemozhetbyt Mar 18, 2019

Show resolved Hide resolved doc/api/cli.md Outdated
Show resolved Hide resolved doc/api/cli.md Outdated
Show resolved Hide resolved doc/api/esm.md Outdated
Show resolved Hide resolved lib/internal/errors.js Outdated
Show resolved Hide resolved doc/api/cli.md Outdated
Show resolved Hide resolved doc/api/esm.md Outdated
Show resolved Hide resolved doc/api/esm.md Outdated

@MylesBorins MylesBorins force-pushed the MylesBorins:modules-lkgr branch from a21e48c to 682f125 Mar 18, 2019

@WebReflection

This comment has been minimized.

Copy link
Contributor

WebReflection commented Mar 18, 2019

FWIW I think entry-type makes little sense ...

  • it's just about the main field, really
  • entry means nothing in both CJS, ESM, and NodeJS or npm world
  • --package-type needs zero explanation if its meaning is to mimic the type field in the package file that would anyway just specify the parse goal of the main field

I'd honestly go either --type or --package-type ... any new term seems overly-superfluous in both specs and devs comprehension, IMHO

guybedford and others added some commits Aug 28, 2018

esm: irp type implementation
Refs: https://github.com/GeoffreyBooth/node-import-file-specifier-resolution-proposal
Refs: nodejs/modules#180
Refs: nodejs/ecmascript-modules#6
Refs: nodejs/ecmascript-modules#12
Refs: nodejs/ecmascript-modules#28
Co-authored-by: Myles Borins <MylesBorins@google.com>
Co-authored-by: John-David Dalton <john.david.dalton@gmail.com>
esm: add experimental .json support to loader
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: w3c/webcomponents#770
esm: add --es-module-specifier-resolution
There are currently two supported values
"explicit" and "node"
esm: scoped --type, cpp refactoring
Co-authored-by: Myles Borins <mylesborins@google.com>
doc: phase 2 documentation
Co-authored-by: Myles Borins <mylesborins@google.com>
Co-authored-by: Evan Plaice <evanplaice@gmail.com>
Co-authored-by: James M Snell <jasnell@gmail.com>

@MylesBorins MylesBorins force-pushed the MylesBorins:modules-lkgr branch from 173c8a3 to f180ddc Mar 18, 2019

added: REPLACEME
-->

To be used in conjunction with `--experimental modules`. Sets the resolution

This comment has been minimized.

@hiroppy

hiroppy Mar 18, 2019

Member

Please add -

- --experimental modules
+ --experimental-modules
@bcoe

bcoe approved these changes Mar 19, 2019

Copy link
Member

bcoe left a comment

Just a first pass reading through everything, but so far it's looking great to me.

Good work Myles! 🎉

else if (conflict === 'scope')
return `Cannot use ${typeString} because nearest parent package.json ` +
((typeFlag === 'module') ?
'includes "type": "commonjs"' : 'includes "type": "module",') +

This comment has been minimized.

@bcoe

bcoe Mar 19, 2019

Member

If there's no index.mjs, and type is module will it treat index.js as the entry-point, or would I need to set main: 'index.js?

SyntaxError: Unexpected reserved word
at internal/modules/esm/translators.js:*:*
at Loader.<anonymous> (internal/modules/esm/translators.js:*:*)

This comment has been minimized.

@bcoe

bcoe Mar 19, 2019

Member

nit: is it worth making the loader method a named function, just to make this exception a bit clearer?


const entry = fixtures.path('/es-modules/noext-esm');

const child = spawn(process.execPath, [

This comment has been minimized.

@bcoe

bcoe Mar 19, 2019

Member

I might add a comment describing the purpose of this test (it's relatively clear from the filename, but might still be nice; I like the approach I've seen in a few sections of the codebase of:

// description of test
{
  // test content, in closure to keep it contained.
}
@@ -301,6 +333,10 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
"show stack traces on process warnings",
&EnvironmentOptions::trace_warnings,
kAllowedInEnvironment);
AddOption("--entry-type",
"top-level module type name",

This comment has been minimized.

@bcoe

bcoe Mar 19, 2019

Member

nit: I find this description slightly confusing ... maybe:

default module syntax supported by application entry-point ... a little bit verbose, but maybe something like this?

@@ -29,11 +24,6 @@ enum HostDefinedOptions : int {
kLength = 10,
};

v8::Maybe<url::URL> Resolve(Environment* env,

This comment has been minimized.

@bcoe

bcoe Mar 19, 2019

Member

was PackageMainCheck and Resolve just vestigial?

process.exit(1);
});
// Handle any nextTicks added in the first tick of the program.
process._tickCallback();

This comment has been minimized.

@bcoe

bcoe Mar 19, 2019

Member

this feels a little fragile, do we have a unit test around this specific behavior (I think it would be worthwhile because someone will eventually try removing this function).

@@ -101,8 +101,7 @@ class ModuleJob {

async run() {
const module = await this.instantiate();
module.evaluate(-1, false);
return module;
return { module, result: module.evaluate(-1, false) };

This comment has been minimized.

@bcoe

bcoe Mar 19, 2019

Member

it might be worth documenting what -1 and false` indicate, it's a little magical ... I believe it's?

module.evaluate(timeout, breakOnSigint)?

@bcoe

This comment has been minimized.

Copy link
Member

bcoe commented Mar 19, 2019

not to paint a shed, but I don't love --entry-type either; better name isn't immediately jumping out at me though ... --module-type?

@richardlau richardlau referenced this pull request Mar 19, 2019

Closed

esm: Implement esm mode flag #18392

3 of 4 tasks complete
@WebReflection

This comment has been minimized.

Copy link
Contributor

WebReflection commented Mar 19, 2019

After thoughts: --package-type makes also no much sense when you are just running a file, instead of an npm package, and in this scenario --entry-type wins.

However, the way current dual modules are published is through the module field, while the main one is kept to specify the CJS entry.

Accordingly, to provide the least possible migration friction, how about the flag is kept as --entry-type but the package field related to such flag is entry instead of main?

In this way:

  • old modules won't ever break
  • current dual modules won't break neither
  • present module can still be published as dual
  • future modules will forgot main existed

If this is not OK, I wonder how current dual modules could be published / migrate.

edit following a couple of packages example (with comments)

the one with entry

{
  // the current/common dual module strategy
  "main": "cjs/index.js",
  "module": "esm/index.js",

  // the present + the future
  "type": "module",
  "entry": "esm/index.js"
}

Alternative to entry, one with type => field

{
  // preserved as is for backward compat
  "main": "cjs/index.js",

  // the type fields, also compatible with current ecosystem
  "module": "esm/index.js",
  // if omitted, fallback to "main"
  "commonjs": "cjs/index.js",

  // the type field, either module or commonjs
  "type": "module" // or "commonjs"
}
@GeoffreyBooth

This comment has been minimized.

Copy link
Contributor

GeoffreyBooth commented Mar 19, 2019

@WebReflection let’s discuss in nodejs/modules#296

@michael-ciniawsky

This comment has been minimized.

Copy link

michael-ciniawsky commented Mar 19, 2019

node -m|--module file.js // === node file.mjs
node file.js // === node file.cjs (default)

The concept of 'package scopes' is generally questionable compared to an explicit way of being able to deteremine the module type of a particular file e.g pkg.module && pkg.exports. entry-type, package-type && module-type are all terrible... (verbose, the prefixes to -type don't add much more clarity if not even the opposite). What in particular is confusing about --type="module|commonjs"? Seems to be pretty straightforward for what it is...

@joyeecheung

This comment has been minimized.

Copy link
Member

joyeecheung commented Mar 19, 2019

What in particular is confusing about --type="module|commonjs"? Seems to be pretty straightforward for what it is...

It's taking a very generic name, so it's going to be confusing when we add other flags that contain the word type in the future - or it forbids us to use the word type in any other future flags which doesn't seem justifiable. In the same spirit we have --openssl-config instead of --config, --inspect-port instead of --port, --http-parser instead of --parser,

@WebReflection

This comment has been minimized.

Copy link
Contributor

WebReflection commented Mar 19, 2019

@joyeecheung we should be a bit more pragmatic here, 'cause there are also perfectly valid cases such as:

  • --check
  • --eval
  • --help
  • --interactive
  • --print
  • --require
  • --version

Decontextualized, all of the above flags could be considered ambiguous, but these are actually simply the most commonly used one, and indeed they all have their counter shortcut.

Accordingly, since --type will be soon one of the most used flags too, and for a topic/switch that's been discussed for years now, and that also has already a meaning on the Web, reserving it for the sake of it, doesn't seem like a win for anyone, just augmented verbosity with no real benefits.

--inspect also has an --inspect-brk, so that having --type today that is about the least surprising configuration, wouldn't exclude the usage of --engine-type in the future (i.e. use v8 or something else).

Prefixes are also very good to explicit meanings in general, so I don't see why --type would be an issue here.

@michael-ciniawsky

This comment has been minimized.

Copy link

michael-ciniawsky commented Mar 19, 2019

It's taking a very generic name, so it's going to be confusing when we add other flags that contain the word type in the future

While I generally understand and not disagree, this actually doesn't seem to be an strictly 'enforced' naming convention atm either e.g node -c|--check -e|--eval -r|--require -l|--loader are all generic names used without a namespace... It also imho does make sense to use plain generic names for the main application itself where appropiated, simply by omitting the main application namespace

main --config ... # instead of main --main-config
main --component-config ...

Following the --ns-name convention the most explicit would be something like module(s)-*, more concretely

node --module-type="module" file.js
node --module-type="commonjs" file.js

which is perfectly fine (explicit, extensible). But since the (implicit) default

node file.js === node --module-type="commonjs" file.js`

already exists and to improve usability, it maybe justifiable to add a shorthand

node -m|--module file.js === node --module-type="module" file.js

for the common ESM usecase aswell ¯_(ツ)_/¯

@joyeecheung

This comment has been minimized.

Copy link
Member

joyeecheung commented Mar 19, 2019

Accordingly, since --type will be soon one of the most used flags too

Is the flag expected to be used frequently in the presence of the type field in package.json? What's the expected percentage of users who are going to use this?

that also has already a meaning on the Web

But it's not <type> on the Web, it's part of the <script> tag, which gives it context. type can be also set for <input> etc.

@devsnek

This comment has been minimized.

Copy link
Member

devsnek commented Mar 19, 2019

--loader is probably going to be renamed or removed anyway.

I guess you can argue that --check should be --check-entry

but overall, why would you use "we named things poorly in the past" as an argument? let's do better in the present, right?

@joyeecheung

This comment has been minimized.

Copy link
Member

joyeecheung commented Mar 19, 2019

Most of the single-worded flags are pretty old, except for --loader which is experimental as it's part of the ESM implementation.

Note that we just added

  • --diagnostic-report-directory
  • --diagnostic-report-filename
  • --diagnostic-report-on-fatalerror
  • --diagnostic-report-on-signal
  • --diagnostic-report-signal=
  • --diagnostic-report-uncaught-exception

to our CLI options (they are also still experimental). I'd like to see that we have some consistency around brevity of these flags before they go out of experiments at which point they are going to stay with us. (although in general, we have more verbose flags than succinct ones)

@michael-ciniawsky

This comment has been minimized.

Copy link

michael-ciniawsky commented Mar 19, 2019

I guess you can argue that --check should be --check-entry

No, it checks a 'script' without executing it. 'script' strictly doesn't need to be an entrypoint or the like... This is a good example where pseudo explicitness may make things actually harder to understand in the end...

@WebReflection

This comment has been minimized.

Copy link
Contributor

WebReflection commented Mar 19, 2019

Is the flag expected to be used frequently in the presence of the type field in package.json?

package.json is not always relevant (i.e. standalone CLI scripts)

What's the expected percentage of users who are going to use this?

I guess everyone that used @std/esm these years, or never cared about .mjs, will switch to node -m anything.

Various, including me, already said that alias node=node -m would be a no brainer for modern code

But it's not on the Web, it's part of the <script> tag, which gives it context.

the script context, in node, is replaced by node

  • script type module
  • node type module

The node is the script executor, and <input> has no practical meaning in node, but when/if it will, --input-type=radio will disambiguate pretty well.

I've said "we should be a bit more pragmatic here" not by accident, I literally meant it.

If you think people can get confused about HTML <input> type when they run node then I'd say everything we do has meanings in other worlds, right?

@MylesBorins

This comment has been minimized.

Copy link
Member Author

MylesBorins commented Mar 19, 2019

Hey y'all can we please move the discussion about flag names and behavior to nodejs/modules#296 in order to keep the comments in the PR focused on actionable changes? Thanks 😇

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.