Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign uptools: produce JSON documentation using unified/remark/rehype #21697
Conversation
nodejs-github-bot
added
doc
tools
labels
Jul 7, 2018
Trott
added
the
work in progress (WIP)
label
Jul 7, 2018
This comment has been minimized.
This comment has been minimized.
|
This pull request will require a lot of discussion. Other puzzles:
My current code is rejecting the constructor for console as the first signature defines parameters that are included in the list list below (at least not as first level items):
I can fix this up, but not seeing the Console constructor in the current output means that I'm flying blind. |
rubys
referenced this pull request
Jul 9, 2018
Open
Creating a branch of the Docs using RunKit #21723
jasnell
reviewed
Jul 10, 2018
| @@ -0,0 +1,491 @@ | |||
| // Copyright Joyent, Inc. and other Node contributors. | |||
This comment has been minimized.
This comment has been minimized.
jasnell
Jul 10, 2018
Member
Is this a brand new file or is it pulled from another source? brand new files do not require the copyright header to be added.
This comment has been minimized.
This comment has been minimized.
rubys
Jul 10, 2018
Author
Member
Ultimately, it will be renamed to tools/doc/json2.js, and contains significant content from the original source.
Currently separate so that side by side comparisons of the output can be produced, via jsondiff.js, which will be removed when this effort is complete.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Latest issue: files like buffer.json sometimes parses stability correctly, e.g.:
And other times misses it entirely:
The issue here is the embedded newline. The bug here is that |
This comment has been minimized.
This comment has been minimized.
|
This pull request is no longer Work remaining to be done before this lands: rename json2.js to json.js (overwriting the original source) and deleting jsondiff.js. For the moment, I'm leaving both sources and the diff tool with the thought that it would help reviewers. Once #21490 lands, further refactoring and cleanup is possible. |
Trott
added
blocked
and removed
work in progress (WIP)
labels
Jul 12, 2018
This comment has been minimized.
This comment has been minimized.
rubys
referenced this pull request
Jul 20, 2018
Closed
Produce HTML documentation using unified/remark/rehype #21490
Trott
removed
the
blocked
label
Jul 20, 2018
This comment has been minimized.
This comment has been minimized.
|
Removed the |
rubys
force-pushed the
rubys:remark-json
branch
from
edb8cc6
to
b80719b
Jul 21, 2018
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
CI is all green. |
This comment has been minimized.
This comment has been minimized.
|
I've compared the diff in <!--introduced_in=v...-->
<!-- type=misc -->
<!--type=class-->
<!--type=module-->
<!-- type=global -->
<!-- type=var -->
<!--type=event-->
<!--type=example-->
<!-- name=dgram -->
<!--name=esm-->
<!--name=fs-->
<!--name=module-->
<!--name=SIGINT, SIGHUP, etc.-->
<!--name=querystring-->
<!--name=vm-->Not sure if this is breaking in any aspect. Maybe we can ignore this change as these comments seem needed at the generation stage only. |
This comment has been minimized.
This comment has been minimized.
|
Ignorable note: nodes.forEach((node, i) => {-> nodes.forEach(({ type, value, children }, i) => {or: return nodes.map((node) => {-> return nodes.map(({ type, position, value, children }) => {etc. |
vsemozhetbyt
approved these changes
Jul 22, 2018
|
Just some comment nits. |
| const metaExpr = /<!--([^=]+)=([^-]+)-->\n*/g; | ||
| const stabilityExpr = /^Stability: ([0-5])(?:\s*-\s*)?(.*)$/s; | ||
|
|
||
| // extract definitions |
This comment has been minimized.
This comment has been minimized.
| // Extract (and remove) metadata that is not directly inferable | ||
| // from the markdown itself. | ||
| nodes.forEach((node, i) => { | ||
| // Input: <!-- name=module -->; output: {name: module} |
This comment has been minimized.
This comment has been minimized.
| values.push(item); | ||
| current = item; | ||
| // Pluralize type to determine which 'bucket' to put this section | ||
| // in. |
This comment has been minimized.
This comment has been minimized.
|
|
||
| delete section.list; | ||
| // Add this section to the parent. Sometimes we have two headings with a |
This comment has been minimized.
This comment has been minimized.
|
|
||
| delete section.list; | ||
| // Add this section to the parent. Sometimes we have two headings with a | ||
| // single blob of description. If the preceding entry at this level |
This comment has been minimized.
This comment has been minimized.
| ` > ${text}` | ||
| ); | ||
|
|
||
| // At this point, the name should match. If it doesn't find one that does. |
This comment has been minimized.
This comment has been minimized.
| } | ||
| return; | ||
| } | ||
| // Stablility marker: > Stability: ... |
This comment has been minimized.
This comment has been minimized.
| return src; | ||
| } | ||
|
|
||
|
|
||
| // This section parse out the contents of an H# tag. | ||
|
|
||
| // To reduse escape slashes in RegExp string components. |
This comment has been minimized.
This comment has been minimized.
| return src; | ||
| } | ||
|
|
||
|
|
||
| // This section parse out the contents of an H# tag. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
CC @nodejs/documentation @nodejs/build @nodejs/build-files: please, let us know if you plan to review this PR in the near future. Otherwise, I will land it, say, on Wednesday. |
This comment has been minimized.
This comment has been minimized.
Trott
approved these changes
Jul 23, 2018
|
Rubber-stamp LGTM |
This comment has been minimized.
This comment has been minimized.
rubys
added some commits
Jul 7, 2018
rubys
added some commits
Jul 21, 2018
rubys
force-pushed the
rubys:remark-json
branch
from
a5ee58c
to
93d2044
Jul 24, 2018
This comment has been minimized.
This comment has been minimized.
|
rebased |
This comment has been minimized.
This comment has been minimized.
vsemozhetbyt
added a commit
that referenced
this pull request
Jul 25, 2018
This comment has been minimized.
This comment has been minimized.
|
Landed in f41dd55 |
vsemozhetbyt
closed this
Jul 25, 2018
This comment has been minimized.
This comment has been minimized.
|
Depends on #21616 to land on |
rubys commentedJul 7, 2018
work in progress. tools/doc/json2.js is a first pass attempt to convert tools/doc/json.js to a new markdown toolchain. The original intent was to produce functionally identical JSON, but some oddities have been found along the way. Examples: (highest priority first):
The current (soon to be previous?) processing is based on a stream of tokens and a state machine. The new process is based on a pipeline of processors that have access to a tree of tokens. Care has been taken to not modify the stream so that this processor can be added to the pipeline that produces HTML.
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passes