|
@@ -10,6 +10,66 @@
|
|
|
//
|
|
|
// Another Codec for rendering Markdown in the terminal will be added in future.
|
|
|
//
|
|
|
+// # Why another Markdown implementation?
|
|
|
+//
|
|
|
+// The Elvish project uses Markdown in the documentation ("[elvdoc]") for the
|
|
|
+// functions and variables defined in builtin modules. These docs are then
|
|
|
+// converted to HTML as part of the website; for example, you can read the docs
|
|
|
+// for builtin functions and variables at https://elv.sh/ref/builtin.html.
|
|
|
+//
|
|
|
+// We used to use [Pandoc] to convert the docs from their Markdown sources to
|
|
|
+// HTML. However, we would also like to expand the elvdoc system in two ways:
|
|
|
+//
|
|
|
+// - We would like to support elvdocs in user-defined modules, not just
|
|
|
+// builtin modules.
|
|
|
+//
|
|
|
+// - We would like to be able to read elvdocs directly from the Elvish
|
|
|
+// program, without requiring a browser.
|
|
|
+//
|
|
|
+// With these requirements, Elvish itself needs to know how to parse and render
|
|
|
+// Markdown sources, so we need a Go implementation instead. There is a good Go
|
|
|
+// implementation, [github.com/yuin/goldmark], but it is quite large: linking it
|
|
|
+// into Elvish will increase the binary size by more than 1MB. In contrast,
|
|
|
+// including [Render] and [HTMLCodec] in Elvish only increases the binary size
|
|
|
+// by 150KB.
|
|
|
+//
|
|
|
+// By having a more narrow focus, this package is much smaller than goldmark,
|
|
|
+// and can be easily optimized for Elvish's use cases. That said, the
|
|
|
+// functionalities provided by this package still try to be as general as
|
|
|
+// possible, and can potentially be used by other people interested in a small
|
|
|
+// Markdown implementation.
|
|
|
+//
|
|
|
+// Besides elvdocs, all the other content on the Elvish website (https://elv.sh)
|
|
|
+// is also converted to HTML using Pandoc; additionally, they are formatted with
|
|
|
+// [Prettier]. Now that Elvish has its own Markdown implementation, we can use
|
|
|
+// it not just for rendering elvdocs, but also replace the use of Pandoc and
|
|
|
+// Prettier. These external tools are decent, but using them still came with
|
|
|
+// some frictions:
|
|
|
+//
|
|
|
+// - Even though both are relatively easy to set up, they can still be a
|
|
|
+// hindrance to casual contributors.
|
|
|
+//
|
|
|
+// - Since different versions of the same tool can behave differently,
|
|
|
+// we explicit specify their versions in both CI configurations and
|
|
|
+// [contributing instructions]. But this creates another problem: every time
|
|
|
+// these tools release new versions, we have to manually bump the versions,
|
|
|
+// and every contributor also needs to manually update them in their
|
|
|
+// development environments.
|
|
|
+//
|
|
|
+// Replacing external tools with this package removes these frictions.
|
|
|
+//
|
|
|
+// Additionally, this package is very easy to extend and optimize to suit
|
|
|
+// Elvish's needs:
|
|
|
+//
|
|
|
+// - We used to custom Pandoc using a mix of shell scripts, templates and Lua
|
|
|
+// scripts. While these customization options of Pandoc are well documented,
|
|
|
+// they are not something people are likely to be familiar with.
|
|
|
+//
|
|
|
+// With this implementation, everything is now done with Go code.
|
|
|
+//
|
|
|
+// - The Markdown formatter is much faster than Prettier, so it's now feasible
|
|
|
+// to run the formatter every time when saving a Markdown file.
|
|
|
+//
|
|
|
// # Which Markdown variant does this package implement?
|
|
|
//
|
|
|
// This package implements a large subset of the [CommonMark] spec, with the
|
|
@@ -46,17 +106,18 @@
|
|
|
// [CommonMark 0.30], with two minor changes in the syntax of [HTML blocks] and
|
|
|
// [inline HTML comments].
|
|
|
//
|
|
|
-// # Is this package relevant if I don't contribute to Elvish?
|
|
|
+// # Is this package useful outside Elvish?
|
|
|
//
|
|
|
-// You may still find this package interesting for the following reasons:
|
|
|
+// Yes! Well, hopefully. Assuming you don't use the features this package omits,
|
|
|
+// it can be useful in at least the following ways:
|
|
|
//
|
|
|
-// - The implementation is quite lightweight.
|
|
|
+// - The implementation is quite lightweight, so you can use it instead of a
|
|
|
+// more full-features Markdown library if small binary size is important.
|
|
|
//
|
|
|
-// A rough test shows that including the code to render Markdown into HTML
|
|
|
-// adds about 150KB to the binary size, while including just the parser of
|
|
|
-// [github.com/yuin/goldmark] adds more than 1MB to the binary size. (The
|
|
|
-// binary size increase depends on which packages the binary is already
|
|
|
-// including though, so your mileage may vary.)
|
|
|
+// As shown above, the increase in binary size when using this package in
|
|
|
+// Elvish is about 150KB, compared to more than 1MB when using
|
|
|
+// [github.com/yuin/goldmark]. You mileage may vary though, since the binary
|
|
|
+// size increase depends on which packages the binary is already including.
|
|
|
//
|
|
|
// - The formatter implemented by [FmtCodec] is heavily fuzz-tested to ensure
|
|
|
// that it does not alter the semantics of the Markdown.
|
|
@@ -76,64 +137,6 @@
|
|
|
// correctness, the corner cases will be interesting, regardless of which
|
|
|
// language you are using to implement the formatter.
|
|
|
//
|
|
|
-// # Why another Markdown implementation?
|
|
|
-//
|
|
|
-// The Elvish project uses Markdown in the documentation ("[elvdoc]") for the
|
|
|
-// functions and variables defined in builtin modules. These docs are then
|
|
|
-// converted to HTML as part of the website; for example, you can read the docs
|
|
|
-// for builtin functions and variables at https://elv.sh/ref/builtin.html.
|
|
|
-//
|
|
|
-// We used to use [Pandoc] to convert the docs from their Markdown sources to
|
|
|
-// HTML. However, we would also like to expand the elvdoc system in two ways:
|
|
|
-//
|
|
|
-// - We would like to support elvdocs in user-defined modules, not just
|
|
|
-// builtin modules.
|
|
|
-//
|
|
|
-// - We would like to be able to read elvdocs directly from the Elvish
|
|
|
-// program, without requiring a browser.
|
|
|
-//
|
|
|
-// With these requirements, Elvish itself needs to know how to parse and render
|
|
|
-// Markdown sources, so we need a Go implementation instead. There is a good Go
|
|
|
-// implementation, [github.com/yuin/goldmark], but it is quite large: linking it
|
|
|
-// into Elvish will increase the binary size by more than 1MB.
|
|
|
-//
|
|
|
-// By having a more narrow focus, this package is much smaller than goldmark,
|
|
|
-// and can be easily optimized for Elvish's use cases. That said, the
|
|
|
-// functionalities provided by this package still try to be as general as
|
|
|
-// possible, and can potentially be used by other people interested in a small
|
|
|
-// Markdown implementation.
|
|
|
-//
|
|
|
-// Besides elvdocs, all the other content on the Elvish website (https://elv.sh)
|
|
|
-// is also converted to HTML using Pandoc; additionally, they are formatted with
|
|
|
-// [Prettier]. Now that Elvish has its own Markdown implementation, we can use
|
|
|
-// it not just for rendering elvdocs, but also replace the use of Pandoc and
|
|
|
-// Prettier. These external tools are decent, but using them still came with
|
|
|
-// some frictions:
|
|
|
-//
|
|
|
-// - Even though both are relatively easy to set up, they can still be a
|
|
|
-// hindrance to casual contributors.
|
|
|
-//
|
|
|
-// - Since different versions of the same tool can behave differently,
|
|
|
-// we explicit specify their versions in both CI configurations and
|
|
|
-// [contributing instructions]. But this creates another problem: every time
|
|
|
-// these tools release new versions, we have to manually bump the versions,
|
|
|
-// and every contributor also needs to manually update them in their
|
|
|
-// development environments.
|
|
|
-//
|
|
|
-// Replacing external tools with this package removes these frictions.
|
|
|
-//
|
|
|
-// Additionally, this package is very easy to extend and optimize to suit
|
|
|
-// Elvish's needs:
|
|
|
-//
|
|
|
-// - We used to custom Pandoc using a mix of shell scripts, templates and Lua
|
|
|
-// scripts. While these customization options of Pandoc are well documented,
|
|
|
-// they are not something people are likely to be familiar with.
|
|
|
-//
|
|
|
-// With this implementation, everything is now done with Go code.
|
|
|
-//
|
|
|
-// - The Markdown formatter is much faster than Prettier, so it's now feasible
|
|
|
-// to run the formatter every time when saving a Markdown file.
|
|
|
-//
|
|
|
// [all the corner cases found by the fuzzer]: https://github.com/elves/elvish/tree/master/pkg/md/testdata/fuzz/FuzzFmtPreservesHTMLRender
|
|
|
// [fuzzing support]: https://go.dev/security/fuzz/
|
|
|
// [CommonMark 0.30]: https://spec.commonmark.org/0.30/
|