Table table parsers parsers parsers parsers local table pipe

1 Introduction

The Markdown package1converts markdown2markup to TEX commands. The functionality is provided both as a Lua module and as plain TEX, LATEX, and ConTEXt macro packages that can be used to directly typeset TEX documents containing markdown markup. Unlike other convertors, the Markdown package does not require any external programs, and makes it easy to redefine how each and every markdown element is rendered. Creative abuse of the markdown syntax is encouraged.

implementation of the package. The technical documentation contains only a limited number of tutorials and code examples. You can find more of these in the user manual.3

1 local metadata = {
2 version = "(((VERSION)))",
3 comment = "A module for the conversion from markdown to plain TeX", 4
author = "John MacFarlane, Hans Hagen, Vít Novotný", 5 copyright = {"2009-2016 John MacFarlane, Hans Hagen",
6 "2016-2022 Vít Novotný"},
7 license
= "LPPL 1.3c" 8 }
9

The Lua part of the package requires that the following Lua modules are available from within the LuaTEX engine:

LPeg ě 0.10 A pattern-matching library for the writing of recursive descent parsers via the Parsing Expression Grammars (pegs). It is used by the Lunamark library to parse the markdown input. LPeg ě 0.10 is included in LuaTEX ě 0.72.0 (TEXLive ě 2013).

17 if not ran_ok then

3See .

21 local md5 = require("md5")

All the abovelisted modules are statically linked into the current version of the LuaTEX engine [1, Section 4.3]. Beside these, we also carry the following third-party Lua libraries:

22 x@@=markdowny 23 \ifx\ExplSyntaxOn\undefined

24 \input expl3-generic\relax
25 \fi

3

The Lua File System module is statically linked into the LuaTEX engine [1, Section 4.3].

26 \NeedsTeXFormat{LaTeX2e}%

a TEX engine that extends ε-TEX, and all the plain TEX prerequisites (see Section 1.1.2):

graphicx A package that provides the \includegraphics macro for the typesetting

of images.

It is used in the witiko/dot LATEX theme (see Section 2.3.2.2).

fancyvrb A package that provides the \VerbatimInput macros for the verbatim

TEXLive ě 2016.

4

soulutf8 A package that is used in the default renderer prototype for strike-throughs.

27 \RequirePackage{expl3}

Please use the Markdown project page on GitHub4to report bugs and submit feature requests. If you do not want to report a bug or request a feature but are simply in need of assistance, you might want to consider posting your question to the TEX-LATEX Stack Exchange.5community question answering web site under the markdown tag.

1.3 Acknowledgements

Funding by the the Faculty of Informatics at the Masaryk University in Brno [2] is gratefully acknowledged.

Support for content slicing (Lua options shiftHeadings and slice) and pipe tables (Lua options pipeTables and tableCaptions) was graciously sponsored by David Vins and Omedym.

Figure 1 shows the high-level structure of the Markdown package: The translation from markdown to TEX token renderers is exposed by the Lua layer. The plain TEX layer exposes the conversion capabilities of Lua as TEX macros. The LATEX and ConTEXt layers provide syntactic sugar on top of plain TEX macros. The user can interface with any and all layers.

2.1 Lua Interface

The Lua interface exposes the new(options) function. This function returns a conversion function from markdown to plain TEX according to the table options that contains options recognized by the Lua interface (see Section 2.1.3). The options parameter is optional; when unspecified, the behaviour will be the same as if options were an empty table.

6

2.1.2 User-Defined Syntax Extensions

For the purpose of user-defined syntax extensions, the Lua interface also exposes the reader object, which performs the lexical and syntactic analy-sis of markdown text and which exposes the reader->insert_pattern and reader->add_special_character methods for extending the peg grammar of markdown.

32 "Verbatim",

7

For example. if we’d like to insert pattern into the grammar between the Inline -> Emph and Inline -> Link rules, we would call reader->insert_pattern with "Inline after Emph" (or "Inline before Link") and pattern as the argu-ments.

The reader->add_special_character method adds a new character with special meaning to the grammar of markdown. The method receives the character as its only argument.

To enable the enumeration of Lua options, we will maintain the \g_@@_lua_options_seq sequence.

62 \ExplSyntaxOn
63 \seq_new:N \g_@@_lua_options_seq

138 }
139 \cs_generate_variant:Nn
140 \str_case_e:nn
141 { Vn }
142 \cs_generate_variant:Nn
143 \msg_error:nnnn

10

211 }
212 \cs_new:Nn
213 \@@_get_default_option_value:nN
214 {
215
\bool_set_false:N \l_tmpa_bool 216
217 \seq_map_inline:Nn
218 \g_@@_option_layers_seq
219
{ \prop_get:cnNT 220
221 { g_@@_default_ ##1 _options_prop } 222 { #1 }
223
#2 { 224
225 \bool_set_true:N
226 \l_tmpa_bool
227
\seq_map_break: } 228
229 }
230 \bool_if:nF
231
\l_tmpa_bool { 232
233 \msg_error:nnn
234 { @@ }
235
{ undefined-option } { #1 } 236
237 }

12

332 \seq_gput_right:Nn \g_@@_cases_seq { @@_snake_case:N }

2.1.4 File and Directory Names

334 { cacheDir }

335 { path }

338 \@@_add_lua_option:nnn

339 { contentBlocksLanguageMap }

The filename of the json file that will be produced when the debugExtensions option is enabled. This file will contain the extensible subset of the peg grammar of markdown (see the walkable_syntax hash table) after built-in syntax extensions (see Section 3.1.6) and user-defined syntax extensions (see Section 2.1.2) have been applied.

15

The frozen cache makes it possible to later typeset a plain TEX document that contains markdown documents without invoking Lua using the frozenCache plain TEX option. As a result, the plain TEX document becomes more portable, but further changes in the order and the content of markdown documents will not be reflected.

348 \@@_add_lua_option:nnn
349 { frozenCacheFileName }
350 { path }
351 { \markdownOptionCacheDir / frozenCache.tex }

false Do not require a blank line between a paragraph and the following blockquote.

353 \@@_add_lua_option:nnn
354 { blankBeforeBlockquote }
355 { boolean }
356 { false }

code block.

false Do not require a blank line between a paragraph and the following

361 { false }

362 defaultOptions.blankBeforeCodeFence = false

364 { blankBeforeDivFence }

365 { boolean }

false Do not require a blank line between a paragraph and the following

header.

372 defaultOptions.blankBeforeHeading = false

17

citationNbsps=true, false default: false

true Replace regular spaces with non-breaking spaces inside the prenotes and postnotes of citations produced via the pandoc citation syntax extension.

386 { true }

387 defaultOptions.citationNbsps = true

392 defaultOptions.citations = false

19

false Do not produce a json file with the peg grammar of markdown.

403 \@@_add_lua_option:nnn

This behavior will always be used if the finalizeCache option is enabled.

21

422 defaultOptions.expectJekyllData = false

24

437 defaultOptions.fancyLists = false

Whether an output file specified with the frozenCacheFileName option (frozen cache) that contains a mapping between an enumeration of markdown documents and their auxiliary cache files will be created.

The frozen cache makes it possible to later typeset a plain TEX document that contains markdown documents without invoking Lua using the frozenCache plain TEX option. As a result, the plain TEX document becomes more portable, but further changes in the order and the content of markdown documents will not be reflected.

Each frozen cache entry will define a TEX macro \markdownFrozenCachexnumbery that will typeset markdown document number xnumbery.

453 \@@_add_lua_option:nnn
454 { frozenCacheCounter }
455 { counter }
456 { 0 }

of spaces.

false Interpret all newlines within a paragraph as spaces.

462 defaultOptions.hardLineBreaks = false

472 defaultOptions.headerAttributes = false

473 \@@_add_lua_option:nnn

474 { html }

true Disable the escaping of special plain TEX characters, which makes it possible to intersperse your markdown markup with TEX code. The intended usage is in documents prepared manually by a human author. In such documents, it can often be desirable to mix TEX and markdown markup freely.

false Enable the escaping of special plain TEX characters outside verbatim environments, so that they are not interpretted by TEX. This is encour- aged when typesetting automatically generated content or markdown documents that were not prepared with this package in mind.

482 defaultOptions.hybrid = false

The inlineFootnotes option has been deprecated and will be removed in Markdown 3.0.0.

483 \@@_add_lua_option:nnn
484 { inlineFootnotes }
485 { boolean }
486 { false }
487 \@@_add_lua_option:nnn
488 { inlineNotes }
489 { boolean }
490 { false }

29

506 defaultOptions.footnotes = false
507 defaultOptions.notes = false

512 defaultOptions.pipeTables = false

To enable raw blocks, the fencedCode option must also be enabled:

518 \@@_add_lua_option:nnn
519 { rawAttribute }
520 { boolean }
521 { false }

522 defaultOptions.rawAttribute = true

All headings will be shifted by xshift amounty, which can be both positive and negative. Headings will not be shifted beyond level 6 or below level 1. Instead, those headings will be shifted to level 6, when xshift amounty is positive, and to level 1, when xshift amounty is negative.

528 \@@_add_lua_option:nnn
529 { shiftHeadings }
530 { number }
531 { 0 }

Two space-separated selectors that specify the slice of a document that will be

processed, whereas the remainder of the document will be ignored. The following

•$xidentifiery selects the end of a section with the html attribute #xidentifiery.•xidentifiery corresponds to ^xidentifiery for the first selector and to $xidentifiery for the second selector.

Specifying only a single selector, xidentifiery, is equivalent to specifying the two selectors xidentifiery xidentifiery, which is equivalent to ^xidentifiery $xidentifiery, i.e. the entire section with the html attribute #xidentifiery will be selected.

537 defaultOptions.slice = "^ $"

smartEllipses=true, false default: false

539 { smartEllipses }

540 { boolean }

34

567 defaultOptions.superscripts = false

572 defaultOptions.tableCaptions = false

36

The high-level operation of the Markdown package involves the communication between several programming layers: the plain TEX layer hands markdown documents to the Lua layer. Lua converts the documents to TEX, and hands the converted documents back to plain TEX layer for typesetting, see Figure 2.

This procedure has the advantage of being fully automated. However, it also has several important disadvantages: The converted TEX documents are cached on the file system, taking up increasing amount of space. Unless the TEX engine includes a Lua interpreter, the package also requires shell access, which opens the door for a malicious actor to access the system. Last, but not least, the complexity of the procedure impedes debugging.

\input\jobname.markdown.out

\jobname.pdf

600 When OUTPUT_FILE is unspecified, the result of the conversion will be 601 written to the standard output. When INPUT_FILE is also unspecified, the 602 result of the conversion will be read from the standard input.

603

Figure 3: A sequence diagram of the Markdown package typesetting a markdown document using the Lua command-line interface

607 local VERSION_STRING = [[
608 markdown-cli.lua (Markdown) ]] .. metadata.version .. [[ 609
610 Copyright (C) ]] .. table.concat(metadata.copyright, 611 "\nCopyright (C) ") .. [[ 612
613 License: ]] .. metadata.license
614
615 local function warn(s)
616 io.stderr:write("Warning: " .. s .. "\n") end
617
618 local function error(s)
619 io.stderr:write("Error: " .. s .. "\n")
620 os.exit(1)
621 end

650 if arg[i] == "--" then
651 process_options = false
652 goto continue
Unless the -- argument has been specified before, an argument containing the equals sign (=) is assumed to be an option specification in a xkeyy=xvaluey format. The available options are listed in Section 2.1.3.

653 elseif arg[i]:match("=") then
654 local key, value = arg[i]:match("(.-)=(.*)")
655 if defaultOptions[key] == nil then
656 key = various_case_options[key]
657 end
The defaultOptions table is consulted to identify whether xvaluey should be parsed as a string, number, table, or boolean.

Unless the -- argument has been specified before, an argument --version, or -v

687 os.exit()

688 end

690 if input_filename == nil then

691 input_filename = arg[i]

693 output_filename = arg[i]

694 else

41

The command-line Lua interface is implemented by the markdown-cli.lua file that can be invoked from the command line as follows:

The plain TEX interface provides macros for the typesetting of markdown input from within plain TEX, for setting the Lua interface options (see Section 2.1.3) used during the conversion from markdown to plain TEX and for changing the way markdown the tokens are rendered.

699 \def\markdownLastModified{(((LASTMODIFIED)))}% 700 \def\markdownVersion{(((VERSION)))}%

The interface exposes the \markdownBegin, \markdownEnd, \markdownInput, and \markdownEscape macros.

The \markdownBegin macro marks the beginning of a markdown document frag-ment and the \markdownEnd macro marks its end.

Another limitation concerns spaces at the right end of an input line. In mark-down, these are used to produce a forced line break. However, any such spaces are removed before the lines enter the input buffer of TEX [6, p. 46]. As a corrolary, the \markdownBegin macro also ignores them.

The \markdownBegin and \markdownEnd macros will also consume the rest of the lines at which they appear. In the following example plain TEX code, the characters c, e, and f will not appear in the output.

The \markdownInput macro accepts a single parameter with the filename of a markdown document and expands to the result of the conversion of the input markdown document to plain TEX.

703 \let\markdownInput\relax

43

fragment. Unlike the \input built-in of TEX, \markdownEscape guarantees that the standard catcode regime of your TEX format will be used.

705 \ExplSyntaxOn
706 \seq_new:N \g_@@_plain_tex_options_seq

To enable the reflection of default plain TEX options and their types, we will maintain the \g_@@_default_plain_tex_options_prop and \g_@@_plain_tex_option_types_prop property lists, respectively.

44

723 { false }

6. Publish the source code of the plain TEX document and the cacheDir directory.

2.2.2.2 File and Directory Names The \markdownOptionHelperScriptFileName macro sets the filename of the helper Lua script file that is created during the conversion from markdown to plain TEX in TEX engines without the \directlua primitive. It defaults to \jobname.markdown.lua, where \jobname is the base name of the document being typeset.

727 { \jobname.markdown.lua }

The helperScriptFileName macro has been deprecated and will be removed in Markdown 3.0.0. To control the filename of the helper Lua script file, use the \g_luabridge_helper_script_filename_str macro from the lt3luabridge package.

The \markdownOptionInputTempFileName macro sets the filename of the temporary input file that is created during the buffering of markdown text from a TEX source. It defaults to \jobname.markdown.in. The same limitations as in the case of the helperScriptFileName macro apply here.

733 \@@_add_plain_tex_option:nnn

737 \@@_add_plain_tex_option:nnn

738 { outputTempFileName }

741 \str_new:N
742 \g_luabridge_standard_output_filename_str 743 \tl_gset:Nn
744 \g_luabridge_standard_output_filename_str 745 { \markdownOptionOutputTempFileName }

The \markdownOptionErrorTempFileName macro sets the filename of the temporary output file that is created when a Lua error is encountered during the conver-sion from markdown to plain TEX in \markdownMode other than 2. It defaults to \jobname.markdown.err. The same limitations apply here as in the case of the helperScriptFileName macro.

755 \@@_add_plain_tex_option:nnn
756 { outputDir }
757 { path }
758 { . }

Here, we automatically define plain TEX macros for the above plain TEX options. Furthemore, we also define macros that map directly to the options recognized by the Lua interface, such as \markdownOptionHybrid for the hybrid Lua option (see Section 2.1.3), which are not processed by the plain TEX implementation, only passed along to Lua.

803 \l_tmpa_tl
804 \cs_set:cpn
805 { \l_tmpa_tl }
806 { #2 }
807 }
808 }
809 \cs_generate_variant:Nn
810 \@@_set_option_value:nn
811
{ nV } 812 \cs_new:Nn
813 \@@_define_option:n
814 {
815 \@@_option_tl_to_csname:nN
816 { #1 }
817 \l_tmpa_tl
818 \cs_if_free:cT
819 { \l_tmpa_tl }
820 {
821 \@@_get_option_type:nN
822 { #1 }
823 \l_tmpb_tl
824 \str_if_eq:NNT
825 \c_@@_option_type_counter_tl
826 \l_tmpb_tl
827 {
828 \@@_option_tl_to_csname:nN
829 { #1 }
830 \l_tmpa_tl
831 \int_new:c
832 { \l_tmpa_tl }
833 }
834 }
835 }
836 \@@_plain_tex_define_option_commands:

2.2.2.3 Miscellaneous Options The \markdownOptionStripPercentSigns macro controls whether a percent sign (%) at the beginning of a line will be discarded when buffering Markdown input (see Section 3.2.4) or not. Notably, this enables the use of markdown when writing TEX package documentation using the Doc LATEX package [7] or similar. The recognized values of the macro are true (discard) and false (retain). It defaults to false.

844 \prop_gput:Nnx

845 \g_@@_default_plain_tex_options_prop

The following TEX macros may occur inside the output of the converter functions

exposed by the Lua interface (see Section 2.1.1) and represent the parsed markdown

\g_@@_renderers_seq sequence.

849 \ExplSyntaxOn

852 \ExplSyntaxOff

2.2.3.1 Attribute Renderers The following macros are only produced, when the

\markdownRendererAttributeClassName represents the xclass namey of a mark-

down element (class="xclass namey ..." in HTML and .xclass namey in Markdown’s headerAttributes syntax extension). The macro receives a single attribute that

854 \markdownRendererAttributeIdentifierPrototype}%

855 \ExplSyntaxOn

49

860 \g_@@_renderer_arities_prop
861 { attributeIdentifier }
862 { 1 }
863 \ExplSyntaxOff
864 \def\markdownRendererAttributeClassName{%
865 \markdownRendererAttributeClassNamePrototype}% 866 \ExplSyntaxOn
867 \seq_gput_right:Nn
868 \g_@@_renderers_seq
869 { attributeClassName }
870 \prop_gput:Nnn
871 \g_@@_renderer_arities_prop
872 { attributeClassName }
873 { 1 }
874 \ExplSyntaxOff
875 \def\markdownRendererAttributeKeyValue{%
876 \markdownRendererAttributeKeyValuePrototype}% 877 \ExplSyntaxOn
878 \seq_gput_right:Nn
879 \g_@@_renderers_seq
880
{ attributeKeyValue } 881 \prop_gput:Nnn
882 \g_@@_renderer_arities_prop
883 { attributeKeyValue }
884 { 2 }
885 \ExplSyntaxOff

50

900 \seq_gput_right:Nn
901 \g_@@_renderers_seq
902 { blockQuoteEnd }
903 \prop_gput:Nnn
904 \g_@@_renderer_arities_prop
905 { blockQuoteEnd }
906 { 0 }
907 \ExplSyntaxOff

930 \def\markdownRendererUlBegin{%
931 \markdownRendererUlBeginPrototype}%
932 \ExplSyntaxOn
933 \seq_gput_right:Nn
934 \g_@@_renderers_seq

51

952 \def\markdownRendererUlItem{%
953 \markdownRendererUlItemPrototype}%
954 \ExplSyntaxOn
955 \seq_gput_right:Nn
956 \g_@@_renderers_seq
957
{ ulItem } 958 \prop_gput:Nnn
959 \g_@@_renderer_arities_prop
960 { ulItem }
961 { 0 }
962 \ExplSyntaxOff

The \markdownRendererUlItemEnd macro represents the end of an item in a bulleted list. The macro receives no arguments.

974 \def\markdownRendererUlEnd{%
975 \markdownRendererUlEndPrototype}%
976 \ExplSyntaxOn
977 \seq_gput_right:Nn
978 \g_@@_renderers_seq
979
{ ulEnd } 980 \prop_gput:Nnn
981 \g_@@_renderer_arities_prop
982 { ulEnd }
983 { 0 }
984 \ExplSyntaxOff

The \markdownRendererUlEndTight macro represents the end of a bulleted list that contains no item with several paragraphs of text (the list is tight). This macro will only be produced, when the tightLists option is disabled. The macro receives no arguments.

1002 \prop_gput:Nnn
1003 \g_@@_renderer_arities_prop
1004 { inputVerbatim }
1005 { 1 }
1006 \ExplSyntaxOff

The \markdownRendererInputFencedCode macro represents a fenced code block. This macro will only be produced, when the fencedCode option is enabled. The macro receives two arguments that correspond to the filename of a file contaning the code block contents and to the code fence infostring.

1029 \def\markdownRendererContentBlock{%
1030 \markdownRendererContentBlockPrototype}%

54

1062 \def\markdownRendererDlBegin{%
1063 \markdownRendererDlBeginPrototype}%
1064 \ExplSyntaxOn
1065 \seq_gput_right:Nn
1066 \g_@@_renderers_seq
1067
{ dlBegin } 1068 \prop_gput:Nnn
1069 \g_@@_renderer_arities_prop
1070 { dlBegin }
1071 { 0 }
1072 \ExplSyntaxOff

The \markdownRendererDlBeginTight macro represents the beginning of a defi-nition list that contains an item with several paragraphs of text (the list is not tight). This macro will only be produced, when the tightLists option is disabled. The macro receives no arguments.

1085 \markdownRendererDlItemPrototype}%
1086 \ExplSyntaxOn
1087 \seq_gput_right:Nn

1088 \g_@@_renderers_seq

The \markdownRendererDlItemEnd macro represents the end of a list of definitions for a single term.

1095 \def\markdownRendererDlItemEnd{%

1103 { dlItemEnd }

1104 { 0 }
1105 \ExplSyntaxOff

1111 { dlDefinitionBegin }
1112 \prop_gput:Nnn

1113 \g_@@_renderer_arities_prop

1118 \markdownRendererDlDefinitionEndPrototype}% 1119 \ExplSyntaxOn
1120 \seq_gput_right:Nn

1121 \g_@@_renderers_seq

1128 \def\markdownRendererDlEnd{%
1129 \markdownRendererDlEndPrototype}%
1130 \ExplSyntaxOn
1131 \seq_gput_right:Nn
1132 \g_@@_renderers_seq
1133
{ dlEnd } 1134 \prop_gput:Nnn
1135 \g_@@_renderer_arities_prop
1136 { dlEnd }
1137 { 0 }
1138 \ExplSyntaxOff

The \markdownRendererDlEndTight macro represents the end of a definition list that contains no item with several paragraphs of text (the list is tight). This macro will only be produced, when the tightLists option is disabled. The macro receives no arguments.

1155 { ellipsis }
1156 \prop_gput:Nnn
1157 \g_@@_renderer_arities_prop
1158 { ellipsis }
1159
{ 0 } 1160 \ExplSyntaxOff

2.2.3.10 Emphasis Renderers The \markdownRendererEmphasis macro represents an emphasized span of text. The macro receives a single argument that corresponds to the emphasized span of text.

The \markdownRendererFencedDivAttributeContextBegin and \markdownRendererFencedDi macros represent the beginning and the end of a div in which the attributes of the
div apply. The macros receive no arguments.

1183 \def\markdownRendererFencedDivAttributeContextBegin{%
1184 \markdownRendererFencedDivAttributeContextBeginPrototype}%

1205 \def\markdownRendererHeaderAttributeContextBegin{%
1206 \markdownRendererHeaderAttributeContextBeginPrototype}% 1207 \ExplSyntaxOn
1208 \seq_gput_right:Nn
1209 \g_@@_renderers_seq
1210
{ headerAttributeContextBegin } 1211 \prop_gput:Nnn
1212 \g_@@_renderer_arities_prop
1213 { headerAttributeContextBegin }
1214 { 0 }
1215 \ExplSyntaxOff
1216 \def\markdownRendererHeaderAttributeContextEnd{%
1217 \markdownRendererHeaderAttributeContextEndPrototype}% 1218 \ExplSyntaxOn
1219 \seq_gput_right:Nn
1220 \g_@@_renderers_seq
1221 { headerAttributeContextEnd }
1222 \prop_gput:Nnn
1223 \g_@@_renderer_arities_prop
1224 { headerAttributeContextEnd }

60

1238 \def\markdownRendererHeadingTwo{%
1239 \markdownRendererHeadingTwoPrototype}% 1240 \ExplSyntaxOn
1241 \seq_gput_right:Nn
1242 \g_@@_renderers_seq
1243
{ headingTwo } 1244 \prop_gput:Nnn
1245 \g_@@_renderer_arities_prop
1246 { headingTwo }
1247 { 1 }
1248 \ExplSyntaxOff

The \markdownRendererHeadingThree macro represents a third level heading. The macro receives a single argument that corresponds to the heading text.

The \markdownRendererHeadingFive macro represents a fifth level heading. The macro receives a single argument that corresponds to the heading text.

1271 \def\markdownRendererHeadingFive{%
1272 \markdownRendererHeadingFivePrototype}% 1273 \ExplSyntaxOn
1274 \seq_gput_right:Nn
1275 \g_@@_renderers_seq
1276
{ headingFive } 1277 \prop_gput:Nnn
1278 \g_@@_renderer_arities_prop
1279 { headingFive }
1280 { 1 }
1281 \ExplSyntaxOff

The \markdownRendererBlockHtmlCommentBegin and \markdownRendererBlockHtmlCommentE macros represent the beginning and the end of a block html comment. The macros
receive no arguments.

1293 \def\markdownRendererInlineHtmlComment{%

1301 { inlineHtmlComment }

1302 { 1 }
1303 \ExplSyntaxOff
1304 \def\markdownRendererBlockHtmlCommentBegin{%

1312 { blockHtmlCommentBegin }

1313 { 0 }
1314 \ExplSyntaxOff
1315 \def\markdownRendererBlockHtmlCommentEnd{%

1323 { blockHtmlCommentEnd }

1324 { 0 }
1325 \ExplSyntaxOff

2.2.3.16 Image Renderer The \markdownRendererImage macro represents an image. It receives four arguments: the label, the fully escaped uri that can be directly typeset, the raw uri that can be used outside typesetting, and the title of the link.

1348 \def\markdownRendererImage{%
1349 \markdownRendererImagePrototype}%
1350 \ExplSyntaxOn
1351 \seq_gput_right:Nn
1352 \g_@@_renderers_seq
1353
{ image } 1354 \prop_gput:Nnn
1355 \g_@@_renderer_arities_prop
1356 { image }
1357 { 4 }
1358 \ExplSyntaxOff

2.2.3.18 Line Break Renderer The \markdownRendererLineBreak macro repre-sents a forced line break. The macro receives no arguments.

1370 \def\markdownRendererLineBreak{%
1371 \markdownRendererLineBreakPrototype}%
1372 \ExplSyntaxOn
1373 \seq_gput_right:Nn
1374 \g_@@_renderers_seq
1375
{ lineBreak } 1376 \prop_gput:Nnn
1377 \g_@@_renderer_arities_prop
1378 { lineBreak }
1379 { 0 }
1380 \ExplSyntaxOff

65

documents may also be nested. Redefinitions of the macros should take this into account.

1399 \g_@@_renderer_arities_prop

1400 { documentBegin }

1410 \g_@@_renderer_arities_prop

1411 { documentEnd }

2.2.3.22 Note Renderer The \markdownRendererNote macro represents a note. This macro will only be produced, when the notes option is enabled. The macro receives a single argument that corresponds to the note text.

The \markdownRendererFootnote and \markdownRendererFootnotePrototype macros have been deprecated and will be removed in Markdown 3.0.0.

67

sents the beginning of an ordered list that contains an item with several paragraphs of text (the list is not tight). This macro will only be produced, when the fancyLists option is disabled. The macro receives no arguments.

1492 \def\markdownRendererFancyOlBegin{%
1493 \markdownRendererFancyOlBeginPrototype}% 1494 \ExplSyntaxOn
1495 \seq_gput_right:Nn
1496 \g_@@_renderers_seq
1497
{ fancyOlBegin } 1498 \prop_gput:Nnn
1499 \g_@@_renderer_arities_prop

68

1504 \markdownRendererFancyOlBeginTightPrototype}% 1505 \ExplSyntaxOn
1506 \seq_gput_right:Nn

1507 \g_@@_renderers_seq

The \markdownRendererOlItem macro represents an item in an ordered list. This macro will only be produced, when the startNumber option is disabled and the fancyLists option is disabled. The macro receives no arguments.

1514 \def\markdownRendererOlItem{%

1522 { olItem }

1523 { 0 }
1524 \ExplSyntaxOff

1530 { olItemEnd }

69

1547 \def\markdownRendererFancyOlItem{%
1548 \markdownRendererFancyOlItemPrototype}% 1549 \ExplSyntaxOn
1550 \seq_gput_right:Nn
1551 \g_@@_renderers_seq
1552
{ fancyOlItem } 1553 \prop_gput:Nnn
1554 \g_@@_renderer_arities_prop
1555 { fancyOlItem }
1556 { 0 }
1557 \ExplSyntaxOff

The \markdownRendererFancyOlItemEnd macro represents the end of an item in a fancy ordered list. This macro will only be produced, when the fancyLists option is enabled. The macro receives no arguments.

1569 \def\markdownRendererFancyOlItemWithNumber{%
1570 \markdownRendererFancyOlItemWithNumberPrototype}% 1571 \ExplSyntaxOn
1572 \seq_gput_right:Nn
1573 \g_@@_renderers_seq
1574
{ fancyOlItemWithNumber } 1575 \prop_gput:Nnn
1576 \g_@@_renderer_arities_prop
1577 { fancyOlItemWithNumber }
1578 { 1 }
1579 \ExplSyntaxOff

The \markdownRendererOlEnd macro represents the end of an ordered list that contains an item with several paragraphs of text (the list is not tight). This macro will only be produced, when the fancyLists option is disabled. The macro receives no arguments.

1595 \g_@@_renderers_seq
1596 { olEndTight }
1597 \prop_gput:Nnn
1598 \g_@@_renderer_arities_prop
1599
{ olEndTight } 1600 { 0 }
1601 \ExplSyntaxOff

The \markdownRendererFancyOlEnd macro represents the end of a fancy ordered list that contains an item with several paragraphs of text (the list is not tight). This macro will only be produced, when the fancyLists option is enabled. The macro receives no arguments.

{xprenotey}{xpostnotey}{xnamey} repeated xnumber of citationsy times. ceives the parameter {xnumber of citationsy} followed by xsuppress authory The

72

1627 \seq_gput_right:Nn

1628 \g_@@_renderers_seq

1633 { 1 }

1634 \ExplSyntaxOff

when the rawAttribute option is enabled.

1635 \def\markdownRendererInputRawInline{%

1640 { inputRawInline }

1641 \prop_gput:Nnn

The \markdownRendererInputRawBlock macro represents a raw block. The macro

receives two arguments: the filename of a file contaning the raw block and the raw

1648 \ExplSyntaxOn

1649 \seq_gput_right:Nn

1654 { inputRawBlock }

1655 { 2 }

1658 \markdownRendererLeftBracePrototype}%
1659 \ExplSyntaxOn
1660 \seq_gput_right:Nn

1661 \g_@@_renderers_seq

1669 \markdownRendererRightBracePrototype}% 1670 \ExplSyntaxOn
1671 \seq_gput_right:Nn

1672 \g_@@_renderers_seq

1680 \markdownRendererDollarSignPrototype}% 1681 \ExplSyntaxOn
1682 \seq_gput_right:Nn

1683 \g_@@_renderers_seq

1691 \markdownRendererPercentSignPrototype}% 1692 \ExplSyntaxOn
1693 \seq_gput_right:Nn

1694 \g_@@_renderers_seq

1699 { 0 }
1700 \ExplSyntaxOff
1701 \def\markdownRendererAmpersand{%

1702 \markdownRendererAmpersandPrototype}% 1703 \ExplSyntaxOn
1704 \seq_gput_right:Nn

1710 { 0 }
1711 \ExplSyntaxOff
1712 \def\markdownRendererUnderscore{%

1713 \markdownRendererUnderscorePrototype}% 1714 \ExplSyntaxOn
1715 \seq_gput_right:Nn

1721 { 0 }
1722 \ExplSyntaxOff
1723 \def\markdownRendererHash{%

1724 \markdownRendererHashPrototype}%
1725 \ExplSyntaxOn
1726 \seq_gput_right:Nn

1732 { 0 }
1733 \ExplSyntaxOff
1734 \def\markdownRendererCircumflex{%

1735 \markdownRendererCircumflexPrototype}% 1736 \ExplSyntaxOn
1737 \seq_gput_right:Nn

1743 { 0 }
1744 \ExplSyntaxOff

75

1786 { strikeThrough }
1787 { 1 }
1788 \ExplSyntaxOff

2.2.3.28 Subscript Renderer The \markdownRendererSubscript macro repre-sents a subscript span of text. The macro receives a single argument that cor-responds to the subscript span of text. This macro will only be produced, when the subscripts option is enabled.

77

• l – The corresponding column is left-aligned.

2.2.3.32 Thematic Break Renderer The \markdownRendererThematicBreak macro represents a thematic break. The macro receives no arguments.

The \markdownRendererHorizontalRule and \markdownRendererHorizontalRulePrototype macros have been deprecated and will be removed in Markdown 3.0.0.

1877 \def\markdownRendererTickedBox{%
1878 \markdownRendererTickedBoxPrototype}%
1879 \ExplSyntaxOn

79

1885 { tickedBox }

1886 { 0 }
1887 \ExplSyntaxOff
1888 \def\markdownRendererHalfTickedBox{%

1896 { halfTickedBox }

1897 { 0 }
1898 \ExplSyntaxOff
1899 \def\markdownRendererUntickedBox{%

1907 { untickedBox }

1908 { 0 }
1909 \ExplSyntaxOff

1915 { jekyllDataBegin }
1916 \prop_gput:Nnn

1917 \g_@@_renderer_arities_prop

1921 \def\markdownRendererJekyllDataEnd{%
1922 \markdownRendererJekyllDataEndPrototype}% 1923 \ExplSyntaxOn
1924 \seq_gput_right:Nn
1925 \g_@@_renderers_seq
1926
{ jekyllDataEnd } 1927 \prop_gput:Nnn
1928 \g_@@_renderer_arities_prop
1929 { jekyllDataEnd }
1930 { 0 }
1931 \ExplSyntaxOff

The \markdownRendererJekyllDataMappingBegin macro represents the begin-ning of a mapping in a yaml document. This macro will only be produced when the jekyllData option is enabled. The macro receives two arguments: the scalar key in the parent structure, cast to a string following yaml serialization rules, and the number of items in the mapping.

The \markdownRendererJekyllDataSequenceBegin macro represents the begin-ning of a sequence in a yaml document. This macro will only be produced when the jekyllData option is enabled. The macro receives two arguments: the scalar key in the parent structure, cast to a string following yaml serialization rules, and the number of items in the sequence.

1954 \def\markdownRendererJekyllDataSequenceBegin{%

1962 { jekyllDataSequenceBegin }

1963 { 2 }
1964 \ExplSyntaxOff

1970 { jekyllDataSequenceEnd }
1971 \prop_gput:Nnn

1972 \g_@@_renderer_arities_prop

1977 \markdownRendererJekyllDataBooleanPrototype}% 1978 \ExplSyntaxOn
1979 \seq_gput_right:Nn

1980 \g_@@_renderers_seq

1985 { 2 }

1986 \ExplSyntaxOff

rules.

1987 \def\markdownRendererJekyllDataNumber{%

1992 { jekyllDataNumber }

1993 \prop_gput:Nnn

The \markdownRendererJekyllDataString macro represents a string scalar value

in a yaml document. This macro will only be produced when the jekyllData option

2000 \ExplSyntaxOn

2001 \seq_gput_right:Nn

2006 { jekyllDataString }

2007 { 2 }

cast to a string following yaml serialization rules.

See also Section 2.2.4.1 for the description of the high-level expl3 interface that

2011 \ExplSyntaxOn
2012 \seq_gput_right:Nn
2013 \g_@@_renderers_seq
2014 { jekyllDataEmpty }
2015 \prop_gput:Nnn
2016
\g_@@_renderer_arities_prop 2017 { jekyllDataEmpty }
2018 { 1 }
2019 \ExplSyntaxOff

2.2.4 Token Renderer Prototypes

The \markdownRendererFootnote and \markdownRendererFootnotePrototype macros have been deprecated and will be removed in Markdown 3.0.0.

84

2040 \cs_new:Nn \@@_plaintex_define_renderer_prototype:n 2041 {
2042 \@@_renderer_prototype_tl_to_csname:nN
2043 { #1 }
2044
\l_tmpa_tl 2045 \prop_get:NnN
2046 \g_@@_renderer_arities_prop
2047 { #1 }
2048
\l_tmpb_tl 2049 \@@_plaintex_define_renderer_prototype:cV
2050 { \l_tmpa_tl }
2051 \l_tmpb_tl
2052
} 2053 \cs_new:Nn \@@_renderer_prototype_tl_to_csname:nN 2054 {
2055 \tl_set:Nn
2056 \l_tmpa_tl
2057 { \str_uppercase:n { #1 } }
2058 \tl_set:Nx
2059 #2
2060 {
2061 markdownRenderer
2062 \tl_head:f { \l_tmpa_tl }
2063 \tl_tail:n { #1 }
2064 Prototype
2065 }
2066 }
2067 \cs_new:Nn \@@_plaintex_define_renderer_prototype:Nn 2068 {
2069
\cs_generate_from_arg_count:NNnn 2070 #1
2071 \cs_set:Npn
2072 { #2 }
2073
{ } 2074 }
2075 \cs_generate_variant:Nn
2076 \@@_plaintex_define_renderer_prototype:Nn
2077 { cV }
2078 \@@_plaintex_define_renderer_prototypes:
2079 \ExplSyntaxOff

2.2.6 Miscellanea

The \markdownMakeOther macro is used by the package, when a TEX engine that does not support direct Lua access is starting to buffer a text. The plain TEX implementation changes the category code of plain TEX special characters to other, but there may be other active characters that may break the output. This macro should temporarily change the category of these to other.

2083 \catcode`\|=0\catcode`\\=12%

2084 |gdef|markdownBegin{%

86

The \markdownMode macro has been deprecated and will be removed in Markdown 3.0.0. The code that corresponds to \markdownMode value of 3 will be the only implementation.

The LATEX interface provides LATEX environments for the typesetting of markdown input from within LATEX, facilities for setting Lua, plain TEX, and LATEX options used during the conversion from markdown to plain TEX, and facilities for changing the way markdown tokens are rendered. The rest of the interface is inherited from the plain TEX interface (see Section 2.2).

The LATEX implementation redefines the plain TEX logging macros (see Sec-tion 3.2.1) to use the LATEX \PackageInfo, \PackageWarning, and \PackageError macros.

2120 \input markdown/markdown

The LATEX interface is implemented by the markdown.sty file, which can be loaded from the LATEX document preamble as follows:

2.3.1 Typesetting Markdown

The interface exposes the markdown and markdown* LATEX environments, and rede-fines the \markdownInput command.

You may prepend your own code to the \markdown macro and append your own code

to the \endmarkdown macro to produce special effects before and after the markdown

markdown document.

The following example LATEX code showcases the usage of the \markdownInput macro:

The LATEX options may be specified when loading the LATEX package, when using the markdown* LATEX environment or the \markdownInput macro (see Section 2.3), or via the \markdownSetup macro. The \markdownSetup macro receives the options

to set up as its only argument:

2127 \keys_set:nn

2128 { markdown/latex-options }

We may also store LATEX options as setup snippets and invoke them later using the \markdownSetupSnippet macro. The \markdownSetupSnippet macro receives two

arguments: the name of the setup snippet and the options to store:

See Section 2.3.2.2 for information on interactions between setup snippets and LATEX
themes. See Section 2.3.2.3 for information about invoking the stored setup snippets. To enable the enumeration of LATEX options, we will maintain the \g_@@_latex_options_seq sequence.

2150 \ExplSyntaxOn
2151 \seq_new:N \g_@@_latex_options_seq

Additionally, if we redefine token renderers and renderer prototypes ourselves, the default definitions will bring no benefit to us. Using the plain package option, we can keep the default definitions from the plain TEX implementation (see Section 3.2.2) and prevent the soft LATEX prerequisites in Section 1.1.3 from being loaded: The plain option must be set before or when loading the package. Setting the option after loading the package will have no effect.

91

typesets fenced code blocks with the dot infostring as images of directed graphs rendered by the Graphviz tools. The following code would first load the Markdown package, then the markdownthemewitiko_beamer_MU.sty LATEX package, and finally the markdownthemewitiko_dot.sty LATEX package:

2207 theme .code:n = { \@@_set_latex_theme:n { #1 } },

2208 }
2209 \ExplSyntaxOff

witiko/dot A theme that typesets fenced code blocks with the dot … infostring as images of directed graphs rendered by the Graphviz tools. The right tail of the infostring is used as the image title.

Typesetting the above document produces the output shown in Figure 4.

Symbol Layout Tree

Infix

Figure 4: Various formats of mathemathical formulae

witiko/graphicx/http A theme that adds support for downloading images whose

URL has the http or https protocol.

witiko/tilde A theme that makes tilde (~) always typeset the non-breaking space

even when the hybrid Lua option is disabled.

2216 \ProvidesPackage{markdownthemewitiko_tilde}[2021/03/22]%

Please, see Section 3.3.2.1 for implementation details of the example themes.

96

2258 }
2259 }
2260 }
2261 \cs_new:Nn \@@_latex_define_option_keyval:nnn 2262
{ 2263 \prop_get:cnN
2264 { g_@@_ #1 _option_types_prop }
2265 { #2 }
2266
\l_tmpa_tl 2267 \keys_define:nn
2268 { markdown/latex-options }
2269 {
2270
#3 .code:n = { 2271 \@@_set_option_value:nn
2272 { #2 }
2273 { ##1 }
2274
}, 2275 }
2276 \str_if_eq:VVT
2277 \l_tmpa_tl
2278
\c_@@_option_type_boolean_tl 2279 {
2280 \keys_define:nn
2281 { markdown/latex-options }
2282
{ 2283 #3 .default:n = { true },
2284 }
2285 }

For options of type clist, we assume that xkeyy is a regular English noun in plural (such as extensions) and we also define the xsingular keyy=xvaluey interface, where xsingular keyy is xkeyy after stripping the trailing -s (such as extension). Rather than setting the option to xvaluey, this interface appends xvaluey to the current value as the rightmost item in the list.

2356 \DeclareOption{finalizecache}{\markdownSetup{finalizeCache}}
2357 \DeclareOption{frozencache}{\markdownSetup{frozenCache}}
The following example LATEX code showcases a possible configuration of plain TEX interface options hybrid, smartEllipses, and cacheDir.

101

2405 { ####1 }
2406 },
2407 }
2408 }
2409 }
2410 \cs_generate_variant:Nn
2411 \@@_latex_define_renderer:nNn
2412 { ncV }
2413 \ExplSyntaxOff

102

2434 }
2435 \cs_new:Nn \@@_latex_define_renderer_prototype:nNn 2436 {
2437 \@@_with_various_cases:nn
2438
{ #1 } 2439 {
2440 \keys_define:nn
2441 { markdown/latex-options/renderer-prototypes } 2442
{ 2443 ##1 .code:n = {
2444 \cs_generate_from_arg_count:NNnn
2445 #2
2446
\cs_set:Npn 2447 { #3 }
2448 { ####1 }
2449 },
2450
} 2451 }
2452 }
2453 \cs_generate_variant:Nn
2454 \@@_latex_define_renderer_prototype:nNn
2455 { ncV }
2456 \ExplSyntaxOff

The ConTEXt interface provides a start-stop macro pair for the typesetting of mark-down input from within ConTEXt and facilities for setting Lua, plain TEX, and ConTEXt options used during the conversion from markdown to plain TEX. The rest of the interface is inherited from the plain TEX interface (see Section 2.2).

2457 \writestatus{loading}{ConTeXt User Module / markdown}% 2458 \startmodule[markdown]
2459 \unprotect

The ConTEXt interface is implemented by the t-markdown.tex ConTEXt module file that can be loaded as follows:

You may prepend your own code to the \startmarkdown macro and redefine the \stopmarkdown macro to produce special effects before and after the markdown block.

Note that the \startmarkdown and \stopmarkdown macros are subject to the same limitations as the \markdownBegin and \markdownEnd macros exposed by the plain TEX interface.

104

provided by the plain TEX interface, this macro also accepts ConTEXt interface options (see Section 2.4.2) as its optional argument. These options will only influnce this markdown document.

The ConTEXt options may be specified when using the \inputmarkdown macro (see Section 2.4), or via the \setupmarkdown macro. The \setupmarkdown macro receives the options to set up as its only argument:

2468 \ExplSyntaxOn
2469 \cs_new:Nn
2470 \@@_setup:n
2471 {
2472 \keys_set:nn
2473 { markdown/context-options }
2474 { #1 }
2475 }
2476 \long\def\setupmarkdown[#1]
2477
{ 2478 \@@_setup:n
2479 { #1 }
2480 }
2481 \ExplSyntaxOff

To make it easier to copy-and-paste options from Pandoc [4] such as fancy_lists, header_attributes, and pipe_tables, we accept snake_case in addition to camel-Case variants of options. As a bonus, studies [5] also show that snake_case is faster to read than camelCase.

2527 { ##1 }
2528 {
2529 { yes } { true }
2530 { no } { false }
2531 }
2532 { ##1 }
2533 }
2534 \@@_set_option_value:nV
2535 { #2 }
2536 \l_tmpa_tl
2537 },
2538 }
2539 \str_if_eq:VVT
2540 \l_tmpa_tl
2541 \c_@@_option_type_boolean_tl
2542 {
2543 \keys_define:nn
2544 { markdown/context-options }
2545 {
2546 #3 .default:n = { true },
2547 }
2548 }
2549 }
2550 \cs_generate_variant:Nn
2551
\@@_set_option_value:nn 2552 { nV }
2553 \@@_context_define_option_commands_and_keyvals: 2554 \ExplSyntaxOff

3 Implementation

107

The Lunamark Lua module implements writers for the conversion to various other formats, such as DocBook, Groff, or html. These were stripped from the module and the remaining markdown reader and plain TEX writer were hidden behind the converter functions exposed by the Lua interface (see Section 2.1).

The util.err method prints an error message msg and exits. If exit_code is provided, it specifies the exit code. Otherwise, the exit code will be 1.

2561 function util.err(msg, exit_code)
2562 io.stderr:write("markdown.lua: " .. msg .. "\n") 2563 os.exit(exit_code or 1)
2564 end

2581 function util.cache_verbatim(dir, string)
2582 string = string:gsub('[\r\n%s]*$', '')
2583 local name = util.cache(dir, string, nil, nil, ".verbatim") 2584 return name
2585 end

The util.table_copy method creates a shallow copy of a table t and its metatable.

2596 util.lookup_files = (function()
2597 local ran_ok, kpse = pcall(require, "kpse")
2598 if ran_ok then
2599 kpse.set_program_name("luatex")
2600
else 2601 kpse = { lookup = function(f, _) return f end } 2602 end
2603
2604 local function lookup_files(f, options)
2605 return kpse.lookup(f, options)
2606 end
2607
2608 return lookup_files
2609 end)()

The util.expand_tabs_in_line expands tabs in string s. If tabstop is specified, it is used as the tab stop width. Otherwise, the tab stop width of 4 characters is used. The method is a copy of the tab expansion algorithm from Ierusalimschy [11, Chapter 21].

2619 function util.walk(t, f)
2620 local typ = type(t)
2621 if typ == "string" then
2622 f(t)
2623
elseif typ == "table" then 2624 local i = 1
2625 local n
2626 n = t[i]
2627
while n do 2628 util.walk(n, f)
2629 i = i + 1
2630 n = t[i]
2631
end 2632 elseif typ == "function" then
2633 local ok, val = pcall(t)
2634 if ok then
2635
util.walk(val,f) 2636 end
2637 else
2638 f(tostring(t))
2639
end 2640 end

The util.flatten method flattens an array ary that does not contain cycles and returns the result.

2654 function util.rope_to_string(rope)
2655 local buffer = {}
2656 util.walk(rope, function(x) buffer[#buffer + 1] = x end) 2657 return table.concat(buffer)
2658 end

The util.rope_last method retrieves the last item in a rope. For the definition of a rope, see the definition of the util.walk method.

2683 function util.map(ary, f)
2684 local new = {}

111

Given a table char_escapes mapping escapable characters to escaped strings and optionally a table string_escapes mapping escapable strings to escaped strings, the util.escaper method returns an escaper function that escapes all occurances of escapable strings and characters (in this order).

The method uses LPeg, which is faster than the Lua string.gsub built-in method.

Create an LPeg capture escapable that produces the escaped string corresponding to the matched escapable character.

2695 local escapable = S(char_escapes_list) / char_escapes

pk, vq P string_escapes. Note that the pattern summation is not commutative and its operands are inspected in the summation order during the matching. As a

corrolary, the strings always take precedence over the characters.

2700 end

Create an LPeg capture escape_string that captures anything escapable does and

2702 return function(s)

2703 return lpeg.match(escape_string, s)

2706 function util.pathname(dir, file)
2707 if #dir == 0 then
2708 return file
2709 else
2710
return dir .. "/" .. file 2711 end
2712 end

3.1.2 HTML Entities

114

116

118

120

122

124

126

128

130

132

134

136

138

140

142

144

146

148

150

152

154

Given a string s of decimal digits, the entities.dec_entity returns the corre-sponding utf8-encoded Unicode codepoint.

4752 return unicode.utf8.char(tonumber("0x"..s)) 4753 end

Given a character entity name s (like ouml), the entities.char_entity returns the corresponding utf8-encoded Unicode codepoint.

4757 return "&" .. s .. ";"

4758 end

4761 M.writer = {}

The writer.new method creates and returns a new TEX writer object associated with the Lua interface options (see Section 2.1.3) options. When options are unspecified, it is assumed that an empty table was passed to the method.

4764 self.options = options

Parse the slice option and define writer->slice_begin, writer->slice_end, and writer->is_writing. The writer->is_writing member variable is mutable.

4792 self.nbsp = "\\markdownRendererNbsp{}"
Define writer->plain as a function that will transform an input plain text block s to the output format.

Define writer->pack as a function that will take the filename name of the output file prepared by the reader and transform it to the output format.

Define writer->linebreak as the output format of a forced line break.

4807 self.linebreak = "\\markdownRendererLineBreak\n{}"

Define a table writer->escaped_chars containing the mapping from special plain TEX characters (including the active pipe character (|) of ConTEXt) that need to be escaped for typeset content.

159

4843 else

4844 self.string = self.escape

Define writer->link as a function that will transform an input hyperlink to the output format, where lab corresponds to the label, src to uri, and tit to the title of the link.

Define writer->image as a function that will transform an input image to the output format, where lab corresponds to the label, src to the url, and tit to the title of the image.

160

Define writer->inline_html_comment as a function that will transform the con-tents of an inline html comment, to the output format, where contents are the contents of the html comment.

4928 function self.tickbox(f)

162

Define writer->blockquote as a function that will transform an input block quote s to the output format.

Define writer->document as a function that will transform a document d to the output format.

5094 function self.get_state()

166

Define writer->set_state as a function that restores the input state s and

returns the previous state of the writer.

writer->defer_call.

5107 function self.defer_call(f)

5112 self.set_state(state)

5113 return return_value

5118 end

3.1.4 Parsers

5162
5163 parsers.doubleasterisks 5164 parsers.doubleunderscores 5165 parsers.doubletildes
5166 parsers.fourspaces

5167
5168 parsers.any
5169 parsers.succeed
5170 parsers.fail

= P(1)
= P(true)
= P(false)

= S("!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~") = parsers.backslash / "" * parsers.escapable

5182 + parsers.spacing

5183 + parsers.tightblocksep) 5184 parsers.eof = -parsers.any
5185 parsers.nonindentspace = parsers.space^-3 * - parsers.spacechar 5186 parsers.indent = parsers.space^-3 * parsers.tab

5197 * C(parsers.linechar^1 * parsers.newline^- 1)
5198 parsers.sp = parsers.spacing^0
5199 parsers.spnl = parsers.optionalspace

5200 * (parsers.newline * parsers.optionalspace)^- 1
5201 parsers.line = parsers.linechar^0 * parsers.newline 5202 parsers.nonemptyline = parsers.line - parsers.blankline

match [^\]
for %, capture \k%
for [^%], capture \2k`1xmatchy reset k

ϵ

170

5215 * Cg((parsers.backslash -- even backslash
5216 * parsers.backslash)^1, "backslashes")
5217 + (parsers.backslash
5218
* (#parsers.percent 5219 * Cb("backslashes")
5220 / function(backslashes)
5221 return string.rep("\\", #backslashes / 2) 5222
end 5223 * C(parsers.percent)
5224 + #parsers.commented_line_letter
5225 * Cb("backslashes")
5226
* Cc("\\") 5227 * C(parsers.commented_line_letter))
5228 * Cg(Cc(""), "backslashes")))^0
5229 * (#parsers.percent
5230
* Cb("backslashes") 5231 / function(backslashes)
5232 return string.rep("\\", #backslashes / 2)
5233 end
5234
* ((parsers.percent -- comment 5235 * parsers.line
5236 * #parsers.blankline) -- blank line
5237 / "\n"
5238 + parsers.percent -- comment
5239 * parsers.line
5240 * parsers.optionalspace) -- leading tabs and spac 5241 + #(parsers.newline)
5242 * Cb("backslashes")
5243 * C(parsers.newline))
5244
5245 parsers.chunk
= parsers.line * (parsers.optionallyindentedline 5246 - parsers.blankline)^0
5247
5248 parsers.attribute_key_char = parsers.alphanumeric + S("_-")
5249 parsers.attribute_key = (parsers.attribute_key_char
5250 - parsers.dash - parsers.digit)
5251 * parsers.attribute_key_char^0
5252 parsers.attribute_value = ( (parsers.dquote / "")
5253 * (parsers.anyescaped - parsers.dquote)^0
5254 * (parsers.dquote / ""))
5255 + ( parsers.anyescaped - parsers.dquote - parsers.rbra 5256 - parsers.space)^0
5257
5258 parsers.attribute = (parsers.dash * Cc(".unnumbered"))
5259 + C((parsers.hash + parsers.period)
5260 * parsers.attribute_key)
5261 + Cs( parsers.attribute_key

5279 parsers.bulletchar = C(parsers.plus + parsers.asterisk + parsers.dash) 5280
5281 parsers.bullet = ( parsers.bulletchar * #parsers.spacing
5282 * (parsers.tab + parsers.space^- 3)
5283 + parsers.space * parsers.bulletchar * #parsers.spacing 5284 * (parsers.tab + parsers.space^-2) 5285 + parsers.space * parsers.space * parsers.bulletchar 5286 * #parsers.spacing
5287 * (parsers.tab + parsers.space^-1) 5288 + parsers.space * parsers.space * parsers.space
5289 * parsers.bulletchar * #parsers.spacing 5290 )
5291
5292 local function tickbox(interior)
5293 return parsers.optionalspace * parsers.lbracket
5294 * interior * parsers.rbracket * parsers.spacechar^1
5295 end
5296
5297 parsers.ticked_box = tickbox(S("xX")) * Cc(1.0)
5298 parsers.halfticked_box = tickbox(S("./")) * Cc(0.5)
5299 parsers.unticked_box = tickbox(parsers.spacechar^1) * Cc(0.0)
5300

3.1.4.3 Parsers Used for Markdown Code Spans

3.1.4.4 Parsers Used for Fenced Code Blocks

5318 local function captures_geq_length(_,i,a,b)
5319 return #a >= #b and i
5320 end
5321
5322 parsers.tilde_infostring
5323 = C((parsers.linechar
5324 - (parsers.spacechar^1 * parsers.newline))^0)
5325 * parsers.optionalspace
5326 * (parsers.newline + parsers.eof)
5327
5328 parsers.backtick_infostring
5329 = C((parsers.linechar
5330 - (parsers.backtick
5331 + parsers.spacechar^1 * parsers.newline))^0)
5332 * parsers.optionalspace
5333 * (parsers.newline + parsers.eof)
5334
5335 local fenceindent
5336 parsers.fencehead = function(char, infostring)
5337 return
C(parsers.nonindentspace) / function(s) fenceindent = #s end 5338 * Cg(char^3, "fencelength")
5339 * parsers.optionalspace * infostring
5340 end
5341
5342 parsers.fencehead_with_attributes
5343 = function(char)
5344 return C(parsers.nonindentspace) / function(s) fenceindent = #s end 5345 * Cg(char^3, "fencelength")
5346 * parsers.optionalspace * Ct(parsers.attributes)

= function(char)
C(parsers.line - parsers.fencetail(char)) / function(s)
local i = 1
local remaining = fenceindent
while true do
local c = s:sub(i, i)
if c == " " and remaining > 0 then
remaining = remaining - 1
i = i + 1
elseif c == "\t" and remaining > 3 then remaining = remaining - 4
i = i + 1
else
break
end
end
return s:sub(i)
end

3.1.4.5 Parsers Used for Markdown Tags and Links

5438 parsers.title = parsers.title_d + parsers.title_s + parsers.title_p 5439
5440 parsers.optionaltitle
5441 = parsers.spnl * parsers.title * parsers.spacechar^0 5442 + Cc("")

3.1.4.6 Parsers Used for HTML

5529 + (parsers.less - parsers.closeelt_exact(s)))^0
5530 * parsers.closeelt_exact(s) }
5531 end
5532
5533 local function parse_matched_tags(s,pos)
5534
local t = string.lower(lpeg.match(C(parsers.keyword),s,pos)) 5535 return lpeg.match(parsers.in_matched(t),s,pos-1)
5536 end
5537
5538 parsers.in_matched_block_tags = parsers.less
5539 * Cmt(#parsers.openelt_block, parse_matched_tags) 5540

3.1.4.7 Parsers Used for HTML Entities

5551 parsers.Inline = V("Inline")
5552 parsers.IndentedInline = V("IndentedInline")
5553
5554 -- parse many p between starter and ender
5555 parsers.between = function(p, starter, ender)
5556 local ender2 = B(parsers.nonspacechar) * ender
5557 return (starter * #parsers.nonspacechar * Ct(p * (p - ender2)^0) * ender2) 5558 end
5559
5560 parsers.urlchar = parsers.anyescaped - parsers.newline - parsers.more

3.1.4.10 Block Elements

5578 -- parse Atx heading start and return level
5579 parsers.heading_start = #parsers.hash * C(parsers.hash^-6)
5580 * -parsers.hash / length
5581
5582 -- parse setext header ending and return level
5583 parsers.heading_level = parsers.equal^1 * Cc(1) + parsers.dash^1 * Cc(2) 5584
5585 local function strip_atx_end(s)
5586 return s:gsub("[#%s]*\n$","")
5587 end

3.1.5 Markdown Reader

Make reader->parsers available as a local parsers variable that will shadow the global parsers table and will make reader->parsers easier to type in the rest of the reader code.

5601 local parsers = self.parsers

5605 end

Define iterlines as a function that iterates over the lines of the input string s, transforms them using an input function f, and reassembles them into a new string, which it returns.

180

If the parser function is top-level and the stripIndent Lua option is enabled, we will first expand tabs in the input string str into spaces and then we will count the minimum indent across all lines, skipping blank lines. Next, we will remove the minimum indent from all lines.

181

3.1.5.4 Parsers Used for Blockquotes (local)

184

3.1.5.6 Inline Elements (local)

185

188

3.1.5.9 Blank (local)

3.1.5.10 Headings (local)

6050 / strip_atx_end

6051 / self.parser_functions.parse_inlines)

6056 * Ct(parsers.linechar^1

6057 / self.parser_functions.parse_inlines)

6062 / writer.heading

6063

plain TEX output.

6065 function self.finalize_grammar(extensions)

use this table to override existing rules using the reader->update_rule method.

6066 local walkable_syntax = (function(global_walkable_syntax)

6071 return local_walkable_syntax

6072 end)(walkable_syntax)

191

192

Define a hash table of all characters with special meaning and add method reader->add_special_character that extends the hash table and updates the peg grammar of markdown.

195

6258 syntax[lhs] = parsers.fail

6259 for _, rhs in ipairs(rule) do

names that reference the peg grammar of Markdown.

6261 if type(rhs) == "string" then

6266 pattern = V(pattern)

6267 end

Finalize the parser by reacting to options and by producing special parsers for

difficult edge cases such as blocks nested in definition lists or inline content nested in

196

Return a function that converts markdown string input into a plain TEX output

out of the package version and the passed options recognized by the Lua interface

(see Section 2.1.3). The cacheDir option is disregarded.

6340 for _, i in ipairs(v) do

6341 opt_string[#opt_string+1] = k .. "=" .. tostring(i)

6346 end

6347 table.sort(opt_string)

6350 local function convert(input)

6351 local document = self.parser_functions.parse_blocks(input)

6356 ".md" .. writer.suffix)

6357 output = writer.pack(name)

If the finalizeCache option is enabled, populate the frozen cache in the

file frozenCacheFileName with an entry for markdown document number

6364 mode = "a"

6365 else

6370 .. [[" for writing]])

6371 assert(file:write([[\expandafter\global\expandafter\def\csname ]]

6376 return output

6377 end

Create extensions hash table that contains built-in syntax extensions. Syntax

extensions are functions that produce objects with two methods: extend_writer

3.1.6.1 Bracketed Spans The extensions.bracketed_spans function implements

the Pandoc bracketed spans syntax extension.

Define writer->span as a function that will transform an input bracketed span s

with attributes attr to the output format.

6390 "\\markdownRendererBracketedSpanAttributeContextEnd{}"}

6391 end

6396 local Span = parsers.between(parsers.Inline,

6397 parsers.lbracket,

6402 self.insert_pattern("Inline after Emph",

6403 Span, "Span")

citation syntax extension. When the citation_nbsps parameter is enabled, the

syntax extension will replace regular spaces with non-breaking spaces inside the

200

Define writer->citation as a function that will transform an input cita-tion name c to the output format. If option hybrid is enabled, use the writer->escape_minimal function. Otherwise, use the escape_citation function.

6427 end

Define writer->citations as a function that will transform an input array of citations cites to the output format. If text_cites is enabled, the citations should be rendered in-text, when applicable. The cites array contains tables with the following keys and values:

203

6547 Citations, "Citations")

6548

6553 end

3.1.6.3 Content Blocks The extensions.content_blocks function implements

The languages_json table maps programming language filename extensions to

fence infostrings. All language_map files located by the kpathsea library are loaded

204

Define writer->definitionlist as a function that will transform an input defi-nition list to the output format, where items is an array of tables, each of the form { term = t, definitions = defs }, where t is a term and defs is an array of definitions. tight specifies, whether the list is tight or not.

207

Define writer->fancylist as a function that will transform an input ordered list to the output format, where:
• items is an array of the list items,
• tight specifies, whether the list is tight or not,
• startnum is the number of the first list item,
• numstyle is the style of the list item labels from among the following: – Decimal – decimal arabic numbers,
– LowerRoman – lower roman numbers,
– UpperRoman – upper roman numbers,
– LowerAlpha – lower ASCII alphabetic characters, and
– UpperAlpha – upper ASCII alphabetic characters, and
• numdelim is the style of delimiters between list item labels and texts from among the following:
– Default – default style,
– OneParen – parentheses, and
– Period – periods.

Define writer->fancyitem as a function that will transform an input fancy ordered list item to the output format, where s is the text of the list item. If the optional parameter num is present, it is the number of the list item.

6916 M.extensions.fenced_code = function(blank_before_code_fence) 6917 return {
6918 name = "built-in fenced_code syntax extension",
6919 extend_writer = function(self)
6920
local options = self.options 6921

Define writer->codeFence as a function that will transform an input fenced code block s with the infostring i to the output format.

213

6950 parsers.tilde_infostring)

6951 end

6956 end

6957 return previous_pattern + fencestart

6962 }

6963 end

6964 M.extensions.fenced_divs = function(blank_before_div_fence)

6965 return {

214

216

7067 - parsers.hash
7068 - parsers.lbrace)^0)^1)
7069 / self.parser_functions.parse_inlines) 7070 * Cg(Ct(parsers.newline
7071 + (parsers.hash^1
7072 * parsers.optionalspace
7073 * parsers.attributes^-1
7074 + parsers.attributes)
7075 * parsers.optionalspace
7076 * parsers.newline), "attributes") 7077 * Cb("level")
7078 * Cb("attributes")
7079 / writer.heading
7080
7081 local SetextHeading = #(parsers.line * S("=-"))
7082 * (C(((parsers.linechar
7083
- (parsers.attributes 7084 * parsers.optionalspace
7085 * parsers.newline))
7086 * (parsers.linechar
7087
- parsers.lbrace)^0)^1) 7088 / self.parser_functions.parse_inlines) 7089 * Cg(Ct(parsers.newline
7090 + (parsers.attributes
7091
* parsers.optionalspace 7092 * parsers.newline)), "attributes") 7093 * parsers.heading_level
7094 * Cb("attributes")
7095
* parsers.optionalspace 7096 * parsers.newline
7097 / writer.heading
7098
7099 local Heading = AtxHeading + SetextHeading
7100 self.update_rule("Heading", function() return Heading end) 7101 end
7102 }
7103 end

Define writer->note as a function that will transform an input note s to the

output format.

218

7152 return ""
7153 end
7154
7155 local NoteRef = RawNoteRef / lookup_note
7156
7157 local NoteBlock
7158 = parsers.leader * RawNoteRef * parsers.colon
7159 * parsers.spnl * parsers.indented_blocks(parsers.chunk) 7160 / register_note
7161
7162 local Blank = NoteBlock + parsers.Blank
7163 self.update_rule("Blank", function() return Blank end)
7164
7165 self.insert_pattern("Inline after Emph",
7166 NoteRef, "NoteRef")
7167 end
7168
7169 self.add_special_character("^")
7170
end 7171 }
7172 end

7314 M.extensions.raw_attribute = function()
7315 return {
7316 name = "built-in raw_attribute syntax extension", 7317 extend_writer = function(self)
7318
local options = self.options 7319

Define writer->rawInline as a function that will transform an input inline raw span s with the raw attribute attr to the output format.

3.1.6.12 Strike-Through The extensions.strike_through function implements the Pandoc strike-through syntax extension.

7373 M.extensions.strike_through = function()
7374 return {
7375 name = "built-in strike_through syntax extension", 7376 extend_writer = function(self)

Define writer->subscript as a function that will transform a subscript span s of input text to the output format.

7404 local parsers = self.parsers

7405 local writer = self.writer

7410

7411 self.insert_pattern("Inline after Emph",

7416 }

7417 end

7420 name = "built-in superscripts syntax extension",

7421 extend_writer = function(self)

expect_jekyll_data parameter is true, then a markdown document may be-

gin directly with yaml metadata and may contain nothing but yaml metadata.

Define writer->jekyllData as a function that will transform an input yaml table

d to the output format. The table is the value for the key p in the parent table; if p

226

227

7571 function M.new(options)
Make the options table inherit from the defaultOptions table.

229

230

Finally, cast the user-defined syntax extension to the internal format of user extensions

7719 extend_writer = function() end,

7720 }

Produce and return a conversion function from markdown to plain TEX.

7725 local writer = M.writer.new(options)

7730 end

7731

7733
7734 local input
7735 if input_filename then
7736 local input_file = assert(io.open(input_filename, "r"),
7737 [[Could not open file "]] .. input_filename .. [[" for reading]]) 7738
input = assert(input_file:read("*a")) 7739 assert(input_file:close())
7740 else
7741 input = assert(io.read("*a"))
7742 end
7743

First, ensure that the options.cacheDir directory exists.

Since the Lua converter expects unix line endings, normalize the input. Also add a line ending at the end of the file in case the input file has none.

7757 local output = convert(input:gsub("\r\n?", "\n") .. "\n")
7758
7759 if output_filename then
7760 local output_file = assert(io.open(output_filename, "w"),
7761 [[Could not open file "]] .. output_filename .. [[" for writing]]) 7762
assert(output_file:write(output)) 7763 assert(output_file:close())
7764 else
7765 assert(io.write(output))
7766 end

7767 \ifx\markdownInfo\undefined

7768 \def\markdownInfo#1{%

7777 \errhelp{#2.}%

7778 \errmessage{(l.\the\inputlineno) markdown.tex error: #1}}% 7779 \fi

7800 \markdownRendererImage}%

234

235

7848 \def\markdownRendererCitePrototype#1{}%
7849 \def\markdownRendererTextCitePrototype#1{}%
7850 \def\markdownRendererTickedBoxPrototype{[X]}% 7851 \def\markdownRendererHalfTickedBoxPrototype{[/]}% 7852 \def\markdownRendererUntickedBoxPrototype{[ ]}% 7853 \def\markdownRendererStrikeThroughPrototype#1{#1}% 7854 \def\markdownRendererSuperscriptPrototype#1{#1}% 7855 \def\markdownRendererSubscriptPrototype#1{#1}%

7860 \str_case:nn

7861 { #2 }

7866 }
7867 \cs_gset_eq:NN

7868 \markdownRendererInputRawBlockPrototype

\c_@@_jekyll_data_scalar_tl The currently traversed branch of the yaml docu- ment contains a scalar value at depth p.

7871 \ExplSyntaxOn
7872 \seq_new:N \g_@@_jekyll_data_datatypes_seq

To keep track of our current place when we are traversing a yaml document, we will maintain the \g_@@_jekyll_data_wildcard_absolute_address_seq stack of keys using the \markdown_jekyll_data_push_address_segment:n macro.

7876 \seq_new:N \g_@@_jekyll_data_wildcard_absolute_address_seq 7877 \cs_new:Nn \markdown_jekyll_data_push_address_segment:n

7882 \seq_get_right:NN

7883 \g_@@_jekyll_data_datatypes_seq

For example, the name key in the following yaml document would correspond to the /*/person/name absolute wildcard:

237

We will construct \g_@@_jekyll_data_wildcard_absolute_address_tl using the \markdown_jekyll_data_concatenate_address:NN macro and we will con-struct both token lists using the \markdown_jekyll_data_update_address_tls: macro.

7900 \tl_new:N \g_@@_jekyll_data_wildcard_absolute_address_tl 7901 \tl_new:N \g_@@_jekyll_data_wildcard_relative_address_tl 7902 \cs_new:Nn \markdown_jekyll_data_concatenate_address:NN 7903 {
7904
\seq_pop_left:NN #1 \l_tmpa_tl 7905 \tl_set:Nx #2 { / \seq_use:Nn #1 { / } }
7906 \seq_put_left:NV #1 \l_tmpa_tl
7907 }
7908 \cs_new:Nn \markdown_jekyll_data_update_address_tls: 7909 {
7910 \markdown_jekyll_data_concatenate_address:NN
7911 \g_@@_jekyll_data_wildcard_absolute_address_seq 7912 \g_@@_jekyll_data_wildcard_absolute_address_tl 7913 \seq_get_right:NN
7914 \g_@@_jekyll_data_wildcard_absolute_address_seq 7915 \g_@@_jekyll_data_wildcard_relative_address_tl 7916 }

7936 \cs_new:Nn \markdown_jekyll_data_set_keyval:nn
7937 {
7938 \keys_set_known:nn
7939 { markdown/jekyllData }
7940
{ { #1 } = { #2 } } 7941 }
7942 \cs_generate_variant:Nn
\markdown_jekyll_data_set_keyval:nn 7943
7944 { Vn }
7945 \cs_new:Nn \markdown_jekyll_data_set_keyvals:nn
7946 {
7947 \markdown_jekyll_data_push:nN
7948 { #1 }
7949
\c_@@_jekyll_data_scalar_tl 7950 \markdown_jekyll_data_set_keyval:Vn
7951 \g_@@_jekyll_data_wildcard_absolute_address_tl
7952 { #2 }
7953
\markdown_jekyll_data_set_keyval:Vn 7954 \g_@@_jekyll_data_wildcard_relative_address_tl
7955 { #2 }
7956 \markdown_jekyll_data_pop:
7957
} Finally, we will register our macros as token renderer prototypes to be able to react to the traversal of a yaml document.

7958 \def\markdownRendererJekyllDataSequenceBeginPrototype#1#2{ 7959 \markdown_jekyll_data_push:nN
7960 { #1 }
7961 \c_@@_jekyll_data_sequence_tl
7962 }
7963 \def\markdownRendererJekyllDataMappingBeginPrototype#1#2{ 7964 \markdown_jekyll_data_push:nN
7965 { #1 }
7966 \c_@@_jekyll_data_mapping_tl
7967 }

7991 \ExplSyntaxOn
7992 \tl_new:N \g_@@_formatted_lua_options_tl 7993 \cs_new:Nn \@@_format_lua_options:
7994 {
7995 \tl_gclear:N
7996
\g_@@_formatted_lua_options_tl 7997 \seq_map_function:NN
7998 \g_@@_lua_options_seq
7999 \@@_format_lua_option:n
8000
} 8001 \cs_new:Nn \@@_format_lua_option:n
8002 {
\@@_typecheck_option:n 8003
8004 { #1 }
8005 \@@_get_option_type:nN
8006 { #1 }
\l_tmpa_tl 8007

240

The \markdownPrepare macro contains the Lua code that is executed prior to any conversion from markdown to plain TEX. It exposes the convert function for the use by any further Lua code.

8068 \def\markdownPrepare{%

8074 local md = require("markdown")
8075 local convert = md.new(\markdownLuaOptions) 8076 }%

3.2.4 Buffering Markdown Input

8109 }
8110 \let\markdownIfOption=\@@_if_option:nTF
8111 \ExplSyntaxOff

The macros \markdownInputFileStream and \markdownOutputFileStream con-tain the number of the input and output file streams that will be used for the IO operations of the package.

8118 \begingroup

Make the newline and tab characters active and swap the character codes of the backslash symbol (\) and the pipe symbol (|), so that we can use the backslash as an ordinary character inside the macro definition. Likewise, swap the character codes of the percent sign (%) and the ampersand (@), so that we can remove percent signs from the beginning of lines when stripPercentSigns is enabled.

If we are not reading markdown documents from the frozen cache, open the inputTempFileName file for writing.

The \markdownReadAndConvertStripPercentSigns macro will process the individ-ual lines of output, stipping away leading percent signs (%) when stripPercentSigns is enabled. Notice the use of the comments (@) to ensure that the entire macro is at a single line and therefore no (active) newline symbols (^^M) are produced.

If we are not reading markdown documents from the frozen cache and the ending token sequence does not appear in the line, store the line in the inputTempFileName file. If we are reading markdown documents from the frozen cache and the ending token sequence does not appear in the line, gobble the line.

Repeat with the next line.

8171 ^^M}@

8180 |endgroup

The following two sections of the implementation have been deprecated and will be removed in Markdown 3.0.0. The code that corresponds to \markdownMode value of 3 will be the only implementation.

The package assumes that although the user is not using the LuaTEX engine, their TEX distribution contains it, and uses shell access to produce and execute Lua scripts using the TEXLua interpreter [1, Section 4.1.1].

8192 \ifnum\markdownMode<2\relax
8193 \ifnum\markdownMode=0\relax
8194 \markdownWarning{Using mode 0: Shell escape via write18
8195 (deprecated, to be removed in Markdown 3.0.0)}% 8196 \else
\markdownWarning{Using mode 1: Shell escape via os.execute 8197 8198 (deprecated, to be removed in Markdown 3.0.0)}% 8199 \fi

The \markdownExecuteDirect macro executes the code it has received as its first argument by writing it to the output file stream 18, if Lua is unavailable, or by using the Lua os.execute method otherwise.

8214 \ifnum\markdownMode=0\relax
8215 \def\markdownExecuteDirect#1{\immediate\write18{#1}}% 8216 \else
8217 \def\markdownExecuteDirect#1{%
8218 \directlua{os.execute("\luaescapestring{#1}")}}% 8219 \fi

Swap the category code of the backslash symbol and the pipe symbol, so that we may use the backslash symbol freely inside the Lua code.

The following TEX code is intended for TEX engines that provide direct access to Lua (LuaTEX). The macro \markdownLuaExecute defined here and in Section 3.2.5

248

Swap the category code of the backslash symbol and the pipe symbol, so that we may use the backslash symbol freely inside the Lua code.

8270 \catcode`|=0%
8271 \catcode`\\=12%
8272 |gdef|markdownLuaExecute#1{%
8273 |directlua{%
8274 local function print(input)
8275 local output = {}
8276 for line in input:gmatch("[^\r\n]+") do 8277 table.insert(output, line)
8278 end
8279 tex.print(output)
8280 end
8281 #1
8282 }%
8283 }%
8284 |endgroup
8285 \fi

If we are reading from the frozen cache, input it, expand the corresponding

\markdownFrozenCachexnumbery macro, and increment frozenCacheCounter.

8298 |input|markdownOptionFrozenCacheFileName|relax

8299 |fi

8304 }{%

8305 |markdownInfo{Including markdown document "&1"}%

Since the Lua converter expects unix line endings, normalize the input. Also add a

250

8321 \gdef\markdownEscape#1{%

8322 \catcode`\%=14\relax

8327 }%

3.3 LATEX Implementation

8330 \markdownVersion\markdownVersionSpace markdown renderer]%

Use reflection to define the renderers and rendererPrototypes keys of

8334 \ExplSyntaxOff

3.3.1 Logging Facilities

8335 \let\markdownInputPlainTeX\markdownInput

8336 \renewcommand\markdownInput[2][]{%

The markdown, and markdown* LATEX environments are implemented using the \markdownReadAndConvert macro.

8355 \ExplSyntaxOn

To keep track of our current place when packages themes have been nested, we will maintain the \g_@@_latex_themes_seq stack of theme names.

8386 \markdownSetup{fencedCode}%

We load the ifthen and grffile packages, see also Section 1.1.3:

8390 \renewcommand\markdownRendererInputFencedCode[2]{%
8391 \def\next##1 ##2\relax{%
8392 \ifthenelse{\equal{##1}{dot}}{%
8393 \markdownIfOption{frozenCache}{}{%
8394
\immediate\write18{% 8395 if ! test -e #1.pdf.source || ! diff #1 #1.pdf.source; 8396 then
8397 dot -Tpdf -o #1.pdf #1;
8398
cp #1 #1.pdf.source; 8399 fi}}%

We include the typeset image using the image token renderer:

253

8405 \next#2 \relax}%

We load the catchfile and grffile packages, see also Section 1.1.3:

8408 \RequirePackage{catchfile,grffile}

8409 \newcount\markdown@witiko@graphicx@http@counter

8410 \markdown@witiko@graphicx@http@counter=0

receive two arguments that correspond to the URL of the online image and to the

pathname, where the online image should be downloaded. The command will produce

character, so that we can use percentage signs in the shell code:

8416 \begingroup

8419 \global\def\markdownRendererImagePrototype#1#2#3#4{^^A

8420 \begingroup

8423 \immediate\write18{^^A

8424 mkdir -p "\markdownOptionCacheDir";

254

If the image does not have the http or https protocols or the image has already been downloaded, the URL will be stored as-is:

8445 \renewcommand\markdownRendererTildePrototype{~}%

3.3.3 Options

255

8457 },
8458 }
8459 \@@_with_various_cases:nn
8460 { rendererPrototypes }
8461
{ 8462 \keys_define:nn
8463 { markdown/latex-options }
8464 {
8465
#1 .code:n = { 8466 \keys_set:nn
8467 { markdown/latex-options/renderer-prototypes } 8468 { ##1 }
8469
}, 8470 }
8471 }

To ensure that keys containing forward slashes get passed correctly, we replace all forward slashes in the nput with backslash tokens with category code letter and then undo the replacement. This means that if any unbraced backslash tokens with category code letter exist in the input, they will be replaced with forward slashes. However, this should be extremely rare.

The following configuration should be considered placeholder. If the plain package option has been enabled (see Section 2.3.2.1), none of it will take effect.

8528 \markdownIfOption{plain}{\iffalse}{\iftrue}

If we loaded the paralist package, define the respective renderer prototypes to make use of the capabilities of the package. Otherwise, define the renderer prototypes to fall back on the corresponding renderers for the non-tight lists.

8532 \ExplSyntaxOn
8533 \@ifpackageloaded{paralist}{
8534 \tl_new:N
8535 \l_@@_latex_fancy_list_item_label_number_style_tl 8536 \tl_new:N
8537 \l_@@_latex_fancy_list_item_label_delimiter_style_tl 8538 \cs_new:Nn
8539 \@@_latex_fancy_list_item_label_number:nn
8540 {
8541 \str_case:nn
8542 { #1 }
8543 {
8544 { Decimal } { #2 }
8545 { LowerRoman } { \int_to_roman:n { #2 } } 8546 { UpperRoman } { \int_to_Roman:n { #2 } } 8547 { LowerAlpha } { \int_to_alph:n { #2 } } 8548 { UpperAlpha } { \int_to_alph:n { #2 } } 8549 }
8550 }
8551 \cs_new:Nn
8552 \@@_latex_fancy_list_item_label_delimiter:n
8553 {
8554 \str_case:nn
8555 { #1 }
8556 {
8557 { Default } { . }
8558 { OneParen } { ) }
8559 { Period } { . }
8560 }
8561 }
8562 \cs_new:Nn
8563 \@@_latex_fancy_list_item_label:nnn
8564 {
8565 \@@_latex_fancy_list_item_label_number:nn
8566 { #1 }
8567 { #3 }
8568 \@@_latex_fancy_list_item_label_delimiter:n
8569 { #2 }
8570 }
8571 \cs_new:Nn
8572 \@@_latex_paralist_style:nn

with symbols to be used for tickboxes.

8666 \@ifpackageloaded{unicode-math}{

8671 \RequirePackage{amssymb}

8672 \markdownSetup{rendererPrototypes={

8677 \RequirePackage{fancyvrb}

8678 \RequirePackage{graphicx}

8683 dollarSign = {\textdollar},

8684 underscore = {\textunderscore},

We can capitalize on the fact that the expansion of renderers is performed by TEX

during the typesetting. Therefore, even if we don’t know whether a span of text is

8689 codeSpan = {%

8690 \ifmmode

8695 }}}

8696 \ExplSyntaxOn

Then, whether the span of text is part of a math formula or not depends on where the macro is

later used, which may easily be both inside and outside a math formula.

262

When the minted package is loaded, use it for syntax highlighting. The minted package is preferred over listings.

8788 \ExplSyntaxOn
8789 \def\markdownLATEXStrongEmphasis#1{%
8790 \str_if_in:NnTF
8791 \f@series
8792 { b }
8793 { \textnormal{#1} }
8794 { \textbf{#1} }
8795 }
8796 \ExplSyntaxOff
8797 \markdownSetup{rendererPrototypes={strongEmphasis={% 8798 \protect\markdownLATEXStrongEmphasis{#1}}}}

Support LATEX document classes that do not provide chapters.

8827 \ifx\markdownLaTeXCheckbox\markdownRendererHalfTickedBox 8828 \item[\markdownLaTeXCheckbox]%
8829 \expandafter\expandafter\expandafter\@gobble
8830 \else
8831 \ifx\markdownLaTeXCheckbox\markdownRendererUntickedBox 8832 \item[\markdownLaTeXCheckbox]%
8833 \expandafter\expandafter\expandafter\expandafter 8834 \expandafter\expandafter\expandafter\@gobble 8835 \else
8836 \item{}%
8837 \fi
8838 \fi
8839 \fi
8840 }

3.3.4.2 HTML elements If the html option is enabled and we are using TEX4ht9, we will pass HTML elements to the output HTML document unchanged.

8863 \newcount\markdownLaTeXCitationsCounter
8864
8865 % Basic implementation
8866 \RequirePackage{gobble}
8867 \def\markdownLaTeXBasicCitations#1#2#3#4#5#6{%
8868 \advance\markdownLaTeXCitationsCounter by 1\relax
8869
\ifx\relax#4\relax 8870 \ifx\relax#5\relax
8871 \ifnum\markdownLaTeXCitationsCounter>\markdownLaTeXCitationsTotal\relax 8872 \cite{#1#2#6}% Without prenotes and postnotes, just accumulate cites 8873 \expandafter\expandafter\expandafter
8874 \expandafter\expandafter\expandafter\expandafter
8875 \@gobblethree
8876 \fi
8877 \else% Before a postnote (#5), dump the accumulator
8878 \ifx\relax#1\relax\else
8879 \cite{#1}%
8880
\fi 8881 \cite[#5]{#6}%
8882 \ifnum\markdownLaTeXCitationsCounter>\markdownLaTeXCitationsTotal\relax 8883 \else
8884
\expandafter\expandafter\expandafter 8885 \expandafter\expandafter\expandafter\expandafter
8886 \expandafter\expandafter\expandafter
8887 \expandafter\expandafter\expandafter\expandafter
8888
\markdownLaTeXBasicCitations 8889 \fi
8890 \expandafter\expandafter\expandafter
8891 \expandafter\expandafter\expandafter\expandafter{%
8892
\expandafter\expandafter\expandafter 8893 \expandafter\expandafter\expandafter\expandafter}%
8894 \expandafter\expandafter\expandafter
8895 \expandafter\expandafter\expandafter\expandafter{%
8896
\expandafter\expandafter\expandafter 8897 \expandafter\expandafter\expandafter\expandafter}%
8898 \expandafter\expandafter\expandafter
8899 \@gobblethree
8900
\fi 8901 \else% Before a prenote (#4), dump the accumulator
8902 \ifx\relax#1\relax\else
8903 \cite{#1}%
8904 \fi
8905 \ifnum\markdownLaTeXCitationsCounter>1\relax
8906 \space % Insert a space before the prenote in later citations
8907
\fi 8908 #4~\expandafter\cite\ifx\relax#5\relax{#6}\else[#5]{#6}\fi
8909 \ifnum\markdownLaTeXCitationsCounter>\markdownLaTeXCitationsTotal\relax

266

9004 \ifx\relax#1\relax\relax\else
9005 \citet{#1}%
9006 \fi
9007 , \citet[#3][#4]{#5}%
9008 \ifnum\markdownLaTeXCitationsCounter<\markdownLaTeXCitationsTotal\relax 9009 ,
9010 \else
9011 \ifnum\markdownLaTeXCitationsCounter=\markdownLaTeXCitationsTotal\relax 9012 ,
9013 \fi
9014 \fi
9015 \expandafter\expandafter\expandafter
9016 \markdownLaTeXNatbibTextCitations
9017 \expandafter\expandafter\expandafter{%
9018 \expandafter\expandafter\expandafter}%
9019 \expandafter
9020 \@gobbletwo
9021 \fi\markdownLaTeXNatbibTextCitations{#1,#5}}
9022
9023 % BibLaTeX implementation
9024 \def\markdownLaTeXBibLaTeXCitations#1#2#3#4#5{%
9025
\advance\markdownLaTeXCitationsCounter by 1\relax 9026 \ifnum\markdownLaTeXCitationsCounter>\markdownLaTeXCitationsTotal\relax 9027 \autocites#1[#3][#4]{#5}%
9028 \expandafter\@gobbletwo
9029
\fi\markdownLaTeXBibLaTeXCitations{#1[#3][#4]{#5}}} 9030 \def\markdownLaTeXBibLaTeXTextCitations#1#2#3#4#5{%
9031 \advance\markdownLaTeXCitationsCounter by 1\relax
9032 \ifnum\markdownLaTeXCitationsCounter>\markdownLaTeXCitationsTotal\relax 9033 \textcites#1[#3][#4]{#5}%
9034 \expandafter\@gobbletwo
9035 \fi\markdownLaTeXBibLaTeXTextCitations{#1[#3][#4]{#5}}}
9036
9037 \markdownSetup{rendererPrototypes = {
9038 cite = {%
9039 \markdownLaTeXCitationsCounter=1%
9040 \def\markdownLaTeXCitationsTotal{#1}%
9041 \@ifundefined{autocites}{%
9042 \@ifundefined{citep}{%
9043 \expandafter\expandafter\expandafter
9044 \markdownLaTeXBasicCitations
9045 \expandafter\expandafter\expandafter{%
9046 \expandafter\expandafter\expandafter}%
9047 \expandafter\expandafter\expandafter{%
9048 \expandafter\expandafter\expandafter}%
9049 }{%
9050 \expandafter\expandafter\expandafter

269

9148 \newcount\markdownLaTeXRowCounter
9149 \newcount\markdownLaTeXRowTotal
9150 \newcount\markdownLaTeXColumnCounter
9151 \newcount\markdownLaTeXColumnTotal
9152 \newtoks\markdownLaTeXTable
9153 \newtoks\markdownLaTeXTableAlignment
9154 \newtoks\markdownLaTeXTableEnd
9155 \AtBeginDocument{%
9156 \@ifpackageloaded{booktabs}{%
9157 \def\markdownLaTeXTopRule{\toprule}%
9158 \def\markdownLaTeXMidRule{\midrule}%
9159 \def\markdownLaTeXBottomRule{\bottomrule}% 9160 }{%
9161 \def\markdownLaTeXTopRule{\hline}%
9162 \def\markdownLaTeXMidRule{\hline}%
9163 \def\markdownLaTeXBottomRule{\hline}% 9164 }%
9165 }
9166 \markdownSetup{rendererPrototypes={
9167 table = {%
9168 \markdownLaTeXTable={}%
9169 \markdownLaTeXTableAlignment={}%
9170 \markdownLaTeXTableEnd={%
9171 \markdownLaTeXBottomRule
9172 \end{tabular}}%
9173 \ifx\empty#1\empty\else
9174 \addto@hook\markdownLaTeXTable{%
9175 \begin{table}
9176 \centering}%

272

9226 \ExplSyntaxOn
9227 \keys_define:nn
9228 { markdown/jekyllData }
9229 {
9230 author
.code:n = { \author{#1} }, 9231 date .code:n = { \date{#1} },
9232 title .code:n = { \title{#1}
}, 9233 }

To complement the default setup of our key–values, we will use the \maketitle macro to typeset the title page of a document at the end of yaml metadata. If we are in the preamble, we will wait macro until after the beginning of the document. Otherwise, we will use the \maketitle macro straight away.

3.3.4.8 Raw Attribute Renderer Prototypes In the raw block and inline raw span renderer prototypes, execute the content with TeX when the raw attribute is tex or latex, display the content as markdown when the raw attribute is md, and ignore the content otherwise.

9267 \ExplSyntaxOn
9268 \cs_gset:Npn
9269 \markdownRendererInputRawInlinePrototype#1#2
9270 {
9271 \str_case:nn
9272 { #2 }
9273 {
9274 { tex } { \markdownEscape{#1} }
9275
{ latex } { \markdownEscape{#1} } 9276 { md } { \markdownInput{#1} }
9277 }
9278
} 9279 \cs_gset_eq:NN
9280 \markdownRendererInputRawBlockPrototype
9281 \markdownRendererInputRawInlinePrototype
9282 \ExplSyntaxOff
9283 \fi % Closes `\markdownIfOption{Plain}{\iffalse}{iftrue}`

3.4 ConTEXt Implementation

The ConTEXt implementation makes use of the fact that, apart from some subtle differences, the Mark II and Mark IV ConTEXt formats seem to implement (the documentation is scarce) the majority of the plain TEX format required by the plain TEX implementation. As a consequence, we can directly reuse the existing plain TEX implementation after supplying the missing plain TEX macros.

3.4.1 Typesetting Markdown

The \inputmarkdown is defined to accept an optional argument with options recog-nized by the ConTEXt interface (see Section 2.4.2).

9307 \ifx\startluacode\undefined % MkII
9308 \begingroup
9309 \catcode`\|=0%
9310 \catcode`\\=12%
9311 |gdef|startmarkdown{%
9312 |markdownReadAndConvert{\stopmarkdown}%
9313 {|stopmarkdown}}%
9314 |gdef|stopmarkdown{%
9315 |markdownEnd}%
9316 |endgroup
9317 \else % MkIV
9318 \startluacode
9319 document.markdown_buffering = false
9320 local function preserve_trailing_spaces(line)
9321 if document.markdown_buffering then
9322 line = line:gsub("[ \t][ \t]$", "\t\t")
9323 end
9324 return line
9325 end
9326 resolvers.installinputlinehandler(preserve_trailing_spaces) 9327 \stopluacode
9328 \begingroup
9329 \catcode`\|=0%
9330 \catcode`\\=12%
9331 |gdef|startmarkdown{%
9332 |ctxlua{document.markdown_buffering = true}%
9333 |markdownReadAndConvert{\stopmarkdown}%
9334 {|stopmarkdown}}%
9335 |gdef|stopmarkdown{%
9336 |ctxlua{document.markdown_buffering = false}%
9337 |markdownEnd}%
9338 |endgroup
9339 \fi

3.4.2 Token Renderer Prototypes

278

9397 \def\markdownRendererDlBeginTightPrototype{%

9406 \stopMarkdownConTeXtDlTightPrototype}%
9407 \def\markdownRendererEmphasisPrototype#1{{\em#1}}%
9408 \def\markdownRendererStrongEmphasisPrototype#1{{\bf#1}}%
9409 \def\markdownRendererBlockQuoteBeginPrototype{\startquotation}% 9410 \def\markdownRendererBlockQuoteEndPrototype{\stopquotation}% 9411 \def\markdownRendererInputVerbatimPrototype#1{\typefile{#1}}% 9412 \def\markdownRendererInputFencedCodePrototype#1#2{%

9413 \ifx\relax#2\relax

9425 \blackrule[height=1pt, width=\hsize]}%
9426 \def\markdownRendererNotePrototype#1{\footnote{#1}}%
9427 \def\markdownRendererTickedBoxPrototype{$\boxtimes$}
9428 \def\markdownRendererHalfTickedBoxPrototype{$\boxdot$}
9429 \def\markdownRendererUntickedBoxPrototype{$\square$}
9430 \def\markdownRendererStrikeThroughPrototype#1{\overstrikes{#1}} 9431 \def\markdownRendererSuperscriptPrototype#1{\high{#1}}
9432 \def\markdownRendererSubscriptPrototype#1{\low{#1}}

3.4.2.1 Tables There is a basic implementation of tables.

9498 \ExplSyntaxOn
9499 \cs_gset:Npn
9500 \markdownRendererInputRawInlinePrototype#1#2 9501 {
9502 \str_case:nn
9503 { #2 }
9504 {
9505 { tex } { \markdownEscape{#1} } 9506
{ context } { \markdownEscape{#1} } 9507 { md } { \markdownInput{#1} } 9508 }
9509 }

281

debugExtensions
debugExtensionsFileName defaultOptions
definitionLists

eagerCache
\endmarkdown
entities.char_entity
entities.dec_entity
entities.hex_entity
errorTempFileName
escape_citation
escaped_citation_chars
expandtabs
expectJekyllData
extensions
extensions.bracketed_spans extensions.citations
extensions.content_blocks extensions.definition_lists extensions.fancy_lists

hardLineBreaks
hashEnumerators
headerAttributes
helperScriptFileName html
hybrid

inlineNotes
\inputmarkdown
inputTempFileName iterlines

\markdown_jekyll_data_set_keyval:Nn
\markdown_jekyll_data_set_keyvals:nn
\markdown_jekyll_data_update_address_tls: \markdownBegin
\markdownEnd
\markdownError
\markdownEscape
\markdownExecute
\markdownExecuteDirect
\markdownExecuteShellEscape
\markdownIfOption
\markdownIfSnippetExists
\markdownInfo
\markdownInput
\markdownInputFileStream
\markdownInputPlainTeX
\markdownLuaExecute
\markdownLuaOptions
\markdownLuaRegisterIBCallback
\markdownLuaUnregisterIBCallback
\markdownMakeOther
\markdownMode
\markdownOptionErrorTempFileName
\markdownOptionFinalizeCache
\markdownOptionFrozenCache
\markdownOptionHelperScriptFileName
\markdownOptionHybrid
\markdownOptionInputTempFileName
\markdownOptionOutputDir
\markdownOptionOutputTempFileName
\markdownOptionStripPercentSigns
\markdownOutputFileStream
\markdownPrepare
\markdownPrepareLuaOptions
\markdownReadAndConvert
\markdownReadAndConvertProcessLine
\markdownReadAndConvertStripPercentSigns \markdownReadAndConvertTab
\markdownRendererAttributeClassName
\markdownRendererAttributeIdentifier
\markdownRendererAttributeKeyValue
\markdownRendererBlockHtmlCommentBegin

285

31, 31, 73
7, 107, 167, 179, 199 7, 8, 194
181
191
194
7, 8, 191, 196
180
179
181
181
180, 180
191, 193, 196
179
179, 179
32

tableCaptions
taskLists
texComments
tightLists

6, 35
36, 79, 264
36, 181
37, 52, 53, 56, 58, 68, 69, 71, 72, 257