Index of /CTAN/macros/latex/contrib/l3build

(1)

The l3build package

Checking and building packages

The L

^A

TEX Project

^∗

Released 2022-09-15

Index 36

∗E-mail: [email protected]

(2)

1 The l3build system

1.1 Introduction

Thel3buildsystem is a Lua script for building TEX packages, with particular emphasis on regression testing. It is written in cross-platform Lua code, so can be used by any modern TEX distribution with thetexluainterpreter. Wrapper functions/binaries are distributed in the standard TEX distributions so that the script can be called usingl3buildon the command line; run without arguments it prints a brief synopsis of its usage.

Thel3buildsystem is designed for packages written in any TEX dialect; its defaults are set up for L^ATEX packages written in the DocStrip style. (Caveat: minimal testing has yet been performed for non-L^ATEX packages.)

Test files are written as standalone TEX documents using theregression-test.tex setup file; documentation on writing these tests is discussed in Section2.

Each package will define its ownbuild.luaconfiguration file which both sets variables (such as the name of the package) and may also provide custom functions.

A standard package layout might look something like the following:

abc/

abc.dtx abc.ins build.lua README.md support/

testfiles/

Most of this should look fairly self-explanatory. The top level support/ directory (optional) would contain any necessary files for compiling documentation, running regression tests, and so on.

Thel3buildsystem is also capable of building and checkingbundles of packages. To avoid confusion, we refer to either a standalone package or a package within a bundle as a module.

For example, within the L^ATEX project we have thel3packagesbundle which contains the xparse, xtemplate, etc., modules. These are all built and distributed as one bundle for installation, distributionviaCTAN and so forth.

Each module in a bundle will have its own build script, and a bundle build script brings them all together. A standard bundle layout would contain the following structure.

mybundle/

build.lua support/

yyy/ zoo/

build.lua build.lua

README.md README.md

testfiles/ testfiles/

yyy.dtx zoo.dtx

yyy.ins zoo.ins

All modules within a bundle must use the same build script name.

In a small number of cases, the name used by CTAN for a module or bundle is different from that used in the installation tree. For example, the L^ATEX 2ε kernel is

(3)

called latex-baseby CTAN but is located inside⟨texmf⟩/tex/latex/base. This can be handled by usingctanpkgfor the name required by CTAN to override the standard value.

Thetestfiles/folder is local to each module, and its layout consists of a series of regression tests with their outputs.

testfiles/

test1.lvt test1.tlg ...

support/

my-test.cls

Again, the support/ directory contains any files necessary to run some or all of these tests.

When the build system runs, it creates a directory build/ for various unpacking, compilation, and testing purposes. For a module, this build folder can be in the main directory of the package itself, but for a bundle it should be common for the bundle itself and for all modules within that bundle. Abuild/ folder can be safely deleted; all material within is re-generated for each command of the l3buildsystem.

1.2 The build.lua file

The build.lua file used to control l3build is a simple Lua file which is read during execution. In the current release of l3build, build.lua is read automatically and can access all of the global functions provided by the script. Thus it may contain a simple list of variable settingsor additional code to customize the build process.

The example scripts given in Section1.4largely cover the required knowledge in Lua programing. For a more advanced usage, one may consult general Lua documentations includinghttp://www.lua.org/manual/5.3/manual.htmland for the fewtexluaspe- cific additions see section 4.2 of the LuaTEX manual available locally withtexdoc luatex command line or athttps://www.pragma-ade.com/general/manuals/luatex.pdf.

1.3 Main build targets

In the working directory of a bundle or module,l3build is run by executing l3build ⟨target⟩ [⟨option(s)⟩]

where⟨target⟩can be one of the following:

• check

• check ⟨name(s)⟩

• clean

• ctan

• doc

• doc ⟨name(s)⟩

• install

• manifest

• save ⟨name(s)⟩

• tag [⟨tag name⟩]

• uninstall

• unpack

(4)

• upload [⟨version⟩]

These targets are described below.

As well as these targets, the system recognises the options

• --config(-c) Configuration(s) to use for testing

• --date Date to use when tagging data

• --debugRuns the target in debug mode (not supported by all targets)

• --dirtySkip cleaning up of the test area

• --dry-runRuns theinstalltarget but does not copy any files: simply lists those that would be installed

• --emailSets the email address for CTAN upload

• --engine(-e) Sets the engine to use for testing

• --epochSets the epoch for typesetting and testing

• --file (-F) Take the upload announcement from the given file

• --firstName of the first test to run

• --force(-f) Force checks to run even if sanity checks fail,e.g.when --engineis not given in checkengines

• --full Instructs theinstalltarget to include thedocandsourcetrees

• --halt-on-error(-H) Specifies that checks should stop as soon as possible, rather than running all requested tests; the difference file is printed in the terminal directly in the case of failure

• --last Name of the last test to run

• --message(-m) Text for upload announcement

• --quiet(-q) Suppresses output from unpacking

• --rerunRun tests without unpacking/set up

• --show-log-on-error To be used in addition to –halt-on-error and results in the full.log file of a failed test to be shown on the console

• --show-saves (-S) When tests fail, print thel3build savecommands needed to regenerate the tests assuming that the failures were false negatives.

• --shuffleShuffle the order in which tests run

• --texmfhome Sets the location of the user tree for installing

(5)

$ l3build check

Thecheckcommand runs the entire test suite. This involves iterating through each.lvt file in the test directory (specified by thetestfiledirvariable), compiling each test in a “sandbox” (a directory specified bytestdir), and comparing the output against each matching predefined .tlgfile.

If changes to the package or the typesetting environment have affected the results, the check for that file fails. A diff of the expected to actual output should then be inspected to determine the cause of the error; it is located in the testdir directory (defaultbuilddir .. "/test").

On Windows, thediffprogram is not available and sofcis used instead (generating an.fc file). Setting the environmental variables diffexeand diffextcan be used to adjust the choice of comparison made: the standard values are

Windows diffext = fc,diffexe = fc /n

*nix diffext = diff,diffexe = diff -c --strip-trailing-cr

The following files are moved into the “sandbox” for thecheckprocess:

• allinstallfilesafter unpacking;

• allcheckfiles after unpacking;

• any files in the directory testsuppdir;

• any files that matchchecksuppfilesin thesupportdir.

Thetexmfdiris also made available to the tests (if defined and non-empty). This range of possibilities allow sensible defaults but significant flexibility for defining your own test setups.

Checking can be performed with any or all of the ‘engines’ pdftex, xetex, and luatex. By default, each test is executed with all three, being compared against the.tlg file produced from thepdftexengine (these defaults are controlled by thecheckengines andstdenginevariable respectively). The format used for tests can be altered by setting checkformat: the default setting latex means that tests are run using e.g. pdflatex, whereas setting to texwill run tests using e.g.pdftex. (Currently, this should be one of latex ortex.) To perform the check, the engine typesets each test up tocheckruns times. More detail on this in the documentation onsave. Options passed to the binary are defined in the variablecheckopts.

By default, texmf trees are searched for input files when checking. This can be disabled by setting checksearch to false: isolation provides confidence that the tests cannot accidentally be running with incorrect files installed in the main distribution or hometexmf.

Thetexmfdirvariable sets a directory which is made available for recursive searching in additionto any files copied fromsupportdir. No subdivison oftexmfdiris attempted, thus it should not contain multiple files with the same name. The texmfdir is made available both to checking and typesetting.

$ l3build check ⟨name(s)⟩

Checks only the test⟨name(s)⟩.lvt. All engines specified bycheckengines are tested unless the command line option --engine(or -e) has been given to limit testing to a single engine. Normally testing is preceded by unpacking source files and copying the

(6)

result plus any additional support to the test directory: this may be skipped using the -soption.

$ l3build clean

This command removes all temporary files used for package bundling and regression testing. In the standard layout, these are all files within the directories defined bylocaldir, testdir,typesetdirandunpackdir, as well as all files defined in thecleanfilesvari- able in the same directory as the script. The defaults are .pdf files from typesetting (doc) and.zip files from bundling (ctan).

$ l3build ctan

Creates an archive of the package and its documentation, suitable for uploading to CTAN.

The archive is compiled indistribdir, and if the results are successful the resultant.zip file is moved into the same directory as the build script. If packtdszip is set true then the building process includes a .tds.zip file containing the ‘TEX Directory Structure’

layout of the package or bundle. The archive therefore may contain two ‘views’ of the package:

abc.zip/

abc/

abc.dtx abc.ins abc.pdf README.md abc.tds.zip/

doc/latex/abc/

abc.pdf README.md source/latex/abc/

abc.dtx abc.ins tex/latex/abc/

abc.sty

The files copied into the archive are controlled by a number of variables. The ‘root’ of the TDS structure is defined bytdsroot, which is"latex"by default. Plain users would redefine this to "plain"(or perhaps"generic"), for example. The build process for a .tds.zipfile currently assumes a ‘standard’ structure in which all extracted files should be placed inside thetextree in a single directory, as shown above. If the module includes any BibTEX or MakeIndex styles these will be placed in the appropriate subtrees.

Thedoctree is constructed from:

• all files matched bydemofiles,

• all files matched bydocfiles,

• all files matched bytypesetfileswith their extension replaced with.pdf,

• all files matched bytextfiles,

• all files matched bybibfiles.

(7)

Thesourcetree is constructed from all files matched bytypesetfilesandsourcefiles.

Thetextree from all files matched byinstallfiles.

The special case ctanreadme is used to allow renaming of a local foo.xyz file to README.xyz. The localfoo.xyzshould be listed intextfiles, and will be renamed as part of constructing the CTAN structure. The file extension will be unchanged by this process.

Files that should always be excluded from the archive are matched against the excludefilesvariable; by default this is {"*~"}, which match Emacs’ autosave files.

Binary files should be specified with thebinaryfiles variable (default{"*.pdf",

"*.zip"}); these are added to the zip archive without normalising line endings (text files are automatically converted to Unix-style line endings).

The intermediate build directories ctandir and tdsdir are used to construct the archive.

$ l3build doc

Compiles documentation files in thetypesetdirdirectory. In the absence of one or more file names, all documentation is typeset; a file list may be given at the command line for selective typesetting. If the compilation is successful the .pdf is moved back into the main directory.

The documentation compilation is performed with thetypesetexe binary (default pdflatex), with optionstypesetopts. Additional TEX material defined intypesetcmds is passed to the document (e.g., for writing\\PassOptionsToClass{l3doc}{letterpaper}, and so on—note that backslashes need to be escaped in Lua strings).

Files that match typesetsuppfiles in the support directory (supportdir) are copied into the build/docdirectory (typesetdir) for the typesetting compilation process. Additional dependencies listed in thetypesetdepsvariable (empty by default) will also be installed.

Source files specified insourcefilesandtypesetsourcefilesare unpacked before the typesetting takes place. (In most casestypesetsourcefileswill be empty, but may be used where there are files to unpackonly for typesetting.)

If typesetsearch is true (default), standard texmf search trees are used in the typesetting compilation. If set to false,allnecessary files for compilation must be included in thebuild/local sandbox.

$ l3build doc ⟨name(s)⟩

Typesets only the files with the⟨name(s)⟩given, which should be the basename without any extension.

$ l3build install

Copies all package files (defined by installfiles) into the user’s home texmf tree in the form of the TEX Directory Structure. The location of the user tree can be adjusted using the--texmfhomeswitch: the standard setting is the location set asTEXMFHOME.

$ l3build save ⟨name(s)⟩

This command runs through the same execution as check for a specific test(s)

⟨name(s)⟩.lvt. This command saves the output of the test to a .tlg file. This file is then used in all subsequent checks against the⟨name⟩.lvt test.

(8)

If the --engine(or -e) is specified (one of pdftex, xetex, or luatex), the saved output is stored in⟨name⟩.⟨engine⟩.tlg. This is necessary if running the test through a different engine produces a different output. A normalisation process is performed when checking to avoid common differences such as register allocation; full details are listed in section1.12.

If the recordstatus variable is set true, additional information will be added to the.tlgto record the “exit status” of the typesetting compilation of the.lvtfile. If the typesetting compilation completed without throwing an error (due to TEX programming errors, for example), the “exit status” is zero, else non-zero.

$ l3build manifest

Generates a ‘manifest’ file which lists the files of the package as known tol3build. The filename of this file (by default"MANIFEST.md") can be set with the variablemanifestfile.

The intended purpose of this manifest file is to include it within a package as metadata. This would allow, say, for the copyright statement for the package to refer to the manifest file rather than requiring the author to manually keep a file list up-to-date in multiple locations. The manifest file can be structured and documented with a degree of flexibility. Additional information is described in Section5.7.

In order for manifest to detect derived and typeset files, it should be run after runningunpack anddoc. If manifest is run after also runningctanit will include the files included in the CTAN and TDS directories as well.

Presently, this means that if you wish to include an up-to-date manifest file as part of actanrelease, you must run ctan/ manifest/ctan. Improvements to this process are planned for the future.

$ l3build tag [⟨tag name⟩]

Apply the Lua update_tag() function to modify the contents of files specified by tagfiles to update the ‘release tag’ (or package version) and date. The tag is given as the optional command line argument⟨tag name⟩and the date using--date(or -d).

If not given, the date will default to the current date in ISO format (YYYY-MM-DD).

If no ⟨tag name⟩is given, the tag will default to nil. Both are passed as arguments to theupdate_tag()function.

The standard setup does nothing unless tag update is set up by defining a custom update_tag()function. See Section4.2 for full details on this feature.

$ l3build unpack

This is an internal target that is normally not needed on user level. It unpacks all files into the directory defined byunpackdir. This occurs before other build commands such asdoc,check, etc.

The unpacking process is performed by executing theunpackexe(defaulttex) with optionsunpackopts on all files defined by theunpackfilesvariable; by default, all files that match{"*.ins"}.

If additional support files are required for the unpacking process, these can be enu- merated in theunpacksuppfilesvariable. Dependencies for unpacking are defined with unpackdeps.

By default this process allows files to be accessed in all standardtexmf trees; this can be disabled by settingunpacksearchtofalse.

(9)

$ l3build upload [⟨version⟩]

This target usescurl to upload the package zip file (created usingctan) to CTAN. To control the metadata used to upload the package, theuploadconfigtable should be pop- ulated with a number of fields. These are documented in Table2. Missing required fields will result in an interactive prompt for manual entry. When given, ⟨version⟩ overrides uploadconfig.version.

See Section4.6for full details on this feature.

1.4 Example build scripts

An example of a standalone build script for a package that uses self-contained.dtxfiles is shown in Figure1. Here, themoduleonly is defined, and since it doesn’t use.insfiles so the variableunpackfilesis redefined to runtexon the.dtxfiles instead to generate the necessary.styfiles. There are some PDFs in the repository that shouldn’t be part of a CTAN submission, so they’re explicitly excluded, and here unpacking is done ‘quietly’

to minimise console output when building the package.

An example of a bundle build script for l3packages is shown in Figure2. Note for L^ATEX we use a common file to set all build variables in one place, and the path to the l3build.lua script is hard-coded so we always use our own most recent version of the script. An example of an accompanying module build script is shown in Figure3.

A collection of full examples (source files in various layouts) are available athttps:

//github.com/latex3/l3build/tree/master/examples.

1.5 Variables

This section lists all variables defined in the l3build.lua script that are available for customisation.

Variable Default Description

module "" The name of the module

bundle "" The name of the bundle in which the module belongs

(where relevant)

ctanpkg module/bundle Name of the CTAN package matching this module

modules {} The list of all modules in a bundle (when not

auto-detecting)

exclmodules {} Directories to be excluded from automatic module detection

1 - - B u i l d c o n f i g u r a t i o n for b r e q n

2

3 m o d u l e = " b r e q n "

4

5 u n p a c k f i l e s = { " * . d t x " }

6 e x c l u d e f i l e s = { " */ breqn - abbr - t e s t . p d f " ,

7 " */ e q b r e a k s . p d f " }

8 u n p a c k o p t s = " - i n t e r a c t i o n = b a t c h m o d e "

Figure 1: The build configuration for the breqnpackage.

(10)

maindir "." Top level directory for the module/bundle

docfiledir "." Directory containing documentation files

sourcefiledir "." Directory containing source files

supportdir maindir .. "/support" Directory containing general support files testfiledir "./testfiles" Directory containing test files

testsuppdir testfiledir .. "/support" Directory containing test-specific support files texmfdir maindir .. "/texmf" Directory containing support files in tree form textfiledir "." Directory containing plain text files

builddir maindir .. "/build" Directory for building and testing

distribdir builddir .. "/distrib" Directory for generating distribution structure localdir builddir .. "/local" Directory for extracted files in “sandboxed” TEX runs resultdir builddir .. "/result" Directory for PDF files when using PDF-based tests testdir builddir .. "/test" Directory for running tests

typesetdir builddir .. "/doc" Directory for building documentation unpackdir builddir .. "/unpacked" Directory for unpacking sources ctandir distribdir .. "/ctan" Directory for organising files for CTAN

tdsdir distribdir .. "/tds" Directory for organised files into TDS structure

tdsroot "latex" Root directory of the TDS structure for the bundle/module to be installed into

auxfiles {"*.aux", "*.lof", "*.lot",

"*.toc"}

Secondary files to be saved as part of running tests

bibfiles {"*.bib"} BibTEX database files

binaryfiles {"*.pdf", "*.zip"} Files to be added in binary mode to zip files

bstfiles {"*.bst"} BibTEX style files

checkfiles { } Extra files unpacked purely for tests

checksuppfiles Files needed for performing regression tests

cleanfiles {"*.log", "*.pdf", "*.zip"} Files to delete when cleaning

demofiles {} Files which show how to use a module

docfiles {} Files which are part of the documentation but should not

be typeset

dynamicfiles { } Secondary files to cleared before each test is run

excludefiles {"*~"} Files to ignore entirely (default for Emacs backup files) installfiles {"*.sty","*.cls"} Files to install to thetexarea of thetexmftree makeindexfiles {"*.ist"} MakeIndex files to be included in a TDS-style zip

scriptfiles { } Files to install to thescriptsarea of thetexmftree

scriptmanfiles { } Files to install to thedoc/manarea of thetexmftree sourcefiles {"*.dtx", "*.ins",

"*-????-??-??.sty"}

Files to copy for unpacking

tagfiles {"*.dtx"} Files for automatic tagging

textfiles {"*.md", "*.txt"} Plain text files to send to CTAN as-is

typesetdemofiles {} Files to typeset before the documentation for inclusion in main documentation files

typesetfiles {"*.dtx"} Files to typeset for documentation

typesetsuppfiles {} Files needed to support typesetting when “sandboxed”

typesetsourcefiles{} Files to copy to unpacking when typesetting

unpackfiles {"*.ins"} Files to run to perform unpacking

unpacksuppfiles {} Files needed to support unpacking when “sandboxed”

(11)

includetests {"*"} Test names to include when checking

excludetests {} Test names to exclude when checking

checkdeps {} List of dependencies for running checks

typesetdeps {} List of dependencies for typesetting docs

unpackdeps {} List of dependencies for unpacking

checkengines {"pdftex", "xetex", "luatex"} Engines to check withcheckby default stdengine "pdftex" Engine to generate.tlgfile from

checkformat "latex" Format to use for tests

specialformats ⟨table⟩ Non-standard engine/format combinations

test_types ⟨table⟩ Custom test variants

test_order {"log", "pdf"} Which kinds of tests to evaluate

checkconfigs {} Configurations to use for tests

typesetexe "pdflatex" Executable for compilingdoc(s)

unpackexe "pdftex" Executable for runningunpack

biberexe "biber" Biber executable

bibtexexe "bibtex8" BibTEX executable

makeindexexe "makeindex" MakeIndex executable

curlexe "curl" Curl executable forupload

checkopts "-interaction=nonstopmode" Options passed to engine when running checks typesetopts "-interaction=nonstopmode" Options passed to engine when typesetting

unpackopts "" Options passed to engine when unpacking

biberopts "" Biber options

bibtexopts "-W" BibTEX options

makeindexopts "" MakeIndex options

checksearch true Switch to search the systemtexmffor during checking

typesetsearch true Switch to search the systemtexmffor during typesetting

unpacksearch true Switch to search the systemtexmffor during unpacking

glossarystyle "gglo.ist" MakeIndex style file for glossary/changes creation indexstyle "gind.ist" MakeIndex style for index creation

specialtypesetting⟨table⟩ Non-standard typesetting combinations forcecheckepoch "true" Force epoch when running tests forcedocepoch "false" Force epoch when typesetting

asciiengines {"pdftex"} Engines which should log as pure ASCII

checkruns 1 Number of runs to complete for a test before comparing the

log

forcecheckruns false Always runcheckrunsruns and never stop early

ctanreadme "README.md" Name of the file to send to CTAN asREADME.⟨ext⟩

ctanzip ctanpkg ... "-ctan" Name of the zip file (without extension) created for upload to CTAN

epoch 1463734800 Epoch (Unix date) to set for test runs

flatten true Switch to flatten any source structure when sending to

CTAN

flattentds true Switch to flatten any source structure when creating a TDS

structure

(12)

maxprintline 79 Length of line to use in log files

packtdszip false Switch to build a TDS-style zip file for CTAN

ps2pdfopts "" Options forps2pdf

typesetcmds "" Instructions to be passed to TEX when doing typesetting

typesetruns 3 Number of cycles of typesetting to carry out

recordstatus false Switch to include error level from test runs in.tlgfiles manifestfile "MANIFEST.md" Filename to use for the manifest file

tdslocations { } Map for non-standard file installations

tdsdirs { } List of ready-to-use source locations

uploadconfig ⟨table⟩ Metadata to describe the package for CTAN (see Table2)

uploadconfig.pkg ctanpkg Name of the CTAN package

bakext ".bak" Extension of backup files

dviext ".dvi" Extension of DVI files

lvtext ".lvt" Extension of log-based test files

tlgext ".tlg" Extension of test file output

tpfext ".tpf" Extension of PDF-based test output

lveext ".lve" Extension of auto-generating test file output

logext ".log" Extension of checking output, before processing it into a .tlg

pvtext ".pvt" Extension of PDF-based test files

pdfext ".pdf" Extension of PDF file for checking and saving

psext ".ps" Extension of PostScript files

1.6 Interaction between tests

Tests are run in a single directory, so whilst they are may be isolated from the system TEX tree they do share files. This may be significant if installation-type files are generated during a test, for example by a filecontents environment in L^ATEX. Typically, you should set up your tests such that they do not use the same names for such files: this may lead to variable outcomes depending on the order in which tests are run.

1.7 Selective running of tests

The variablesincludetestsandexcludetestsmay be used to select which tests are run:

these variables take raw testnamesnot full file names. The list of tests inexcludetests overrides any matches in includetests, meaning that tests can be disabled selectively.

It also makes it possible to disable test on for example a platform basis: the texlua specific variableos.typemay be used to setexcludetests only on some systems.

1.8 Multiple sets of tests

In most cases, a single set of tests will be appropriate for the module, with a common set of configuration settings applying. However, there are situations where you may need entirely independent sets of tests which have different setting values, for example using different formats or where the entire set will be engine-dependent. To support this,l3build offers the possibility of using multiple configurations for tests. This is supported using

(13)

thecheckconfigstable. This is used to list the names of each configuration (.lua file) which will be used to run tests.

For example, for the core L^ATEX 2ε tests the main test files are contained in a directory testfiles. To test font loading for X E TEX and LuaTEX there are a sec- ond set of tests in testfiles-TU which use the short build-TU.lua file shown in Figure 4. To run both sets of tests, the main build.lua file contains the setting checkconfigs = {"build", "config-TU"}. This will cause l3build to run first using no additional settings (i.e.reading the normal build.luafile alone), then runningalso loading the settings fromconfig-TU.lua.

To allow selection of one or more configurations, and to allow saving of .tlg files in non-standard configurations, the --config(-c) option may be used. This works in the same way as --engine: it takes a comma list of configurations to apply, overriding checkconfigs.

1.9 Dependencies

If you have multiple packages that are developed separately but still interact in some way, it’s often desirable to integrate them when performing regression tests. For L^ATEX, for example, when we make changes tol3kernelit’s important to check that the tests for l3packages still run correctly, so it’s necessary to include the l3kernel files in the build process forl3packages.

In other words, l3packages is dependent on l3kernel, and this is specified in l3build by setting appropriately the variables checkdeps, typesetdeps, andunpackdeps. The relevant parts of the L^ATEX repository is structured as the following.

l3/

l3kernel/

build.lua expl3.dtx expl3.ins ...

testfiles/

l3packages/

build.lua xparse/

build.lua testfiles/

xparse.dtx xparse.ins support/

For L^ATEX build files,maindiris defined as top level folderl3, so all support files are located here, and the build directories will be created there. To setl3kernelas a dependency of l3package, within l3packages/xparse/build.lua the equivalent of the following is set:

maindir = "../.."

checkdeps = {maindir .. "/l3kernel"}

This ensures that the l3kernel code is included in all processes involved in unpacking and checking and so on. The name of the script file in the dependency is set with the scriptname variable; by default these are"build.lua".

(14)

1 - - B u i l d s c r i p t for L a T e X " l 3 p a c k a g e s " f i l e s

2

3 - - I d e n t i f y the b u n d l e : t h e r e is no m o d u l e as t h i s is the " d r i v e r "

4 b u n d l e = " l 3 p a c k a g e s "

5

6 - - L o c a t i o n of m a i n d i r e c t o r y : use Unix - s t y l e p a t h s e p a r a t o r s

7 m a i n d i r = " .. "

Figure 2: The build script for thel3packagesbundle.

1 - - B u i l d s c r i p t for L a T e X " x p a r s e " f i l e s

2

3 - - I d e n t i f y the b u n d l e and m o d u l e :

4 b u n d l e = " l 3 p a c k a g e s "

5 m o d u l e = " x p a r s e "

6

7 - - L o c a t i o n of m a i n d i r e c t o r y : use Unix - s t y l e p a t h s e p a r a t o r s

8 - - S h o u l d m a t c h t h a t d e f i n e d by the b u n d l e .

9 m a i n d i r = " .. / .. "

Figure 3: The build script for thexparsemodule.

1 - - S p e c i a l c o n f i g for t h e s e t e s t s

2 c h e c k s e a r c h = t r u e

3 c h e c k e n g i n e s = { " x e t e x " , " l u a t e x " }

4 t e s t f i l e d i r = " t e s t f i l e s - TU "

Figure 4: Example of using additional (or overriding) settings for configuring tests in a different subdirectory.

(15)

1.10 Non-standard source layouts

A variety of source layouts are supported. In general, a “flat” layout with all source files “here” is most convenient. However, l3build supports placement of both code and documentation source files in other locations using thesourcefiledir,docfiledirand textfiledir variables. For pre-built trees, the glob syntax **/*.⟨ext⟩may be useful in these cases: this enables recursive searching in the appropriate tree locations. With the standard settings, this structure will be removed when creating a CTAN release: the variableflattenmay be used to control this behavior. Theflattentdssetting controls the same concept for TDS creation.

Notice that text files are treated separately from documentation files when splitting trees: this is to allow for the common case where files such asREADME andLICENSEare at the top level even when other documentation files are in a sub-directory.

A series of example layouts and matchingbuild.luafiles are available fromhttps:

//github.com/latex3/l3build/tree/master/examples.

For more complex layouts in which sources are laid out in TDS format and should be used directly, the tabletdsdirsis available. Each entry is a source directory and the matching installation target, for example

tdsdirs = {sources = "tex"}

This would enable a directorysourcesin the development area to be used for testing and typesetting, and for it to be installed into thetex tree when building a release. When this method is used, the sources are not copied into the local tree: liketexmfdir, they are added directly to the areas accessible during a testing or typesetting run. When using this approach, the files listed intypesetfiles must still be included indocfiles: they have to be directly visible tol3build, not found by kpsewhichsearching.

1.11 Non-standard formats/binaries

The standard approach used by l3build is to use a combination of engine and checkformat to generate the binary and format combination used for tests. For example, when pdftexis theengineandlatex is thecheckformat, the system call used is

pdftex --fmt=pdflatex

i.e.the binary names is the same as theengine, and the format is a simple substitution of thecheckformatintoengine, replacingtex.

For more complex set ups,specialformatsshould be used. This is a table with one entry percheckformat. Each entry is itself a table, and these contain a list of engines and settings for binary, format and options. For example, for ConTEXt and appropriate set up is

specialformats.context = {

luatex = {binary = "context", format = ""}, pdftex = {binary = "texexec", format = ""},

xetex = {binary = "texexec", format = "", options = "--xetex"}

}

Additional tokens can also be injected before the loading of a test file using thetokens entry: this might for example be used to select a graphics driver with a DVI-based route.

(16)

1.12 Output normalisation

To allow test files to be used between different systems (e.g. when multiple developers are involved in a project), the log files are normalised before comparison during checking.

This removes some system-dependent data but also some variations due to different engines. This normalisation consists of two parts: removing (“ignoring”) some lines and modifying others to give consistent test. Currently, the following types of line are ignored:

• Lines before the\START, after the \ENDand within\OMIT/\TIMOblocks

• Entirely blank lines, including those consisting only of spaces.

• Lines related to loading.fdfiles (from(⟨name⟩.fdto the matching)).

• Lines starting \openinor \openout.

Modifications made in lines are:

• Removal spaces at the start of lines.

• Removal of./at start of file names.

• Standardisation of the list of units known to TEX (pdfTEX and LuaTEX add a small number of additional units which are not known to TEX90 or X E TEX, (u)pTEX adds some additional non-standard ones)

• Standardisation of\csname\endcsname␣to\csname\endcsname(the former is for- mally correct, but the latter was produced for many years due to a TEX bug).

• Conversion of on line ⟨number⟩toon line ... to allow flexibility in changes to test files.

• Conversion of file dates to....-..-.., and any version numbers on the same lines to v....

• Conversion of register numbers in assignment lines\⟨register⟩=\⟨type⟩⟨number⟩ to \⟨type⟩⟨...⟩

• Conversion of box numbers in \showlines> \box⟨number⟩=to > \box...=

• Conversion of Lua data reference ids <lua data reference ⟨number⟩> to <lua data reference ...>

• Removal of some (u)pTEX data where it is equivalent to pdfTEX (yoko direction,

\displace 0.0)

• Removal of various\speciallines inserted due to the build process

LuaTEX makes several additional changes to the log file. As normalising these may not be desirable in all cases, they are handled separately. When creating LuaTEX-specific test files (either with LuaTEX as the standard engine or saving a LuaTEX-specific.tlgfile) no further normalisation is undertaken. On the other hand, for cross-engine comparison the following normalisation is applied:

• Removal of additional (unused) \discretionarypoints.

• Normalisation of some \discretionarydata to a TEX90 form.

(17)

• Removal ofỰ..notation for missing characters.

• Removal ofdisplayfor display math boxes (included by TEX90/pdfTEX/X E TEX).

• Removal of Omega-likedirection TLTinformation.

• Removal of ađitional whatsit containing local paragraph information (\localinterlinepenalty, etc.).

• Rounding of glue set to four decimal places (glue set may be slightly different in LuaTEX compared to other engines).

• Conversion of low chars (0 to 31) to ^^notation.

When making comparisons between 8-bit and Unicode engines it is useful to format the top half of the 8-bit range such that it appears in the log as ^^⟨char⟩ (the exact nature of the 8-bit output is otherwise dependent on the active code page). This may be controlled using theasciienginesoption. Any engines named here will use a .tcxfile to produce only ASCII chars in the log output, whilst for other engines normalisation is carried out from UTF-8 to ASCIỊ If the option is set to an empty table the latter process is skipped: suitable for cases where only Unicode engines are in usẹ

2 Writing test files

Test files are written in a TEX dialect using the support file regression-test.tex, which should be \input at the very beginning of each test. Ađitional customisations to this driver can be included in a localregression-test.cfgfile, which will be loaded automatically if found.

The macros loaded byregression-test.tex set up the test system and provide a number of commands to aid the production of a structured test suitẹ The basis of the test suite is to output material into thẹlogfile, from which a normalised test output (.tlg) file is produced by the build command savẹ A number of commands are provided for this; they are all written in uppercase to help avoid possible conflicts with other package commands.

2.1 Metadata and structural commands

Any commands that write content to the .log file that should be ignored can be sur- rounded by\OMIT. . . \TIMỌ At the appropriate location in the document where thẹlog comparisons should start (say, after\begin{document}), the test suite must contain the

\STARTmacrọ

The \END command signals the end of the test (but read on). Some ađitional diagnostic information is printed at this time to debug if the test did not complete

‘properly’ in terms of mismatched brace groups or \if. . .\figroups.

In a L^ATEX document, \end{document}will implicitly call\END at the very end of the compilation process. If\ENDis used directly (replacing\end{document}in the test), the compilation will halt almost immediately, and various tasks that \end{document}

usually performs will not occur (such as potentially writing to the various.tocfiles, and so on). This can be an advantage if there is ađitional material printed to the log file in this stage that you wish to ignore, but it is a disadvantage if the test relies on various auxiliary data for a subsequent typesetting run. (See the checkruns variable for how these tests would be test up.)

(18)

2.2 Commands to help write tests

\TYPE is used to write material to the .log file, like L^ATEX’s \typeout, but it allows

‘long’ input. The following commands are defined to use\TYPEto output strings to the .logfile.

• \SEPARATOR inserts a long line of =symbols to break up the log output.

• \NEWLINEinserts a linebreak into the log file.

• \TRUE,\FALSE,\YES, \NOoutput those strings to the log file.

• \ERROR is not defined but is commonly used to indicate a code path that should never be reached.

• The \TEST{⟨title⟩}{⟨contents⟩} command surrounds its ⟨contents⟩ with some

\SEPARATORs and a⟨title⟩.

• \TESTEXP surrounds its contents with\TYPEand formatting to match\TEST; this can be used as a shorthand to test expandable commands.

• \BEGINTEST{⟨title⟩} . . .\ENDTESTis an environment form of \TEST, allowing ver- batim material, etc. to appear.

• \SHOWFILE(ε-TEX only) Shows the content of the file given as an argument.

• \ASSERTand\ASSERTSTRAsserts if the full expansion of the two required arguments are the same: the \ASSERT function is token-based, the \ASSERTSTR works on a string basis.

An example of some of these commands is shown following.

\TEST{bool_set,~lazy~evaluation}

{

\bool_set:Nn \l_tmpa_bool {

\int_compare_p:nNn 1=1

&& \bool_lazy_any_p:n {

{ \int_compare_p:nNn 2=3 } { \int_compare_p:nNn 4=4 }

{ \int_compare_p:nNn 1=\ERROR } % is skipped }

&& \int_compare_p:nNn 2=2 }

\bool_if:NTF \l_tmpa_bool \TRUE \FALSE }

This test will produce the following in the output.

==========================================

TEST 8: bool_set, lazy evaluation

==========================================

TRUE

==========================================

(Only if it’s the eighth test in the file of course, and assumingexpl3 coding conventions are active.)

(19)

2.3 Showing box content

The commands introduced above are only useful for checking algorithmic or logical cor- rectness. Many packages should be tested based on their typeset output instead; TEX provides a mechanism for this by printing the contents of a box to the log file. The regression-test.tex driver file sets up the relevant TEX parameters to produce as much output as possible when showing box output.

A plain TEX example of showing box content follows.

\input regression-test.tex\relax

\START

\setbox0=\hbox{\rm hello \it world $a=b+c$}

\showbox0

\END

This produces the output shown in Figure5 (left side). It is clear that if the definitions used to typeset the material in the box changes, the log output will differ and the test will no longer pass.

The equivalent test in L^ATEX 2ε usingexpl3 is similar.

\input{regression-test.tex}

\documentclass{article}

\usepackage{expl3}

\START

\ExplSyntaxOn

\box_new:N \l_tmp_box

\hbox_set:Nn \l_tmp_box {hello~ \emph{world}~ $a=b+c$}

\box_show:N \l_tmp_box

\ExplSyntaxOff

\END

The output from this test is shown in Figure5 (right side). There is marginal difference (mostly related to font selection and different logging settings in L^ATEX) between the plain andexpl3versions.

When examples are not self-contained enough to be typeset into boxes, it is possible to ask TEX to output the entire contents of a page. Insert \showoutput for L^ATEX or set \tracingoutputpositive for plain TEX; ensure that the test ends with\newpageor equivalent because TEX waits until the entire page is finished before outputting it.

TODO: should we add something like \TRACEPAGES to be format-agnostic here?

Should this perhaps even be active by default?

2.4 Testing entire pages

There may be occasions where creating entire test pages is necessary to observe the test output required. That is best achieved by applying\showoutputand forcing a complete page to be produced, for example

\input{regression-test.tex}

\documentclass{article}

\usepackage{expl3}

\START

\showoutput

(20)

> \box0=

\hbox(6.94444+0.83333)x90.56589 .\tenrm h

.\tenrm e .\tenrm l .\tenrm l .\tenrm o

.\glue 3.33333 plus 1.66666 minus 1.11111 .\tenit w

.\tenit o .\tenit r .\tenit l .\tenit d

.\glue 3.57774 plus 1.53333 minus 1.0222 .\mathon

.\teni a

.\glue(\thickmuskip) 2.77771 plus 2.77771 .\tenrm =

.\glue(\thickmuskip) 2.77771 plus 2.77771 .\teni b

.\glue(\medmuskip) 2.22217 plus 1.11108 minus 2.22217 .\tenrm +

.\glue(\medmuskip) 2.22217 plus 1.11108 minus 2.22217 .\teni c

.\mathoff

! OK.

l.9 \showbox0

> \box71=

\hbox(6.94444+0.83333)x91.35481 .\OT1/cmr/m/n/10 h

.\OT1/cmr/m/n/10 e .\OT1/cmr/m/n/10 l .\OT1/cmr/m/n/10 l .\OT1/cmr/m/n/10 o

.\glue 3.33333 plus 1.66666 minus 1.11111 .\OT1/cmr/m/it/10 w

.\OT1/cmr/m/it/10 o .\OT1/cmr/m/it/10 r .\OT1/cmr/m/it/10 l .\OT1/cmr/m/it/10 d .\kern 1.03334

.\glue 3.33333 plus 1.66666 minus 1.11111 .\mathon

.\OML/cmm/m/it/10 a

.\glue(\thickmuskip) 2.77771 plus 2.77771 .\OT1/cmr/m/n/10 =

.\glue(\thickmuskip) 2.77771 plus 2.77771 .\OML/cmm/m/it/10 b

.\glue(\medmuskip) 2.22217 plus 1.11108 minus 2.22217 .\OT1/cmr/m/n/10 +

.\glue(\medmuskip) 2.22217 plus 1.11108 minus 2.22217 .\OML/cmm/m/it/10 c

.\mathoff

! OK.

<argument> \l_tmp_box l.12 \box_show:N \l_tmp_box

Figure 5: Output from displaying the contents of a simple box to the log file, using plain TEX (left) andexpl3(right). Some blank lines have been added to the plain TEX version to help with the comparison.

(21)

1 f u n c t i o n r u n t e s t _ t a s k s ( name , run )

2 if run == 1 t h e n

3 r e t u r n " b i b e r ␣ " .. n a m e

4 e l s e

5 r e t u r n 0

6 end

7 end

Figure 6: Exampleruntest_tasksfunction.

% Test content here

\vfil\break

\END

2.5 Pre-check hook

To allow complex set up for tests, a hookcheckinit_hook()is available to be executed once all standard set up is complete but before any tests are run. This should return an integer value: 0 indicates no error.

2.6 Additional test tasks

A standard test will run the file⟨name⟩.lvtusing one or more engines, but will not carry out any additional processing. For some tests, for example bibliography generation, it may be desirable to call one or more tools in addition to the engine. This can be arranged by defining runtest_tasks, a function taking two arguments, the name of the current test (this is equivalent to TEX’s \jobname, i.e. it lacks an extension) and the current run number. The function runtest_tasksis run after the main call to the engine for a test cycle. It should return an errorlevel value. If more than one task is required, these should be separated by use ofos_concat, a string variable defined byl3buildas the correct concatenation marker for the system. An example ofruntest_taskssuitable for calling Biber is shown in Listing6.

2.7 Instructions for rebuilding test output

Sometimes changes to fundamental parts of the code can cause a lot of tests to fail even though the actually tested systems are still working correctly. This is especially common when the logging and error reporting systems changes and therefore all log file based tests using the component fail with these changes.

In these cases, the option --show-saves can be passed to l3build check in order to generate a list ofl3build save commands which can be executed to regenerate the expected output of all tests which fail. Additionally it sometimes prints a list of l3build checkcommands for tests which might still fail due to engine differences after running thesavecommands. After running all thesel3build checkcommands and all l3build savecommands listed by them, all tests will succeed.

When bundles are usedl3build check --show-saveshas to be executed separately for every module in the bundle.

This option is potentially dangerous and therefore should only be used with care. It can easily hide valid test failures between a bunch of spurious changes. Therefore you

(22)

should always take a close look at the difference files generated byl3build checkbefore running the generated l3build save commands. Additionally it should only be used when you are aware of the reason why a large number of tests failed and the change causing the failures has been tested separately to have no unintended side effects.

2.8 Epoch setting

To produce predictable output when using dates, the test system offers the ability to set the epoch to a known value. The epochvariable may be given as a raw value (a simple integer) or as a date in ISO format. The two flagsforcecheckepochandforcedocepoch then determine whether this is applied in testing and typesetting, respectively.

The epoch may also be given as a command line option,-E, which again takes either a date or raw epoch. When given, this will automatically activate forcing of the epoch in both testing and typesetting.

2.9 Settings in texmf.cnf

To allow application of non-standard TEX trees or similar non-standard settings,l3build enables searching for a texmf.cnf file by setting the envirnmental variable TEXMFCNF.

This might for example be used with a file containing TEXMFAUXTREES = ../../texmf,

for adding a local tree within the development repository (assuming the typical l3build layout).

3 Alternative test formats

3.1 Generating test files with DocStrip

It is possible to pack tests inside source files. Tests generated during the unpacking process will be available to thecheckand savecommands as if they were stored in the testfiledir. Any explicit test files inside testfiledir take priority over generated ones with the same names.

3.2 Specifying expectations

Regression tests check whether changes introduced in the code modify the test output.

Especially while developing a complex package there is not yet a baseline to save a test goal with. It might then be easier to formulate the expected effects and outputs of tests directly. To achieve this, you may create an.lveinstead of a.tlg file.¹ It is processed exactly like the.lvtto generate the expected outcome. The test fails when both differ.

Combining both features enables contrasting the test with its expected outcome in a compact format. Listing7exemplary tests TEXs counters. Listing8shows the relevant part of an.ins file to generate it.

1Mnemonic: lvt:test,lve:expectation

(23)

1 \i n p u t r e g r e s s i o n - t e s t . tex \r e l a x

2 \ S T A R T

3 \ T E S T { counter - m a t h }{

4 % <* test >

5 \ O M I T

6 \ n e w c o u n t e r { n u m b e r s }

7 \ s e t c o u n t e r { n u m b e r s } { 2 }

8 \ a d d t o c o u n t e r { n u m b e r s } { 2 }

9 \ s t e p c o u n t e r { n u m b e r s }

10 \ T I M O

11 \ t y p e o u t {\ a r a b i c { n u m b e r s }}

12 % </ test >

13 % < expect > \ t y p e o u t {5}

14 }

15 \ END

Figure 7: Test and expectation can be specified side-by-side in a single.dtxfile.

1 \ g e n e r a t e {\ f i l e {\j o b n a m e. lvt }{\ f r o m {\j o b n a m e. dtx }{ t e s t }}

2 \ f i l e {\j o b n a m e. lve }{\ f r o m {\j o b n a m e. dtx }{ e x p e c t }}}

Figure 8: Test and expectation are generated from a.dtx file of the same name.

3.3 PDF-based tests

In most cases, testing is best handled by using the text-based methods outlined above.

However, there are cases where the detail of output structure is important. This can only be fully tested by comparing PDF structure. To support this,l3buildcan be instructed to build and compare PDF files by setting up tests in.pvtfiles. The following normalization takes place:

• Replacement of binary streams by the marker[BINARY STREAM]

• Replacement of /IDvalues byID-STRING

• Removal of blank lines

• Removal of comment (%%) lines

After this normalization takes place, the file can not usually be rendered properly.

To check if the build system has produced a correct PDF, the pre-normalization PDF can be found in thebuildfolder.

To allow platform-independence, PDF-based tests must use only Type 1 or Open- Type fonts: Type3 fonts are system-dependent. PDF files are engine-specific, thus one .tpffile should be stored per engine to be tested.

3.4 Custom tests

If neither the text-based methods nor PDF-based tests are sufficient, there is the additional option of defining custom variants with individual normalization rules.

For this, the variant has to be registered in thetest_typestable and then activated in test_order.

(24)

Every element intest_types is a table with fieldstest (the extension of the test file),reference(the extension of the file the output is compared with),generated(extension of the analyzed L^ATEX output file) andrewrite(A Lua function for normalizing the output file, taking as parameters the name of the unnormalized L^ATEX output file to be read, the name of the normalized file to be written, the engine name and a potential errorcode).

For example:

test_types = { mytest = {

test = ".mylvt", reference = ".mytlg", generated = ".log",

rewrite = function(source, normalized, engine, errorcode)

-- In this example we just copy the logfile without any normalization os.execute(string.format("cp %s %s", source, normalized)

end, }, }

test_order = {"mylvt", "log", "pdf"}

4 Release-focussed features

4.1 Installation structure

With the standard settings, l3build will install files within the TEX directory structure (TDS) as follows

• installfileswithin a⟨bundle⟩/⟨module⟩(or⟨module⟩) directory insidetex/⟨format⟩

• sourcefileswithin a⟨bundle⟩/⟨module⟩(or⟨module⟩) directory insidesource/⟨format⟩

• Typeset PDFs within a⟨bundle⟩/⟨module⟩(or⟨module⟩) directory insidedoc/⟨format⟩

• bstfileswithin a⟨bundle⟩/⟨module⟩(or⟨module⟩) directory insidebibtex/bst

• bibfileswithin a⟨bundle⟩/⟨module⟩(or⟨module⟩) directory insidebibtex/bib

• makeindexfiles within a ⟨bundle⟩/⟨module⟩ (or ⟨module⟩) directory inside makeindex

For more complex set ups, this can be customised using the tdslocations table.

Each entry there should be a glob specifying the TDS position of a file or files. Any files not specified in the table will use the standard locations above. For example, to place some files in the generic tree, some in the plain TEX tree and some in the L^ATEX tree, one might use the set up shown in Figure9.

The table is read in order, and thus specific file names should come before potential wild-card matches.

(25)

1 t d s l o c a t i o n s =

2 {

3 " tex / g e n e r i c / m y p k g /* . g e n e r i c . t e x " ,

4 " tex / p l a i n / m y p k g /* . p l a i n . t e x " ,

5 " tex / l a t e x / m y p k g /* . l a t e x . t e x "

6 }

Figure 9: Exampletdslocationstable.

1 - - D e t a i l how to set the v e r s i o n a u t o m a t i c a l l y

2 f u n c t i o n u p d a t e _ t a g ( file , content , tagname , t a g d a t e )

3 if s t r i n g . m a t c h( file , " % . d t x $ " ) t h e n

4 r e t u r n s t r i n g . g s u b( content ,

5 " \ n %% ␣ \\ d a t e { R e l e a s e d ␣ % d % d % d % d /% d % d /% d % d }\ n " ,

6 " \ n %% ␣ \\ d a t e { R e l e a s e d ␣ " .. t a g n a m e .. " }\ n " )

7 e l s e i f s t r i n g . m a t c h( file , " % . m d $ " ) t h e n

9 " \ n R e l e a s e ␣ % d % d % d % d /% d % d /% d % d \ n " ,

10 " \ n R e l e a s e ␣ " .. t a g n a m e .. " \ n " )

11 e l s e i f s t r i n g . m a t c h( file , " % . l u a $ " ) t h e n

13 ’ \ n r e l e a s e _ d a t e ␣ = ␣ "% d % d % d % d /% d % d /% d % d "\ n ’ ,

14 ’ \ n r e l e a s e _ d a t e ␣ = ␣ " ’ .. t a g n a m e .. ’ "\ n ’ )

15 end

16 r e t u r n c o n t e n t

17 end

Figure 10: Exampleupdate_tagfunction.

4.2 Automatic tagging

Thetagtarget can automatically edit source files to modify date and release tag name.

As standard, no automatic replacement takes place, but setting up a update_tag() function will allow this to happen. This function takes four input arguments:

1. file name

2. full content of the file 3. tag name

4. tag date

The update_tag() function should return the (modified) contents for writing to disk.

For example, the function used byl3builditself is shown in Figure10.

To allow more complex tasks to take place, a hooktag_hook()is also available. It will receive the tag name and date as arguments, and may be used to carry out arbitrary tasks after all files have been updated. For example, this can be used to set a version control tag for an entire repository.

4.3 Typesetting documentation

As part of the overall build process,l3buildwill create PDF documentation as described earlier. The standard build process for PDFs will attempt to run Biber, BibTEX and MakeIndex as appropriate (the exact binaries used are defined bybiberexe,bibtexexe

(26)

and makeindexexe). However, there is no attempt to create an entire PDF creation system in the style of latexmkor similar.

For package authors who have more complex requirements than those covered by the standard set up, the Lua script offers the possibility for customisation. The Lua function typeset may be defined before reading l3build.lua and should take one argument, the name of the file to be typeset. Within this function, the auxiliary Lua functions biber, bibtex, makeindex and tex can be used, along with custom code, to define a PDF typesetting pathway. The functionsbiberandbibtextake a single argument: the name of the file to work with minus any extension. The textakes as an argument the full name of the file. The most complex function makeindexrequires the name, input extension, output extension, log extension and style name. For example, Figure11shows a simple script which might apply to a case where multiple BibTEX runs are needed (perhaps where citations can appear within other references).

Where there are complex requirements for pre-compiled demonstration files, the hooktypeset_demo_tasks() is available: it runs after copying files to the typesetting location but before the main typesetting run. This may be used for example to script a very large number of demonstrations using a single source (see thebeamerpackage for an example of this). Note that this hook is intended for use filesnotlisted intypesetfiles ortypesetdemofiles.

4.4 Pre-typesetting hook

To allow complex set up for typesetting, a hook docinit_hook() is available to be executed once all standard set up is complete but before any typesetting is run.

1 #!/ usr / bin / env t e x l u a

2

3 - - B u i l d s c r i p t w i t h c u s t o m PDF r o u t e

4

5 m o d u l e = " m y m o d u l e "

6

7 f u n c t i o n t y p e s e t ( f i l e )

8 l o c a l n a m e = j o b n a m e ( f i l e )

9 l o c a l e r r o r l e v e l = tex ( f i l e )

10 if e r r o r l e v e l == 0 t h e n

11 - - R e t u r n a non - z e r o e r r o r l e v e l if a n y t h i n g g o e s w r o n g

12 e r r o r l e v e l =(

13 b i b t e x ( n a m e ) +

14 tex ( f i l e ) +

15 b i b t e x ( n a m e ) +

16 tex ( f i l e ) +

17 tex ( f i l e )

18 )

19 end

20 r e t u r n e r r o r l e v e l

21 end

Figure 11: A customised PDF creation script.

(27)

4.5 Non-standard typesetting

To allow non-standard typesetting combinations, for example per-file choice of engines, the table specialtypesetting may be used. This is a table with one entry per file.

Each entry is itself a table, and these contain a list of engines and settings forcmdand func. For example, to choose to use LuaTEX for one file when typesetexeispdftex specialtypesetting = specialtypesetting or {}

specialtypesetting["foo.tex"] = {cmd = "luatex -interaction=nonstopmode"}

or to select an entirely different typesetting function specialtypesetting = specialtypesetting or {}

specialtypesetting["foo.tex"] = {func = typeset_foo}

4.6 Automated upload to CTAN

The CTAN upload process is backed by an API, whichl3buildcan use to send zip files for release. Along with the file, a variety of metadata must be specified about the package, including the version, license, and so on, explained athttps://www.ctan.org/upload.

A description of this metadata is outlined in Table2, and a simple example of an extract from abuild.luafile using this is shown in Figure12.

Note that theuploadtarget will not execute thectantarget first.

This upload facility assumes availablity of curl on your system. In the case of Windows, the system curl will not be available if you are using a 32 bit TEX im- plementation. Curl executables are available for a variety of operating systems from https://curl.haxx.se/download.html.

Announcement text It can be convenient not to include the announcement text within the build.luafile directly. The command line argument--message(-m) allows the announcement to be included as part of the l3build arguments, and --file (-F) reads the announcement from a specified file. Thebuild.luafile may also specify that this text is to be taken from the file specified by uploadconfig.announcement_file, this allows the release-specific announcement to be specified outside the mainbuild.lua file. Ifuploadconfig.announcement_fileisnilor specifies a file that can not be read, and no announcement is provided by theannouncementfield or commandline arguments, l3buildwill interactively prompt for text (which may be empty).

Note that if the announcement text is empty a ‘silent update’ is performed; this should usually be performed for minor bug or documentation fixes only.

Note text This optional field is for passing notes to the CTAN maintainers. As for announcements, the text may be set inuploadconfig.noteor perhaps more usefully, if uploadconfig.note_file is the filename of a readable file the file text is used as the note.

Uploader details The CTAN team use the uploader email address as a form of low- security sanity check that the upload is coming from a reputable source. Therefore, it is advisable not to store this information within a publicbuild.luafile. It can be set on the command line with the--emailoption tol3build; alternatively, a private configuration file could be used to add this information at upload time.

(28)

The update field In most scenarios the update field does not need to be explicitly set. By defaultl3buildassumes that the package being uploaded already exists on CTAN (update=true). If it does not, this is caught in the validation process before uploading and automatically corrected. If you setupdate explicitly this will be passed directly to CTAN in all circumstances, leading to errors if you attempt to update a non-existing package or if you attempt to upload a new package with the same name as a pre-existing one.

The curl options file Thel3build upload options are passed tocurl by writing the fields to a text file with a default name being ⟨package⟩-ctan.curlopt. This is then passed to curl using its--configcommandline option. (Using an intermediate file helps keepl3buildportable between systems using different commandline quoting conventions.

Any backslashes are doubled when writing to this file, so they do not need to be doubled in announcement and note texts.)

By default the file is written into the current directory alongside the zip file to be uploaded. You may wish to specify that this file is ignored by any version control in that directory (using .gitignore or similar). Or alternatively you can use the uploadconfig.curl_filefield in thebuild.lua file to specify an alternative name or location for this file.

Validating To validate your upload but not actually submit to CTAN, you may use the--dry-runcommand-line option.

Debugging If you have have difficulty with the upload process, add the option--debug to divert the request from CTAN to a service that redirects the input back again so it can be examined. It can also be useful to check the contents of the curloptsfile which has a record of the options passed to curl.

5 Lua interfaces

Whilst for the majority of users the simple variable-based control methods outlined above will suffice, for more advanced applications there will be a need to adjust behavior by using interfaces within the Lua code. This section details the global variables and functions provided.

Index of /CTAN/macros/latex/contrib/l3build

The l3build package

Checking and building packages

The L

TEX Project

Released 2022-09-15

Contents

Index 36

1 The l3build system

1.1 Introduction

1.2 The build.lua file

1.3 Main build targets

1.4 Example build scripts

1.5 Variables

1.6 Interaction between tests

1.7 Selective running of tests

1.8 Multiple sets of tests

1.9 Dependencies

1.10 Non-standard source layouts

1.11 Non-standard formats/binaries

1.12 Output normalisation

2 Writing test files

2.1 Metadata and structural commands

2.2 Commands to help write tests

2.3 Showing box content

2.4 Testing entire pages

2.5 Pre-check hook

2.6 Additional test tasks

2.7 Instructions for rebuilding test output

2.8 Epoch setting

2.9 Settings in texmf.cnf

3 Alternative test formats

3.1 Generating test files with DocStrip

3.2 Specifying expectations

3.3 PDF-based tests

3.4 Custom tests

4 Release-focussed features

4.1 Installation structure

4.2 Automatic tagging

4.3 Typesetting documentation

4.4 Pre-typesetting hook

4.5 Non-standard typesetting

4.6 Automated upload to CTAN

5 Lua interfaces