QuartoTools.jl

Utilities for working with Quarto notebooks in Julia

This package provides several utilities that can be used in conjuction with Quarto notebooks when using the engine: julia setting, which executes your notebook code with QuartoNotebookRunner.jl.

"Expandables"

QuartoNotebookRunner.jl has a special feature called cell expansion. It allows you to have a single code cell that outputs what looks like multiple code cells and their outputs to Quarto.

What can you use this feature for?

Quarto has many advanced options which allow you to create richer output, for example tabsets which can group several separate sections of a quarto notebook into selectable tabs. These features are controlled with markdown annotations, for example, a tabset follows this structure:

::: {.panel-tabset}

## Tab 1

Content of tab 1

## Tab 2

Content of tab 2

... possibly more tabs ...

:::

As you can see, the tabset begins and ends with a pandoc ::: div fence and consists of sections demarcated by markdown headings. This mechanism has two drawbacks for the user:

It can be tricky to get the syntax right, especially with multiple nested ::: fences that need to be closed correctly. (This also applies when you generate markdown programmatically by printing out snippets in loops and using the output: asis cell option.)
It is static. Each tab has to be written into the source markdown explicitly, so you cannot easily create a tabset with a dynamic number of tabs. For example, a tabset with one plot per tab where the number of plots depends on runtime information and therefore is not known in advance.

Cell expansion can solve both of these problems. It relies on a function called QuartoNotebookWorker.expand, which is defined within the notebook worker process for every notebook that you execute.

When a notebook cell returns a Julia value from a cell, such that it is displayed but Quarto, the expand function will first be called on that value. By default this returns nothing and so we just display the original value. But if an expand method is defined for that type, then it should return a Vector{QuartoNotebookWorker.Cell} object which is then evaluated as if they were real cells. This feature is recursive, so Cells can themselves return more vectors of Cells.

Thus we can use these cells to build the structures Quarto expects programmatically, instead of having to hardcode them into the notebook.

For example, a tabset with plots could be generated by expanding into:

a cell with markdown output ::: {.panel-tabset}
a cell with markdown output ## Tab 1
a cell with a plot output, for example a Makie.Figure
more headings and plots
a cell with markdown output :::

Note

Cell expansion is not code generation. We do not generate and evaluate arbitrary code. Instead, we create objects describing code cells together with their outputs which is easier to reason about and more composable.

Each QuartoNotebookWorker.Cell has three fields:

thunk stores a function which returns the fake cell's output value when run. This value is treated as any other code cell output value, so it may be of any type that the display system can handle, and it may even be expandable itself (allowing for recursive expansion).
code may hold a string which will be rendered as the code of the fake cell (this code is not run).
options is a dictionary of quarto cell options, for example "echo" => false to hide the source code section.

QuartoTools defines a set of helper objects that can serve as building blocks that can be composed further. For example, a Tabset may contain multiple Divs, each describing a two-column layout which is populated with two plots.

Caching

QuartoTools provides a caching mechanism that can be used to save the results of expensive function calls in your notebook cells. Once loaded into a notebook via a cell containing import QuartoTools you can annotate any subsequent cell with the julia.cache.enabled key as follows:

---
engine: julia
---

```{julia}
import QuartoTools
```

```{julia}
#| julia:
#|   cache:
#|     enabled: true
result = expensive_func(arg)
```

The first time that expensive_func is called with any specific arg value the result will be saved to disk using Serialization. Subsequent calls with the same arg value will return the cached result rather than re-running expensive_func. This can be useful for long-running computations that slow down the rendering of your notebook.

Avoid using the feature on cells that only take a few seconds that run, since the overhead of saving and loading cached results can be larger than the time saved. If you have a particularly complex cell that contains some fast calls and some slow ones, try to factor them out into separate cells and only run the caching on the slow ones.

The cache for each notebook is stored alongside it in a folder called .cache. Removing this folder will clear the cache for the notebook. Do not commit the contents of this folder to version control.

Serialization

When working with serialized data in Quarto notebooks users must use the QuartoTools.serialize and QuartoTools.deserialize functions provided by the QuartoTools package rather than the Serialization package. This is due to the differences in the behaviour of code evaluation between the Julia REPL and that of Quarto. These two functions are drop-in replacements for those provided by Serialization and fall back on the implementation provided by it when not run in a Quarto notebook. This means that simply replacing using Serialization with using QuartoTools should be sufficient to allow for transparent serialization and deserialization between notebooks, batch scripts, and the REPL.

Note that if both QuartoTools and Serialization are imported with using in the same session then the functions serialize and deserialize will need to be prefixed with their package name due to the name collisions between the two packages. Typically users should only need to import QuartoTools.

Docstrings

QuartoTools.Cell — Type

struct Cell

Cell(content::Function; code = nothing, options = Dict{String,Any}(), lazy = true)
Cell(content; code = nothing, options = Dict{String,Any}(), lazy = false)

The most basic expandable object, representing a single code cell with output.

If code === nothing, the code cell will be hidden by default using the quarto option echo: false. Note that code is never evaluated, merely displayed in code cell style. Only content determines the actual cell output.

All options are written into the YAML options header of the code cell, this way you can use any cell option commonly available in code cells for your generated cells. For example, options = Dict("echo" => false) will splice #| echo: false into the code cell's options header.

If lazy === true, the output will be treated as a thunk, which has to be executed by QuartoNotebookRunner to get the actual output object that should have display called on it. Accordingly, you will get an error if the output object is not a Base.Callable. If lazy === false, the output will be used as the actual output object directly by QuartoNotebookRunner. As an example, if you generate a hundred plot output cells, it is probably better to generate the plots using lazy functions, rather than storing all of them in memory at once. The lazy option is set to true by default when a Function is passed to the convenience constructor, and to false otherwise.

source

QuartoTools.Div — Type

struct Div

Div(children::Vector; id=[], class=[], attributes=Dict())
Div(child; kwargs...)

Construct a Div which is an expandable that wraps its child cells with two markdown fence cells to create a pandoc div using ::: as the fence delimiters. Div optionally allows to specify one or more ids, classes and key-value attributes for the div.

id and class should each be either one AbstractString or an AbstractVector of those. attributes should be convertible to a Dict{String,String}.

Examples

Div(Cell(123))
Div(
    [Cell(123), Cell("ABC")];
    id = "someid",
    class = ["classA", "classB"],
    attributes = Dict("somekey" => "somevalue"),
)

source

QuartoTools.Expand — Type

struct Expand

Expand(expandables::AbstractVector)

Construct an Expand which is an expandable that wraps a vector of other expandable. This allows to create multiple output cells using a single return value in an expanded quarto cell.

Example

Expand([Cell(123), Cell("ABC")])

source

QuartoTools.Tabset — Type

struct Tabset

Tabset(pairs; group = nothing)

Construct a Tabset which is an expandable that expands into multiple cells representing one quarto tabset (using the ::: {.panel-tabset} syntax).

pairs should be convertible to a Vector{Pair{String,Any}}. Each Pair in pairs describes one tab in the tabset. The first element in the pair is its title and the second element its content.

You can optionally pass some group id as a String to the group keyword which enables quarto's grouped tabset feature where multiple tabsets with the same id are switched together.

Example

Tabset([
    "Tab 1" => Cell(123),
    "Tab 2" => Cell("ABC")
])

source

QuartoTools.MarkdownCell — Method

MarkdownCell(s::String)

A convenience function which constructs a Cell that will be rendered by quarto with the output: asis option. The string s will be interpreted as markdown syntax, so the output will look as if s had been written into the quarto notebook's markdown source directly.

source

QuartoTools.cacheable — Method

cacheable(f) -> Bool

Determine if a function is cacheable. By default all functions are cacheable. Use this function to override that behaviour, for example to make Base.read uncacheable:

QuartoTools.cacheable(::typeof(Base.read)) = false

source

QuartoTools.content_hash — Method

content_hash(object)

Compute a content hash for the given object. This should result in hashes that match between different instances of identical objects. Used for cache keys.

source

QuartoTools.deconstruct — Method

deconstruct(value::T) -> S

An extension function for turning values of type T into a type S such that they can be serialized properly.

source

QuartoTools.deserialize — Function

deserialize(s::IO)
deserialize(filename::AbstractString)

Deserialize a value from the given IO stream or file using Julia's built-in serialization while correctly handling differences in "root" evaluation module between the REPL and Quarto notebooks.

source

QuartoTools.reconstruct — Method

reconstruct(value::S) -> T

An extension function for turning values of type S back into a type T after they have been deserialized.

source

QuartoTools.serialize — Function

serialize(s::IO, x)
serialize(filename::AbstractString, x)

Serialize x to the given IO stream or file using Julia's built-in serialization while correctly handling differences in "root" evaluation module between the REPL and Quarto notebooks.

source

QuartoTools.toggle_cache — Method

toggle_cache()

Switch caching of function calls in the REPL on/off.

source

QuartoTools.@cache — Macro

@cache func(args...; kws...)

Cache the result of a function call with the given arguments and keyword arguments.

The caching key is based on:

The full VERSION of Julia.
The Function being called.
The Module in which the function is called.
The file in which the function is called.
The active Project.toml.
The argument values and keyword argument values passed to the function.

The cache is stored in a .cache directory in the same directory as the file in which the function is called. Deleting this directory will clear the cache.

source

QuartoTools.@nc_cmd — Macro

nc`variable_name` = func(args...)

Mark the given variable as non-cachable. This means that assigning to this variable from a function call will not cache the function call. This is equivalent to using the julia.cache.ignored array in cell options or notebook frontmatter in a Quarto notebook.

source