Motivating example

Below is a reduced version of the cranview Shiny app that allows you to enter an R package name to generate a plot of its CRAN downloads over the past year. This app provides a nice example of how to modify an existing Shiny app so that it can generate code to reproduce what a user sees in the app:

library(shiny)
library(tidyverse)

ui <- fluidPage(
  textInput("package", "Package name", value = "ggplot2"),
  plotOutput("plot")
)

server <- function(input, output, session) {
  
  downloads <- reactive({
    cranlogs::cran_downloads(input$package, from = Sys.Date() - 365, to = Sys.Date())
  })
  
  downloads_rolling <- reactive({
    validate(need(sum(downloads()$count) > 0, "Input a valid package name"))
    
    downloads() %>% 
      mutate(count = zoo::rollapply(count, 7, mean, fill = "extend"))
  })
  
  output$plot <- renderPlot({
    ggplot(downloads_rolling(), aes(date, count)) + geom_line()
  })
}

shinyApp(ui, server)

Below is a modified version of the app that generates code to reproduce output$plot outside of the shiny session (via shinymeta). In the screencast of the app below, note how both output$plot and output$code update dynamically in response to user input. To keep the focus on code generation, we’ve presented the output$code as simple as possible here (by using verbatimTextOutput() and renderPrint()), but the next article outlines the various options distributing code to users.

library(shiny)
library(tidyverse)
library(shinymeta)

ui <- fluidPage(
  textInput("package", "Package name", value = "ggplot2"),
  verbatimTextOutput("code"),
  plotOutput("plot")
)

server <- function(input, output, session) {
  
  downloads <- metaReactive({
    cranlogs::cran_downloads(..(input$package), from = Sys.Date() - 365, to = Sys.Date())
  })
  
  downloads_rolling <- metaReactive2({
    validate(need(sum(downloads()$count) > 0, "Input a valid package name"))
    
    metaExpr({
      ..(downloads()) %>% 
        mutate(count = zoo::rollapply(count, 7, mean, fill = "extend"))
    })
  })
  
  output$plot <- metaRender(renderPlot, {
    ggplot(..(downloads_rolling()), aes(date, count)) + geom_line()
  })
  
  output$code <- renderPrint({
    expandChain(
      quote(library(tidyverse)), 
      output$plot()
    )
  })
}

shinyApp(ui, server)

Overview

There are roughly 3 main steps required to get an existing Shiny app generating reproducible code via shinymeta (well, 4 steps if you want to generate ‘readable’ code). Those steps are illustrated in the video below:

Step 1: Identify and capture domain logic

Each reactive building block that contains domain logic must be replaced by a suitable meta-counterpart (i.e., reactive() -> metaReactive(), renderPlot() -> metaRender(), observe() -> metaObserve(), etc). In situations where a reactive building block contains non-domain logic that you don’t want to capture (e.g., Shiny specific code, like validate()), shinymeta provides a second version (e.g. metaReactive2(), metaRender2(), metaObserve2(), etc) that allows you to ignore code (by wrapping only the code that you care about in metaExpr()). When using these -2 variants, make sure the return value of the expression is a metaExpr() object (In practice, the code you want to capture might depend on other input value(s). In that case, you can use control flow similar to this, just make sure to return a metaExpr()!).

Step 2: Identify and mark reactive reads

To substitute reactive reads (e.g., input$package, downloads()) with a suitable value or name (e.g., "ggplot2", downloads), mark them with ..(). When ..() is applied to something other than a reactive read, it’s treated as an unquoting operator, which is discussed more in The execution model.

Step 3: Generate code with expandChain()

The expandChain() function generates code from any combination of meta-counterparts (i.e., metaReactive(), metaRender(), etc) and other quoted code. Supplying quoted code is primarily useful for supplying setup code that the user needs but isn’t captured by meta-reactives (e.g., loading of libraries).

# Imagine we've added this output to our example
output$summary <- metaRender(renderPrint, {
  summary(..(downloads())$count)
})

expandChain(
  quote({
    library(dplyr)
    library(ggplot2)
  }),
  output$plot(),
  output$summary()
)
library(dplyr)
library(ggplot2)
downloads <- cranlogs::cran_downloads("ggplot2", from = Sys.Date() - 365, to = Sys.Date())
downloads_rolling <- downloads %>%
  mutate(count = zoo::rollapply(count, 7, mean, fill = "extend"))
ggplot(downloads_rolling, aes(date, count)) + geom_line()
summary(downloads$count)

If we expand these outputs separately, expandChain() won’t automatically know to avoid duplicating code for dependencies that they share. In this case, both of these outputs depend on downloads, so if we expand them in subsequent calls to expandChain(), we’ll be producing code that calls cranlogs::cran_downloads() twice:

expandChain(output$plot())
downloads <- cranlogs::cran_downloads("ggplot2", from = Sys.Date() - 365, to = Sys.Date())
downloads_rolling <- downloads %>%
  mutate(count = zoo::rollapply(count, 7, mean, fill = "extend"))
ggplot(downloads_rolling, aes(date, count)) + geom_line()
downloads <- cranlogs::cran_downloads("ggplot2", from = Sys.Date() - 365, to = Sys.Date())
summary(downloads$count)

Fortunately, there is a way to avoid this redundant code caused by shared dependencies by sharing an ‘expansion context’ between subsequent calls to expandChain(). This is especially useful for generating reports where you might want to spit code out into separate knitr chunks.

ec <- newExpansionContext()
expandChain(output$plot(), .expansionContext = ec)
downloads <- cranlogs::cran_downloads("ggplot2", from = Sys.Date() - 365, to = Sys.Date())
downloads_rolling <- downloads %>%
  mutate(count = zoo::rollapply(count, 7, mean, fill = "extend"))
ggplot(downloads_rolling, aes(date, count)) + geom_line()
expandChain(output$summary(), .expansionContext = ec)
summary(downloads$count)

Expansion contexts are also useful for cases where you need to redefine a meta-reactive’s logic. This is useful in at least two scenarios:

  1. For efficiency or privacy reasons, you may not want to provide the “rawest” form of the data in your app to users. Instead, you might want to only provide a transformed and/or summarized version of the data. For example, instead of providing the user with downloads, we could provide downloads_rolling as file to be included as part of a download bundle.
saveRDS(downloads_rolling(), "d.rds")
ec <- newExpansionContext()
ec$substituteMetaReactive(downloads_rolling, function() {
  metaExpr(readRDS("d.rds"))
})

expandChain(
  quote(library(ggplot2)),
  output$plot(),
  .expansionContext = ec
)
library(ggplot2)
downloads_rolling <- readRDS("d.rds")
ggplot(downloads_rolling, aes(date, count)) + geom_line()
  1. Apps that allow users to upload a file: the location of the file on the server won’t be available to users, so it may be easier just to substitute the reactive that reads the uploaded file. For an example, see this example in the next vignette.

Step 4: Improving the readability of generated code

There’s a few different techniques you can leverage to improve the quality of the generated code, including:

  • Comment preservation: Surround comments in quotes to ensure they appear in the generated code. This works with any meta-reactive as well as expandChain():
mr <- metaReactive({
  "# comment"
  1 + 1
})
expandChain("# another comment", mr())
# another comment
# comment
mr <- 1 + 1
mr
  • Controlling names: In some cases, meta-reactive name inference fails1 and/or isn’t quite the name you want to appear in the generated code. In those cases, you can specify the name via the varname argument.
mr <- metaReactive(1 + 1, varname = "two")
expandChain(mr())
two <- 1 + 1
two
  • Controlling scope: Meta-reactive expressions that use intermediate variable names may generate code that introduces those names into the global scope. For example, the code generated from this three meta-reactive introduces two into the global scope:
three <- metaReactive({
  two <- 1 + 1
  two + 1
})
expandChain(three())
three <- {
  two <- 1 + 1
  two + 1
}
three

If you want to be careful not to unnecessarily introduce names into the users namespace, you can force the generated code expressions to be wrapped in local() which ensures intermediate variables aren’t bound to the global environment:

three <- metaReactive({
  two <- 1 + 1
  two + 1
}, localize = TRUE)
expandChain(three())
three <- local({
  two <- 1 + 1
  two + 1
})
three

Another option is to bind the meta-reactive’s name to the last call of the meta-expression expression. This option has the benefit of generating the most readable code, but also has the downside of introducing intermediate variables into the global namespace.

three <- metaReactive({
  two <- 1 + 1
  two + 1
}, bindToReturn = TRUE)
expandChain(three())
two <- 1 + 1
three <- two + 1
three

The execution model

For most existing Shiny applications, you should be able to follow the steps outlined above in the Overview section, and the code generation should “just work”. In some scenarios, however, you may have to tweak or debug your Shiny app logic, and in doing so, it’ll be helpful to understand shinymeta’s model for execution.

Meta-reactives (e.g., metaReactive(), metaRender(), etc) can be invoked in two different modes: meta or normal (the default). In normal mode, the behavior of a meta-reactive is essentially the same as the non-meta version (e.g., downloads() still evaluates and caches results just like a normal reactive() does). The only subtle difference is that, in normal execution, meta-reactives know to (silently) ignore ..():

downloads <- metaReactive({
  cranlogs::cran_downloads(
    ..(input$package), 
    from = Sys.Date() - 365, 
    to = Sys.Date()
  )
})
downloads()
        date count package
1 2018-08-06 17220 ggplot2
2 2018-08-07 18114 ggplot2
3 2018-08-08 17990 ggplot2
4 2018-08-09 17736 ggplot2
5 2018-08-10 14900 ggplot2
6 2018-08-11  8218 ggplot2
[...plus 360 more rows...]

When invoked in meta mode, meta-counterparts return a code expression instead of fully evaluating the expression. shinymeta currently provides two ways to invoke meta-reactives in meta mode: withMetaMode() and expandChain(). In practice, you’ll almost always want to use expandChain() over withMetaMode(): the former has a special understanding of marked reactive reads, whereas the latter is a less intelligent quasi-quotation interface. More specifically, expandChain() intelligently substitutes marked reactive reads with suitable value(s) or name(s) (and reuses those names to avoid redundant computation), whereas withMetaMode() does nothing more than evaluate what appears in ..().

withMetaMode(downloads())
cranlogs::cran_downloads("ggplot2", from = Sys.Date() - 365, to = Sys.Date())

When applied to arbitrary code expression, ..() works like an unquoting operator (similar to rlang’s !! operator), regardless of whether expandChain() or withMetaMode() is used. That is, it evaluates the code that appears in ..() and inlines the result in the generated code. This makes it possible, for instance, to ‘hard-code’ a dynamic result (e.g., use the date the code was generated instead of when the generated code is actually evaluated).

downloads <- metaReactive({
  cranlogs::cran_downloads(
    ..(input$package), 
    from = ..(format(Sys.Date() - 365)), 
    to = Sys.Date()
  )
})
expandChain(downloads())
downloads <- cranlogs::cran_downloads("ggplot2", from = "2018-08-06", to = Sys.Date())
downloads

When it comes to -2 variants (e.g. metaReactive2(), metaRender2(), etc), only the code that appears inside metaExpr() can execute in meta mode. That means, among other things, that the read of downloads() that appears outside of metaExpr() always returns a data frame (the validate() wouldn’t make sense if downloads() returned code!). It also means that ..() isn’t defined outside of metaExpr().

downloads_rolling <- metaReactive2({
  # Using ..() here would produce an error
  validate(need(sum(downloads()$count) > 0, "Input a valid package name"))
  
  metaExpr({
    ..(downloads()) %>% 
      mutate(count = zoo::rollapply(count, 7, mean, fill = "extend"))
  })
})

expandChain(downloads_rolling())
downloads <- cranlogs::cran_downloads("ggplot2", from = "2018-08-06", to = Sys.Date())
downloads_rolling <- downloads %>%
  mutate(count = zoo::rollapply(count, 7, mean, fill = "extend"))
downloads_rolling

  1. Name inference depends on a srcref of the expr argument being available. There are at least a couple different ways name inference can fail: (1) The keep.source option is FALSE (2) expr does not appear as the first argument.↩︎