shinyloadtest package and the accompanying
shinycannon software enable load testing deployed Shiny applications.
Load testing helps developers and administrators estimate how many users their application can support. If an application requires tuning, load testing and load test result analysis can be used to identify performance bottlenecks and to guide changes to infrastructure, configuration, or code.
It’s a common misconception that “Shiny doesn’t scale.” In actuality, properly-architected Shiny applications can be scaled horizontally, a fact which Sean Lopp was recently able to demonstrate at rstudio::conf 2018. We used
shinycannon to simulate 10,000 concurrent users interacting with an application deployed to AWS. You can see a recording of Sean’s talk and the load test demonstration here: Scaling Shiny
To get started with
shinyloadtest, read through the quick start guide below.
To perform a load test you’ll need two pieces of software:
shinyloadtestis an R package used to generate recordings and analyze results. You should install it on your development machine. If you are installing
shinyloadtestwithin a Linux environment, you will need to install
OpenSSL >= 1.0.1and its associated development libraries as those are required by the
websocketpackage. On Debian/Ubuntu systems, install the
libssl1.0-devsoftware packages. On Redhat/Fedora/CentOS systems, install the
shinycannonis the command-line replay tool. You can install it on your development machine for testing, but for best results we recommend installing it on a server, and preferably not the one the application under test is also on.
As opposed to the
shinyloadtest R package,
shinycannon is installed and run differently depending on platform.
Depending on your distribution,
shinycannon can be installed using one of the following packages.
|Distribution||Download Link||Install Command|
The process for load testing a Shiny application consists of three steps:
Rinse and repeat as necessary. Each step is described below.
Record a session using the
record_session function in the
shinyloadtest package, which takes the URL of the deployed application as an argument:
You should substitute
https://shinyapp.example.com/ for the URL of the Shiny app you’d like to record. If your application requires authentication you should consult the authentication article. You should also be aware that certain Shiny features are not compatible with shinyloadtest.
Running the function will open a browser displaying the app. Once open, interact with the application as a normal user would and then close the browser. After closing the browser window or tab, a file will be created that contains a recording of the session. This recording, written to
recording.log by default, will serve as the basis for the load test.
With the recording in hand, we’re ready to run the load test. The actual test is conducted outside of R using the
shinycannon command-line tool. You can run it using your system’s terminal or console program, or you can run it from the RStudio terminal tab.
shinycannon accepts two required positional arguments:
In addition to these two required arguments,
shinycannon accepts a number of optional arguments that can be specified with flags. Of these, the most interesting are:
shinycannonuses threads to represent each user. It defaults to 1.
shinycannonwill re-run the recording as necessary to fill the time.
As an example, to run a load test simulating 5 concurrent users for at least 2 minutes, outputting to the directory
$ shinycannon recording.log https://shinyapp.example.com/ --workers 5 --loaded-duration-minutes 2 --output-dir run1 2018-08-29 15:06:14.191 INFO [progress] - Running: 0, Failed: 0, Done: 0 2018-08-29 15:06:14.193 INFO [thread01] - Warming up 2018-08-29 15:06:14.195 INFO [thread00] - Waiting for warmup to complete 2018-08-29 15:06:19.193 INFO [progress] - Running: 1, Failed: 0, Done: 0 2018-08-29 15:06:24.194 INFO [progress] - Running: 1, Failed: 0, Done: 0 2018-08-29 15:06:29.083 INFO [thread02] - Warming up 2018-08-29 15:06:29.195 INFO [progress] - Running: 1, Failed: 0, Done: 0 2018-08-29 15:06:34.195 INFO [progress] - Running: 2, Failed: 0, Done: 0 2018-08-29 15:06:39.196 INFO [progress] - Running: 2, Failed: 0, Done: 0 2018-08-29 15:06:43.973 INFO [thread03] - Warming up 2018-08-29 15:06:44.196 INFO [progress] - Running: 2, Failed: 0, Done: 0 2018-08-29 15:06:49.196 INFO [progress] - Running: 3, Failed: 0, Done: 0 2018-08-29 15:06:54.201 INFO [progress] - Running: 3, Failed: 0, Done: 0 2018-08-29 15:06:58.862 INFO [thread04] - Warming up 2018-08-29 15:06:59.201 INFO [progress] - Running: 3, Failed: 0, Done: 0 2018-08-29 15:07:04.201 INFO [progress] - Running: 4, Failed: 0, Done: 0 2018-08-29 15:07:09.202 INFO [progress] - Running: 4, Failed: 0, Done: 0 2018-08-29 15:07:13.751 INFO [thread05] - Warming up 2018-08-29 15:07:13.752 INFO [thread00] - Maintaining for 2 minutes (120000 ms) 2018-08-29 15:07:14.202 INFO [progress] - Running: 4, Failed: 0, Done: 0 2018-08-29 15:07:19.202 INFO [progress] - Running: 5, Failed: 0, Done: 0 2018-08-29 15:07:24.202 INFO [progress] - Running: 5, Failed: 0, Done: 0 ...
shinycannon includes detailed help documentation explaining the other arguments:
During the test,
shinycannon reports the progress and number of simulated users. The result of the test is an output directory (
run1 in the example above) which includes timing information for each session.
The output directory created in step 2 (
run1) is the focus of step 3. The
shinyloadtest R package provides functions for analyzing the output files from the test.
A tidy data frame of run simulation information can be created using
df <- shinyloadtest::load_runs("5 workers" = "./run1")
Once the data is loaded into R, a report can be generated using:
This self contained html report will be opened in your browser for inspection. For further analysis explaination, please visit ‘Analysing load test logs’.