This vignette answers frequently asked technical questions about {safetyGraphics}. It addressees questions on a variety of technical topics including Qualification and Validation status, Common Data Pipelines and Security.
Whenever new questions come in, we’ll update the version of this FAQ in our wiki - so check there first if you have a question. We’ll update the vignette on CRAN whenever a new version of the package is released.
A: Yes. Check out our contributor guidelines. Feel free to follow-up with questions on the discussion board.
A: Yes. Fill out this Google Form and we’ll get in touch.
A: As of the version 2 release, the safetyGraphics package is intended for exploratory use only and is not validated or qualified per 21 CFR Part 11. No warranty or guarantees are included as part of the package. Further, any formal validation should be fit for purpose and follow your organization’s procedures. That said, extensive quality checks are built in to the package (see the question below for details) and into many of charts that are included by default. The R Consortium has guidance on usage of R in Regulated Trials and we also follow the work of R Validation hub closely, and may release validation guidance based on the approach described in their white paper at a future date.
A: Study-specific instances of most safetyGraphics charts can be exported either as an R script or as a standalone html report. It may be possible to treat those outputs as standard TLFs (Tables, Listings and Figures) and conduct QC/Validation on them using standard statistical SOPs. Consult with your companies procedures to confirm.
A: Several layers of quality control are included in
our standard development workflow including: - Over 200 automated unit
tests with {testthat} that run automatically via Continuous
integration with GitHub actions. - Automated unit tests for shiny
modules run via a headless browser using {shinytest}. -
Pass all standard R checks in R cmd check - Full code
review of all new functionality documented in GitHub PR. - Issue
tracking in GitHub - Formal Alpha/Beta user testing before major
releases - Basic user tests conducted before minor release
A: As of the v2 release, safetyGraphics is mostly used in an exploratory fashion for a variety of use cases. The details of a ‘production’ implementation of safetyGraphics in a GxP environment would largely be dictated the intended use of the tool. For example, using safetyGraphics as your primary safety oversight platform would likely require more work than using it as a supplemental exploratory tool for biostatisticians.
That said, the issues surrounding a ‘production’ deployment are mostly technical and operational at this point and could likely be overcome by a motivated organization with ample technical expertise. Some of the issues that would need to be addressed in a production deployment:
Many of these issues aren’t specific to safetyGraphics and may be easier to address for an organization that has experience using R and Shiny in production. As discussed above, there is a significant push towards using R for many aspects of clinical trials. We plan to keep safetyGraphics up to date with emerging best practices and will provide supporting documentation whenever possible.
Finally, it is worth noting “Productionize” and “Validate” are slightly different. Joe Cheng (the primary author of Shiny) has a nice talk on this topic from a software engineering perspective.
A: Yes. The package is open source and free to use. Here’s a link to the license and a quick summary of how it works.
A: safetyGraphics graphics has been used in a variety of ways. Some of the most common use cases thus far are: - Analysts exploring safety data - Clinicians monitoring ongoing studies - Analysts and Clinicians evaluating safety signals in completed studies
As an open source tool with a flexible data pipeline, many other use cases have been discussed: - Data review by Data Safety Monitoring Boards (link) - Visualizing Analysis results data (link) - Risk based monitoring
A: No. Any standard (or non-standard) data can be
loaded as long as it meets the minimum data
requirements for the selected data domain. Metadata capturing
default CDISC standards are included with the app (see
?safetyGraphics::meta) so that data mappings can be
automatically populated when AdAM and SDTM data are loaded. Other data
standards require the user to manually complete the data mapping in the
mapping tab - see the cookbook vignette for
examples.
A: This topic is covered in detail in the Loading data section of the
Introductory vignette. safetyGraphics is designed to
support a flexible data pipeline that supports many data types. In
short, data can be loaded using the dataDomains parameter
in the safetyGraphicsApp() function or via the
safetyGraphicsInit() graphical user interface.
safetyGraphicsApp() - custom data can be loaded via the
dataDomains parameter, which should be a list containing
dataframes or tibbles for each clinical
domain; that list can be populated by importing data from any number of
sources including databases, sas files or any number of other
sources. See the cookbook vignette for
some basic examples of loading custom data.
safetyGraphicsInit() - allows users to load tabular data
from a variety of sources using the point-and-click interface provided
in the {datamods} package.
More detail is provided in the Loading data section of the Introductory vignette
Since the safetyGraphics app typically uses data subject to GxP regulations, data security is extremely important and should always be discussed with your organizations IT and regulatory departments before loading any study data in to the application. No warranty or guarantees are included as part of the package.
Just like the discussion regarding Validation and Quality Control, data security requirements are dictated by the intended use of the app and should be fit for purpose. There are many different ways to run shiny applications, and the security implications of each approach varies. For example, having statisticians run the app locally using R Studio is quite different than deploying the app on a service like shinyapps.io. This complexity is all the more reason to discuss with IT. There are many resources - related to data security for clinical trials in general (1, 2, 3) and discussing data security in shiny (4, 5) - that could help to facilitate these discussion.
A: It depends a bit on your use-case and how the app is hosted. For example, analysts using the data in an exploratory fashion can probably just run it from RStudio, but if multiple medical monitors using the app for medical monitoring in active studies probably need a more robust (and possibly validated) set up using Shiny Server, RStudio Connect or something similar.
safetyGraphics to shinyapps.io to
explore trial data from my organization?A: We advise against loading non-authorized, private, or non-deidentified patient data outside of your organization’s firewall. Consult with your IT and QA first. There is huge risk associated with confidentiality, IP, and patient privacy. Also refer to ShinyApps.io Chapter 8 Security and Compliance.
safetyGraphics to an
internal RStudio
Connect server?A: Yes. The script
below should be easy to deploy via the RStudio interface or by
running rsconnect::deployApp() and can easily be customized
to support custom data and charts.
# Launch the ShinyApp (Do not remove this comment)
library(safetyGraphics)
safetyGraphics::safetyGraphicsApp(runNow = FALSE) A: Several of the JavaScript charts build using htmlwidgets do have performance issues with very large data sets. Focusing on specific toxicities helps, but probably isn’t enough for really big studies. In those cases, I think the most important thing is to design a your data pipeline to include both a database backend (as opposed to loading all of your study data each time you initialize the app) and visualizations that summarize the data in a reasonable way (as opposed to just plotting every single data point in a scatter plot no matter what). Fortunately this is all doable in R, and improvements in this area are on our road map for future releases of safetyGraphics.
safetyGraphics to shinyapps.io to explore
trial data from my organization?safetyGraphics to an internal RStudio Connect
server?