1
Examples of Funded Grants in Implementation Science
Overview
The National Cancer Institute (NCI) frequently receives requests for examples of funded grant
applications. Several investigators and their organizations agreed to let Implementation Science
(IS) post excerpts of their dissemination and implementation (D&I) grant applications online.
About
We are grateful to the investigators and their institutions for allowing us to provide this important
resource to the community. To maintain confidentiality, we have redacted some information
from these documents (e.g., budgets, social security numbers, home addresses, introduction to
revised application), where applicable. In addition, we only include a copy of SF 424 R&R Face
Page, Project Summary/Abstract (Description), Project Narrative, Specific Aims, and Research
Strategy; we do not include other SF 424 (R&R) forms or requisite information found in the full
grant application (e.g., performance sites, key personnel, biographical sketches).
Copyright Information
The text of the grant applications is copyrighted. Text from these applications can only be used
for nonprofit, educational purposes. When using text from these applications for nonprofit,
educational purposes, the text cannot be changed and the respective Principal Investigator,
institution, and NCI must be appropriately cited and credited.
Accessibility
Individuals using assistive technology (e.g., screen reader, Braille reader, etc.) who experience
difficulty accessing any information should send an email to the Implementation Science Team
([email protected]).
2
424 R&R and PHS-398 Specific
Table Of Contents
Examples of Funded Grants in Implementation Science 1
Table Of Contents 2
SF 424 R&R Face Page 3
Project Summary 4
Project Narrative 5
Specific Aims 6
Research Strategy 7
References 22
3
SF 424 R&R Face Page
PI: Rahm, Alanna K
Grant Number: 1 R01 CA211723-01A1
Title: Implementing Universal Lynch Syndrome Screening across Multiple Healthcare Systems:
Identifying Strategies to Facilitate and Maintain Programs in Different Organizational Contexts
FOA: PAR16-238
FOA Title: DISSEMINATION AND IMPLEMENTATION RESEARCH IN HEALTH (R01)
Organization: GEISINGER CLINIC
Department: Genomic Medicine Institute
Senior/Key Personnel: Alanna Rahm Ph.D
Organization: GEISINGER CLINIC
Role Category: PD/PI
4
Project Summary
Lynch syndrome (LS) is the most common form of inherited colorectal cancer risk. People with
Lynch syndrome are also at increased risk for endometrial, ovarian, gastric, small bowel, and
renal cancers. Importantly, well-established clinical guidelines with strong evidence exist for
cancer treatment, screening, and prevention in individuals with LS. Identification of individuals
with LS is accomplished through a variety of techniques, including family and medical history
evaluation, computational models, or tumor testing. The systematic screening of all colorectal
tumors for LS was first recommended by the Evaluation of Genetic Application in Practice and
Prevention (EGAPP) working group in 2009 and has been designated high priority by the
National Academies of Science, Engineering, and Medicine working group and by the Blue
Ribbon Panel. The potential public health impact to reduce cancer morbidity and mortality of this
intervention supports this priority, as effective implementation of LS screening will help meet the
goals of the Cancer Moonshot as well as demonstrate the promise of precision medicine.
Currently, implementation of LS screening in healthcare systems remains suboptimal for a
variety of reasons. LS screening involves the coordination of multiple departments and
individuals across an organization, which is often difficult in large, complex, healthcare systems.
Therefore, the overarching goal of this project is to utilize tools from implementation science to
describe, explain, and compare decision making and other variations in LS screening
implementation across multiple healthcare systems to create and evaluate in a real world setting
an organizational toolkit to facilitate implementation of LS screening. Our specific aims are to (1)
Describe variation in LS screening implementation across multiple healthcare systems; then (2)
Explain practice variation and determine factors associated with optimal implementation; and (3)
Determine the relative effectiveness, efficiency, and costs of different LS screening protocols by
healthcare system; and finally to (4) Develop and test in a natural environment an organizational
toolkit for LS screening. This toolkit will enable effective implementation of LS screening
programs; ultimately preventing needless suffering of patients and their family members from
preventable cancers, decreasing waste in healthcare system costs, and informing strategies to
facilitate the promise of precision medicine.
5
Project Narrative
The overarching goal of this project is to create an organization-level toolkit for implementing,
maintaining and improving Lynch syndrome (LS) screening by using tools from implementation
science to describe, explain, and compare decision making and other variations in LS screening
implementation across multiple healthcare systems. We will accomplish this through analyzing
variation in LS screening implementation across diverse healthcare systems, estimating costs of
different protocols by healthcare system, synthesizing this information into an organizational
implementation toolkit, and testing the toolkit within the healthcare systems. This model will
enable more effective and efficient implementation of LS screening; ultimately preventing
needless suffering of patients and their family members from preventable cancers, decreasing
waste in healthcare system costs, and informing strategies to facilitate the promise of precision
medicine.
6
Specific Aims
The goal of precision medicine is to improve individual health outcomes by tailoring healthcare based on genomic and
other relevant information
1
. One such example is the use of systematic tumor screening to identify all patients newly
diagnosed with colorectal cancer whose cancer may be related to Lynch syndrome (LS)
2,3
. Estimates suggest about one
million people in the US have LS, of which only about 2% are aware
4,5
; therefore, most are not receiving life-saving
surveillance and treatment. LS screening includes evaluating tumors for mismatch repair gene deficiency and offering
genetic counseling and confirmatory germline genetic testing to individuals who screen positive. LS screening is one of
the first cost-effective
6,7
genomic medicine interventions with top-tier evidence
8
for reducing cancer morbidity and mortality
and improving quality of life
9
. In September, 2016 the Blue Ribbon Panel Report recommended LS screening as a
high priority intervention with the potential to achieve the goals of the Cancer Moonshot
4
.
Implementation of new technologies into clinical practice, however, is challenging
10,11
. Contextual factors such as
organization mission, organization structure, economic impact, providers, and patient population, all influence
implementation decisions in healthcare systems
12-14
. Therefore, analysis of these contextual factors and their effects is
critical to our understanding of variability in implementation
15-18
. The Consolidated Framework for Implementation
Research (CFIR) is designed to guide multi-level evaluation of implementation, and has been used successfully to
evaluate variation in program implementation
16,19
. The CFIR, along with Qualitative Comparative Analysis (QCA), can
identify which implementation strategies are more likely to work under which circumstances; resulting in an organizational
toolkit for implementing complex interventions in complex health care delivery systems
20-24
.
Implementing LS screening involves multiple stakeholders and customization to local contextual factors such as individual
organizational processes, patients, and costs. Because LS screening is infrequently and inconsistently implemented,
there is poor understanding of how these contextual factors impede or facilitate implementation in healthcare systems and
under what circumstances
15,22,25-27
. The goal of this proposal is to utilize the CFIR and other tools from implementation
science to describe, compare, and explain variations in LS screening implementation across multiple healthcare systems
and create a comprehensive, customizable organizational toolkit for implementing LS screening. Our specific aims are to:
Aim 1: Describe variations in LS screening implementation across multiple healthcare systems. Guided by the
CFIR, we will conduct interviews with key stakeholders from multiple sites within members of the Healthcare Systems
Research Network (HCSRN). We will describe variations in LS screening processes, organizational structure and
resources, organizational decision making, and barriers and facilitators related to implementing LS screening as
recommended by published guidelines.
Aim 2: Explain current practice variation and determine factors associated with optimal LS screening
implementation. Through cross-case analysis and Qualitative Comparative Analysis (QCA) guided by the CFIR, examine
associations between contextual factors and LS screening implementation to determine factors associated with
implementation, maintenance, and improvement. We will conduct analyses to determine factors associated with
implementing LS screening or not and analyses to determine factors associated with optimal and sub-optimal
implementation across healthcare systems.
Aim 3: Determine the relative effectiveness, efficiency, and costs of different LS screening protocols. Using
decision analysis models developed from previous work
28,29
and data specific to each healthcare system, we will
demonstrate the relative effectiveness and efficiency of various LS screening protocols used by healthcare systems based
on their local costs.
Aim 4: Develop and test in a natural environment an organizational toolkit to facilitate LS screening
implementation and improvement. A draft toolkit will be disseminated to all sites. Additional interviews and analyses will
assess utility for facilitating LS screening implementation or improvement.
Through systematic comparison and in-depth analyses of implementation across multiple healthcare systems, this study
will create a comprehensive toolkit for organization-level decision-making to facilitate LS screening implementation and
improvement and lead to testable hypotheses about associations between specific organizational contextual factors and
implementation. This organizational toolkit will enable more effective and efficient implementation of LS screening;
ultimately preventing needless suffering of patients and their family members from preventable cancers, decreasing waste
in healthcare system costs, and informing strategies to facilitate the promise of other precision medicine initiatives.
7
Research Strategy
A. BACKGROUND AND SIGNIFICANCE
Colorectal cancer (CRC) is the third leading cause of cancer deaths in the US
30
. Importantly, colonoscopy
is effective for both screening and primary prevention, particularly when those with hereditary risk can be
identified and cared for appropriately
30,31
. Lynch Syndrome (LS) is the most common form of inherited CRC risk
and includes significant risk for second primary cancer
31,32
. Cost-effective evidence-based systematic
screening strategies to identify CRC patients with LS exist
6,7,28,29,33
, yet this precision medicine approach for
cancer prevention is inconsistently (if at all) applied within healthcare systems
15,22,27,34
, resulting in unwarranted
suffering and death from preventable cancers in cancer patients and their families
35
.
Estimates indicate about one million people in the US have LS
4,5
. LS accounts for 3-5% of all newly
diagnosed CRC
31
; yet only about 2% of individuals are diagnosed
5
. Individuals with LS have an increased risk
of endometrial, ovarian, gastric, small bowel, and renal cancers, among others
31,32
. Diagnosis is confirmed
when a germline genetic mutation is detected in any one of four DNA mismatch repair genes (MLH1, MLH2,
MLH6, and PMS2). Importantly, well-established clinical guidelines with strong evidence exist for screening
and prevention of cancers in individuals with LS
31
. Earlier (prior to population screening age) and more
frequent colonoscopies in individuals with LS can reduce CRC risk by 62%
36
and CRC mortality by 70%
2,37-40
.
Identification of individuals with LS can be accomplished through a variety of techniques, including family and
medical history evaluation, computational models, or tumor testing
31,32
. However, clinical and family history-
based methods alone, even if optimally applied, fail to identify at least one-third of LS patients
31,41
.
Importance of LS screening is recognized by the Blue Ribbon Panel to save lives from cancer
4
.
Systematic screening for LS has clear evidence supporting broad implementation in healthcare systems
9
. This
“universal” approach was first recommended by the Evaluation of Genetic Application in Practice and
Prevention (EGAPP) working group in 2009
2,42
, has CDC-ranked top-tier evidence
8
for reducing cancer
morbidity and mortality and improving quality of life
2,9
, is currently recommended by multiple professional
organizations
31,43-47
, is endorsed by the National Comprehensive Cancer Network (NCCN)
48
, is an objective of
the Healthy People 2020 initiative
3
, and was recently recommended by the Blue Ribbon Panel Precision
Prevention and Early Detection Working Group to meet the goals of the Cancer Moonshot
4
. LS screening
involves evaluating all CRC tumors for evidence of mismatch repair gene deficiency via immunohistochemistry
testing or molecular testing for microsatellite instability. Individuals whose tumors screen positive are then
offered confirmatory germline sequencing to diagnose LS
31,32,49
. For patients with cancer, diagnosis of LS
changes surgical options, treatment and medical management, and additional screening and prevention
requirements (especially for women). Emerging evidence suggests CRC patients with LS may benefit from
treatment with certain immunotherapy options
50
. CRC patients with less than total colectomy have about a 20%
risk for metachronous tumors in 10 years and therefore require more frequent screening
31,32
. Likewise,
endometrial cancer occurs in 54% of women with LS, a risk that can be significantly reduced (90-100%) with
prophylactic surgery
31,40,51
.
Public health impact of LS screening cannot be realized without effective implementation. First degree
relatives of patients with LS are at 50% risk to also have LS, and have an 85% lifetime risk of cancer
31,32,52
.
Therefore, the cost effectiveness of LS screening is greatest when cascade testing identifies at-risk
relatives
6,7,28,29,53
. When individuals with cancer are identified through LS screening, they are more likely to
follow up with genetic counseling and diagnostic gene sequencing
54,55
. Likewise, evaluation and oversight of
LS screening by genetic counselors results in higher patient follow through to gene sequencing
22,56
. However,
unless the first individual is identified through effectively implemented LS screening, additional family members
cannot be found and overall impact of this precision medicine intervention will be greatly reduced.
8
Implementation of LS screening in healthcare systems has been slow
4,8,27
. Only half of all genetic
counselors report LS screening of some type at their institution
57
and more academic medical centers report
implementing LS screening than other types of cancer centers
58
. This gap between evidence-based guidelines
and their implementation into routine clinical practice is emblematic of one of the most critical issues in
healthcare and public health today
25,59
. Therefore, this research proposal seeks to understand organizational
factors impacting implementation and create an organizational toolkit to guide implementation, evaluation,
maintenance, and improvement of LS screening - a recognized area of genomic medicine ready for national
implementation with known variability and incomplete implementation across healthcare systems.
LS screening offers a prime opportunity to study and develop new models for implementation.
Contextual factors such as organization mission, patient population, and economic impact of policies all
influence decisions to implement genomic technologies in healthcare systems
12-14
. Factors specific to LS
screening implementation may include: involvement of multiple key stakeholders and champions, availability of
genetic counseling, and genetic testing costs. This complexity contributes to existing variability across
healthcare systems, making it unlikely a single strategy or inflexible process will lead to successful LS
screening implementation in all systems. In fact, there are multiple evidence-based protocols that are
acceptable for use in LS screening
31,44
. Choosing the most appropriate protocol suited to the organization may
determine the success or failure of implementation. Therefore, an organizational-level toolkit informed by the
principles of implementation science can better facilitate LS screening implementation in healthcare systems
by providing guidance on which protocol is best suited to specific organizational contexts and costs
25
.
Evaluation of LS screening implementation and toolkit development will be guided by a framework
from implementation science. The Consolidated Framework for Implementation Research (CFIR) uses
constructs from multiple implementation science theories to guide multi-level evaluation of implementation.
CFIR has been used successfully to evaluate variation in program implementation in the VA system
16,19
and
variation in patient follow-through to confirmatory gene sequencing in LS screening
22
. The CFIR guides
assessment of implementation barriers and facilitators at the individual, organizational, and external levels, and
can also guide data gathering and structuring for Qualitative Comparative Analysis (QCA). QCA is useful for
studying causal complexity in organizational implementation (LS screening implementation in this study)
20-23
.
CFIR constructs also include cost, a critical component of implementation in healthcare systems. Business
case analysis algorithms to understand local costs associated with LS screening were created by members of
the study team and utilized by one healthcare system
28,29
, but have not yet been widely disseminated.
The resulting organizational toolkit will provide guidance for the evaluation, maintenance, and
improvement of LS screening in the face of organizational context and changes in scientific evidence.
Most studies of implementation focus on barriers and facilitators in individual organizations, or across a few
organizations, without providing guidance for other organizations. In addition, cost-effectiveness studies are
usually performed from the societal perspective; which do not provide useful insights for local decision makers
about the cost impact within a specific organization
60,61
. Finally, little attention is paid to the maintenance and
improvement of programs in the face of changing organization contexts and scientific evidence. This point is
critical, as the evidence base for new technologies, particularly genomic technologies, is likely to increase
substantially due to the national precision medicine research initiative
62
. By creating an organizational toolkit
that includes guidance for implementation, maintenance, and improvement, this project could accelerate
optimal implementation of LS screening; benefitting patients, families, the healthcare system, and society; thus
meeting a goal of the Cancer Moonshot and demonstrating the promise of precision medicine.
B. INNOVATION
This study will conduct an in-depth assessment of contextual factors impacting implementation across
an unprecedented number of sites representing diverse healthcare systems, geographies, and patient
populations served. This data will provide significant information for the Precision Prevention and Early
9
Detection Working Group of the Blue Ribbon Panel to successfully address the Cancer Moonshot
4
, and for the
working group of National Academies of Sciences, Engineering, and Medicine (NASEM) to successfully
facilitate their goal of implementing LS screening
63
. The in-depth qualitative assessment of key stakeholders
across this number of diverse sites is possible because of the collaborations and processes built within the
Healthcare Systems Research Network (HCSRN), as well as the experience of this study team in researching
LS and in conducting centralized qualitative studies of this magnitude. Finally, in creating an organizational
toolkit guided by the CFIR, this study will also contribute to implementation science more generally.
This study combines multiple methods of exploring implementation in the complex environment of the
healthcare system. Traditional case-based in-depth analyses of individual healthcare system barriers and
facilitators will be conducted, followed by cross-case and Qualitative Comparative Analysis (QCA) to determine
combinations of conditions necessary and/or sufficient for implementing LS screening in the presence of
different organizational contextual factors, and cost-consequences modeling for local-level decision making.
This process goes beyond a typical “lessons learned” approach to a comprehensive and critical analysis of
implementation and non-implementation that is only possible because of the number of participating sites, a
number of which (N=10) have not yet implemented LS screening and others with sub-optimal implementation
at this time.
This study combines key stakeholder information with business case decision models populated with
local data. Relevance of general societal cost to organizational decision-making has been questioned
60,61
;
therefore, we will model site-specific costs of LS screening. Prior studies also indicate that organization-
specific costs to screen and cost to detect LS cases for different protocols is critical information for health
systems to make decisions about LS screening implementation
18,28,29
. To our knowledge, no studies have
synthesized in-depth cross-site comparison of context, barriers and facilitators with local business case
analyses into a comprehensive toolkit for organizations and that can be utilized beyond initial implementation.
This study will produce an innovative organizational toolkit to inform maintenance and improvement in
addition to initial implementation. Because of the number of sites in various stages of implementation
available for evaluation, this study will result in an organizational toolkit that informs initial LS screening
implementation, ongoing evaluation and maintenance, improvement of sub-optimal implementation, and
adaptation of optimal implementation to changing evidence. Traditional approaches to implementation lack
flexibility to incorporate emerging evidence and are therefore less likely to be successful in the era of precision
medicine. Likewise, most studies focus on strategies for initial implementation, rather than evaluation,
maintenance, and improvement in the face of organizational context or evidence changes. For example,
emerging evidence supports including evaluation of EC tumors when implementing LS screening
55,64,65
. Some
early adopters of LS screening have adapted their LS screening to include EC tumors (Table 1, section C.1.2).
How these sites adapted to new evidence will provide important information that has not been previously
available in implementation science or in organizational implementation toolkits.
This study will demonstrate how an implementation toolkit can be used in organizational decision
making to implement and improve LS screening. The greatest cost-effectiveness and cancer prevention
benefit of LS screening will be realized only after effective cascade testing of at-risk relatives can be
incorporated into optimally implemented programs. Providing a toolkit for organizational decision makers to
guide implementation based on system-specific contextual factors and costs is a critical first step towards
optimal implementation of LS screening through which familial cascade testing can be facilitated and studied.
Additionally, this toolkit may be generalizable to implementing screening for other genomic conditions with top-
tier evidence for effectiveness of familial cascade screening (e.g. Familial Hypercholesterolemia and
Hereditary Breast and Ovarian Cancer).
C. APPROACH
C.1. Preliminary Studies
10
C.1.1 Universal LS screening. The most recent
multi-society guidelines for CRC recommend
systematic or “universal” LS screening programs
test all CRCs, regardless of patient age, by IHC
with reflex testing for BRAF V600E mutation and
promoter hypermethylation (PHM) when there is
MLH1 protein loss (Figure 1)
31,32
. Testing tumors
with MLH1 protein loss for evidence of BRAF
v600E point mutation or PHM identifies sporadic
cancers not related to LS
31,32
. Confirmatory
germline sequencing follows for screen positive
individuals and medical management and
additional cancer risk management for the patient
as well as their at-risk family members is
determined
31
. Figure 1 also includes screening EC,
as some guidelines now recommend this
31,64,66,67
and some healthcare systems have already
implemented EC screening into their LS screening
protocols
68
(see Table 1, C.1.2). We will refer to this
as “optimal implementation” in this application and
study.
However, guidelines recognize the difficulty implementing LS screening in clinical practice and suggest that if
all CRCs cannot be tested, then testing tumors under age 70 and using family history and other risk models to
evaluate patients with CRC over age 70 is acceptable
31
; despite evidence demonstrating that age limits are not
cost effective and that clinical assessments fail to result in genetic testing for LS
5,34
. Likewise, other guidelines
advocate for maintaining age and tumor morphology limitations to EC screening
69
. These conflicting guidelines
in the face of evolving evidence are therefore likely key contributors to variability in implementation
25
.
C.1.2. Current LS screening across healthcare systems. Dr.
Rahm recently surveyed leaders or delegates of sites in the
HCSRN. Twelve sites responded (63% response rate) with
information about their current LS screening protocols (Table 1).
Two out of 12 systems reported no LS screening and only 4 sites
included, or were in process of including, EC tumor screening
C.1.3. Pilot feasibility. To further develop our research approach,
Dr. Rahm conducted exploratory interviews with one key
stakeholder from each of 5 HCSRN sites with LS screening
programs. These interviews were conducted to: a) determine
feasibility of a CFIR-based interview guide to gather information
Table 1. Survey of LS Screening in the HSCRN
Site
LS Screening Implementation
1
CRC only - no age limits but no enforcement
2
No Implementation
3
No Implementation
4
CRC only - no age limits w/ reflex
5
CRC only <75 unless high risk
6
CRC only - no age limit, reflex ordered by oncologist
7
CRC and EC - no age limits with reflex
8
CRC only no age limits, EC under60, adding all EC
9
CRC only no age limits, with reflex, adding EC
11
CRC under research protocol, no clinical program
12
CRC and EC - no age limits with reflex
about LS screening implementation, organizational context, barriers, and facilitators (see Appendix-Pilot
Interview Guide), b) begin to understand the complexity of factors contributing to variability between sites, c)
guide analysis plan development for the larger study, and d) determine the breadth of LS screening
implementation processes available for analysis. While the survey (Table 1) provided a cross section of LS
screening implementation and non-implementation across the HCSRN, interviews indicate that information
from multiple stakeholders across a number of healthcare systems is necessary to fully understand the multi-
level complexities of implementing LS screening.
11
Despite existing evidence supporting LS screening
2,27,42
, recent calls to action by the NASEM and the Blue
Ribbon Panel indicate a pressing need for increased efforts to improve implementation
4,63
. Our preliminary
data indicate persistent variability in LS implementation across healthcare systems and demonstrates the need
for additional data and organizational toolkits to facilitate implementation. The cross-site comparison and QCA
proposed here will synthesize for organizational decision-makers the breadth of barriers to implementation,
different solutions for those barriers, and which solutions are most likely to work under which conditions. This
data will be the foundation of a toolkit that will more effectively guide future implementation efforts in these and
other healthcare systems. One key example demonstrating this need is that one site recently implemented LS
screening as a randomized trial
70,71
. However, rather than adopting protocols developed by researchers,
clinical operations staff implemented by repeating work previously conducted by researchers. Stakeholders
from this site will provide critical information on factors hindering direct adoption of research protocols (see
Letters of Support KPNW). Without a toolkit to guide implementation based on strategies and contextual
information from multiple systems, as proposed by this project, other healthcare systems will proceed with ad-
hoc implementation, increasing risk of ineffective or unsustainable LS screening, or will simply continue to
avoid implementation all together.
C.1.4. Guiding framework. All five pilot interviewees described LS screening implementation as an ongoing
process; including improvement and adaptation over time as new evidence arises. LS screening
implementation may be a continuum between no LS screening to the current optimal implementation of
screening all CRC and EC tumors with reflexive BRAF/PHM testing
31,32
(Figure 1). Healthcare systems appear
to utilize multiple approaches for starting, adapting, and optimizing LS screening, as some healthcare systems
began with optimal implementation while others chose sub-optimal implementation (Table 1). Still others began
with sub-optimal implementation and improved to optimal screening.
This preliminary indication that LS screening implementation is not a static endpoint is consistent with other
reports of LS implementation
18,54,65
, and is a critical concept for genomic medicine, as new evidence and
technologies are constantly emerging. For example, the decreasing cost of gene sequencing could soon lead
to sequencing of all CRC patients as the most cost-effective screening option
31
. The CFIR is therefore an ideal
guiding framework for this study, as it describes implementation as an active process that changes and adapts
over time
72
. We believe utilizing the CFIR to guide study design, data collection, and analyses, will result in the
development of an organizational toolkit that will facilitate initial implementation, as well as maintenance and
optimization of LS screening as evidence in genomic medicine changes over time.
C.1.5. Cost-consequences analysis. Value, determined by the relationship between a set of health outcomes
and the costs associated with achieving those outcomes, is also critical to decision-making in healthcare
systems
10
, as each clinical site assesses value based on its individual mission and patient population
13
.
Previous experience of members of this research team
18,28,29
in implementing LS screening identified that cost
to the institution of the different testing protocols and of screening older cancer patients were key barriers
28,29
.
The latter concern was based on provider perception that excluding older CRC patients from LS screening
would substantially reduce total costs to the organization and increase efficiency with negligible impact on
detection of LS
28
. This provider perception remains an oft cited barrier to screening all ages of CRC and EC
patients in healthcare systems, despite evidence to the contrary
28,29,64,65
. Because local data including
institutional testing costs, number of CRC patients diagnosed per year, and local prevalence of LS, have been
reported as key inputs for creating budget impact and cost consequences models for organizational decision
makers
21,28,29
, we believe this information is a critical component of an implementation toolkit for LS screening.
C.1.5 Generalizability. The number and diversity of healthcare systems and clinical sites included in this study
will enhance generalizability of the toolkit to guide LS screening implementation, maintenance, and
improvement nationally and internationally. Evidence suggests issues and experiences of implementation are
not unique to the HCSRN. To enhance generalizability, we have included one clinical site outside the HCSRN
(Northwell Health). Other healthcare systems, as well as other countries are also seeking to implement LS
12
screening, yet these efforts continue similar inefficient ad-hoc implementation
56,73
. We have included
collaborators from Cancer Care Ontario and the Lynch Syndrome Screening Network (LSSN) as part of an
external project advisory panel to help ensure study results and implementation toolkit are broadly applicable
to LS screening implementation.
C.2. Setting
We will study the contextual factors of 23 clinical sites across 8 healthcare systems; seven of which are
members of the HCSRN. The HCSRN has standard data models and processes to facilitate IRB approval and
data sharing to improve efficiency for conducting multi-site studies such as this. Some of the healthcare
systems are also members of the Cancer Research Network (CRN), a subgroup of the HCSRN, which also
includes a scientific working group specific to communication and dissemination research (C&D SWG); led by
established experts in implementation science. The C&D SWG will serve as an additional venue for presenting
preliminary study results to a multi-disciplinary group of scientists and for additional dissemination of study
results (Letters of Support-C&D SWG).
C.3. Participants
Table 2. Lynch Syndrome Screening Implementation Across Healthcare Systems and Clinical Sites
Clinical Site
NO SCREENING
CRC Screening
BRAF Reflex
PHM Reflex
EC Screening
PHM Reflex
Geisinger
All Ages
x
x
All Ages
x
Geisinger-Holy Spirit
x
PAMF
All Ages
KP-Colorado (KPCO)
x
KP-Northwest (KPNW)
All Ages
x
x
MCPI
All Ages
Health Partners
All Ages
x
x
All Ages
x
Harvard Pilgrim
x
Northwell Health
All Ages
CHI-Franciscan WA
All Ages
x
x
All Ages
x
CHI- Tri-Health OH
All Ages
x
x
All Ages
x
CHI-Mercy Des Moines IA
All Ages
x
x
All Ages
CHI-Kentucky One KY
All Ages
CHI-Chattanooga TN
All Ages
x
x
All Ages
x
CHI-Good Samaratin NE
x
CHI-Lincoln NE
<70 years
x
CHI-St Francis NE
x
CHI-St Joes Bryan TX
x
CHI-St. Vincent AR
x
CHI-Centura CO
<70 years
<60
CHI-Alegent Creighton OH
x
CHI- St. Alexius ND
x
CHI- Mercy ND
x
C.3.1. Healthcare systems. For this project, the unit of analysis is the clinical site through which LS screening
is or can be implemented. Participating healthcare systems (N=8; Table 2) have been purposively selected to
maximize the number of clinical sites (N=23) in various stages of implementing LS screening, as well as to
maximize diversity of location, system structures, and patient populations. Sampling selection also includes
one HCSRN system with LS screening recently acquired a smaller system that has not implemented LS
screening (see Letter of Support - Geisinger Holy Spirit). Another HCSRN system, Catholic Health Initiatives
(CHI) has a centralized research structure but the clinical sites (N=14) operate independently and have
different clinical structures, patient populations, and LS screening implementation. CHI has the ability to
influence clinical implementation in the organization both at the local site levels and from an overall policy level.
C.3.2. Patient and organizational stakeholders. In-depth qualitative interviews with key stakeholders will be
utilized to elicit the information important to organizational decision making about LS screening implementation
13
and provide data for the site-level analysis. Key stakeholder opinions important to LS screening
implementation include patient and organizational stakeholders. Patient opinion is important to organizational
decision-making, as anticipation of patient reactions can be a barrier to implementation for some clinical sites.
We will therefore interview newly diagnosed cancer patients (10 per site) and cancer patients who have
received a positive LS screening result (N=25 total from sites with LS screening at the beginning of the study)
in order to provide this information for organizational decision-making. Organizational stakeholders (N=10 per
site) important to LS screening implementation include individuals from health plan leadership, pathology,
genetics, surgery, oncology, and others
70,74
(Table 3 Section C.5.1). Patient and organizational stakeholder
recruitment and data collection is described in detail in C.5.1.2 and C.5.1.3.
C.4. Study Design
Implementation, especially of complex interventions such as LS screening, is highly context dependent
15,18,74,75
.
Therefore, we propose a multiple-case study design with a mixed-methods approach to analyses followed by a
naturalistic observational evaluation. Study design and analyses are informed by the CFIR. The multiple case-
study design utilizes purposive selection of cases (N=23 clinical sites; Table 2) with known variability in LS
screening implementation, including cases (N=10 sites) without LS screening implementation at present. It is
possible that any of these 10 sites may implement LS screening prior to the beginning of the study, however,
given the significant barriers to implementation in healthcare systems, it is unlikely that all 10 sites would begin
LS screening implementation prior to the beginning of the study. This study is specifically designed to evaluate
cases (sites) based on their implementation status at the beginning of and throughout the study, and will thus
provide information critical to LS screening specifically and implementation science in general.
Data for the multiple-case study design will be gathered via in-depth qualitative interviews with key
stakeholders, including patients and organizational stakeholders (Table 3). Stakeholders will be identified
through purposive and snowball sampling to provide the most in-depth information for describing variation in
practice and factors influencing implementation, evaluation, maintenance, and improvement of LS screening at
each site (Aim 1). The CFIR provides a process for analyzing qualitative data from a multiple case-study
design to look for associations across cases (sites) in order to identify factors associated with where, when,
and under which conditions different processes for implementing or improving LS screening might be
successful (detail in C.7.2.1). Further in-depth Qualitative Comparative Analysis (QCA) will be utilized to
develop a model of conditions necessary and/or sufficient for implementing and improving LS screening under
different organizational conditions (detail in C.7.2.3). The large number of sites (N=23) increases the
opportunity to measure outcomes common to multiple sites and compare with other sites (Aim 2).
Cost-consequence modeling
28,29
and other quantitative analyses will be utilized to address concerns of
organizational decision-makers and the CFIR construct of intervention cost by modeling intermediate
parameters in the LS screening pathway (e.g. results of different assays in a protocol). While Quality Adjusted
Life Years (QALYs) are an accepted measure of cost-effectiveness on a population level, the relevance of this
measure to healthcare decision-making has been questioned
60,61
. Testing costs for each clinical site, as well as
tumor registry data for incident CRC and EC cases will be used to support economic model assumptions and
values used to provide site-specific estimates of costs associated with the different LS screening protocols, the
site-specific incremental costs of detecting LS cases, and the economic impact to the site of imposing of age
cutoffs. Additional costs important to stakeholders that arise from Aim 1 interviews will be included as
appropriate to provide the most locally relevant cost information for organizational decision-makers to compare
implementation options and make informed decisions based on local clinical costs and impact. (Aim 3).
An organizational implementation toolkit will be developed from the data in Aims 1-3 and provided to all
participating sites (Aim 4). We will utilize a naturalistic observational design with qualitative evaluation to
assess utility of the tool for facilitating implementation at sites without LS screening and optimization in sites
with screening. Additional organizational stakeholder interviews will be conducted to determine the utility of the
14
toolkit to facilitate organizational decision-making regarding LS screening implementation and improvement.
C.4.1. External Advisory Panel. A project-specific External Advisory Panel (EAP) has been created. The EAP
has been involved in the development of this proposal and will continue to meet twice yearly via teleconference
with study investigators for the duration of the project to provide guidance on data analysis and reporting, and
assist in dissemination of study findings. This process will guide strategies for broader dissemination of the
organizational toolkit to other healthcare systems. The EAP consists of individuals from the CRN C&D SWG,
the Hereditary Colon Cancer Foundation, The Lynch Syndrome Screening Network (LSSN), and Cancer Care
Ontario (See Letters of support and Budget Justification).
C.5. Procedures
C.5.1. Data collection - Aims 1 and 2:
Organizational and patient stakeholders will be recruited to participate in telephone interviews to provide data
for Aims 1 and 2. Sample size, sampling plan, and other aspects of data collection are detailed in Table 3,
while recruitment and interviewing is described in detail in the sections following. Interviews will be conducted
centrally by experienced staff at either Geisinger or KPNW as detailed in Table 3.
Table 3. Data Collection Plan for Key Stakeholders
Key Stakeholder Type
Sample Size
Sampling Plan
Data Site
Interview Site
Organizational Stakeholders-Aim 1
10 per site
Purposive with snowball sampling
All sites
Geisinger
Newly Diagnosed CRC Patients
10 per site
Prospective
All sites
KPNW
CRC patients with Positive LS Screen
25 total
Retrospective
Sites with LS Screening ONLY
KPNW
Organizational Stakeholder-Aim 4
5 per site
Purposive with snowball sampling
All sites
Geisinger
C.5.1.2. Organizational stakeholder recruitment: Up to 10 organizational stakeholders per site will be
recruited through purposive role-based recruitment and snowball sampling
70,76
. The actual number of and
specific individual key stakeholders invited to be interviewed will depend on each site’s organizational
structure, however, based on previous research
19,22,70,76
it is anticipated that 10
organizational stakeholders per site will provide sufficient information about LS
screening for analysis and that the stakeholder types will be relatively standard
across sites. Standard role-based stakeholders relevant to LS screening include:
pathology, genetic counselors, gastroenterology, gynecology, surgery, and health
plan leaders (Table 4). Additional site-specific role-based stakeholders will be
identified through snowball sampling. Research staff from each site will reach out to
initial stakeholders from their organization via email or other methods, such as
attending department meetings, to alert them to the study and invite them to
participate in a telephone interview. At the end of each completed interview, the
interviewee will be asked to identify any additional organizational stakeholders
necessary for implementing new processes generally and LS screening specifically
at the site. Additional stakeholders will be sent an email indicating that they were
nominated to be invited into the study and offered the opportunity to participate in a
telephone interview.
Table 4. Role-Based Key
Stakeholder Types in LS
Screening
Organizational Stakeholders
Pathology
Genetic counselors
Gastroenterology
Gynecology
Oncology
Surgery
Health Plan leadership
Other key stakeholders
(as identified by each site)
Patient Stakeholders
Newly diagnosed CRC patients
LS screen positive patients
15
C.5.1.3. Patient stakeholder recruitment: Two different groups of patient stakeholders will be invited to
participate in this study: (1) patients newly diagnosed with CRC (N=10 per site) and (2) patients who have
been notified of a positive LS screen result and were recommended for additional genetic counseling and
testing to confirm diagnosis (N=25 total across sites). For the patients with newly diagnosed CRC (group 1),
study staff at each site will determine the best way to identify and contact patients up to one month post-
diagnosis and offer the opportunity to participate in this one-time telephone interview. This group will illuminate
for organizational decision-makers local patient attitudes and opinions about LS screening, while the diversity
of these patients across all sites will provide insight into patient attitudes in general towards LS screening.
Additionally, patients with CRC who have been notified of a positive LS screen result (group 2) will also be
invited to participate in telephone interviews. A total of 25 patients will be recruited only from sites with LS
screening at the start of the study (Table 3) to provide insight into patient experiences with a positive LS screen
across different sites and different LS screening implementation protocols.
C.5.1.4. Qualitative data collection (patients and organizational stakeholders): Semi-structured interviews
will be conducted via telephone centrally by staff experienced in qualitative data collection. Centralized
telephone interviewing and data analysis is an efficient and effective way for qualitative data collection from
multiple stakeholders across multiple sites, and has been used successfully by this project team and
others
75,77-79
. Utilizing the telephone allows interviews to be conducted at a time convenient to the key
stakeholders and centralized processes reduces variability in interviewing. Finally, interviews conducted by
personnel external to the interviewee’s organization may facilitate more candid discussion regarding
organizational facilitators and barriers
80
. The interview guide is described in more detail in section C.6.1.
A summary will be created immediately after each interview and reviewed with site investigators during regular
study meetings. These summaries will be used to iterate the sampling procedure or interview guides, if
necessary, and to create the initial coding schema and analytic framework. Summaries allow for high-level
analysis during on-going data collection, facilitate initial codebook development, and reduce the number of de
novo codes requiring re-review and re-coding of transcripts during data analysis
81,82
.
Interviews with organizational stakeholders will be conducted centrally by staff at Geisinger led by Dr. Rahm.
Patient interviews will be conducted centrally by staff at KPNW led by Dr. Hunter and Ms. Schneider.
Interviewees will receive a $25 gift card upon completion of the interview.
C.5.2. Data collection - Aim 3: Addressing
Aim 3 requires estimates of annual number of
cancer cases, LS prevalence or the
assumption of an equivalent rate for all
populations, and cost of tests included in
screening protocols from each site (collected in
Aim 1) to populate decision analysis models.
Data sources for Aim 3 are detailed in Table 5.
All data will be summarized in aggregate for
each site, creating de-identified data sets. In
most instances, this data is available from
electronic data stores and tumor registry.
Table 5. Data Sources for Aim 3
Input Variables
Definition
Data Source
# CRC cases
per year
incidental CRC cases by year averaged over
a 5 year period stratified by age at diagnosis
Site Electronic
Data
# EC cases
per year
incidental EC cases by year averaged over a
5 year period stratified by age at diagnosis
Site Electronic
Data
Local testing
costs
cost to institution of each test of the site-
specific screening protocol
Billing or
Contracts
Prevalence of LS in
unselected CRC cases
number of LS cases detected through
screening program if available
Site Electronic
Data
Prevalence of LS in
unselected EC cases
number of LS cases detected in screening
protocol (if available)
Site Electronic
Data
ite-specific LS
screening protocol
site-specific LS screening protocol at the
beginning of the study
Site Stakeholders
-Aim 1 Interview
While letters of support detail commitment of clinical sites to obtain institutional cost data, we will use
alternative methods when this data is not available due to proprietary reasons. Alternatives to local test cost
may include using a test cost range based on the other participating clinical sites, or regional test cost figures if
publicly available from testing companies, Medicare reimbursement, or other sources. We also recognize that
reliable estimates of LS prevalence specific to each site may not be available; therefore, this model parameter
may be estimated from sites with such data and/or the most current estimates for U.S. populations
33
.
16
During year 1 initial exploration of site testing costs, cancer cases, and LS cases detected (if available) will be
determined with preliminary data pulls and tested for accuracy. For sites with HCSRN VDW (virtual data
warehouse) capability, we will use the standard distributed code process, where code is written and tested at
one site and distributed to the other sites, where it is used within the new site’s VDW. Because clinical sites,
their LS screening protocols, and scientific evidence are dynamic, we do not expect testing costs or LS
screening guidelines to be static. Therefore, this basic analytic framework will be updated to account for any
evidence that may have emerged, and a final data pull will be conducted and aggregated data per site will be
sent to Geisinger for economic analysis described in C.7.3 just prior to creating the draft organizational toolkit
and distributing to participating sites.
C.5.3. Data collection - Aim 4: Data from Aims 1, 2, and 3 will be used to generate a working organizational
toolkit to guide implementation, maintenance, and improvement of LS screening. Because healthcare systems
are not static and guidelines are changing rapidly, additional observational data will be collected over the entire
project period from monthly project meetings, communications from site investigators, pertinent data regarding
site-specific screening changes, and external evidence or guideline changes for LS screening will be recorded
in a project specific database created for tracking such information related to implementation
83
. Importantly,
this tracking database of all other factors impacting implementation will provide additional information for the
toolkit development should any sites begin to implement LS screening based on being interviewed for Aim 1,
but prior to receiving the toolkit.
In year 5, additional qualitative interviews will be conducted with up to 5 organizational stakeholders at each
site using the same processes described in section C.5.1.2 and analyzed as in C.5.1.4. Stakeholders will be
contacted for interviews 6 months after distribution of the toolkit. Stakeholders from sites without LS screening
and those with sub-optimal implementation will be interviewed about the utility of the tool to facilitate
implementation and improvement. Stakeholders from sites with optimally implemented programs will be
interviewed about the utility of the tool for improvement or adaptation to emerging evidence.
C.6. Measurement
C.6.1. Measurement - Aims 1 and 2: In years 2 and 3,
we will conduct qualitative semi-structured interviews with
patients and organizational stakeholders from each site
to measure current LS screening protocols, attitudes
towards LS screening, and specific implementation
strategies employed (successfully and unsuccessfully). A
draft semi-structured interview guide for organizational
key stakeholders has been developed using the CFIR
question bank
72
and pretested with 5 key stakeholders
from different sites (Results presented in C.1.2). The
patient interview guide will be adapted from a prior study
which utilized similar constructs
71
. Interview guides (See
Appendix for draft interview guides) will be reviewed at
the start of the study with the site investigators and the
project EAP. The CFIR-guided constructs to be assessed
Table 6. CFIR Constructs by Domain Specific to
LS Screening to be Assessed in Stakeholder Interviews
CFIR Domain
CFIR Constructs Specific to LS Screening
Intervention
Characteristics
Adaptability of LS screening to local context
Perceived difficulty implementing LS screening
Cost to the organization associated with screening
Outer Setting
Patient needs and resources
Competitive pressure to implement screening
Impact of external policies on organization
Inner Setting
Organization structure
Perceived organizational priority to implement
Implementation climate in organization
Characteristics
of Individuals
LS knowledge and beliefs, perception of evidence
Individual readiness to implement screening
Self-efficacy to complete actions in screening
Implementation
Process
Planning process to implement LS screening
Champions, opinion leaders, and other stakeholders
Tracking and feedback processes for LS screening
through the patient and organizational stakeholder interview guides are detailed according to CFIR domain in
Table 6. For organizational stakeholders, interview guides will be further tailored to the position of the key
stakeholder as necessary. For example, system leaders may be asked more questions about engagement of
leadership, external influences such as pressure to be like other institutions, and reimbursement incentives.
Tailoring questions to the position of the key stakeholder was found necessary in a similar study of
organizational implementation
76,78
.
17
C.6.2. Measurement - Aim 3: Aim 3 will measure, via simulation modeling, 1) total testing costs and
incremental testing costs by healthcare system for LS screening programs, 2) total costs to screen for site-
specific protocol compared to all other possible protocols, and 3) site-specific costs to screen by age cutoff
categories. The models previously developed by Dr. Williams and others
28,29
will be adjusted as necessary to
appropriately reflect site-specific LS screening protocols as determined from data collected from site
organizational stakeholder interviews in Aim 1.
C.6.3. Measurement - Aim 4: To measure facilitation of implementation and LS screening improvement in the
natural environment after receiving the toolkit, data from sources listed in section C.5.3 will be coded for
information regarding to whom the organizational toolkit was distributed at each site, questions that were asked
by key stakeholders, and whether and how the toolkit was used by organizational decision makers to facilitate
LS screening implementation and/or improvement. The interview guide for the additional post toolkit
organizational stakeholder interviews will be adapted from the Aim 1 interview guide (C.6.1) and adjusted to
gather information on ability of the toolkit to facilitate or improve implementation.
C.7. Analyses by Study Aim
C.7.1. Describe variation in LS screening implementation across multiple healthcare systems (Aim 1)
Qualitative analysis for Aim 1 will be led by Dr. Rahm and other study team members experienced in
qualitative analysis. During the entire analytic process, progress, codebooks, and analytic framework will be
reviewed and cross-checked with site investigators during monthly project meetings, with the C&D SWG
leaders, and with the project EAP as part of their scheduled meetings.
C.7.1.1. Coding to describe LS screening implementation and contextual factor variation. All patient and
organizational stakeholder interviews will be digitally recorded and transcribed verbatim. Transcripts will be
uploaded into Atlas.ti (www.atlas.ti.com
) for qualitative analysis. Interview transcripts will be initially coded
using an a priori codebook developed from the semi-structured interview guides, interview summaries, and
CFIR constructs. This first round of coding will look for description of LS screening, process of implementing
LS screening, champions, and external factors important to the key stakeholders and other constructs
described in Figure 2. Emergent (de novo) codes will be added to any other relevant sections of transcript text
not fitting the a priori codes. This coding is an iterative process that will involve team members independently
coding 2-3 transcripts at a time, then discussing their coding to adjust the codebook and to create a working
analytic framework by grouping codes into categories or themes. This process will continue until the code list is
static, all transcripts are coded, and the analytic framework is finalized. Geisinger team members experienced
in qualitative analysis under direction of Dr. Rahm will analyze the organizational stakeholder interviews while
Dr. Hunter and Ms. Schneider at KPNW will lead the coding and analysis of patient interviews. Both coding
teams will coordinate to create the final analytic framework.
C.7.2. Explain current practice variation and determine factors associated with optimal implementation
of LS screening. (Aim 2)
18
C.7.2.1. Coding for presence
and impact of CFIR
constructs. Transcripts from
stakeholder interviews will be
coded to capture selected CFIR
constructs (Table 6 section
C.6.1 and Figure 2) present and
whether that construct was a
barrier or facilitator of LS
screening implementation,
evaluation, maintenance, or
improvement at each site.
Transcript sections coded for the presence of specific CFIR constructs will be coded for whether the construct
impacted implementation and/or choice of implementation strategy. If the construct was impactful, the study
team will code for direction (positive or negative) and for magnitude of impact (small vs. large). This will allow
analyses of which factors are important to implementation in different organizational contexts and provide initial
information for the QCA. This coding for construct presence and impact will follow protocols detailed in the
CFIR technical assistance website
72
and will be conducted by individuals described in C.7.1.1 and led by Drs.
Cragun and Rahm with input from Dr. Mittman.
C.7.2.2. Framework matrix construction and cross-case analysis. A framework matrix will be created to
summarize the completed coding of all interviews. This matrix will also facilitate comparison of the data across
sites (the cross-case analysis) and will determine set membership for the QCA in Aim 2 (section C.7.2).
Because this summary matrix maintains the link to the original coded data, the matrix and conclusions can be
revised and restructured as needed based on feedback and insight from the larger project team, the individual
site investigators, the CRN C&D SWG leaders, and the project EAP as part of a constant comparative process
to minimize bias in qualitative data coding.
C.7.2.3. Qualitative Comparative Analysis (QCA). Conditions associated with LS screening implementation
and conditions associated with optimal or sub-optimal implementation will be determined using QCA. QCA is a
well-established methodology arising from political science research
21,23,24
. QCA uses set theory to identify
combinations of conditions that are associated with an outcome and is particularly suitable when there is
causal complexity (multiple conditions may lead to the same outcome)
24
, as in LS screening implementation
where a facilitator in the presence of one contextual factor may be a barrier to implementation in another. QCA
is well-suited for case-oriented research and uses Boolean algebra instead of statistical correlation to
determine which combinations of conditions (CFIR constructs) are consistently associated with an outcome
(LS screening implementation)
23
. The QCA process is comprised of multiple steps
21
that can be summarized as
follows: a) code the outcome (Figure 2), b) code the conditions (CFIR constructs; Figure 2) and calibrate if
necessary, c) determine which conditions (CFIR constructs) are necessary and sufficient for the outcome and
d) interpret solutions to create a model. Specific software (http://www.compasss.org/software.htm
) designed for
QCA is used to conduct this analysis.
C.7.2.4. QCA outcome definitions. Two different outcomes analyses will be conducted (Figure 2). Initial
analysis will describe the outcome of no implementation of LS screening vs. any implementation of screening
across sites. Secondary analysis will describe outcomes associated with optimal implementation (Figure 1;
CRC and EC tumor screening with reflex testing) vs. sub-optimal implementation across sites with LS
screening only. Additional analysis of this outcome will also examine conditions reported in previous studies to
be more cost-effective, result in better patient ascertainment and completion of germline genetic testing, and
show effective use of genetic services. Such conditions appear to include multidisciplinary involvement,
effective tracking, and reflex testing for BRAF and PHM
22,28,29,54 68,84-86
.
19
C.7.2.5. QCA analyses. In Aim 1 all CFIR constructs assessed are analyzed to ensure in-depth understanding
of each site. In Aim 2, an iterative process will be used to evaluate and assign values to the CFIR constructs,
which will serve as conditions in QCA
20,21,23
. For conditions (CFIR constructs) exhibited by more than 2 sites,
QCA can be considered. Constructs relevant to each outcome are represented by a natural number (0,1,2, etc)
based on the degree to which the construct falls within a particular set (i.e., whether or not screening is
present, or degree to which screening has been optimized). Necessary and sufficiency analyses will be run
using QCA software to determine conditions that are necessary or sufficient for each outcome. A resulting
“truth table” is created in the sufficiency analysis, which is then analyzed for contradictions. In line with best
QCA practices, the research team will resolve any contradictions by returning to the original data and using the
in-depth knowledge of the cases (sites) from Aim 1 to determine if key conditions may be missing from the
model. Once all contradictions are resolved, QCA software will make multiple comparisons of the data to
create solutions, which will again be evaluated by the research team, with review and cross-checking of
assumptions by the site investigators and the project EAP during scheduled meetings. Dr. Cragun has
extensive experience in this analytic method and will direct these analyses in collaboration with Dr. Rahm. The
resulting solutions will be the basis for the toolkit to help organizational decision makers determine what
implementation strategies are more likely to work given their organizational context.
C.7.3. Determine the relative effectiveness, efficiency, and costs of different LS screening protocols.
(Aim 3)
C.7.3.1. Efficiencies of different LS
implementation strategies. Simulation
modeling will be used to estimate multiple
factors identified as important to stakeholders
during Aim 1 and from previous modeling
conducted for Intermountain Healthcare
28,29
.
Table 7 details the parameters and data
sources included in the models. The models
will be populated with data described in C.5.2.
The model used to address screening protocol
efficiencies will estimate for each site:
sensitivity of the different LS screening
protocols (e.g. with or without reflex testing),
average number of LS cases expected to be
identified, total costs for each screening
protocol for a defined cohort size (e.g. 500
cases per year), cost-per case-screened, cost-
per-LS diagnosis, and incremental cost, case
identification, and detection of an additional LS
case between protocols. All analyses for this
Aim will be conducted under the direction of
Drs. Hao, Snyder, Williams, with input from
other project team members experienced in
economic analyses.
Table 7. Parameters and Data Source for Economic Modeling
Input Variable
Data Definition
Data Level
Data Source
#CRC cases per year
Incidental CRC cases
Local Site
Electronic Data
#EC cases per year
Incidental EC cases
Local Site
Electronic Data
% Appropriate tissue
avialable
Eligible cases with tissue available
for LS screening
Local Site
Pathology
Prevalence of LS
in population
Number of LS cases in population
(actual or estimate)
Local Site
or Literature
Electronic Data
Or Literature
Cost of IHC test
Institutional cost of test as available
Local Site
Billing Data
Sensitivity of IHC screen
From Laboratory or Local site as
available
Local Site
or Literature
Test information
Specificity of IHC screen
from Laboratory or Local site as
available
Local Site
or Literature
Test information
% IHC screens that
are positive
Incidental CRC or EC cases with
positive IHC screening tests
Lab or
Local Site
Electronic Data
% IHC screens with
MLH1 loss
IHC positive screens that
demonstrate loss of MLH1 activity
Lab or
Local Site
Electronic Data
% MLH1 positive Ruled out
by BRAF
MLH1 loss cases due to BRAF
mutation
Lab or
Local Site
Electronic Data
%MLH1 positive ruled out by
PHM
MLH1 loss cases due to PHM
Lab or
Local Site
Electronic Data
Cost of BRAF test
Institutional cost of test as available
Local Site
Billing Data
Cost of PHM test
Institutional cost of test as available
Local Site
Billing Data
% Patients referred
to genetics
Screen positive patients sent to
genetics for follow up
Local Site
determined
by site
% Patients offered
germline testing
Screen positive patients offered
confirmatory sequencing
Local Site
determined
by site
% Patients with
germline testing
Screen positive patients with an
order for germline testing
Local Site
electronic data
Cost of sequencing test
Institutional cost of test as available
Local Site
Billing Data
Sensitivity of sequence test
From sequencing laboratory used
Literature
Literature
Specificity of sequence test
From sequencing laboratory used
Literature
Literature
C.7.3.2. Site-specific age cut-off modeling. Additional modeling of different LS screening age cut-off policies
will also be conducted to estimate their impact on effectiveness, efficiency, and cost to each site using local-
level data whenever practical (see C.5.2 for alternatives). Outcomes that will be simulated in this model
include: total cost to screen age cutoff cohort vs. no age cutoff, LS cases expected in the age cutoff category,
cost-per-LS case detected in each age category, and total number and percent of LS cases missed when age
cutoff is applied
28
. This modeling will provide objective metrics, driven by local data, of the impacts of applying
20
age-cutoffs in LS screening implementation.
All modeling will be conducted using TreeAge (https://www.treeage.com/
) or Microsoft Excel with the @Risk
software add-on for Excel (Palisade) for sensitivity analyses. The purpose of these analyses is to provide
information previously determined to be important to healthcare system stakeholders to inform initial LS
screening implementation decisions or to improve existing LS screening. Acceptable variability associated with
clinical/ business costs will also be illuminated by performing these analyses across multiple sites using local
data. These results will provide site-specific cost information most relevant to organizational decision-makers,
contribute to our overall understanding of variation in LS screening implementation, and highlight acceptable
variation in LS screening related to different organizational costs. This process of reviewing local
implementation costs of a complex intervention to illuminate acceptable variability may also be generalizable to
other precision medicine programs and will contribute to the field of implementation science in general.
C.7.4. Develop and test in a natural environment an organizational toolkit to facilitate LS screening
implementation and improvement. (Aim 4)
C.7.4.1. Toolkit Creation. An organizational toolkit will be created based on the CFIR conceptual framework,
the in-depth knowledge of LS screening programs and contextual factors of healthcare systems from Aim 1,
the cross-site comparison and QCA results from Aim 2, and economic modeling with local costs from Aim 3.
This toolkit will be disseminated to all sites through site PIs and the tracking database will record to whom it is
distributed, questions asked by those receiving the toolkit, and actions taken by the site.
C.7.4.2. Analyses of toolkit utility. Utility will be assessed in year 5 through additional post-toolkit stakeholder
interviews. Interview coding and analyses will utilize the same methods described previously to identify
conditions that changed within organizations to allow LS screening implementation, improvement, or
adaptation of optimally implemented programs. The final organizational toolkit for LS screening
implementation, maintenance, and improvement will be modified based on this information prior to broad
dissemination.
C.8. Project timeline
Table 8. Project Timeline Overview
Year 01
Year 02
Year 03
Year 04
Year 05
General Project Tasks
Qtr 1
Qtr 2
Qtr 3
Qtr 4
Qtr 1
Qtr 2
Qtr 3
Qtr 4
Qtr 1
Qtr 2
Qtr 3
Qtr 4
Qtr 1
Qtr 2
Qtr 3
Qtr 4
Qtr 1
Qtr 2
Qtr 3
Qtr 4
IRB approval
x
x
Data Use Agreements
x
x
Exploratory data collection for Aim 3
x
x
x
Finalize interview guides
x
x
key stakeholder recruitment and interviews* (Aim 1)
x
x
x
x
x
x
x
x
CFIR guided cross-case and QCA (Aim2)
x
x
x
x
x
x
x
x
x
Data pull and economic analysis (Aim 3)
x
x
x
x
x
Collection and Tracking of data on change
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Develop and distribute toolkit (Aim 4)
x
x
x
x
x
Natural Experiement with toolkit (aim 4)
x
x
x
x
x
x
Additional organizational stakeholder interviews
x
x
x
Manuscript development and publication
x
x
x
x
x
External Advisory Panel meetings
x
x
x
x
x
x
x
x
x
x
C.9. Dissemination of study results.
The assembled study team will enable broad dissemination of study results and increase the
significance of this work. Dr. Williams and Dr. Mittman are established national leaders in the area of
translating genomics into clinical practice and will help identify opportunities for broad dissemination of study
results. Through the HCSRN, CRN, and LSSN Dr. Rahm and her collaborators can reach additional healthcare
systems to facilitate LS screening implementation. The organizational toolkit will be posted on the LSSN
21
website, making this model available to healthcare organizations nationally and internationally. Dr. Baxter of
the External Advisory Group will be able to utilize the toolkit to further guide her work implementing LS
screening in Ontario. Furthermore, collaborations have been initiated by Dr. Rahm and other study team
members to disseminate results of this study through other networks, including CSER (Clinical Sequencing
Exploratory Research), eMERGE (Electronic Medical Records and Genomics) and IGNITE (Implementing
Genomics in Practice) networks. All three networks have prioritized LS screening for implementation.
Finally, through the new Precision Medicine Initiative more genomic applications with evidence for improving
population will emerge. Now more than ever, a flexible organizational toolkit to guide efficient and effective
implementation, evaluation, maintenance, and improvement of genomic applications is needed. This research
will create an organizational toolkit that addresses a major unmet need identified by the Blue Ribbon
Panel to achieve the goals of the Cancer Moonshot; thus improving our understanding of clinical
implementation of complex interventions and fulfilling the promise of precision medicine to improve
health and prevent disease.
22
References
1.
Collins FS, Varmus H. A new initiative on precision medicine. The New England journal of medicine. Feb
26 2015;372(9):793-795.
2. Palomaki GE, McClain MR, Melillo S, Hampel HL, Thibodeau SN. EGAPP supplementary evidence review:
DNA testing strategies aimed at reducing morbidity and mortality from Lynch syndrome. Genetics in
medicine : official journal of the American College of Medical Genetics. Jan 2009;11(1):42- 65.
3. Healthy People 2020. 2014;
https://www.healthypeople.gov/2020/topics-
objectives/topic/genomics/objectives. Accessed 1/22/2016.
4. Cancer Moonshot Blue Ribbon Panel Report 2016. National Cancer Institute 10/17/2016 2016.
5. Hampel H, de la Chapelle A. The search for unaffected individuals with Lynch syndrome: do the ends
justify the means? Cancer prevention research (Philadelphia, Pa.). Jan 2011;4(1):1-5.
6. Mvundura M, Grosse SD, Hampel H, Palomaki GE. The cost-effectiveness of genetic testing strategies for
Lynch syndrome among newly diagnosed patients with colorectal cancer. Genetics in medicine : official
journal of the American College of Medical Genetics. Feb 2010;12(2):93-104.
7. Ladabaum U, Wang G, Terdiman J, et al. Strategies to identify the Lynch syndrome among patients with
colorectal cancer: a cost-effectiveness analysis. Ann Intern Med. Jul 19 2011;155(2):69-79.
8. Khoury MJ, Coates RJ, Evans JP. Evidence-based classification of recommendations on use of genomic
tests in clinical practice: dealing with insufficient evidence. Genetics in medicine : official journal of the
American College of Medical Genetics. Nov 2010;12(11):680-683.
9. CDC. Genetic Testing: Genomic Tests and Family Health History by Levels of Evidence. 2105;
http://www.cdc.gov/genomics/gtesting/tier.htm
. Accessed 1/22/2016.
10. Williams MS. Genomic medicine implementation: learning by example. American journal of medical
genetics. Part C, Seminars in medical genetics. Mar 2014;166c(1):8-14.
11. IOM. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National
Academy Press:
http://www.nap.edu/catalog/10027/crossing-the-quality-chasm-a-new-health-system-for-
the. Accessed 1/22/2016.
12. Williams MS. Perspectives on what is needed to implement genomic medicine. Molecular genetics &
genomic medicine. May 2015;3(3):155-159.
13. Wade JE, Ledbetter DH, Williams MS. Implementation of genomic medicine in a health care delivery
system: a value proposition? American journal of medical genetics. Part C, Seminars in medical genetics.
Mar 2014;166c(1):112-116.
14. Snyder SR, Mitropoulou C, Patrinos GP, Williams MS. Economic evaluation of pharmacogenomics: a
value-based approach to pragmatic decision making in the face of complexity. Public Health Genomics.
2014;17(5-6):256-264.
15. Khoury MJ, Coates RJ, Fennell ML, et al. Multilevel research and the challenges of implementing genomic
medicine. Journal of the National Cancer Institute. Monographs. May 2012;2012(44):112-120.
16. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of
health services research findings into practice: a consolidated framework for advancing implementation
science. Implementation science : IS. 2009;4:50.
17. Eccles MP, Armstrong D, Baker R, et al. An implementation research agenda. Implementation science : IS.
2009;4:18.
18. Williams M. Delivery of Personalized Medicine in an Integrated Healthcare System. In: Ginsburg GS;
Willard HF, ed. Genomic and Personalized Medicine. 2nd ed. New York: Elsevier Inc; 2013:340-352.
19. Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the
consolidated framework for implementation research (CFIR). Implementation science : IS. 2013;8:51.
20. Kahwati LC, Lewis MA, Kane H, et al. Best practices in the Veterans Health Administration's MOVE!
Weight management program. American journal of preventive medicine. Nov 2011;41(5):457-464.
21. Cragun D, Pal T, Vadaparampil ST, Baldwin J, Hampel H, DeBate RD. Qualitative Comparative Analysis: A
Hybrid Method for Identifying Factors Associated With Program Effectiveness. Journal of Mixed Methods
23
Research. February 25, 2015 2015.
22. Cragun D, DeBate RD, Vadaparampil ST, Baldwin J, Hampel H, Pal T. Comparing universal Lynch
syndrome tumor-screening programs to evaluate associations between implementation strategies and
patient follow-through. Genetics in medicine : official journal of the American College of Medical Genetics.
Oct 2014;16(10):773-782.
23. Devers KJL, Nicole Cafarella; Burton, Rachel A; Kahwati, Leila; McCall, Nancy; Zuckerman, Stephen.
Using Qualitative Comparative Analysis (QCA) to Study Patient-Centered Medical Homes. 2013.
https://innovation.cms.gov/Files/reports/QCA-Report.pdf
. Accessed 1/22/2016.
24. Berg-Schlosser DDM, Gisele; Rihoux, Benoit; Ragin, Charles C;. Qualitative Comparative Analysis (QCA)
as an Approach In: Ragin BRCC, ed. Configurational Comparative Methods: Qualitative Comparative
Analysis (QCA) and Related Techniques Thousand Oaks, CA: Sage Publications, Inc; 2009.
25. Mittman B. Implementation Science in Health Care. In: Brownson RC CG, Proctor EK, ed. Dissemination
and Implementation Research in Health. New York: Oxford University Press; 2012:400- 418.
26. Schully SD, Khoury MJ. What is translational genomics? An expanded research agenda for improving
individual and population health. Applied & translational genomics. Dec 2014;3(4):82-83.
27. Bellcross CA, Bedrosian SR, Daniels E, et al. Implementing screening for Lynch syndrome among patients
with newly diagnosed colorectal cancer: summary of a public health/clinical collaborative meeting. Genetics
in medicine : official journal of the American College of Medical Genetics. Jan 2012;14(1):152-162.
28. Gudgeon JM, Belnap TW, Williams JL, Williams MS. Impact of Age Cutoffs on a Lynch Syndrome
Screening Program. Journal of Oncology Practice. July 1, 2013 2013;9(4):175-179.
29. Gudgeon JM, Williams JL, Burt RW, Samowitz WS, Snow GL, Williams MS. Lynch syndrome screening
implementation: business analysis by a healthcare system. The American journal of managed care.
2011;17(8):e288-300.
30. Colorectal Cancer Facts & Figures 2014-2016. 2014.
http://www.cancer.org/acs/groups/content/documents/document/acspc-042280.pdf
. Accessed 1/30/2016.
31. Giardiello FM, Allen JI, Axilbund JE, et al. Guidelines on Genetic Evaluation and Management of Lynch
Syndrome: A Consensus Statement by the US Multi-Society Task Force on Colorectal Cancer. Am J
Gastroenterol. 08//print 2014;109(8):1159-1179.
32. Lynch HT, Snyder CL, Shaw TG, Heinen CD, Hitchins MP. Milestones of Lynch syndrome: 1895-2015.
Nature reviews. Cancer. Mar 2015;15(3):181-194.
33. Grosse SD, Palomaki GE, Mvundura M, Hampel H. The cost-effectiveness of routine testing for Lynch
syndrome in newly diagnosed patients with colorectal cancer in the United States: corrected estimates.
Genetics in medicine : official journal of the American College of Medical Genetics. Jun 2015;17(6):510-
511.
34. Cross DS, Rahm AK, Kauffman TL, et al. Underutilization of Lynch syndrome screening in a multisite study
of patients with colorectal cancer. Genetics in medicine : official journal of the American College of Medical
Genetics. 12//print 2013;15(12):933-940.
35. Pasche B, Pennison MJ, DeYoung B. Lynch Syndrome Testing: A Missed Opportunity in the Era of
Precision Medicine. Jama. Jul 5 2016;316(1):38-39.
36. Jarvinen HJ, Aarnio M, Mustonen H, et al. Controlled 15-year trial on screening for colorectal cancer in
families with hereditary nonpolyposis colorectal cancer. Gastroenterology. May 2000;118(5):829-834.
37. Jarvinen HJ, Renkonen-Sinisalo L, Aktan-Collan K, Peltomaki P, Aaltonen LA, Mecklin JP. Ten years after
mutation testing for Lynch syndrome: cancer incidence and outcome in mutation-positive and mutation-
negative family members. Journal of clinical oncology : official journal of the American Society of Clinical
Oncology. Oct 1 2009;27(28):4793-4797.
38. de Jong AE, Hendriks YM, Kleibeuker JH, et al. Decrease in mortality in Lynch syndrome families because
of surveillance. Gastroenterology. Mar 2006;130(3):665-671.
39. Stupart DA, Goldberg PA, Algar U, Ramesar R. Surveillance colonoscopy improves survival in a cohort of
subjects with a single mismatch repair gene mutation. Colorectal disease : the official journal of the
Association of Coloproctology of Great Britain and Ireland. Feb 2009;11(2):126-130.
40. Schmeler KM, Lynch HT, Chen LM, et al. Prophylactic surgery to reduce the risk of gynecologic cancers in
24
the Lynch syndrome. The New England journal of medicine. Jan 19 2006;354(3):261-269.
41. Moreira L, Balaguer F, Lindor N, et al. Identification of Lynch syndrome among patients with colorectal
cancer. Jama. Oct 17 2012;308(15):1555-1565.
42. Recommendations from the EGAPP Working Group: genetic testing strategies in newly diagnosed
individuals with colorectal cancer aimed at reducing morbidity and mortality from Lynch syndrome in
relatives. Genetics in medicine : official journal of the American College of Medical Genetics. Jan
2009;11(1):35-41.
43. Rubenstein JH, Enns R, Heidelbaugh J, Barkun A. American Gastroenterological Association Institute
Guideline on the Diagnosis and Management of Lynch Syndrome. Gastroenterology. Sep 2015;149(3):777-
782; quiz e716-777.
44. Stoffel EM, Mangu PB, Gruber SB, et al. Hereditary colorectal cancer syndromes: American Society of
Clinical Oncology Clinical Practice Guideline endorsement of the familial risk-colorectal cancer: European
Society for Medical Oncology Clinical Practice Guidelines. Journal of clinical oncology : official journal of
the American Society of Clinical Oncology. Jan 10 2015;33(2):209-217.
45. Balmana J, Balaguer F, Cervantes A, Arnold D. Familial risk-colorectal cancer: ESMO Clinical Practice
Guidelines. Annals of oncology : official journal of the European Society for Medical Oncology. Oct 2013;24
Suppl 6:vi73-80.
46. Syngal S, Brand RE, Church JM, Giardiello FM, Hampel HL, Burt RW. ACG clinical guideline: Genetic
testing and management of hereditary gastrointestinal cancer syndromes. Am J Gastroenterol. Feb
2015;110(2):223-262; quiz 263.
47. Weissman SM, Burt R, Church J, et al. Identification of individuals at risk for Lynch syndrome using
targeted evaluations and genetic testing: National Society of Genetic Counselors and the Collaborative
Group of the Americas on Inherited Colorectal Cancer joint practice guideline. Journal of genetic
counseling. Aug 2012;21(4):484-493.
48. NCCN. Clinical Practice Guidelines in Oncology - Genetic/Familial High-Risk Assessment: Colorectal.
2015; Version 2.2015. Available at:
http://www.nccn.org/professionals/physician_gls/pdf/genetics_colon.pdf
. Accessed 1/22/2016.
49. Pai RK, Pai RK. A Practical Approach to the Evaluation of Gastrointestinal Tract Carcinomas for Lynch
Syndrome. The American journal of surgical pathology. Apr 2016;40(4):e17-34.
50. Le DT, Uram JN, Wang H, et al. PD-1 Blockade in Tumors with Mismatch-Repair Deficiency. The New
England journal of medicine. Jun 25 2015;372(26):2509-2520.
51. McCann GA, Eisenhauer EL. Hereditary cancer syndromes with high risk of endometrial and ovarian
cancer: surgical options for personalized care. Journal of surgical oncology. Jan 2015;111(1):118-124.
52. Coates R, Williams M, Melillo S, Gudgeon J. Genetic testing for lynch syndrome in individuals newly
diagnosed with colorectal cancer to reduce morbidity and mortality from colorectal cancer in their relatives.
PLoS currents. 2011;3:Rrn1246.
53. Hampel H. Genetic counseling and cascade genetic testing in Lynch syndrome. Familial cancer. Jul
2016;15(3):423-427.
54. Heald B, Plesec T, Liu X, et al. Implementation of Universal Microsatellite Instability and
Immunohistochemistry Screening for Diagnosing Lynch Syndrome in a Large Academic Medical Center.
Journal of Clinical Oncology. April 1, 2013 2013;31(10):1336-1340.
55. Frolova AI, Babb SA, Zantow E, et al. Impact of an immunohistochemistry-based universal screening
protocol for Lynch syndrome in endometrial cancer on genetic counseling and testing. Gynecol Oncol. Apr
2015;137(1):7-13.
56. Corinne Daly CR, Marcia Facey, Natalie A. Baker, Nancy N. Baxter. Reflex Lynch syndrome screening by
example: A review of existing programs. Journal of clinical oncology : official journal of the American
Society of Clinical Oncology. 2015;33(suppl 3):abstr 543.
57. Cohen SA. Current Lynch syndrome tumor screening practices: a survey of genetic counselors. Journal of
genetic counseling. Feb 2014;23(1):38-47.
58. Beamer LC, Grant ML, Espenschied CR, et al. Reflex immunohistochemistry and microsatellite instability
testing of colorectal tumors for Lynch syndrome among US cancer programs and follow-up of abnormal
25
results. Journal of clinical oncology : official journal of the American Society of Clinical Oncology. Apr 1
2012;30(10):1058-1063.
59. Ellis P, Robinson P, Ciliska D, et al. A systematic review of studies evaluating diffusion and dissemination
of selected cancer control interventions. Health psychology : official journal of the Division of Health
Psychology, American Psychological Association. Sep 2005;24(5):488-500.
60. Sculpher MJ, Pang FS, Manca A, et al. Generalisability in economic evaluation studies in healthcare: a
review and case studies. Health technology assessment (Winchester, England). Dec 2004;8(49):iii-iv, 1-
192.
61. Schlander M. The use of cost-effectiveness by the National Institute for Health and Clinical Excellence
(NICE): no(t yet an) exemplar of a deliberative process. J Med Ethics. Jul 2008;34(7):534-539.
62. The Precision Medicine Initiative. 2015; https://www.whitehouse.gov/precision-medicine
. Accessed
1/30/2016.
63. National Academies of Sciences E, and Medicine. Applying an implementation science approach to
genomic medicine: Workshop summary. Washington, DC: The National Academies Press; 2016.
64. Mills AM, Liou S, Ford JM, Berek JS, Pai RK, Longacre TA. Lynch syndrome screening should be
considered for all patients with newly diagnosed endometrial cancer. The American journal of surgical
pathology. Nov 2014;38(11):1501-1509.
65. Moline J, Mahdi H, Yang B, et al. Implementation of tumor testing for lynch syndrome in endometrial
cancers at a large academic medical center. Gynecologic Oncology. 7// 2013;130(1):121-126.
66. Mange S, Bellcross C, Cragun D, et al. Creation of a network to promote universal screening for Lynch
syndrome: the LynchSyndrome Screening Network. Journal of genetic counseling. Jun 2015;24(3):421-
427.
67. Goodfellow PJ, Billingsley CC, Lankes HA, et al. Combined Microsatellite Instability, MLH1 Methylation
Analysis, and Immunohistochemistry for Lynch Syndrome Screening in Endometrial Cancers From
GOG210: An NRG Oncology and Gynecologic Oncology Group Study. Journal of clinical oncology : official
journal of the American Society of Clinical Oncology. Dec 20 2015;33(36):4301-4308.
68. Rahm AK, Kip NS, Williams JL, et al. Coordinating Laboratory and Clinical Data to Include Endometrial
Tumor Testing in the Universal Lynch Syndrome Screening Program at Geisinger Health System. National
Society of Genetic Counselors Annual Education Conference; 2015; Pittsburgh, PA.
69. Meyer LA, Broaddus RR, Lu KH. Endometrial cancer and Lynch syndrome: clinical and pathologic
considerations. Cancer control : journal of the Moffitt Cancer Center. Jan 2009;16(1):14-22.
70. Schneider JL, Davis J, Kauffman TL, et al. Stakeholder perspectives on implementing a universal Lynch
syndrome screening program: a qualitative study of early barriers and facilitators. Genetics in medicine :
official journal of the American College of Medical Genetics. 04/16/online 2015.
71. Hunter JE, Zepp JM, Gilmore MJ, et al. Universal tumor screening for lynch syndrome: Assessment of the
perspectives of patients with colorectal cancer regarding benefits and barriers. Cancer. 2015:n/a- n/a.
72. CFIR Technical Assistance Website. http://cfirguide.org/
. Accessed 1/30/2016.
73. Schofield L, Grieu F, Goldblatt J, Amanuel B, Iacopetta B. A state-wide population-based program for
detection of lynch syndrome based upon immunohistochemical and molecular testing of colorectal
tumours. Familial cancer. Mar 2012;11(1):1-6.
74. Yano EM, Green LW, Glanz K, et al. Implementation and spread of interventions into the multilevel context
of routine practice and policy: implications for the cancer care continuum. Journal of the National Cancer
Institute. Monographs. May 2012;2012(44):86-99.
75. Hamilton AB, Oishi S, Yano EM, Gammage CE, Marshall NJ, Scheuner MT. Factors influencing
organizational adoption and implementation of clinical genetic services. Genetics in medicine : official
journal of the American College of Medical Genetics. Mar 2014;16(3):238-245.
76. Rahm AK, Boggs JM, Martin C, et al. Facilitators and Barriers to Implementing SBIRT in Primary Care in
Integrated Health Care Settings. Substance abuse : official publication of the Association for Medical
Education and Research in Substance Abuse. Aug 15 2014:0.
77. Boggs JM RA, Martin CR, Beck A, Price DW, Backer TE, Gunter M, Ahmedani BK, Dearing JW. Feasibility
of Implementing Screening Brief Intervention and Referral to Treatment (SBIRT) within Multiple Healthcare
26
Settings. 19th Annual HMO Research Network Conference; 2013; San Francisco, CA
78. Rahm AK BJ, Martin CR, Beck A, Pearson M, Backer TE, Ahmedani BK. Systematic Stakeholder
Assessment to Determine Facilitators of and Barriers to SBIRT Implementation in Multiple Integrated
Health Systems. 20th Annual HMO Research Network Conference; 2014; Pheonix, AZ.
79. Harris JN, Liljestrand P, Alexander GL, et al. Oncologists' attitudes toward KRAS testing: a multisite study.
Cancer medicine. Dec 2013;2(6):881-888.
80. Frazier LM, Miller VA, Horbelt DV, Delmore JE, Miller BE, Paschal AM. Comparison of focus groups on
cancer and employment conducted face to face or by telephone. Qualitative health research. May
2010;20(5):617-627.
81. Beebe J. Rapid Assessment Process: An Introduction. Walnut Creek, CA AltaMira Press 2001.
82. Padgett DK. Qualitative and Mixed Methods in Public Health. Thousand Oaks, CA: Sage; 2012.
83. Rahm A, Hawkins R, Dearing J, et al. Implementing an evidence-based breast cancer support and
communication tool to newly diagnosed patients as standard care in two institutions. Translational
behavioral medicine. 2015/06/01 2015;5(2):198-206.
84. Rahm AK, Kip NS, Williams JL, et al. Implementing Genomic Medicine in an Integrated Healthcare System:
Evaluation and Improvement of a Universal Lynch Syndrome Screening Program. Dissemination and
Implementation Conference; 2015; Washington, DC.
85. Rahm AK, Kip NS, Williams JL, et al. Evaluation and Optimization of a Universal Lynch Syndrome
Screening Program at Geisinger Health System. American Society of Human Genetics Annual Meeting;
2015; Baltimore, MD.
86. Kip NS, Rahm AK, Williams JL, et al. Evaluation to Include Endometrial Carcinoma in Universal Lynch
Syndrome Screening at Geisinger Health System. American Society for Clinical Pathology Annual Meeting
2015; Long Beach, CA.