Clinical Trial Details
— Status: Recruiting
Administrative data
| NCT number |
NCT04698967 |
| Other study ID # |
D3401-W |
| Secondary ID |
|
| Status |
Recruiting |
| Phase |
N/A
|
| First received |
|
| Last updated |
|
| Start date |
April 1, 2021 |
| Est. completion date |
March 31, 2026 |
Study information
| Verified date |
April 2024 |
| Source |
VA Office of Research and Development |
| Contact |
Brian J Stevenson, PhD |
| Phone |
(718) 687-3919 |
| Email |
Brian.Stevenson4[@]va.gov |
| Is FDA regulated |
No |
| Health authority |
|
| Study type |
Interventional
|
Clinical Trial Summary
The Veterans Health Administration (VHA) offers robust vocational programming that have
helped countless Veterans obtain competitive employment; however, these services are not
uniformly effective as recent data suggests that only 35 to 43% of Veterans are competitively
employed at time of discharge. For those who become competitively employed, job tenure may be
brief, which is often attenuated by underemployment or poor person-job fit. Moreover, only
3.5% of Veterans experiencing vocational problems engage in vocational services offered by
the VHA. On average, it takes Veterans more than four years to utilize vocational services.
These Veterans are at high risk of acquiring multiple functional losses and developing
chronic disabilities as their vocational needs go unmet for years.
Research suggests that intrinsic factors like lacking clear vocational goals, perceiving
barriers to employment, and negative beliefs about one's ability to work contribute to low
engagement, outcomes, and tenure of some consumers of vocational rehabilitation services.
Thus, the VA may be able to improve vocational engagement, outcomes, and tenure of Veterans
with psychiatric disorders by enhancing vocational services with added interventions
targeting unhelpful psychological factors. Career counseling and development services have
been shown to be effective in helping civilian populations clarify vocational goals and
identity, enhance vocational self-efficacy, and increase proactive vocational behaviors in
the face of obstacles. Additionally, career counseling and development services help
facilitate greater "match" between a person and their job, and person-job match is a key
determinant of long-term career tenure among individual with psychiatric disorders.
The researchers of this project propose a three-aim study to develop a career counseling and
development intervention for Veterans with psychiatric disorders (Purposeful Pathways). The
first aim will focus on the design and development of the Purposeful Pathways intervention
with veteran and provider input (n=16). The second aim will pilot test the intervention in an
open trial (n=10) to gather Veteran input on the initial intervention. The third and final
aim will consist of a feasibility pilot randomized controlled trial (n=50) to examine
acceptability and feasibility outcomes and to explore the impact of the Purposeful Pathways
intervention in terms of functional improvement and other vocational outcomes. Purposeful
Pathways consists of up to 12 individual sessions that will be offered concurrently with
existing VHA vocational rehabilitation services, (e.g., transitional work experience [TWE]).
The final product of this study is to produce a manual of Purposeful Pathways, and
corresponding fidelity monitoring checklist, to be tested later in a larger efficacy trial.
Description:
AIM 1: Develop a career counseling and development treatment manual for Veterans with
psychiatric disorders (i.e., Purposeful Pathways) along with a fidelity monitoring checklist
based on veteran and provider needs assessment (n=16).
Procedures. Manual development will be an iterative process that spans over Aim 1 and Aim 2
(Open Trial) and will include: 1) reviewing clinical procedures and interventions of the
Vocational Evaluation Center (VEC) for career counseling and development in vocational
rehabilitation, related treatment manuals (Opening Doors), and relevant career development
theory; 2) conducting veteran and provider focus groups to gather perspectives on necessary
treatment components; 3) developing a preliminary version of the manual through a two-day
retreat attended by key CDA-2 mentors and consultants, and 4) revising the manual based on
mentor, provider, and veteran feedback. A fidelity monitoring checklist will also be
developed to monitor treatment adherence of clinicians implementing the Purposeful Pathways
protocol. The fidelity monitoring checklist will be used in Aim 2 and Aim 3 of this study to
monitor treatment adherence of the study clinicians.
Needs Assessment Focus Groups. Two focus groups will be conducted with veterans (n=8), and
two focus groups will be conducted with mental health providers (n=8) with knowledge in
vocational rehabilitation practice (psychologists, social workers, and vocational
rehabilitation specialists). The providers will be recruited from outside the Bedford VA to
capture perspectives from clinicians who are not familiar with the Bedford VA Vocational
Evaluation Center. Half of veteran participants will be recruited from currently enrolled TWE
veterans. The other half of veteran participants will be individuals who previously
participated in TWE and were unsuccessful in obtaining employment. Focus groups will be 90
minutes, and will be audio recorded for future transcription and analysis with NVIVO.
Transcription will be completed by a VA approved contractor.
A qualitative interview guide drawing upon Psychology of Working theory, protocol developed
through the Vocational Evaluation Center (VEC) as well as the Opening Doors manual when
conducting provider and veteran focus groups. Overall, the goal is to gain an unfiltered
perspective on career development and factors related to successful vocational recovery,
while also querying on aspects of existing practices An open-ended interview guide consisting
of open-ended questions will focus on 1) barriers, potential solutions, and perceived needs
to promote successful career outcomes among veterans with psychiatric disorders; 2) obtain
feedback on the useful elements of Psychology of Working theory, as well as the VEC and
Opening Doors protocols as it relates to their clinical experience/expertise; 3) explore
issues related to augmenting VA vocational services with career counseling and development
interventions.
The purpose of the veteran focus groups are to gather personal accounts from people with
lived experience on career development/vocational recovery, and to connect that to the best
available research and clinical practice to inform treatment development. The qualitative
interview guide will consist of open-ended questions to gather the following information: 1)
barriers, potential solutions, and perceived needs to promote successful career outcomes
among veterans with psychiatric disorders; 2) obtain feedback on the useful elements of
Psychology of Working theory, as well as the VEC and Opening Doors protocols as it relates to
their lived experiences; 3) gather information regarding preferred methods of implementation
of adjunctive career counseling and development services.
Sample. The eight provider participants will be recruited through emails to vocational
providers and will be scheduled for one of two one hour focus group of providers only during
regular work hours. These focus groups will be held via teleconference. Half of the eight
veteran participants will be recruited from the Compensated Work Therapy TWE program, while
half will be former TWE recipients who did not secure competitive employment. Veteran
participants will be scheduled for a one of two one hour focus group of veterans only. All
participants will be consented for participating in the interview and having their
participation audio-recorded.
Qualitative Analysis. A Hamilton Rapid Turn-around Analytic Approach will be used. This
technique is suited to short-term projects using targeted interview guides. It improves on
past rapid procedures through its systematic, rigorous approach, yielding results that are
slightly less nuanced but more efficient. The PI will develop an analytic template organized
by the topic areas from the semi-structured interview guiding qualitative interviews. The PI
and two other researchers will apply the template to summarize two interviews to test the
usability and relevance of the template to the transcripts. All three researchers will meet
to establish consistency in how the template is used to summarize the transcripts. From here,
all focus groups will be individually summarized (each focus group will be analyzed by two
researchers to ensure reliability). PI will then transfer all the summary points into a
master matrix (respondent x topic area) as an important step in drawing conclusions and
deriving meaning from the transcripts. PI will then condense the summaries in the master
matrix by identifying and organizing recurring themes. The other two researchers will review
the condensed matrix to the original master matrix to confirm all central themes have been
identified and to ensure consensus.
Two-Day Retreat. Data gathered from the veteran and provider focus groups will be used to
inform the design of the treatment protocol as well as the fidelity monitoring checklist. The
retreat will be coordinated at the Bedford VA, and will be facilitated by Drs. Stevenson and
Mueller and attended by mentors (Drs. Bakken, Blustein, Ellison, and Kelly), as well as
consultants (Drs. Millner, Kenny, Duffy, Allan, and Autin). Drs. Stevenson and Mueller will
facilitate the retreat beginning with an opening agenda and ground rules for facilitating
dialogue and resolving conflicting viewpoints about focus group material, expert knowledge,
and research evidence. Every theme will be presented and reviewed. Drs. Stevenson and Mueller
will facilitate conversation about each theme and whether or not thematic content could be
integrated into the treatment protocol considering the integrity of best available research
and practice. In situations where consensus among content experts cannot be reached, the
majority stance will determine inclusion or exclusion of content into the treatment manual.
Content and Structure of the Proposed Intervention. Foremost, the intervention will largely
be based upon the results of the needs assessment focus groups and qualitative analysis
discussed above. Some of the proposed content is based on Psychology of Working Theory (PWT)
and the corresponding Psychology of Working Counseling (PWC) model which identifies critical
areas of intervention in vocationally-oriented treatment (e.g., assessment of work-related
needs in terms of survival/power, social connection, and self-determination; increasing
skills related to proactivity and adaptability to navigate career development processes;
increasing awareness to environmental or other contextual factors influencing career choice
and development; increasing sense of choice and agency over one's career development; and
improving relational support systems to enhance vocational career development outcomes). In
addition, the preliminary treatment goals and interventions for the Purposeful Pathways
intervention take into account current VEC practices among veterans with psychiatric
disabilities (e.g., exploring the impact of psychiatric and physical disability on one's work
like and career choices) as well as the Opening Doors treatment manual (e.g., a
person-environment fit model of career development that espouses a three step process of
knowing oneself, knowing one's work-related options, and embarking on a career path that fits
with self-knowledge). Anticipated length of treatment is eight weekly sessions which is
informed by the average length of treatment veterans receive through the VEC. Sessions will
be delivered to veterans through individual, face-to-face weekly meetings with a
masters-level (or higher) trained counselor.
Qualitative Inquiry of Initial Treatment Protocol. After developing an initial version of the
Purposeful Pathways manual, qualitative interviews will be conducted among the eight mental
health providers who participated in the first set of focus groups. A copy of the Purposeful
Pathways manual will be given to each of these mental health providers. These reviewers will
be asked to evaluate aims, structure, concerns, and other issues of acceptability of this
intervention. After everyone independently reviews the manual, Dr. Mueller and I will set up
two focus groups, which will include open-ended questions and general reactions to a short
walk-through of the intervention. The purpose of these questions is to assess the perceived
feasibility of the intervention (as discussed by Bowen and colleagues). Specifically, our
interview guide will explore perceived acceptability, demand, implementation, and integration
of Purposeful Pathways. Interviews will be audio recorded and will be transcribed by a VA
approved contractor. These transcriptions will be analyzed using NVIVO. The same qualitative
analytic approach previously discussed (Hamilton Rapid Turn-around) will be utilized for this
qualitative inquiry. Any themes suggesting problems with acceptability, demand,
implementation, and integration will be presented (via video conferencing) to the experts
attending the two day manual development retreat. If consensus cannot be reached among the
mentor and content expert about whether or not to make changes to the manual based on this
qualitative inquiry, final decisions about treatment modifications will be based on the
majority perspective.
AIM 2: Pilot test the manual by treating a small number of Veterans (n=10) in an open trial.
Sample/Recruitment/Screening. The sample will consist of 10, newly enrolling, Veterans in
TWE. All Veterans who enter TWE will be informed about this voluntary research study. The
study candidates will be invited to meet personally with a research assistant or the PI who
will conduct the screening, thoroughly explain all procedures, and answer any questions
before obtaining written informed consent on the consent form approved by Bedford VAMC
Institutional Review Board (IRB). Any participant who is found to meet any exclusion criteria
will not be eligible to participate in the study. The inclusion or exclusion of subjects will
not be based on ethnic or racial characteristics. After screening and informed consent
procedures are completed, the PI will begin meeting with the subject for weekly appointments
following the Purposeful Pathways manual. Veterans participating in this phase of research
will complete several surveys and questionnaires throughout the treatment.
It is estimated that two Veterans per month will be recruited (total of 5 months to reach
target sample of 10 subjects). During FY18, the Bedford VA enrolled 148 Veterans through TWE
and 97.7% of these Veterans had a psychiatric diagnosis. Moreover, the average length of stay
of Veterans in CWT programs was 419.6 days. Thus, recruitment of 10 subjects to participate
in an eight-week treatment program, will be an achievable goal.
Assessment Measures and Timeline. A variety of reliable and valid measures will be used in
this phase to assess demographics, career development constructs, vocational behavior,
employment functioning, working alliance, and client satisfaction.
Qualitative Exit Interviews. At termination of treatment, participants will participate in a
semi-structured, 30-minute exit interview with a research assistant who was not involved in
the delivery of treatment. Questions for these qualitative exit interviews were derived from
the items on the Client Satisfaction Questionnaire-8 (CSQ-8) - which will be completed by
research participants at treatment termination - in order to provide more detailed
information about the nature of participants experiences impacting satisfaction with
treatment. The CSQ-8 measures satisfaction, perceived quality, and effectiveness of
treatment. The following content areas will be assessed through these interviews
(corresponding items from the CSQ-8 are listed but the items are not reproduced due to
copyright): (1) general thoughts/experiences with treatment (CSQ-8 Question #1, 2, 3, 5); (2)
perceived benefits of participating in treatment (CSQ-8 Questions #2, 3, 4, 6, 8); (3)
specific changes and outcomes obtained because of participating in treatment (CSQ-8 Questions
#6, 8); (4) perceived limitations of the treatment (CSQ-8 Questions #1, 4, 7, 8); (5)
recommendations for enhancement/improvement of services (CSQ-8 Questions #1, 2, 5, 7, 8). We
will also ask open-ended questions about usability of measures/assessments. Exit interviews
will be audio recorded, for transcription and analysis.
Data Management. Subject files will be kept in a locked cabinet with a locked office at the
Bedford VA. Subjects will be given an ID number unrelated to any identifying information. An
electronic, password protected file linking participants to their ID number will be kept on a
VA computer on a secure, VA approved server. Data gathered through quantitative measures will
be will be stored on a VA computer on a secure, VA approved server. All audiotaped interviews
will be professionally transcribed by a VA approved contractor (see Budget Justification).
These transcriptions will be stored on a VA computer on a secure, VA approved server.
Data Analysis. The same qualitative analysis plan described in Aim 1 (the Hamilton Rapid
Turn-around Analytic Approach) will be used for this aim. NVIVO will be used for analysis.
Quantitative data gathered throughout the open trial will be used to triangulate findings
from the qualitative exit interviews. A weaving approach will be used to integrate
qualitative and quantitative findings through narratives on a theme-by-theme, or
concept-by-concept basis. In this case, the themes/concepts will be based on the CSQ-8 (e.g.,
satisfaction with treatment, perceived quality of treatment and effectiveness of the
treatment). We will assess for fit between the data looking for confirmation of results and
possible expansion of findings. Patterns of concern, challenges, dissatisfaction, or
improvements found in integrated data will be reviewed by the PI and mentors, possibly
resulting in treatment modification prior to pilot RCT in Aim 3. Discordance in qualitative
and quantitative results will also be reviewed by the PI and the primary mentor team,
possibly resulting in modification to the assessment of client satisfaction (i.e., CSQ-8 or
Qualitative Exit Interview questions) prior to pilot RCT in Aim 3. Whether or not to modify
treatment will be based on fidelity to best available research and feasibility of
implementing recommended changes. If consensus cannot be obtained among the mentor team, then
the majority perspective will be used to make final decisions. Patterns in incomplete or
missing data, reported fatigues, or issues with the measurement paradigm in this Aim of the
study will be addressed before implementation during pilot RCT in Aim 3.
AIM 3: Conduct a randomized controlled pilot study comparing Purposeful Pathways (n=25) to
treatment as usual (n=25).
Sample/Recruitment/Screening. The sample will consist of 50, newly enrolling, Veterans in
TWE. Only veterans entering TWE will be recruited for this study, currently enrolled TWE or
SE veterans will not be eligible. Recruitment, screening, and inclusionary/exclusionary
criteria are the same as Aim 2
Study Procedures. Veterans who have completed screening and informed consent procedures will
be randomized to one of two study arms: Purposeful Pathways (intervention) or treatment as
usual (TAU; control). To ensure equal distribution across the intervention and control
conditions, a permutated block randomization method will be utilized. Randomization will use
computer-generated numbered lists; one for each condition (treatment and control). The lists
will be generated through a block design where the block sizes are randomly selected among
block sizes of either 4 or 6. There will be five rounds of each of the block sizes to
randomize the 50 veterans into either the treatment or control condition. With this approach,
every study arm assignment on the randomization list will consist of an even number of
treatment or control assignments in random order. This approach has been discussed by
previous researchers and not only guarantees balance in the two conditions, but also ensures
that the investigator is not able to predict the treatment sequence as they will not know if
the block size is 4 or 6. This procedure will result in 25 participants per condition which
is in line with previous recommendations of a minimum of 15 subjects per condition. Subjects
randomized to the treatment group will receive the full Purposeful Pathways treatment
protocol, in addition to TWE (i.e., treatment as usual). Subjects randomized to the control
group will receive treatment as usual (i.e., TWE). Subjects will be asked to attend a brief
meeting with a researcher at a three-month follow-up. During this follow up meeting,
participants will complete a variety of self-report questionnaires.
Control Condition. Treatment as usual will serve as the control group for this aim of the
study. Treatment as usual will consist of participation in Transitional Work Experience
(TWE). Upon admission, veterans in TWE are quickly placed into a structured work setting.
These time-limited work settings may be within the community, many are within the VA. CWT
pays veterans through funds received by contracts from companies who contact TWE labor. There
is minimal formal assessment or other vocational services prior to a veteran being placed
into a work site. Work sites vary by TWE program. Common TWE placements at the VA include
housekeeping, laundry, grounds maintenance, and food services. TWE program staff, which can
range from peer support staff to masters trained vocational counselors, provide some on-site
and off-site job coaching to help veterans maintain their TWE placement, in addition to some
guidance on competitive job searching. TWE program staff tend to have (1.5 to 2 times larger
caseloads that counselor in supported employment; approximately 25-40 veterans).
Clinician Recruitment. The PI will recruit and train at least four master's-level trained
clinicians, in the delivery of the CCD-V treatment protocol (two clinicians per 15 months).
All clinicians will be employed at the Bedford VA.
Treatment Adherence. There will be multiple fidelity checks to ensure treatment adherence
among study clinicians. First, all Purposeful Pathways clinicians will participate in a
didactic training introducing clinicians to the treatment protocol. Training of study
clinicians is modeled after the VHA evidence-based practice (EBP) national training program.
The national EBP training programs consist of an initial, three-day intensive didactic
training. For this proposed study, Dr. Stevenson, as well as Drs. Mueller and Blustein, will
lead a 9-hour workshop held over the course of three days (3 hours/day) for an initial
didactic overview of the Purposeful Pathways protocol. Secondly, Purposeful Pathways
clinicians will be required to attend weekly group consultation with the Dr. Stevenson (1
hour/week) for the first three months of providing treatment to address technical issues in
the provision of the treatment. Thirdly, all Purposeful Pathways sessions will be video
recorded, and these sessions will be reviewed and rated (on a variable time schedule) for
treatment adherence by two staff researchers using the fidelity checklist developed in Phase
One. Other researchers have noted that using a variable schedule to monitor treatment
adherence among study clinicians can be an effective methodological approach to offset the
time and resources needed to rate every-single session (in this proposed study there will be
25 Veterans assigned to the treatment condition, which consists of eight sessions, totaling
200 sessions). Once a session is rated, researchers will provide in-person, individual
feedback to clinicians.
Assessment Measures and Timeline. The same measurement paradigm described for Aim 2
(screening, first session, last session) will be used for Aim 3 with the addition of a
three-month follow up. At the three-month follow up Veterans will complete the same battery
of measures completed at their final session. Qualitative interviews will be conducted and
will be conducted as described in Aim 2, with some additional (new) questions regarding
acceptability of randomization.
Data Management. The same data management system described for Aim 2 will be used for Aim 3.
Data Analysis. Consistent with the recommended stage model for development of behavioral
therapies, the aim of this Stage I Pilot RCT is to inform future treatment development, and
research and evaluation plans. To have .80 power to detect differences at a two-tailed alpha
level of .05, a medium effect would require N=125. This is beyond the scope of a CDA-2 award
and is more appropriate for a Stage II trial. A total sample size of 50 is feasible and
consistent with recommendations of 15 to 30 subjects per condition for stage I behavioral
treatment development. Outcomes gathered from this study will provide feasibility and
acceptability data, as well as preliminary efficacy.
Primary analyses will involve examination of the feasibility and acceptability of a RCT of
Purposeful Pathways. Feasibility will be verified by adequate 1) recruitment rates (i.e., 2-3
Veterans per month), 2) >90% fidelity of clinician adherence to the treatment protocol, and
3) data retention (i.e., 70% of participants who attend at least 6 sessions of Purposeful
Pathways. An acceptable level of attrition will be 30% in each condition. We will also
examine the willingness of Veterans to be randomized with a goal of 80% of those approached
who agree to randomization, the number of referrals from clinicians, ease of recruitment
based on the average number of contacts needed to get consent and to schedule visits, the
number of eligible participants from the pool of potential participants, and adherence and
compliance to the treatment protocol. We will compare Veteran acceptability between groups by
conducting an independent samples t-test on the CSQ-8. We will compare attrition rates (<15%
vs. >15%) between the two groups using a chi square and logistic regression analysis. We will
also obtain qualitative data about the feasibility and acceptability of the Purposeful
Pathways treatment and randomization procedures to confirm and expand upon quantitative data.
The Hamilton Rapid Turn-around analytic strategy (previously discussed in Aim 1 and 2) will
be utilized for Aim 3 qualitative analysis.
While efficacy is not our major analytic aim, given the preliminary nature of this study, we
will examine differences in measures of vocational identity, vocational functioning,
psychiatric functioning, work importance and confidence, career adaptability, job search
behavior, decent work, and job need satisfaction to identify empirical targets for further
refinement of the treatment and inform the design of the future Merit Review. Based on the
intent-to-treat principle, all participants randomized to treatment will be included in
analyses. We will run descriptive statistics as well as internal reliability (Cronbach's
alpha) for all measures, Pearson's r to look at relationships between variables, and Cohen's
D for any continuous measures and Cramer's v for any categorical measures to look at effect
size. We will also look at individual trajectories of response to gain better insight into
which participants do particularly well (or poorly) in the two interventions. Effect sizes
and 95% confidence intervals will be used as only one element to estimate power and sample
size to help plan for a Merit Review study.
