Service-user Experiences of Text-based Conversations With a Computer About Their Difficulties

Psychological Clinical Trial

Official title:

Understanding What is Helpful and Unhelpful in Intervention With Online Relational Agent MYLO: An Intervention Process Study.

Clinical Trial Details — Status: Completed

Administrative data

• NCT number: NCT03524196
• Other study ID #: 238389
• Status: Completed
• Phase: N/A
• Start date: November 19, 2018
• Est. completion date: July 11, 2019

Study information

• Verified date: September 2019
• Source: University of Manchester
• Contact: n/a
• Is FDA regulated: No
• Study type: Interventional

Clinical Trial Summary

Mental health problems such as anxiety and depression are common. Services are finding it difficult to treat people quickly. This has led to computers being used to treat common problems.

Research has shown that around half of people who have mental health problems have more than one difficulty (e.g.low mood and worry). Many of the computer interventions available are not designed to work with more than one problem at a time. This can make them less helpful and can lead to people stopping intervention early.

A talking intervention called the Method of Levels is suitable for people with more than one problem. A new computer programme that aims to emulate this type of intervention is called Manage Your Life Online (MYLO). MYLO is accessed online. Users have conversations with MYLO by typing on a computer keyboard. MYLO aims to help people to talk freely and experience feelings related to a problem. In this way, people can find new ways of thinking about their problem. MYLO can be used as often as the person wants.

MYLO has been tested twice before with student volunteers. Participants said that it was helpful in solving an everyday problem. The investigators are not aware of any research that has tested a programme like MYLO for people experiencing mental health problems.

24 adult service users will be recruited from Self Help Services, a participating organisation in Manchester contracted to provide primary care mental health provision for NHS patients. Participants will use MYLO for a two week period.

The investigators aim to conduct a detailed examination of what questions service users find helpful and unhelpful in intervention with MYLO and whether service user perceptions are related to key mechanisms of psychological change identified in MOL. The information gathered from this study will help to increase understanding of how this type of intervention works and what makes it most helpful to service users. These findings will inform improvements to the MYLO programme to improve its acceptability to service users.

Description:

BACKGROUND

Mental health problems such as anxiety and depression are prevalent and related to significant personal and economic burden. Discrepancy between supply and demand of effective psychological therapies has led to the proliferation of computerised therapies for psychological intervention. Computerised cognitive behaviour therapy (cCBT) packages such as 'Fearfighter' and 'Beating the Blues' are recommended by NICE for the treatment of depression and phobias. However, studies have demonstrated high levels of attrition and although active telephone support seems to facilitate their effectiveness this has led to concerns about their efficiency and cost effectiveness.

Most available cCBT interventions are disorder specific and may not provide a good fit for the estimated 50% of service users who experience comorbidities. Transdiagnostic psychological interventions delivered in traditional and computerised formats have been shown to provide equivalent effects to disorder specific interventions of their type and have the potential to improve acceptability and efficiency.

A transdiagnostic form of cognitive therapy called the Method of Levels (MOL) has been used in the treatment of a wide range of psychological problems and is suitable for service users experiencing more than one problem and problems that do not fit into pre-defined diagnostic categories. MOL is differentiated from other cognitive therapies as it originates from a unifying theoretical approach known as Perceptual Control Theory (PCT). PCT posits that psychological distress is due to sustained conflict in internal control systems that are essential to all living things. Therefore, the aim of MOL is to increase a client's sense of control through enabling them to talk freely whilst supporting experiences of emotional distress necessary to increase awareness of internal conflict.

The acceptability of computerised interventions to service users is paramount if increased uptake, adherence and 'reach' are to be achieved whilst improving efficiency. Computerised intervention that offers greater collaboration has the potential to increase its acceptability. The investigators propose that one way of achieving greater collaboration in computerised interventions is through relational agents. Relational agents are computer programmes that simulate conversation. A relational online agent called MYLO (Manage Your Life Online) has been developed at the University of Manchester and uses artificial intelligence to simulate a MOL style conversation with a user via typed text. MYLO has been evaluated in two feasibility trials using student samples and has shown promising results.

The development of relational agents for use in mental health care is progressing, however, academic research has struggled to keep pace with developments in industry. The investigators are not aware of any research that has developed or investigated a transdiagnostic relational agent in psychological intervention in a mental health setting.

The evaluation of computerised interventions through standardised criteria has largely neglected the exploration of client centred outcome measures and the processes of psychological change. A novel methodology that is client orientated and theory led was tested in a feasibility study which examined service user perceptions of helpfulness in a single session of therapy. The current study will replicate this design and aims to examine service users' experiences with MYLO.

The investigators aim to conduct a detailed examination of what questions participants find helpful and unhelpful in intervention with MYLO and whether client perceptions are related to key mechanisms of psychological change identified in MOL. The information gathered from this study will help to inform understanding of how this type of intervention works and what makes it most helpful to service users. These findings will inform improvements to the MYLO programme to improve its acceptability to service users.

AIMS

The aim of the study is to conduct a process level examination of what questions clients find helpful and unhelpful in intervention with MYLO and whether client perceptions are related to key mechanisms of psychological change identified in MOL.

RESEARCH OBJECTIVES & HYPOTHESES

The primary objective is to conduct a process level examination of what questions clients find helpful and unhelpful in intervention with MYLO and whether client perceptions of helpfulness are related to key mechanisms of psychological change identified in MOL. To investigate this objective, Hypothesis 1 will be tested:

H1) Four key mechanisms of psychological change (ratings of perceived control, the ability to talk freely, maintain a focus on emotion and see their problem in a new way) will be positively correlated with ratings of the helpfulness of MYLO's questions.

Qualitative data on why participants perceived certain questions to be helpful or unhelpful and what was happening in those moments will also be obtained to enhance the quantitative ratings further.

Secondary research aims are:

To compare Reorganisation of Conflict scale scores pre-and post-intervention and to estimate the size of the effect of the intervention.

To examine how effective MYLO is in reducing psychological distress and resolving service user defined problems hypothesis 2 will be tested:

H2) Psychological distress (as measured by total scores on the GAD-7 & PHQ-9) and the effect of problems (as measured by the PSYCHLOPS Q1b and 2b) will significantly reduce between Time 1 and Time 2.

To explore service user experiences of the MYLO interface qualitative feedback will be obtained and analysed using content analysis.

METHOD

Study design

The study will utilise a multi-methods case series design repeated over several cases (n=24). This design of study allows for examination of the process of psychological change both within and between individuals. Participants will complete questionnaire measures pre-and post-intervention with MYLO. A multi-methods design allows for the examination of theory driven hypotheses about key mechanisms of psychological change with MYLO.

Participants

Participants will be adults accessing a primary care mental health service (Self-Help Services). Participants will be experiencing psychological difficulties and a computerised intervention will have been identified as a potential intervention option. Practitioners at Self-Help Services will establish whether a service user meets study inclusion and exclusion criteria.

Sample

Based on previous research conducted by the supervisors and a power calculation detailed below, it is estimated that 24 participants will be required. Manchester Self-Help Services received approximately 10,000 referrals in 2016 with approximately 3,000 (30%) of these people opting to access a computerised intervention. Therefore, it is anticipated that between 80 and 160 service users will be screened (based on a conservative 50% uptake rate) to recruit 24 participants.

Power calculation

Taking into account the repeated measures design, to estimate a correlation coefficient of 0.5 (a medium effect size) between the explanatory variables and perceived helpfulness of MYLO's questions with 80% power at a significance level of 0.05, a total of 21 participants will be required to complete Time 2 measures. Assuming a 10% attrition prior to Time 2, 24 participants will be recruited.

Recruitment

The investigators aim to recruit from Self-Help Services, an organisation in Manchester contracted to provide primary care mental health provision for NHS patients. All service users eligible to take part will receive information about MYLO and the study from a Self-Help Services practitioner providing an initial appointment. If a service user is interested in taking part (they consider the modality and style of MYLO as being potentially suited to their needs) they will be given an information flyer and Patient Information Sheet (PIS) to keep and asked to contact the investigator or to consent to pass their contact details to the investigator.

Eligible and interested service users will then contact or be contacted by the investigator. Interest will be confirmed and a convenient time and place to meet to gain informed consent, collect Time 1 measures and provide access to the MYLO programme will be arranged.

Participants can withdraw consent at any time without giving any reason, as participation in the research is voluntary, without their care or legal rights being affected.

Intervention

Manage Your Life Online (MYLO). MYLO is accessed online using a username and password. MYLO operates by analysing the client's input of text for key terms and themes. It responds with questions about the problem aimed at encouraging higher level awareness.

Participants will be provided with access to the MYLO programme through a unique username and password. The investigator will ensure that participants are able to access the programme and answer any questions. Based on feedback from a Community Liaison Group (CLG), the investigator will also provide a short demonstration of how to access the programme and provide all participants with printed instructions on how to access the programme as a reminder.

Participants will decide how often to use the MYLO programme over a two week period. This is likely to be a reasonable length of time to allow at least one use of the programme with no upper limit on usage. Service user led appointment scheduling provides clients the autonomy to decide when and how often to access interventions which increases efficiency whilst maintaining efficacy.

DATA ANALYSIS

Statistical analyses will be performed in Stata version 14. Age, gender and MYLO engagement data will be reported descriptively.

To address research question one, separate multi-level mixed effects linear regression analyses will be conducted to test H1, with helpfulness question rating as the dependent variable. This will be followed by a comparison of the unique contributions made by each predictor variable in the helpfulness rating using a mixed-effects regression analysis.

To address H2, a paired samples t-test will be conducted to examine whether scores on the RoC scale have significantly increased between Time 1 and Time 2.

A qualitative interview will examine why participants thought questions were particularly helpful or unhelpful and serve to enhance the quantitative findings.

To address research question two, H3 will be tested. Paired samples t-tests will be conducted to examine whether there is a significant difference in scores on the PHQ-9, GAD-7 and PSYCHLOPS items 1b and 2b scores between Time 1 and Time 2. In addition, reliable change on the PHQ-9 and GAD-7 will be calculated using the Reliable Change Index and used to identify whether any change is clinically significant.

To address research question three, brief qualitative interviews will be analysed using content analysis. This method of analysis allows codes to come from the data and be organised into categories. This method of analysis will facilitate clear recommendations for the enhancement or modification of the MYLO interface.

DATA PROTECTION & GOVERNANCE

University policies and procedures regarding information governance will be adhered to. Hard copy data will be stored in a locked, fire proof filing cabinet at the School of Psychological Sciences, University of Manchester accessible only to members of the research team.

Electronic data will be transported, stored and analysed in accordance with university data protection policies. Electronic data will be stored on a secure storage area on a university provided research drive accessible only to members of the direct study team. Data files will be encrypted with a password using Microsoft Office and data folders will be encrypted using 7-Zip according to University of Manchester information storage policy.

University policy and procedures relating to audio recording will be adhered to. Audio recordings will be made using an audio recorder. The recorder will be locked away securely when not being used and will not be left unattended. Audio files will be transferred as soon as possible to a secure computer storage drive accessible only to members of the research team. Transcription will be conducted by a member of the research team in a secure environment and the identity of the service user will be anonymised.

The MYLO programme will be hosted on a Virtual Machine (VM) set up and maintained by IT services at the University of Manchester. The VM will be connected to university Lightweight Directory Access Protocol (LDAP) so only permitted University of Manchester registered users will be able to access the VM. All University of Manchester employees have to login to university services using GlobalProtect.

The web address for MYLO will only be provided to study participants. Users accessing MYLO will be unable to upload files to MYLO or change any of the interface. User passwords for MYLO are encrypted and usernames and passwords are to be held on a password protected database which is accessible only to the research team and stored on the secure shared drive.

MYLO conversation logs are not encrypted and are stored in text file format in a system folder within the MYLO database. Conversation files will be transferred from the MYLO database by the researcher as soon as practicable following conversation completion to a secure storage area on the university provided research drive. Data folders will be encrypted using 7-ZIP with password protection. Passwords will be strong i.e.: minimum length of 10 characters, case sensitive, include a mixture of alphabetic (a-Z, upper and lower case), numeric (0-9) and special characters.

All data obtained will be securely stored for 5 years in accordance with University policy.

The study will be subject to the audit and monitoring regime of the University of Manchester.

Recruitment information / eligibility

• Status: Completed
• Enrollment: 17
• Est. completion date: July 11, 2019
• Est. primary completion date: May 1, 2019
• Accepts healthy volunteers: No
• Gender: All
• Age group: 16 Years and older

Eligibility

Inclusion Criteria:

• Aged 16+

• Able to converse, read and write in English proficiently

• Service user considers the MYLO programme to offer a potentially helpful format of intervention

• Service user has access to a device connected to the internet

Exclusion Criteria:

• A higher intensity (step 3) psychological intervention has been recommended

• Service user has current suicidal intent or persistent self-injury

• Service user is currently psychotic

• Service user has current substance dependence identified as a primary problem

• There is a known neurological or organic basis for their presentation (e.g. dementia)

• Service user has a moderate to severe learning disability which would impair their ability to use the computer programme and take part

• Service user has a sight difficulty which would impair their ability to use the computer programme and take part

Related Conditions & MeSH terms

• Psychological

Intervention

Other:
• Manage Your Life Online (MYLO)
Manage Your Life Online (MYLO) is accessed online using a username and password. Client's type into the MYLO conversation box about a problem they are currently experiencing. MYLO operates by analysing the client's input of text for key terms and themes. It responds with questions about the problem aimed at encouraging higher level awareness. Participants will decide how often to use the MYLO programme over a two week period. This is likely to be a reasonable length of time to allow at least one use of the programme with no upper limit on usage.

Locations

Country	Name	City	State
United Kingdom	University of Manchester	Manchester

Sponsors (1)

Lead Sponsor	Collaborator
University of Manchester

Country where clinical trial is conducted

United Kingdom, 

References & Publications (14)

Barlow DH, Farchione TJ, Bullis JR, Gallagher MW, Murray-Latin H, Sauer-Zavala S, Bentley KH, Thompson-Hollands J, Conklin LR, Boswell JF, Ametaj A, Carl JR, Boettcher HT, Cassiello-Robbins C. The Unified Protocol for Transdiagnostic Treatment of Emotional Disorders Compared With Diagnosis-Specific Protocols for Anxiety Disorders: A Randomized Clinical Trial. JAMA Psychiatry. 2017 Sep 1;74(9):875-884. doi: 10.1001/jamapsychiatry.2017.2164. — View Citation

Bennion MR, Hardy G, Moore RK, Millings A. E-therapies in England for stress, anxiety or depression: what is being used in the NHS? A survey of mental health services. BMJ Open. 2017 Jan 23;7(1):e014844. doi: 10.1136/bmjopen-2016-014844. — View Citation

Bohannon J. Artificial intelligence. The synthetic therapist. Science. 2015 Jul 17;349(6245):250-1. doi: 10.1126/science.349.6245.250. — View Citation

Carey, T. (2006). The method of levels : how to do psychotherapy without getting in the way. Living Control Systems Pub.

Carey, T. a., & Mullan, R. J. (2008). Evaluating the method of levels. Counselling Psychology Quarterly, 21(October 2014), 247-256. https://doi.org/10.1080/09515070802396012

Cocklin AA, Mansell W, Emsley R, McEvoy P, Preston C, Comiskey J, Tai S. Client Perceptions of Helpfulness in Therapy: a Novel Video-Rating Methodology for Examining Process Variables at Brief Intervals During a Single Session. Behav Cogn Psychother. 2017 Nov;45(6):647-660. doi: 10.1017/S1352465817000273. Epub 2017 May 22. — View Citation

Duarte A, Walker S, Littlewood E, Brabyn S, Hewitt C, Gilbody S, Palmer S. Cost-effectiveness of computerized cognitive-behavioural therapy for the treatment of depression in primary care: findings from the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Psychol Med. 2017 Jul;47(10):1825-1835. doi: 10.1017/S0033291717000289. Epub 2017 Feb 23. — View Citation

Gaffney H, Mansell W, Edwards R, Wright J. Manage Your Life Online (MYLO): a pilot trial of a conversational computer-based intervention for problem solving in a student sample. Behav Cogn Psychother. 2014 Nov;42(6):731-46. doi: 10.1017/S135246581300060X. Epub 2013 Jul 30. — View Citation

Gilbody S, Brabyn S, Lovell K, Kessler D, Devlin T, Smith L, Araya R, Barkham M, Bower P, Cooper C, Knowles S, Littlewood E, Richards DA, Tallon D, White D, Worthy G; REEACT collaborative. Telephone-supported computerised cognitive-behavioural therapy: REEACT-2 large-scale pragmatic randomised controlled trial. Br J Psychiatry. 2017 May;210(5):362-367. doi: 10.1192/bjp.bp.116.192435. Epub 2017 Mar 2. — View Citation

Kessler RC, Aguilar-Gaxiola S, Alonso J, Chatterji S, Lee S, Ormel J, Ustün TB, Wang PS. The global burden of mental disorders: an update from the WHO World Mental Health (WMH) surveys. Epidemiol Psichiatr Soc. 2009 Jan-Mar;18(1):23-33. — View Citation

Knowles SE, Toms G, Sanders C, Bee P, Lovell K, Rennick-Egglestone S, Coyle D, Kennedy CM, Littlewood E, Kessler D, Gilbody S, Bower P. Qualitative meta-synthesis of user experience of computerised therapy for depression and anxiety. PLoS One. 2014 Jan 17;9(1):e84323. doi: 10.1371/journal.pone.0084323. eCollection 2014. Review. — View Citation

Lamers F, van Oppen P, Comijs HC, Smit JH, Spinhoven P, van Balkom AJ, Nolen WA, Zitman FG, Beekman AT, Penninx BW. Comorbidity patterns of anxiety and depressive disorders in a large cohort study: the Netherlands Study of Depression and Anxiety (NESDA). J Clin Psychiatry. 2011 Mar;72(3):341-8. doi: 10.4088/JCP.10m06176blu. Epub 2011 Jan 25. — View Citation

Newby JM, Mewton L, Andrews G. Transdiagnostic versus disorder-specific internet-delivered cognitive behaviour therapy for anxiety and depression in primary care. J Anxiety Disord. 2017 Mar;46:25-34. doi: 10.1016/j.janxdis.2016.06.002. Epub 2016 Jun 23. — View Citation

Powers, W. (1973). Behavior: The control of perception. Chicago: Aldine publishing co. Retrieved from http://doi.wiley.com/10.1002/bs.3830190609

* Note: There are 14 references in all — Click here to view all references

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	MOL helpfulness ratings questionnaire.	Participants will read their first MYLO conversation transcript and be asked to choose two questions which they found helpful and two questions which they found unhelpful. Participants will be interviewed about why they have chosen each of the four questions and this will be audio taped. Participants will complete a MOL helpfulness rating scale for each of the 4 questions. Ratings of helpfulness are on an 11-point likert scale (0, not helpful at all to 10, extremely helpful) and ratings of the degree to which the question enabled 1) a sense of control over what is happening 2) the ability to talk freely 3) the ability to experience emotion and 4) to see the problem in a new way.	2 weeks post baseline
Secondary	Reorganisation of Conflict Scale (RoC)	We will use an 11-item sub-scale of this scale measuring goal conflict reorganisation which is a key mechanism of change in MOL.Participants are requested to rate their belief in each statement on a scale from 0 (I don't believe this at all) to 100 (I believe this completely). An average score across all 11 items is then computed and can range from 0-100.	Baseline & two weeks post
Secondary	Patient Health Questionnaire (PHQ-9)	A nine-item measure of depression with scores from 0 to 27 with a threshold score of 10 indicating clinical intervention.	Baseline & two weeks post
Secondary	Generalised Anxiety Disorder Questionnaire (GAD-7)	A seven-item measure of anxiety with scores from 0 to 21 with a threshold of 8 indicating clinical intervention.	Baseline & two weeks post
Secondary	Psychological Outcome Profiles questionnaire (PSYCHLOPS)	A four-question client-generated outcome measure with scores of 0 to 20. This measure assesses wellbeing, functioning and distress.	Baseline & two weeks post
Secondary	Qualitative feedback about the MYLO interface	A brief audio taped interview will be conducted exploring participants experiences of the MYLO interface and usability.	2 weeks post baseline
Secondary	Frequency of MYLO use	Frequency of use of MYLO	2 weeks post baseline
Secondary	Duration of MYLO use	Total duration in minutes of use of MYLO	2 weeks post baseline