Service-user Experiences of Text-based Conversations With a Computer About Their Difficulties

Psychological Clinical Trial

Official title: Understanding What is Helpful and Unhelpful in Intervention With Online Relational Agent MYLO: An Intervention Process Study.

Status Completed

Clinical Trial Summary

Mental health problems such as anxiety and depression are common. Services are finding it difficult to treat people quickly. This has led to computers being used to treat common problems.

Research has shown that around half of people who have mental health problems have more than one difficulty (e.g.low mood and worry). Many of the computer interventions available are not designed to work with more than one problem at a time. This can make them less helpful and can lead to people stopping intervention early.

A talking intervention called the Method of Levels is suitable for people with more than one problem. A new computer programme that aims to emulate this type of intervention is called Manage Your Life Online (MYLO). MYLO is accessed online. Users have conversations with MYLO by typing on a computer keyboard. MYLO aims to help people to talk freely and experience feelings related to a problem. In this way, people can find new ways of thinking about their problem. MYLO can be used as often as the person wants.

MYLO has been tested twice before with student volunteers. Participants said that it was helpful in solving an everyday problem. The investigators are not aware of any research that has tested a programme like MYLO for people experiencing mental health problems.

24 adult service users will be recruited from Self Help Services, a participating organisation in Manchester contracted to provide primary care mental health provision for NHS patients. Participants will use MYLO for a two week period.

The investigators aim to conduct a detailed examination of what questions service users find helpful and unhelpful in intervention with MYLO and whether service user perceptions are related to key mechanisms of psychological change identified in MOL. The information gathered from this study will help to increase understanding of how this type of intervention works and what makes it most helpful to service users. These findings will inform improvements to the MYLO programme to improve its acceptability to service users.

Clinical Trial Description

BACKGROUND

Mental health problems such as anxiety and depression are prevalent and related to significant personal and economic burden. Discrepancy between supply and demand of effective psychological therapies has led to the proliferation of computerised therapies for psychological intervention. Computerised cognitive behaviour therapy (cCBT) packages such as 'Fearfighter' and 'Beating the Blues' are recommended by NICE for the treatment of depression and phobias. However, studies have demonstrated high levels of attrition and although active telephone support seems to facilitate their effectiveness this has led to concerns about their efficiency and cost effectiveness.

Most available cCBT interventions are disorder specific and may not provide a good fit for the estimated 50% of service users who experience comorbidities. Transdiagnostic psychological interventions delivered in traditional and computerised formats have been shown to provide equivalent effects to disorder specific interventions of their type and have the potential to improve acceptability and efficiency.

A transdiagnostic form of cognitive therapy called the Method of Levels (MOL) has been used in the treatment of a wide range of psychological problems and is suitable for service users experiencing more than one problem and problems that do not fit into pre-defined diagnostic categories. MOL is differentiated from other cognitive therapies as it originates from a unifying theoretical approach known as Perceptual Control Theory (PCT). PCT posits that psychological distress is due to sustained conflict in internal control systems that are essential to all living things. Therefore, the aim of MOL is to increase a client's sense of control through enabling them to talk freely whilst supporting experiences of emotional distress necessary to increase awareness of internal conflict.

The acceptability of computerised interventions to service users is paramount if increased uptake, adherence and 'reach' are to be achieved whilst improving efficiency. Computerised intervention that offers greater collaboration has the potential to increase its acceptability. The investigators propose that one way of achieving greater collaboration in computerised interventions is through relational agents. Relational agents are computer programmes that simulate conversation. A relational online agent called MYLO (Manage Your Life Online) has been developed at the University of Manchester and uses artificial intelligence to simulate a MOL style conversation with a user via typed text. MYLO has been evaluated in two feasibility trials using student samples and has shown promising results.

The development of relational agents for use in mental health care is progressing, however, academic research has struggled to keep pace with developments in industry. The investigators are not aware of any research that has developed or investigated a transdiagnostic relational agent in psychological intervention in a mental health setting.

The evaluation of computerised interventions through standardised criteria has largely neglected the exploration of client centred outcome measures and the processes of psychological change. A novel methodology that is client orientated and theory led was tested in a feasibility study which examined service user perceptions of helpfulness in a single session of therapy. The current study will replicate this design and aims to examine service users' experiences with MYLO.

The investigators aim to conduct a detailed examination of what questions participants find helpful and unhelpful in intervention with MYLO and whether client perceptions are related to key mechanisms of psychological change identified in MOL. The information gathered from this study will help to inform understanding of how this type of intervention works and what makes it most helpful to service users. These findings will inform improvements to the MYLO programme to improve its acceptability to service users.

AIMS

The aim of the study is to conduct a process level examination of what questions clients find helpful and unhelpful in intervention with MYLO and whether client perceptions are related to key mechanisms of psychological change identified in MOL.

RESEARCH OBJECTIVES & HYPOTHESES

The primary objective is to conduct a process level examination of what questions clients find helpful and unhelpful in intervention with MYLO and whether client perceptions of helpfulness are related to key mechanisms of psychological change identified in MOL. To investigate this objective, Hypothesis 1 will be tested:

H1) Four key mechanisms of psychological change (ratings of perceived control, the ability to talk freely, maintain a focus on emotion and see their problem in a new way) will be positively correlated with ratings of the helpfulness of MYLO's questions.

Qualitative data on why participants perceived certain questions to be helpful or unhelpful and what was happening in those moments will also be obtained to enhance the quantitative ratings further.

Secondary research aims are:

To compare Reorganisation of Conflict scale scores pre-and post-intervention and to estimate the size of the effect of the intervention.

To examine how effective MYLO is in reducing psychological distress and resolving service user defined problems hypothesis 2 will be tested:

H2) Psychological distress (as measured by total scores on the GAD-7 & PHQ-9) and the effect of problems (as measured by the PSYCHLOPS Q1b and 2b) will significantly reduce between Time 1 and Time 2.

To explore service user experiences of the MYLO interface qualitative feedback will be obtained and analysed using content analysis.

METHOD

Study design

The study will utilise a multi-methods case series design repeated over several cases (n=24). This design of study allows for examination of the process of psychological change both within and between individuals. Participants will complete questionnaire measures pre-and post-intervention with MYLO. A multi-methods design allows for the examination of theory driven hypotheses about key mechanisms of psychological change with MYLO.

Participants

Participants will be adults accessing a primary care mental health service (Self-Help Services). Participants will be experiencing psychological difficulties and a computerised intervention will have been identified as a potential intervention option. Practitioners at Self-Help Services will establish whether a service user meets study inclusion and exclusion criteria.

Sample

Based on previous research conducted by the supervisors and a power calculation detailed below, it is estimated that 24 participants will be required. Manchester Self-Help Services received approximately 10,000 referrals in 2016 with approximately 3,000 (30%) of these people opting to access a computerised intervention. Therefore, it is anticipated that between 80 and 160 service users will be screened (based on a conservative 50% uptake rate) to recruit 24 participants.

Power calculation

Taking into account the repeated measures design, to estimate a correlation coefficient of 0.5 (a medium effect size) between the explanatory variables and perceived helpfulness of MYLO's questions with 80% power at a significance level of 0.05, a total of 21 participants will be required to complete Time 2 measures. Assuming a 10% attrition prior to Time 2, 24 participants will be recruited.

Recruitment

The investigators aim to recruit from Self-Help Services, an organisation in Manchester contracted to provide primary care mental health provision for NHS patients. All service users eligible to take part will receive information about MYLO and the study from a Self-Help Services practitioner providing an initial appointment. If a service user is interested in taking part (they consider the modality and style of MYLO as being potentially suited to their needs) they will be given an information flyer and Patient Information Sheet (PIS) to keep and asked to contact the investigator or to consent to pass their contact details to the investigator.

Eligible and interested service users will then contact or be contacted by the investigator. Interest will be confirmed and a convenient time and place to meet to gain informed consent, collect Time 1 measures and provide access to the MYLO programme will be arranged.

Participants can withdraw consent at any time without giving any reason, as participation in the research is voluntary, without their care or legal rights being affected.

Intervention

Manage Your Life Online (MYLO). MYLO is accessed online using a username and password. MYLO operates by analysing the client's input of text for key terms and themes. It responds with questions about the problem aimed at encouraging higher level awareness.

Participants will be provided with access to the MYLO programme through a unique username and password. The investigator will ensure that participants are able to access the programme and answer any questions. Based on feedback from a Community Liaison Group (CLG), the investigator will also provide a short demonstration of how to access the programme and provide all participants with printed instructions on how to access the programme as a reminder.

Participants will decide how often to use the MYLO programme over a two week period. This is likely to be a reasonable length of time to allow at least one use of the programme with no upper limit on usage. Service user led appointment scheduling provides clients the autonomy to decide when and how often to access interventions which increases efficiency whilst maintaining efficacy.

DATA ANALYSIS

Statistical analyses will be performed in Stata version 14. Age, gender and MYLO engagement data will be reported descriptively.

To address research question one, separate multi-level mixed effects linear regression analyses will be conducted to test H1, with helpfulness question rating as the dependent variable. This will be followed by a comparison of the unique contributions made by each predictor variable in the helpfulness rating using a mixed-effects regression analysis.

To address H2, a paired samples t-test will be conducted to examine whether scores on the RoC scale have significantly increased between Time 1 and Time 2.

A qualitative interview will examine why participants thought questions were particularly helpful or unhelpful and serve to enhance the quantitative findings.

To address research question two, H3 will be tested. Paired samples t-tests will be conducted to examine whether there is a significant difference in scores on the PHQ-9, GAD-7 and PSYCHLOPS items 1b and 2b scores between Time 1 and Time 2. In addition, reliable change on the PHQ-9 and GAD-7 will be calculated using the Reliable Change Index and used to identify whether any change is clinically significant.

To address research question three, brief qualitative interviews will be analysed using content analysis. This method of analysis allows codes to come from the data and be organised into categories. This method of analysis will facilitate clear recommendations for the enhancement or modification of the MYLO interface.

DATA PROTECTION & GOVERNANCE

University policies and procedures regarding information governance will be adhered to. Hard copy data will be stored in a locked, fire proof filing cabinet at the School of Psychological Sciences, University of Manchester accessible only to members of the research team.

Electronic data will be transported, stored and analysed in accordance with university data protection policies. Electronic data will be stored on a secure storage area on a university provided research drive accessible only to members of the direct study team. Data files will be encrypted with a password using Microsoft Office and data folders will be encrypted using 7-Zip according to University of Manchester information storage policy.

University policy and procedures relating to audio recording will be adhered to. Audio recordings will be made using an audio recorder. The recorder will be locked away securely when not being used and will not be left unattended. Audio files will be transferred as soon as possible to a secure computer storage drive accessible only to members of the research team. Transcription will be conducted by a member of the research team in a secure environment and the identity of the service user will be anonymised.

The MYLO programme will be hosted on a Virtual Machine (VM) set up and maintained by IT services at the University of Manchester. The VM will be connected to university Lightweight Directory Access Protocol (LDAP) so only permitted University of Manchester registered users will be able to access the VM. All University of Manchester employees have to login to university services using GlobalProtect.

The web address for MYLO will only be provided to study participants. Users accessing MYLO will be unable to upload files to MYLO or change any of the interface. User passwords for MYLO are encrypted and usernames and passwords are to be held on a password protected database which is accessible only to the research team and stored on the secure shared drive.

MYLO conversation logs are not encrypted and are stored in text file format in a system folder within the MYLO database. Conversation files will be transferred from the MYLO database by the researcher as soon as practicable following conversation completion to a secure storage area on the university provided research drive. Data folders will be encrypted using 7-ZIP with password protection. Passwords will be strong i.e.: minimum length of 10 characters, case sensitive, include a mixture of alphabetic (a-Z, upper and lower case), numeric (0-9) and special characters.

All data obtained will be securely stored for 5 years in accordance with University policy.

The study will be subject to the audit and monitoring regime of the University of Manchester. ;

Related Conditions & MeSH terms

• Psychological

• NCT number: NCT03524196
• Study type: Interventional
• Source: University of Manchester
• Status: Completed
• Phase: N/A
• Start date: November 19, 2018
• Completion date: July 11, 2019