Focus of Study: Cognitive Surgical Skill Acquisition Clinical Trial
Official title:
The Role of Multimedia in Cognitive Surgical Skill Acquisition in Open and Laparoscopic Colorectal Surgery
To practice independently surgeons require competency in surgical skills, encompassing a
combination of technical and non-technical skills. Cognitive skills, aspects of
non-technical skills, represent an integral component of surgical competency. Cognitive
skills comprise factual knowledge and decision-making.
Changing work patterns in the United Kingdom, as specified by the European Working Time
Directive (EWTD), have had a profound impact on the delivery of surgical skills training.
Surgical trainees are now increasingly removed from normal working hours in which the
majority of traditional operative training and experience is gained, leading to a net
reduction in trainees' operative exposure. This reduction in operative experience means that
surgical competence can no longer be assured on the basis of experience alone.
Although there is no educational technology that can replace the craft apprenticeship
required to train a competent surgeon, reduction in training hours has led to rapid
development of educational tools to augment surgical skills training outside the operating
room environment. These tools tend to concentrate on technical skills performance without
emphasis on cognitive skills.
Trainees in today's era have grown up in a multimedia environment; multimedia is media that
uses a combination of text, voiceover, animation and video. Multimedia is an underdeveloped
educational resource that can supplement cognitive skills training in operative surgery.
The purpose of this study was to design and develop an online multimedia educational tool in
a common colorectal surgery procedure ("Anterior Resection") and determine the effectiveness
of this tool in teaching and assessment of cognitive skills.
Study hypothesis: Multimedia learning is equivalent to conventional teaching "Study Day" in
improving scores in cognitive surgical skills.
This study involved two main objectives:
1. Develop and design multimedia educational tools for open and laparoscopic anterior
resection
- The multimedia tools were developed in collaboration with a professional
multimedia company (Digimed). Initially 31 open and 35 laparoscopic (key-hole)
procedures were filmed after informed, written consent was obtained from patients.
Key procedural steps/ subtasks were identified and integrated onto navigational
interactive interface platforms developed using Adobe® Flash® Professional CS5
10.1.
- Each subtask was composited into succinct video clips using Final Cut Pro®,
supplemented with annotation, animation, text and voiceover. Content addressed
cognitive aspects including factual and anatomical knowledge, decision-making
points, instrumentation required, team positioning and use of assistance. The
multimedia educational tools are available online at following the link
http://www.colorectaltraining.co.uk. The colorectal training website has 1.24GB
including 69 video clips in the open tool and 67 video clips in the laparoscopic
tool.
2. Evaluate the effectiveness of multimedia in teaching/assessment of cognitive surgical
skills
- A randomised control study was conducted to evaluate the effectiveness of online
multimedia in comparison to conventional "Study day" teaching in cognitive
surgical skills acquisition. Enrolled participants were randomly allocated into
either the intervention (Multimedia) or control (Study Day) Group. The methodology
of the assessments was online timed (written) assessments to test cognitive
surgical skills. All trainees were assessed before and after the study period.
- Following ethics approval, permission was obtained from surgical training
programme directors (TPDs) to recruit trainees into the study. The study was
conducted between 6th October 2011 and 23rd December 2011. Trainees invited to
participate were contacted by email. A study flyer attachment and hyperlink
(http://www.colorectaltraining.co.uk) were provided for trainees to access
relevant study information on a 'Participation Information Sheet' and an online
consent form. After submission of the consent form, trainees were directed to
separate web pages on the secure web-based service containing a Trainee Proforma
(to obtain demographic participant information) and Pre-Assessment Test.
- Participants in the multimedia group were provided with unrestricted access for 30
days and the study day was held at the end of this period for the "Study Day"
group. Multimedia group participants were each emailed an individual login and
password to access the multimedia tools on the colorectal training website:
http://www.colorectaltraining.co.uk. The login details were requested each time
the website was accessed.
- The study day involved a series of detailed interactive lectures covering all the
steps of open and laparoscopic anterior resection surgery. The content delivered
was identical to that in the multimedia tool. After a period of one week, all
study participants were required to complete the online Post-Assessment test.
Multimedia group participants emailed a separate hyperlink to complete an online
Evaluation form. All completed forms were stored securely for analysis.
3. Sample size
- Sample size was calculated on the basis of a pilot study of 50 surgical trainees
taking part in an anterior resection study day in January 2009. Trainees were
asked to complete a pre-study day written assessment, followed by a post-study day
assessment immediately after the lectures had been completed. The mean (SD) pre-
and post-assessments scores were 12.4 (4.56) and 26.8 (3.99) respectively.
- The sample size was calculated with the mean change under the assumption that the
baseline (pre-assessment) score for both arms is the same. Mean change in score in
the control is 14.4 (4.9) and an expected mean change in score of 24.4 (SD 4.9) in
the intervention group is assumed. The sample size required to test the hypothesis
at a type I error of 5% and power of 80% was calculated (per group) to be 10. So a
total of 20 trainees need to be recruited. If the response rate is 20% and
assuming a drop off rate of 20% ((20/20%)/ (1-20%)), a total of 125 trainees
needed to be approached.
4. Statistical analysis
- The baseline pre-assessment test ensured comparability of knowledge base and
cognitive skills between study groups and helped to assess construct validity of
the assessment tool. Basic descriptive statistics include the numbers and
percentages for categorical data, mean and standard deviation for normally
distributed continuous data; median and range for skewed data was also included.
All data were analysed on an intention to treat basis.
- Differences in assessment scores between multimedia and control (study day) groups
were compared using either the unpaired Students' t-test (for normally distributed
data) or the Mann Whitney U test (for skewed data). To analyse post-scores versus
pre-test scores within the same group, paired t-test (for normally distributed
data) or the Mann Whitney U test (for skewed data) was used. Data was analysed
using SPSS (statistical package for social sciences version 16.0, Chicago IL).
Significance levels were set at P<0.05.
;
Allocation: Randomized, Intervention Model: Parallel Assignment, Masking: Single Blind (Outcomes Assessor)