We completed a review of Roundabout’s evaluation methods in 2017/18 which was conducted by Dr Emma Ramsden, an independent post-doctoral researcher, which culminated in a report entitled Shine a Light (SaL). The report concluded:
“The SaL project’s main aims were to review the implementation and effectiveness of evaluation tools used by Roundabout dramatherapists over time, to make recommendations for future use, and to undertake preparatory work for a feasibility study. Charting the history of the organisation’s use of evaluation in this report has shown its commitment to prioritising the client voice and to integrating the three strands of evaluative data in order to support the client and the overall system in every setting. Whilst primarily concerned with the business of Roundabout – meeting the needs of clients, developing ways of reaching funders and raising the organisation’s profile as a service provider of excellence – nevertheless, over more than 30 years of successful operation as a charity, the portfolio of validated and non-validated evaluation methods that it has developed and implemented, along with the publications, conference presentations and workshops it has generated, clearly demonstrate Roundabout’s position as a leader in and contributor to theory and practice in the field of UK dramatherapy”.
These conclusions underline the importance of evaluation in Roundabout’s work and the report as a whole successfully underpinned an application to the British Association of Dramatherapists for a formal, academic research project, assessing the impact of dramatherapy on children with autism. This grant of £20,000 will enable us to conduct the research project, called Shine a Light on Autism (SaLoA), over the next two years.
The involvement of the whole team has been very important in both SaL and SaLoA, and these new developments have formed the basis of a number of team meetings held this year.
Analysis of evaluation material generated in the year produced interesting and affirming results, demonstrating the profound impact of Roundabout’s work. For example, at a high school, where we worked with girls with autism, evaluation using Psychlops YP showed the effect size was 0.97, (“An effect size of 0.8 is generally considered to be large and indicates an effective intervention.” (Cohen 1988)), indicating that the girls were reporting that dramatherapy was helping them with their problems and reducing their worries.
Analysis of evaluation using Strengths and Difficulties Questionnaires for our work with primary aged children showed that there was a significant degree of positive change reported by parents, carers and teachers.
“We have noticed very positive changes in Amos since participating in dramatherapy sessions. Amos has slowly returned to his chatty, happy, cheeky… He is giggling a lot more on a daily basis; there was a long period of time when Amos didn’t laugh in school. His peer relationships have grown stronger and he has opened up about the death of his mum and significant changes in his life to his peers”. Parent
Background to our Evaluation and Research
From its earliest days, Roundabout recognised the importance of monitoring and evaluating its work, and developed various approaches to ensure this happened. Initially, this was through the feed-back between service users and dramatherapists and between staff, parents and carers and Dramatherapists, through the supervision of all our work by the Project Directors, and through written reports.
By reflecting on the information we collect, we ensure our projects are meeting the needs of the service users and our own high standards. Because feed-back and supervision happens every week, we can respond quickly to changing situations, areas of difficulty and any other challenges. And because we work as a team, we can draw on the skills of everyone in that team to make sure we respond effectively.
Over the years, monitoring and evaluation has become increasingly important throughout the Voluntary Sector, and the demand from funders of all types for ‘evidence based need’ has increased. The nature of our work – the diversity of our client base, the sensitive nature of the therapeutic process, and the fact that our work generates ‘soft’ rather than ‘hard’ outcomes – has presented Roundabout with some interesting challenges in finding or creating evaluation methods that are sensitive to the needs of our service users and to the dramatherapy process.
Thanks to generous support from the Terpsichore Trust, the Wates Foundation and the City Bridge Trust we have been able to invest the necessary time over the last few years in order to meet that challenge. In work co-ordinated by Susan Crockford, we undertook an audit of all the methods our dramatherapists currently employed, and what was being used in the wider community of arts therapists, and amongst practitioners working with relevant client groups.
It became apparent that there was no single methodology that could be applied across all our projects, and even after sub-dividing our work into service user categories, it was still proving difficult to find suitable methods. Until we heard about Psychlops (see below for a full description). We also began to use, when appropriate, recognised systems such as Strength and Difficulties Questionnaire and, for working with adults with learning disabilities, CORE LD.
As well as working in partnership with King’s College, London, to adapt Psychlops to the school setting, we have experimented with systems we created ourselves. Finding the right Monitoring and Evaluation methodology for Roundabout is still a work in progress. We are currently embarking on a trial of the Goal Attainment Scoring (GAS) methodology.
To summarise, we monitor and evaluate our projects and outcomes through:
1. Project evaluation methods:
- Hope and Wishes
- Behavioural Summarised Evaluation
- CORE questionnaire specially adapted for people with learning disabilities
- Roundabout’s own evaluation tool using ‘faces’ and tick boxes, for adults with more profound learning disabilities
- Roundabout’s own evaluation tool ‘Here and Now’ evaluation for older adults
- Verbal and written feed-back from service users
- Verbal and written feed-back from family, teachers, carers, support workers etc
- Observations of the Dramatherapists
- Written reports
2.Statistical monitoring methods:
- Collecting and collating statistics on the projects we have facilitated, and the numbers of people we have worked with, as well as their diagnosis, ethnicity and sex.
During 2010/11 Roundabout completed the development and piloting of ‘Psychlops Kids’. This was a major undertaking for Roundabout and resulted in the development of a new way to evaluate the impact of dramatherapy. ‘Psychlops Kids’ is a self-report methodology of evaluating therapy designed specifically for children aged between 7 and 13 years old, offering them the opportunity to say in their own words what their worries are and how dramatherapy might have helped. ‘Psychlops Kids’ is based on ‘Psychlops’, a mental health outcome measure for adults developed by primary health-care professionals and therapists based at King’s College London. Between 2008 and 2011 Roundabout worked with this team to develop a version suitable for young service users as research had discovered no other self-report evaluation for under 13 year olds. Roundabout has been in a unique position to develop and pilot such a measure because of the large number of children and young people we work with each year.
The ‘Psychlops Kids’ forms identify, at the beginning of therapy, issues that are bothering the child or young person and also how they are feeling about going to dramatherapy. Through the pilot process the Roundabout team were able to add creative forms of evaluation to the format established by the original team. ‘Psychlops Kids’ was piloted 3 times over 2 years and we were able to elicit feed-back from children, young people, the adults who supported them to complete the forms and the therapists. This led us to the final version which was used during the academic year September 2010 to July 2011. Our work with ‘Psychlops Kids’ has shown that this form of evaluation really does give the children and young people ‘a voice’. The pre-therapy forms provide a helpful starting point for the therapeutic work, the mid-therapy forms provide an opportunity to reflect on change and the impact of therapy, and the forms completed after therapy has ended offer both qualitative and quantitative results.
The results have been very positive indicating that dramatherapy is really making a difference in children and young people’s lives. The results of 66 completed before and after Psychlops Kids forms showed an effect size of 0.9. The effect size calculation is based on the section in Psychlops Kids that can be given a numerical scoring and shows that for those who completed the forms they felt significantly better after the dramatherapy programme. Other data from the forms indicated that the issues children were most worried about were: school problems, peer problems, self-esteem issues, anxiety about school transfer and family problems.
Accessing Psychlops Kids
Forms are free to download from
Dramatherapy with Children, young People and Schools
This book edited by Deborah Haythorne, Lauraine Leigh, Irvine Gersch and Ann Dix is the first book to specifically evaluate the unique value of drama therapy in the education environment. A variety of highly experienced drama therapists, educational psychologists and childhood experts discuss the benefits of dramatherapy to children and young people and also in relation to the involvement of teachers, the multi-disciplinary team and families. See here for further information and to order this book: