Earlier this year we heard from a number of projects as part of an Elluminate session on Using video within the Curriculum Design programme: from personal reflection to evaluation evidence. During the session, Rebecca Galley and her colleague Andrew Charlton-Perez talked about whole project use of video within the Open University Learning Design Initiative project (OULDI).
Picking up from that session, I spoke with Rebecca recently to further explore her views on the pros and cons of using video as part of the evaluation process.
To start can you tell us how video is being used as part of the evaluation of OULDI?
Rebecca: We’ve used video in three different ways:
Firstly, to capture expert evaluation of Cloudworks. Many of these people are very busy, or we just happened to get talking about Cloudworks at a conference. We have found it much easier to grab experts for a quick, fairly informal video ‘chat’ than to book them for a more formal interview. Added to that the results have been just as useful for informing development. On another project, the OU-OPAL project, we’ve used the same ‘on the spot expert video‘ approach to capture barriers and enablers to Open Educational Practices (OEP) prior to the development of an OEP maturity matrix. We’ve also used video to promote wider engagement in the project consultation process through spotlight discussions.
Secondly, we offered a range of reporting options to academics trialling the OULDI tools and resources. One of the options was a reflective video log. One academic at Reading chose to keep a curriculum design video diary. He found it a very rich and interesting process. So much so that he felt it impacted positively on his design process by enabling him to reflect on and consolidate his design activity in a way that he wouldn’t have done otherwise. For the project team, this was especially interesting because the video log captures the learning design process from a personal perspective.
Finally, and more recently, within our Cluster we’ve been developing a series of video podcasts on the theme ‘Design problem, design solution’. These will be uploaded to YouTube in late September. In these four minute videos we have interviewed a range people involved in the project and asked them what they perceived their design problem was and how our tools, resources, methodology helped them solve it. We think that these videos authentically and compellingly show the degree and range of impact the projects have had. (This is an example of an early problem/solution interview.)
That’s a great range of how video is being used. Now let’s talk about the practicalities, what approach have you taken to collecting video data?
Rebecca: We’ve gone for a very low-tech approach, mainly because the primary benefits of video for us are flexibility and authenticity. So, we generally use the flip or iphone camera and keep editing to a minimum. Where appropriate and agreed we have made all interviews, whether audio, video or transcript openly available and have archived videos on YouTube or Vimeo and embedded in Cloudworks.
Are there any practical issues that others looking to use video should be aware of?
Rebecca: Our biggest practical problem has been coping with the wide variety of file types we have been getting because all the partners use different cameras. In the end we have had to purchase specialist editing software. We bought Sony Vegas Movie Studio Platinum.
In your presentation you hinted at the resource implications of managing and analysing video data, and referred to light touch analysis. Could you tell us a bit more about that?
Rebecca: The OULDI project is quite complex with a number of strands across five institutions, for this reason we are using Nvivo to collate and code all our data. The Nvivo software allows for the collation and coding of all sorts of data, which is really useful. For example, it means that we can do a search on data which relates to novice users of Compendiumld and get a whole range of data back including video excerpts, excerpts from blog postings, Compendium designs and textual feedback and compare this with data we have from expert users. For ease, everything has gone into Nvivo but we have had to be really disciplined and focused about what and how we code – and why – otherwise we could spend all our time on analysis that we can’t use.
In terms of presenting video findings, you mentioned archiving using YouTube or Vimeo, with more open presentation on Cloudworks. Why have you taken this approach?
Rebecca: The OULDI project is very much informed by the open education, OER and open scholarship movements. We see value in publicly sharing our conceptual ideas and thinking, research tools, methodologies, and findings, and opening these up for community discussion and critique. We hope that our use of Cloudworks has enabled a community of interest to develop around the project, for example educators who have been involved in earlier project workshops and trials are able to stay in touch with development and give us feedback on these from their own perspective which is really valuable.
Finally, are there any caveats you would put on the use of video?
Rebecca: I think that the use of video for research and evaluation, especially where it is made publicly available, significantly changes the relationships between researchers, project participants (previously ‘subjects’), the audience and the video data itself. I’m not saying this is necessarily a bad thing but the impact of this on the reliability and validity of the data needs to be recognised and critically considered. My personal thinking is that it should only ever be used as a small part of a broader multi-methods approach.