Sharing evaluation | don't reinvent the wheel...

Sharing evaluation | don't reinvent the wheel...

Barriers

For over 12 years now I’ve worked within the museum and heritage sector in the field of evaluation and audience research and I’m reasonably confident the sector can now see the benefits of evaluation. However, I often feel frustrated at the barriers that we, as museum professionals, collectively seem to have raised, which often prevent us from sharing what we find out from the evaluation process.  We are still reticent about telling others what hasn’t worked, we still ask the same old questions, and we don’t look internally or externally as much as we could for useful data.  We just don’t seem to learn as well as we potentially could from the experiences of our colleagues. Recently a colleague suggested that perhaps it’s just part of the human condition that we need to keep asking the same questions of museum audiences.  Does this suggest we need reassurance or is it, fundamentally, about the fact that we believe our organisations are so unique we don’t feel we can apply what others have found out about their audiences to our own situation?  Does it feel easier to keep asking the same questions rather than spending time applying someone else’s research to our own organisation?   I’ve come to the conclusion that breaking down the barriers to sharing is much more than the question of whether we publish our findings. Many museums already do this really well, e.g. the Victoria and Albert, who commission a lot of research and put reports on line and the Natural History Museum, who share evaluations of exhibitions and learning programmes.  I think we need to do more. 

Understanding

We know that carrying out evaluation provides evidence about how projects and programmes are meeting aims and objectives and encourages on-going project improvements. We know it can act as an excellent tool for telling others about the great work being done in the sector, enabling us to share lessons learned with colleagues.  Evaluation can help the project team and participants feel like they ‘own’ the project, it can develop relationships with visitors and play a really important role in developing ideas and planning future projects. [see image]  We also know that if we don’t carry out any evaluation we could waste time and money, produce something that we can’t change, lose interest from our target audience, and waste an opportunity to learn something useful and lose funding.  There are a number of excellent online toolkits readily available which steer willing participants through the whole evaluation process e.g. the East of England Museum Hub’s Evaluation Toolkit for Museum Practitioners and the Smithsonian Centre for Education and Museum Studies has just set up an online community of practice around evaluation, which looks really exciting.

Facilitating a focus group at the Grant Museum of Zoology as part of the London Museums Hub Project ‘Say it again say it differently’ 2004-2006.

Fear

So we understand the benefits of evaluation and how to do it, but why are we not sharing what we find out through the evaluation process more effectively? [see image] One of the important factors may be that many of us are still reticent about telling others about what hasn’t worked so well.  I really understand the fear behind this.  For many reasons we want to portray projects, programmes and events in the best light, to secure future funding or to retain professional standing, but in its purest sense evaluation should be seen as an opportunity for us to learn and improve our professional practice. It should not be done to point fingers or assign blame. Rather it should help create a culture where we are able to take risks and share what hasn’t gone so well. Only then can we learn and move forward as a sector.

Re-packaging

I think a possible way forward is to re-package the data already held within organisations and think more strategically about how we plan evaluation. The organisations I regularly work with are often surprised with how much data they actually have once they start looking! Recently, working with the Education and Interpretation teams in a heritage organisation a colleague and I tried this as a practical exercise.  After some initial hesitation, followed by discussion and deliberation, the team realised they could pull out useful audience information from their existing evaluation data which they could put to efficient use in planning future projects and programmes.  Maybe the next step would be to work with two or three similar organisations to share audience consultation and evaluation data around specific audiences. Maybe a group of museums could agree some overarching research questions so all the evaluation and audience consultation work they do individually could collectively feed into answering these questions.

Facilitating a focus group at the Brunel Museum as part of the London Museums Hub Project ‘Say it again say it differently’ 2004-2006.

Let’s be confident about what we know

As a profession we need to be confident about what we already know.  I am often asked to find out the barriers secondary school teachers have in taking their students out to museums. I can safely say that I’ve asked a lot of teachers this question and I know what the answers usually are, so I encourage the organisations I work with to ask something which might be fundamentally more useful [see image]. A recent project with The National Maritime Museum [NMM] in Greenwich took this a step further when they asked secondary teachers what the NMM could do to help teachers advocate internally for a museum trip with their students. Should the NMM guarantee that the school could bring a whole year group at once or guarantee specialist input from a member of museum staff for the visit? Should they provide evidence of impact on attainment targets or information that highlights the benefits of learning outside the classroom? The Museum plans to share the results of their research with the sector.  Having the confidence that they know about the barriers teachers have to visit the museum with students has enabled the NMM to find out new information which will ultimately improve their school offer.

Evaluating with children at Bromley Museum as part of the London Museums Hub Project ‘Say it again say it differently’ 2004-2006.

Sharing documents is not enough

As a committee member for the London Museums Group there are certainly more conversations we can have as to how Share London, the London Museums Group online forum for sharing practice, can help us to share evaluation and audience consultation findings more widely.  I think we need to remember that posting our analysed and interpreted evaluation and audience research data might not be enough and more work and conversations need to be had about how we stop reinventing the wheel. Ultimately, time is always precious, and if we can stop asking the same questions, use the data we already have and share our findings more efficiently we [and our audiences] can all only benefit in the longer term.  

Author | Nicky Boyd, Museum Consultant [Audience Research & Evaluation] and LMG | nickyboyd@btinternet.com

Images | all photographs were taken by J. Neligan during focus groups as part of the London Museums Hub project Saying it differently 2004-06.  

For more information on the project please go to: http://www.museumoflondon.org.uk/Corporate/About-us/Regional-Programmes/Publications+and+Resources.htm

Nicky Boyd's Blog has been reprinted on the Guardian's Culture Professionals Network.  Sign up free to become a member of the Culture Professionals Network.