Validated instruments for use in arts impact research

I was putting together a table of various validated instruments which have been or could be used in arts impact research, and thought it might be useful for others working in this area. The table is a list of instruments which can be used to measure empathy, self-esteem and self-efficacy, wellbeing and so on. I have also included links to the instruments if they are available for free or purchase online.

You can access the table in html or pdf.

• Validated Arts Research Instruments Table - html
• Validated Arts Research Instruments Table - pdf

Please feel free to comment - the list is not exhaustive and doubtless there are some in there which have more or less efficacy than others!

Happy measuring :-)

Jackie Bailey, Principal, BYP Group

Can we measure the value of enduring artworks?

(This post is a continuation of my ruminations on Ann Markusen's work)

There is something else I saw in Ann Markusen's report on California's arts and cultural ecology which I found really interesting.

I saw these words:

'California’s arts and cultural nonprofits generate new and enduring artworks—they commission an estimated 41,000 theater, dance, musical compositions, and artworks annually (p.33)'

and had one of those moments of nerdy, impact evaluation realisation (you know the ones ;-).

I have never tried to capture the value of the enduring nature of works of art. Sure, we all talk about heritage value, bequest value etc (thanks to John Holden). But normally when I do impact or value assessments for arts clients, I am very focused on the intrinsic experience of the art and the social, educational and personal outcomes for the participants.

I have always included 'contribution to society and culture' as an element of Artistic Vibrancy (a framework for measuring the health and impact of an arts organisation). But I never thought of measuring the contribution to culture through the creation and continued public experience and enjoyment of enduring works.

How can you value the contribution of Shakespeare, for example? Well, it's invaluable - as you can can always say with the arts. But you can try to translate the value of enduring works into a language that funders can work with ('Invaluable' does not cut it in the Cabinet room).

For example, could you start to count the number of works likely to endure over say, 2, 3, 5 years and longer? You could probably come up with a statistical formula based on big data (PhD project anyone?!) And then how many people are likely to see it, engage with it, perform it, derive some meaning from it, and be transformed by it (a kind of pyramid effect)?

I think this would be completely awesome or terribly hard to believe ,depending on how carefully it is done. There is nothing worse for arguments about arts impact than seemingly ambit claims of creating $X in value. But when it's done well, it's - well, invaluable.

I'd love to hear other evaluators' thoughts on this. Do you think it could be done and done well? Or perhaps it has been done?

Jackie Bailey - Principal, BYP Group

Appreciative Inquiry - change and evaluation rolled into one

We are often asked about the different types of evaluation approaches.  We will try to summarise them from time to time here, on our site, and hope this helps people sort through the jargon of evaluation and get to the bottom of what suits you and your needs best. We have started with an explanation of Appreciative Inquiry, because I was asked about it at a presentation in Melbourne to arts organisations.

What is an Appreciative Inquiry Evaluation?[1]

An Appreciative Inquiry evaluation is based on the principles of Appreciate Inquiry (AI), an organizational development method which is focused on building on what an organization does well.  The principles of AI are summarized below.

In an AI evaluation, evaluators and participants work together to share their views of the present, and “co-create” the future. An AI evaluation does not ignore problems, but approaches them as opportunities for change.

In an AI evaluation, the evaluator and participants:

• become fully engaged in the learning journey
• work to “co-create” the future
• acknowledge that there are multiple, equally valid interpretations of reality
• share their individual interpretations of reality, with an aim to gain a shared understanding of experiences
• envision possible positive futures which build on present strengths
• use language and foster relationships which create that positive future

In a pure AI evaluation, traditional evaluative methods – eg qualitative and quantitative research – are used only as the need arises, and are driven by participants.

Principles of Appreciative Inquiry

“Appreciative inquiry” is an approach to evaluation based on the assumption that an organization wants to improve.  Accordingly, the evaluation has a fundamentally positive focus on what the organization does well, and how it can build on this.

The core principles of Appreciative Inquiry are:

1. Constructionist principle: people’s realities are “constructed” through their social interactions.
2. Simultaneity principle: change and inquiry are simultaneous.  Inquiry can itself effect change.
3. Poetic principle: the “story” of an organization is a product of the ongoing narrative of its members and others.
4. Anticipatory principle: envisioning a positive future can help to guide people towards one.
5. Positive principle: focusing on the positive can help create a positive energy for the future.
6. Wholeness principle: wholeness brings out the best in people, so supporting people to share the whole story from a position of individual wholeness can build a “collective capacity for change.”
7. Enactment principle: positive change occurs when people create the future through their words, images and relationships.
8. Free choice principle: free choice stimulates positive change and liberates personal and organizational power.

---

[1] This explanation of AI is drawn from Howieson, Jill, “A Constructive Inquiry approach: blending Appreciative Inquiry with traditional research and evaluation methods,” Evaluation Journal of Australasia, 11(2) 2011.

Evaluating arts and social inclusion projects

Yesterday I gave a presentation at the Creative Victoria Expert Arts Panel session on evaluating arts impact. Along with me, Deakin Uni's Hilary Glow and Anne Kershaw presented about a recent evaluation they did for Vichealth on arts and wellbeing, and Mark Hogan from Regional Development talked about the Clunes Booktown regional transformation story. Once Creative Victoria upload the full session to the web I will upload a link. In the meantime, I have uploaded my powerpoint presentation here.