XXXXXX




Follow Voicebox on Twitter Follow Voicebox on Facebook
Follow Voicebox on Facebook

Great Study. But What's It For?

June 18, 2009

Arts organizations are forever commissioning research papers to test the validity of their various endeavors. Quite often though, the time, money and effort that goes into producing a study fails to be put to good use. The research is published, journalists sometimes write articles about the findings and then, the organization which funded the research might use the results in grant applications. But then, more often than not, the information quietly gathers dust in a filing cabinet or on a hard-drive somewhere.

When I heard about Theatre Bay Area's new study aimed at measuring the impact of the Free Night of Theater (FNOT) -- a now-national, annual event offering free theatre tickets to many shows in the region between designated dates in October and which the Bay Area was the first metropolitan area to pilot several years ago -- I was anxious to find out why TBA had commissioned the research and what the local performing arts umbrella organization hoped to gain by it.

Commissioned by TBA, the "Assessing the Intrinsic Impacts of the Bay Area Free Night of Theatre Program" study results, which were released on June 9, sets out an ambitious goal -- to "measure and demonstrate the inherent value of the arts."

Based on the study results, TBA concludes:

1. Regardless of ticket price, patrons who make it through the doors of the theatre and are captivated by the live performance do exhibit measurable signs of intrinsic impact from that art. The most notable of these signs include increased intellectual and emotional engagement, social bonding and aesthetic growth.

2. The FNOT program serves to create social accessibility by providing an opportunity for theatergoers with unsupportive social networks to attend the theatre. The data reveals that the lack of a social network can be a major barrier to theater attendance. However, by lowering the perceived cost (offering performances for free), theatre companies can generate large numbers of new patrons and foster new social networks - with a high number of patrons promising to return and pay due to the inherent and social impacts of the experience.

All well and good. The study results backing up these claims are published on TBA's website here. But what does TBA hope to gain by all this work? I asked the organization's marketing director, Clay Lord, for his thoughts:

"This first round was really just a pilot study to refine the survey and develop what we're calling the "dashboard," which is a dissemination device designed to be read by non-researcher-types. We're now in the last few days of writing a proposal for phase 2, which would run over the next 2 years in 5 cities across the country and would be used to both gather more data and build the infrastructure for us at Theatre Bay Area to take the data processing and analysis out of the hands of our consultants and make it accessible for a much cheaper price to just about any theatre company in the country. This scaling up is vital to the data actually being useful. We've gotten interest from a variety of foundations both locally and nationally, and assuming the funding comes through, Round 2 would kick off in the Bay Area, LA, Chicago, New York and one other city TBD this fall.

In a larger sense (which you may have been actually asking about), the question "What is this data good for?" is foremost in our minds. We do not believe this is just data for data's sake. As part of phase 2, we'll also be developing with our consultants a pre- and post-survey discussion process with the participating companies to help them understand the usefulness of the information, set goals and expectations on the work, and then look at where the goals and reality matched up (and didn't) post-study. As part of this, we'll be developing a series of potential strategies (and small test projects) to address what we think are going to be some of the issues that emerge from the results based on our consultant's prior work - issues like how to increase captivation/involvement of audiences following the show, which seems to be pretty correlated to audience retention rates, or how to minimize the number of impediments to captivation in the theatre (everything from ushers being friendly to making sure there aren't lights glaring in audiences' eyes to giving out a free glass of wine). By taking these intrinsic impact measurements and beginning to pair them with certain concrete actions, we move from abstract data into actionable steps.

Over time (following this study period), we hope to use a centralized database (to be built as part of this proposal) to be able to see trends, thereby hypothetically helping companies anticipate what types of shows will have what types of impacts (and of course, what outside-theatre impacts are the most affecting on audience engagement).

From my perspective as a marketing director, I can't wait for this data to be more widely gathered and used. To be able to actually see on paper if marketing materials are increasing anticipation, or if certain perks instituted in the venue are actually helping captivation - it would be a whole new way of working with audiences."

Like any study, it will be a while before the research can actually be put to positive use. At least Clay and his colleagues are thinking beyond the limits of the the data.

17 Comments:

  • I'm looking forward to seeing the results of this data. This was a point of conversation in Radiostar's recent interview with Pianofight. There are serious questions in the small theatre community about whether or not TNoT actually works. So far, we've been told it does, but actual hard data is hard to find. Getting some actual, public data on numbers of new patrons, increased repeat attendees, and which theatres benefit most from this kind of program will be VERY useful to the broader community.
    (If you're curious to hear this half of the debate, the interview is up at http://www.radiostarnetwork.com )

    By Blogger Dan Wilson, At June 18, 2009 at 11:13 AM  

  • I guess I would count the number of people in seats on the free night. If it's noticeably more than usual, it's working.

    By Blogger Joe, At June 19, 2009 at 8:28 AM  

  • the question isn't "do they come because it's free", but "do they come back when it isn't?"

    The comparison made in our conversation with Pianofight was the free samples you get in grocery stores. Sure, if someone wants to give you some free sausage, you're gonna eat it. But how many people then go an buy a package of sausage? Those are the raw numbers that matter.

    By Blogger Dan Wilson, At June 19, 2009 at 10:06 AM  

  • I agree that the best measure of impact would be the number of returning new patrons on "paid" nights. If that number increases demonstrably, then it's a success.

    i do wonder, however, how these theaters can take the hit of a night's worth of lost revenue. Perhaps I need to know more about the program - are participating theaters subsidized in anyway to participate?

    By Anonymous Sam Shaw, At June 19, 2009 at 3:52 PM  

  • Nope. It's not subsidized at all.

    By Blogger Dan Wilson, At June 21, 2009 at 10:15 AM  

  • Dan, you're right that Free Night tickets aren't subsidized to the theatres - but here in the Bay Area, a lot of the contingent advertising is. Theatre Bay Area (where I run the Free Night program and work as the Marketing Director) gets, by our count, over $200,000 in in-kind advertising every year from sponsors ranging from SF Weekly to BART.

    You're right that FNOT's not a perfect program - we're constantly battling no-show rates and repeat attendees abusing the program - but the analogy to grocery store sausage is potentially a good one. How much does it cost (actually cost) a grocery store to grill up some sausage bits for customers? Even if only, say, 1 out of every 10 people who sample actually buy a package of sausage, you've got to assume the grocery store has worked it out so they're not losing money on the deal. And what about those people who may not buy the sausage, but linger in the store tasting the various free samples, and eventually build up some level of loyalty to that store?

    We have a return rate (to the same theatre) of 1 out of every 3 patrons within 6 months (self-reported by the patron, which I realize isn't ideal). And we have 5 years of survey data that essentially show the same trends of repeat attendance. Theatre companies rarely give over their entire houses (we recommend a quarter of any given house). Given that the usual cost to the company to get a new person through the door is about 4 to 5 times more than the cost of getting an attendee to come back, that's a pretty good return on investment -- especially since often the free seats are for off-nights that might not have filled anyway.

    There are always exceptions of course, and the program doesn't work for every company, every year -- and we honestly can't find much of a pattern. We've gotten incredibly positive feedback, though, over the life of the program, and plan on continuing to provide the service in 2009 and beyond.

    All that to say, it's great to hear that this data's going to be useful -- the next round (assuming we secure the funding) will actually step away from researching Free Night and start looking at how theatre affects audiences of all price points (in part to see if there's a difference in impact based on price -- which, according to our Free Night research, there is not).

    (as I said, but just to say it again, I'm the Marketing Director at Theatre Bay Area and also run the Free Night of Theater program here. Full disclosure :-)).

    By Blogger Clay Lord, At June 22, 2009 at 10:34 AM  

  • After reading the survey, reading this blog post and the comments, I have some questions of my own:

    1) How much did this survey cost?

    2) How many people of those who signed up for free tickets actually showed up?

    3) Of those people, how many came back and paid full price?

    I realize the third question is tough to answer in that individual theater companies would need to report that data, but essentially, that is the one fact that matters - how many came back and paid.

    My big question for FNOT, is why does it need to be free? Restaurant Week does great, but doesn't devalue the product by giving it away. The whole psychology behind Restaurant Week is that the product is so amazing that it can't be given away, but this time, we'll give you a break and if you like it, maybe you can come back on a special occasion. Patrons are conditioned to believe that the meal they are receiving is so incredibly delicious, that giving it away would be unthinkable.

    For my money (of which, TBA has $300 worth), I think the survey missed the mark by focusing largely on studying the impact of art on the patrons as opposed to the economic viability of the program. Art effects people - we know, that's why we do it. What we don't really know is how awesome or useless Free Night is. Would have preferred some more numbers on that as opposed to the emotional response of audience members.

    Anyway, I've got some beef with Free Night, obviously, but I do appreciate Clay giving us his thoughts on it all, and especially Chloe for providing a forum for discussion.

    By Blogger PianoFight, At June 22, 2009 at 12:31 PM  

  • Hi PianoFight,

    To answer your questions, the best I can:

    1) We did 2 separate surveys, one of which was conducted by Theatre Bay Area only in the Bay Area (the intrinsic impact study posted at www.theatrebayarea.org/datapoint), and one of which is a nation-wide study actually conducted by TCG in New York that looks at more traditional survey-type questions, including the frequency with which people return.

    The intrinsic impact study, which was a pilot, was supported entirely by a grant from the Doris Duke Charitable Foundation to the tune of about $40,000. One thing to point out is that none of this burden was taken on by the companies. Another thing to point out is that, while yes this study is now giving us insight into Free Night vs. non-Free Night behavior, the research into intrinsic impact is a longer-term goal of Theatre Bay Area that extends beyond Free Night (and in fact the next phase of the study won't be conducted during the Free Night period at all).

    The other study, conducted by Shugoll Research in Washington, DC with all participants in Free Night nationally, is paid for by TCG, so I don't have access to that hard cost. But again, the companies don't pay a dime.

    2) No-show rates vary wildly across the country (and within our own community). Here in the Bay Area, our no-show rate is about 30% on Free Night tickets on average. Some companies experience much higher no-show rates (80%), and some experience much lower no-show rates (0%). We're constantly looking into and trying to find patterns around no-show rates - as far as we can tell, it's a combination of some or all of the following:

    - geographic location (specifically in relation to the people who actually take the free tickets from your company)
    - pre-show engagement (both by Theatre Bay Area, who sends out reminder emails, and by companies, many of whom choose not to do any pre-show engagement. It won't surprise you to know that companies that do do pre-show engagement generally have somewhat lower no-show rates)
    - show theme (dramas seem to get hit tougher if other factors are in play, such as...)
    - other factors no one can control, such as weather, quality of the day the person had, level of tiredness in the patron, family issues, etc.

    (A sidenote: when asked, no-shows said first and foremost that they "had a conflict," followed by "was sick," "gave them away," "transportation problems," and then "forgot to go.")

    Partially in response to this and other questions and critiques of the program by companies that have participated, and thanks again to monies from the Doris Duke Charitable Foundation, we're building a comprehensive best practices guide for Free Night that will include 20-30 pages of information on how best to pursue patrons, get them in the door, and get them back, as well as sample materials etc.

    3) Of the people who come through the door, we have only self-reported (by the patron) data on return rates - these surveys are taken 6 months after the fact. The national numbers look as follows:

    - Of those attending, 69% were attending the theatre they went to for the first time (in the Bay Area, that number was 70%)
    - Of those attending, 33% returned to that theatre and bought a ticket within 6 months, another 3% bought a full or partial subscription (in the Bay Area the total returning was 35%)
    - 79% of attendees reported buying a ticket to ANY other theatre in the 6 months since Free Night. (and the majority of tickets purchased were full price, with the next-highest rate being for discount tickets directly from the theatre)
    - 34% report they attend theatre more now than they did before attending Free Night, and of those 34%, 84% say that they go more because of Free Night.

    (my post is too long, so I have to separate it...)

    By Blogger Clay Lord, At June 23, 2009 at 11:09 AM  

  • (continued)

    To address your larger concern - we thought long and hard about whether it was a good idea to do totally free when we were formulating this program. And the answer is that we don't believe that a special, packaged "free" event devalues the product in the long-term. When asked, prospective patrons name the #1 barrier to attendance as the price of admission, with the #2 being "I don't want to spend money on something I might not like." What this means, to me at least, is that there are a lot of people who don't know they like theatre because they don't feel like they can spend the money to try it out. So why not give them a gateway into the experience that removes that barrier? I understand your comparison to Restaurant Week, but will also point out that everyone knows what it's like to eat in a restaurant, at least in the abstract -- I've been to a restaurant in my life, even if I've never been to French Laundry. I know the social mores, I know the order of the evening, the peculiar call-and-response of a meal. This sounds silly, but for light-theatre-users, being uncomfortable with the idea of theatre is a big barrier -- especially when coupled with the barrier of price. Our intrinsic impact research indicates that another big barrier to attendance is that a person's social circle doesn't attend -- that is to say, they don't have anyone to go with, share the experience with, feel less awkward with. We believe that Free Night is an incredibly effective tool at breaking down those barriers by making it an event that is special enough to gather your friends for, low enough in cost that that simply can't be a barrier anymore, and savvy enough in regards to information gathering that companies, if they act on it, can follow up aggressively with those primed patrons for future purchase.

    Your membership dues go to support a lot of different programs at Theatre Bay Area, which we very much appreciate. It did not, however, go to any of the costs of conducting these surveys (we are providing the results to the entire field for free as a service for the betterment of everyone). What I would ask you, though, before you discount intrinsic impact completely, is - you and I agree that art impacts people. But can you tell me how much? Can you tell me how much your art affects your audiences? Can you prove your value in a way that non-artists (say, government officials or non-artist funders) can understand? The impact research is very early, but we're excited about it (as are funders and service organizations across the country) because it will allow companies of all sizes another metric for demonstrating their impact on and value to their communities. And at some point we plan on making the tool available at relatively low cost to anyone who would like to use it, so that they too can have an intrinsic impact dashboard for their use.

    I didn't realize I hadn't posted the other survey surrounding Free Night, which shows a lot of the more demographic numbers you're asking about. It's now available at www.theatrebayarea.org/datapoint -- take a look, and I'd love to engage more in this conversation. As I said in my earlier comment, we don't ever believe that any of the programs we run at Theatre Bay Area are perfect -- we're constantly modifying and updating, and coming up with new programs that serve the community better, so all comments are useful and invited.

    Thanks!

    By Blogger Clay Lord, At June 23, 2009 at 11:09 AM  

  • Hi Clay,

    Responded to your comments with a post of my own. Apologies in advance if the tone of it isn't too sweet, but please know that I really appreciate your even responding to my comments (I have never received a response from TBA to any email I have sent, ever), so I really mean it when I say thanks for talking about all this.

    The post is here.

    Looking forward to your thoughts.

    -PF

    By Blogger Carl Benson, At June 24, 2009 at 1:19 PM  

  • Wow, this is great. Thank you Clay for posting this, and for getting the report with more hard and fast numbers out to be publicly available. That alone goes a long way to creating some more communication and clarity on this issue.

    By Blogger Dan Wilson, At June 24, 2009 at 3:15 PM  

  • This comment has been removed by the author.

    By Blogger Unknown, At June 30, 2009 at 10:29 AM  

  • This comment has been removed by the author.

    By Blogger Unknown, At June 30, 2009 at 10:29 AM  

  • This comment has been removed by the author.

    By Blogger Unknown, At June 30, 2009 at 10:30 AM  

  • (PART1)

    I just got back from a wonderful trip to the Bay Area, reconnecting with the community and seeing some fun shows. Something that kept coming up is the online discussion taking place on various blogs regarding Free Night of Theatre. It was wonderful to see such passion on an important topic and watching the theater community go at it, something I haven’t seen since the space crunch. I realize I’m coming in as a bit of an out-of-towner, but as I’m still putting on shows and running spaces in SF I thought I would chime in.

    I’d like to start with a thank you to Chloe and Clay for this forum and information and to all those making their comments to allow for this conversation. This is online community at its best and you should be applauded.

    For the record, I am neither for nor against the general idea of FNOT. I see the value of mass marketing and the importance of getting new audience members into the theatre. I also respect the fact that it may appear that giving free tickets can decrease the perceived value of a product. I have never participated in FNOT as it never timed out with any applicable show.

    There was much discussion about the value of the data and whether a free night of theater is actually a good or bad thing. This was all very interesting and while I do have a few thoughts on those subjects I am more interested in the real economic value of this program and the surveys attached to them.

    Looking at the numbers--

    Over $200,000 was spent on advertisement for a one-day event. I imagine there were other costs to TBA for this event (staffing, mailings, etc.)
    (Clay is it possible to get this amount?).

    All of this netted the following numbers:

    Theatre Bay Area received 2,657 email requests for FNOT tickets.
    – 2,488 requests for 2 tickets
    – 139 requests for 1 ticket

    I add this up to be: 4976 + 139 = 5115 tickets

    Figuring that there was 30% no show—roughly 3581 people went to FNOT in SF with 2649 seeing a show for that company for the first time (74% of the audience and my perceived important audience. It should be mentioned only 18% of the people seemed to have never seen a show before)

    Is this about right Clay?

    That means, just looking at the ad value it cost – almost $56 per person to get them there. That’s amazing. And that number will only go up with the additional costs of the program not listed and doesn’t reflect the $40K for the survey. And if you figure that the goal of this program is to get new people into the theater, the number goes up to $75 (using 2649 people). I can’t imagine any theater or business for that matter, willing to keep with a program with those returns, even if 1/3 of these people do come back. Something’s not working here. The fact that that number is so high tells me one of three things:

    1. This program isn’t marketing well enough to get the word out.

    2. Perhaps a free night of theater is not the best plan to get new people to see shows.

    3. And if the word is out there, a harder-to-swallow conclusion (and echoing Carl): people, we can’t just blame the recession. Appetites have changed. Whether it’s a free sausage or a cheap meal, people want something else. I agree that the theater community needs to address this issue that people don’t go to theatre for their artistic fulfillment. I’ve been to many meetings where the community has ignored this (“are tastes/attitudes changing?”) and instead concentrate on how to get more money given to them (in the form of grants). That model will not work, especially not in this recession. It is crucial that the theatre community discuss this.

    By Blogger Unknown, At June 30, 2009 at 10:40 AM  

  • PART 2

    One final note. I realize that grant money is often forced to be spent on things addressed by the giver, a problem with many of the grant programs. I have to say though it is such a disappointment to see $40K spent on a survey which I fell has little or no value and that there are plans for further surveys and even a database. Perhaps I should say more, but I don’t see how any sized company can use this to help decide on how to get more people to the theatre. I’d love to be shown how I can use this survey. Perhaps as Chloe states it will take time and discussion to fully realize its importance. I think the money could have been used for other more pertinent projects.

    Having been involved in the community for over 10 years I feel it critical for the bigger questions to be addressed. Free tickets and surveys, can’t solve the problem of people not wanting to go to theater, if, they just don’t want to go. I look forward to the conversation.

    Thank you,
    Matthew Quinn
    Combined Artform
    Off-Market Theater –SF
    Theatre Asylum – LA

    PS- Before I was able to post this it looks like the conversation has already begun regarding theatres relevance on TBA’s Blog. I hope more people chime in.

    By Blogger Unknown, At June 30, 2009 at 10:42 AM  

  • Chloe et al...

    The thread on this has seemed to move over to PianoFight's blog.

    They've been kind enough to let me post my most recent reply here.


    http://piano-fight.blogspot.com/2009/07/matthew-quinns-cost-break-down-of-free.html


    Thanks,

    Matthew Quinn

    By Blogger Unknown, At July 2, 2009 at 4:20 PM  

Post a Comment



<< Home