An Assessment, From a Downtown Perspective, of Economic Impact Studies Done on Downtown Arts Venues That Use I-O Economic Models (Multipliers) cont’d.
The Critical Geographic Problem. Downtowns, even in NYC, Chicago or L.A., are geographically far smaller than counties. Consequently, there is an inherent mismatch between the geographic area the I-O model impact studies utilize and the one downtowners are most concerned about. Furthermore, the nature of the available data needed to construct these models precludes their being built for impact areas smaller than a county. While there are data available for zip code areas — a geographic unit sometimes useful for looking at large downtowns — experts in their use strongly recommend that analyzing a single zip code absolutely should be avoided. Instead, they advise that the zip code data should mainly be used to look at custom defined geographic regions that are not congruent with one or more counties, though sometimes they also can be used to cover part of a county (or counties) if enough zip codes are to be analyzed (7). C3D, for example, has done some studies on arts organizations that use a multiple zip code approach to define their impact areas (8).
According to Ben Davidson, Americans for the Arts’ (AftA) senior director of research services: “Our I-O models are generally constructed at the county level (obviously), and then aggregated for larger regions. But our proprietary model also allows us to customize it to specific local economic characteristics such as a city-based lodging tax“ (9). Each model is the basis of an Excel spreadsheet “calculator” that can estimate the impacts of a specific arts venue or of a community’s entire arts industry on the number of full time jobs, household incomes, local and state tax revenues.
One probably unintended and undesirable consequence of using county-centric I-O models is that the related off-site expenditures of county residents are often basically discarded and the analytical spotlight is focused solely on daytime and overnight tourists. The explanation given for this exclusion in a study done for the Los Angeles County Museum of Art (LACMA), which is similar to what can be found in many other large arts venue impact study reports, is that:
“Ancillary spending of local visitors (i.e., residents of Los Angeles County) is excluded, since these visitors would have spent these same monies elsewhere in the region, making their expenditures related to LACMA a displacement of local spending and not new spending attributable to the Museum.” (10).
For downtowners, a significant percentage of their counties’ residents will be in their trade areas and very important customers. They absolutely cannot be ignored. Another fact that cannot be ignored is that making a downtown again successful means it must recapture the expenditure dollars of its trade area residents. PC puffery may attempt to hide that fact, but its veracity will remain unrefuted. Information about an arts venue’s ability to draw trade area residents downtown and their demographic characteristics, lifestyles and spending patterns (how much and for what) would be very valuable pieces of impact information for downtown EDOs and merchants. There are surveys of arts audiences that provide such information. For example, the audience surveys done by AftA, which are done on site and only ask a limited number of questions, has provided such information about many arts venues and, in many ways are a model of how they should be done.
The heavy emphasis the I-O impact studies place on tourists also can be harmful. It can lead to downtown businesses over targeting tourists in their marketing and merchandise selection to the detriment of local shoppers, with seriously negative consequences in terms of sales and customer relations. It can also help generate bad town-tourist vibes that, in turn, can have undesirable political consequences. Over the past decade, DANTH has visited a number of small and medium-sized communities with penny-wise pound-foolish growth strategies: local merchants and politicians mistakenly placed far more attention and resources on attracting more tourists than on improving their abilities to serve local residents. Of course, merchants that fail to well serve their resident customers are not likely to succeed with tourists.
Arts/culture tourism certainly can be a valuable revitalization asset, but not every town can be an arts tourist mecca. As the growing success of downtown Valparaiso’s entertainment/hospitality niche is demonstrating, a strategy targeting that market segment is probably best used when local residents are also being well served by downtown merchants. Getting a good handle on the proportion of downtown retail, hospitality and entertainment sales that arts tourists are most likely to provide can help determine the level of attention and resources it deserves.
Audience Expenditures: It’s Critical to Get Good Data for This Critical Variable. Besides arts venue expenditures, the other variable that arts venue I-O model impact studies start from is audience expenditures. It also is a variable that, in and of itself, is highly likely to be of considerable interest and value to downtowners — even without the benefit of the information an I-O model can provide about how it is respent. Moreover, in many, perhaps most cases, downtowners will benefit more from audience direct expenditures than from the arts venue’s direct expenditures. Both downtowners and I-O models need or want good data on audience expenditures. The I-O model uses estimates of audience expenditures provided by other sources as an input. For downtowners, audience expenditures represent a very important impact. Downtowners do not need, nor can they use, an I-O model to get reliable and accurate information about audience expenditures.
Nonetheless, it is often a very troublesome one to research. The best way of obtaining reliable, acceptably accurate data on how consumers spend their money is to survey them. However, such surveys require substantially more methodological skills and resources than most other kinds of surveys. For many consumer items, the relevant information must be obtained close to the points in time when the expenditures are made, lest memories of them quickly fade. For example, the Bureau of Labor Statistics (BLS) probably sets the gold standard for this type of research and it has half of its 14,000 survey respondents keep diaries of their expenditures and interviews the other half 5 times over 15 months.
In contrast, too many of the arts impact studies do a secondary analysis of surveys conducted by an organization other than the one doing the impact study. These surveys are seldom done to BLS’s methodological standards and usually have far fewer respondents. Their ability to establish the causal connection between respondents expenditures and their arts venue attendance is usually weak or non-existent. Surveys done in the same community can produce very dissimilar expenditure estimates, leaving their wiser clients to be uncertain about their accuracy and reliability.
Forced to used surveys designed by others for other research objectives, the impact analysts often must make numerous assumptions or stipulations if they are to be able to use the “other research house” surveys for their own arts venue related research purposes (11). For example, the original survey may have been of the expenditures of all tourists visiting the community. The secondary analysts must figure out how to deal with an impact attribution problem: how much of this type of spending can be attributed to the tourists visiting the arts venue whose impacts are being studied, but who have not been surveyed? Of course, there will be other unanswerable issues such as: are the venue’s tourist visitors sufficiently similar to those tourists who do not visit the venue that the secondary use of the data is warranted?
Such impact research thereby becomes more and more of a modeling exercise, with each assumption or stipulation bringing in another unknown error factor and their interaction being potentially exponential rather than additive. For example, in the LACMA study:
“(V)isitors who have traveled from other parts of California, as well as from other states …are assumed to spend at least one additional night in Los Angeles County in order to visit LACMA…..International visitors to Los Angeles …are also assumed to have spent one additional day in the region in order to visit LACMA” (12).
The average expenditures for one day for these two types of tourists, as determined by the “outside” general survey of tourists, were then applied to estimate the economic impacts of the comparable tourists who visit LACMA.
Of course, most researchers and decision-makers live in a world of imperfect data and limited financial resources for obtaining the information they would like to use. Doing the necessary survey work may be beyond their allotted time and financial resources. Very often, they then find that making assumptions enables a kind of modeling exercise that, though admittedly imperfect and capable of only yielding ballpark type findings, still may promise sufficient enlightenment to be worth doing and for its findings to be seriously, if cautiously, considered. Such exercises, however, should come with some standards and responsibilities. For example:
- Are the assumptions really reasonable? Re the LACMA study’s assumptions: Are tourists really likely to spend a full day or most of it at LACMA? Could they not also walk to the La Brea Tar Pits next door, or the other museums nearby on the “Miracle Mile or shop at The Grove just a 15 minute walk away– or get in their rented cars to go look at the Hollywood sign or the beach scene in Venice on that same day? Are they really spending a day in town just because of their visit to LACMA – so the motive not the behavior is the justification for the assumption? Or are they going to LACMA to fill part of a day they already planned on spending in LA and they then plan on also eating dinner, say at Spago, because they are not only are art lovers, but also restaurant tourists, etc.? In a world of multi-tasking, is uni-motivation for a day’s schedule really reasonable? Why then should a full day’s spending be attributed to the LACMA visit if its visitors – or at least a good number of them –are likely spending only a fraction of the day there and going to other destinations they might highly value in the remaining hours? Should portions of their daily expenditures be allocated to these other venues? If so, then by what formula? Will the assumptions produce answers that are sufficiently accurate to be usefully informative?
- Are the consumers of the research being told about its data and methodological limitations and overtly adequately cautioned about its conditional value and utility?
Expenditure estimates based on iffy assumptions and that arrive without adequate and overt cautions about their use should be treated as probable horse puckey.
The LACMA study also provides a good example of just how much estimates of visitor expenditures can vary in one community, in this instance Los Angeles. Its analysis uses the following data, based on research published by TNS TravelAmerica in 2010, about visitor off-site spending per day: $53 those from Southern CA; $75 those from other places in CA; $75 those from other places in the US and $116 international visitors (12). However, the authors then note that:
“These estimates are conservative. In comparison, the estimates used by the Los Angeles Convention Center and Visitors Bureau for international visitors are at least four times as high, and those for domestic non-California visitors are five times that shown in the exhibit” (13).
The variation in these survey findings speaks more to the variation in their researchers’ skills and methodologies than to visitor expenditure patterns.
Realizing that its I-O models would be far more accurate and useful with accurate and reliable estimates of audience expenditures, AftA has made an enormous and impressive effort to get to surveys of arts audiences completed in each of the study areas for which it has generated an I-O model. The arts induced daily expenditures reported by AftA for the nation and in its 139 study regions are considerably lower than those cited in the LACMA study, though they still show a good deal of variation. Nationally, nonprofit arts attendees averaged arts induced daily expenditures of $24.60 per person (admission costs excluded) in 2012, with local audience members spending $17.42 and non-locals spending $39.96 (14). As can be seen in Table 6, the average resident daily expenditure across the six study region groups ranged from $16.55 to $21.89, with the highest community average within each group ranging from $24.69 to $39.39. Much the same is the case for non-residents, though the expenditures are, as to be expected, about twice as high. (It should be remembered that the non-arts related downtown expenditures of local residents may well easily erase this discrepancy.) The average individual expenditure ranged among the study region groups from $32.26 to $53.95, with the highest community average within each group ranging from $49.97 to a Colorado ski resort town’s $191.81.
AftA’s survey data across 139 communities and the data cited by the LACMA report indicate that there is so much potential variation in visitor daily expenditure estimates that data not gathered specifically for and in the subject arts venue is likely to be too error prone to warrant any useful and actionable conclusions. Secondary analysis of someone else’s survey of guest expenditures certainly should not be the preferred way to go.
For Downtowners, Audience Surveys Provide Much Richer Information About an Arts Venue Impacts Than I-O Models. Downtowners in the communities that did AftA directed surveys and have arts venues in their districts are likely to find the information collected and the way it is presented to be very useful, indeed. In these instances, the data describe the total arts induced spending that arts visitors bring along with them into the larger community (e.g., the large city, county or region) when they visit downtown arts venues, but not what they spend downtown. In other words, these data detail the sizes and types of spending pies (e.g., meals, clothing, gifts, ground transportation, etc.) downtown merchants can go after (see Table 7). They do not address how big a slice downtown merchants have won of each pie, though the survey has established that the respondents have visited the downtown, and the merchants’ geographic proximity should give them some competitive edge. This is similar to the information benefits merchants in downtown North Adams can get from the I-O model C3D used because AftA survey data for comparably sized communities were entered into that model.
Importantly, the survey data detail a number of other interesting audience characteristics, e.g., the visitors’ household incomes, education levels (see Table 8) and their reasons for visiting the community. For the vast majority of downtown business people and EDOs, these surveys provide much more useful and simple impact-related information (the characteristics and spending patterns of people brought downtown are the impacts, the arts venue is the “impacter”) than any I-O model can. Undeniably, the I-O models cannot function properly without them, since data about audience spending are a critical initial impacting force variable.
Looked at another level, the AftA data on the 139 study regions constitutes a unique database in which the audiences of many more than 139 arts venues (study areas can contain more than one venue) have been surveyed using the same questionnaire and in similar settings. It can facilitate the drawing of general conclusions about the impacts of arts venues. For example, Table 7 indicates that arts venue induced spending is most likely to go for “need” type purchases, such as food and drink away from home and lodging, while apparel and accessory shops might not win as many sales dollars or benefit so much from the arts venue’s downtown presence. Table 8 shows just how strongly arts venues attract well-educated visitors. True, these conclusions were long generally held within the economic development community, but never before has such strong systematic evidence been gathered in their support.
Downtowners and their EDOs also can learn a lot from AftA’s survey techniques:
- Survey people on-site at the venue being studied
- Don’t ask too many questions (but make sure that the respondents zip code is one of them)
- Have a lot of respondents
- Do it over enough calendar time to account for variation in events and weather
- Make it easy for people to respond by asking about recent situations and actions, about things that are easy to know or remember, e.g., about purchases that were made recently.
These are also characteristics that should be found in the surveys downtown EDOs do of people employed in their districts or attending a college there.
A Closer Look at the Audience Expenditures Impact Attribution Problem. Impact attribution is a problem that plagues many entertainment niche element impact analyses, not just those in the arts. It derives from the essential tasks of logically establishing and/or measuring the causal linkage between an entertainment venue and some effect, sometimes when compared to other potential impacting factors or agents. For example, in earlier articles in this series, questions arose about:
- How much of the improvements on Division Street Plaza (in Somerville, NJ), could be attributed to its conversion into a pedestrian mall and how much was the result of the general revitalization of its downtown?
- How much of the revitalization of Manhattan’s Upper West Side, especially the Lincoln Square area, was due to the creation and growth of Lincoln Center and how much was due to the revitalization of Central Park and/or the area’s proximity to the Midtown CBD and strong transportation assets?
A similar problem also frequently arises regarding the spending of arts venue visitors, a critical variable for I-O impact analyses: how much of their local spending can be attributed to their visit to the venue? This was, for example, the challenge that prompted the LACMA study to make various assumptions about non-resident museum visitor spending. Making assumptions is one approach to establishing the causal linkage. A more effective one is to undertake an audience survey structured so that the respondents essentially report the causal linkages they have made between their arts venue visits and the local expenditures. Unfortunately, the impact attribution problem also can crop up in such surveys.
Americans for the Arts, as an advocacy organization, wants to show that arts venues and local arts industries provide strong economic benefits for their communities. It knows that its survey respondents are arts venue guests and so they just could be asked about their purchases in the community that day. But, their concern about arts impact attribution apparently is so strong that their survey asks the following two questions:
“2. Which of the following best describes your primary reason for being in this community today?
— I am a full-time resident/I live here
— I am a part-time resident (e.g., I own a vacation home)
— I am here specifically to attend this arts/cultural event
— I am here on a vacation/holiday
— I am here to conduct business (e.g., meeting)
— I am here for a combination of business & pleasure
— I am here on personal business (e.g., wedding)
— I am here to visit friends or relatives who live here
— Other (Please specify)”
“5. List below the estimated amount of money that you and the members of your immediate travel party have spent or plan to spend in this community specifically as a result of your attendance to this arts event. Remember to include money spent before, during, and after the event. If exact figures are not available, use your best estimates.
- Admission tickets to this event
- Refreshments and/or snacks purchased while at this event
- Food, drinks, or meals purchased before or after this event (i.e., at a local restaurant)
- Souvenirs, gifts, books, recordings, and/or art
- Clothing or accessories specifically for this event
- Local transportation (e.g., gas, parking, tolls, rental car, taxi or bus fare—not air fare)
- Child-care specifically to attend this event
- Overnight accommodations because of this event (e.g.,hotel, motel, bed & breakfast)
- Other (Please specify):” (underline added) (15).
Question 2 allows residents and non-residents to be identified, though that can be equally achieved by another question that asks for the respondents residential zip code. More importantly, it allows the non-residents who are visiting specifically to attend an arts/cultural event to be identified. That helps measure the magnetism of the arts venue to attract tourists to the community. In building a downtown as a tourist center, venues that are strong tourist magnets are very important assets.
The question also enables non-residents who are visiting for other purposes to be identified. For these visitors, the arts venue is not a magnet, but more of a potential amenity. The arts venue, for them, makes the downtown “stickier,” i.e., it can pleasantly occupy visitors’ time and makes them want to stay longer and return. Successful downtowns are sticky, especially those targeting tourists. One well-known axiom about tourism is the “four times one rule, ” i.e., that “people will visit your community if it has activities that interest them, and that keep them busy four times longer than it took to get them there” (16). Entertainment venues are often recommended by tourism experts as good ways to make a downtown stickier.
Notice that the differentiators being looked for here are specific motivations; surveys are essential for capturing information about them. The reported behaviors of traveling to the community and attending the event cannot do the job by then turning into assumption burdened surrogate variables.
Q5 aims at establishing a very clear causal link so AftA can claim their findings show: “The total direct expenditures made by arts and culture audiences in each participating study region as a direct result of their attendance to nonprofit arts and culture events…” (underline added) (17). The question enables the art venue visitors — not the impact analysts — to make the connection between their expenditures and their visits. The arts impact attribution is very clear.
Downtowners, of course, will wish that the AftA surveys had asked about expenditures in the arts venues’ downtowns, something they can fix when they do their own surveys of their arts audiences. For some downtowners, the filter AftA used to establish the impact attribution may be too clear, perhaps finding that the wording in question 5 would exclude purchases that they would want to know about and see as causally related to visits to the arts venue. For instance, what about a desk or antique lamp purchased in the downtown that trip , but shipped to the home of the arts event attendee? Q5 does not ask about home furnishings or antiques. Also, what does “a direct result of your attendance” mean? Is the wording clear enough that respondents will not be confused or puzzled? Does the article of clothing purchased have to be worn at the event or simply purchased on the trip to attend the event for it to be counted? However, issues about question wording are, of course, endemic in survey research. Nevertheless, while the AftA surveys are understandably not perfect for downtowners, downtowners can learn much from their methodology and from the findings of all the surveys done in communities of comparable size to their own.
That said, it is still necessary to also keep in mind that surveys asking about how people spend their money are very error prone because of respondents’ memories and their concerns about confidentiality. Items that cost a lot ( e.g., a TV, computer or car) or are repetitive payments (e.g., rent, car payments) are best remembered. Smaller transactions are less well remembered. Survey questions that ask for expenditures in categories, such as the AftA survey asks, essentially require respondents to work, to remember individual transactions and then add them up in their heads. Consequently, the data produced by AftA type surveys are likely to have more erroneous responses, the extent of which cannot be determined. Nevertheless, they still may be the best information we can get about arts audience expenditures and still useful if we treat them as pretty good ballpark estimates, rather than some kind of precise statistical photograph of the real situation. Such estimates very likely will be good enough to meet the needs of decision-makers who are thinking in the ordinal measurement terms of everyday language (e.g. none, some, most, all) that inherently lack rigorously defined boundaries.
The essential question of whether or not such data are good enough really can only be answered by their potential end users and the people to whom they are accountable. For those of us who live in a world filled with imperfect data, this is not an unfamiliar question.
Up Next: Part 3 of this article will focus on analyzing the economic impacts of downtown parks.
7. Telephone conversation with Implan’s technical support.
8. See, for example, its research on the Casita Maria Center for Arts and Education (Bronx, NY) at: http://web.williams.edu/Economics/ArtsEcon/Casita.html
9. Quoted from an email message to the author
10. Christine Cooper, Shannon M. Sedgwick, and Somjita Mitra THE TRANSFORMATION OF LACMA: AN ECONOMIC IMPACT ANALYSIS. Economic And Policy Analysis Group, Los Angeles County Economic Development Corporation. January 2014, pp.17, p.9. (Hereafter referred to as LACMA study,)
11. For a good example, see: Economic Development Research Group, “The Economic Role and Impact of Lincoln Center,” September 2004, pp.49, p.18
12. LACMA study, p. 11-13
13. LACMA study p. 12
14. Americans for the Arts, “Arts Facts…Spending by Arts Audiences.” http://www.americansforthearts.org/by-program/reports-and-data/legislation-policy/naappd/arts-facts-spending-by-arts-audiences-2012
15. Americans for the Arts, “Arts & Economic Prosperity IV: National Statistical Report,” pp. 377, p. C-27
16. Roger A. Brooks, Maury Forman, “Twenty-five Immutable Rules of Successful Tourism,” Kendall/Hunt Publishing Company, 2003, pp. 55, p. 46.
17. Americans for the Arts, “Arts & Economic Prosperity IV: National Statistical Report,” p. B149