Credibility Roundtables: Brickman Exectuive Summary

Evaluation of the APME Credibility Roundtables Project
Executive Summary
June, 2004

Prepared by: Alan Brickman

APME's Credibility Roundtables: Catalysts for Change

The goals of the Associated Press Managing Editors (APME) National Credibility Roundtables Project, launched in 2001 and supported by grants from The Ford Foundation, are to encourage editors to act to address the decline in public confidence in news media and to emphasize engagement with the public as a core approach to rebuilding regard for the accuracy, fairness and journalistic values of newspapers. The project encourages journalists to meet with community members in a roundtable format to discuss topics chosen for their potential effectiveness as vehicles for raising issues of journalistic credibility. In the first two cycles of the project 113 news organizations sponsored roundtables events starting in 2001 and running through the first half of 2003, and these roundtables were the subject of this study.

The participating news organizations understood and committed to this model of community engagement, and there was an impressive amount of positive institutional change as a result. The roundtables successfully catalyzed a significant amount of change with regard internal policies and standards of practice, approaches to sources and coverage, and mechanisms for ongoing interaction with the community. There are strong themes in the types of changes that the project successfully generated at the participating news organizations. These include a renewed focus on accuracy, increased sensitivity to diversity issues (including both coverage of minority issues and diversifying the composition of the newsroom), and a broader and more inclusive approach to sources. There are also strong themes in the editors' description of the barriers to change at their newspapers. These include: staff turnover (of reporters as well as management staff that can lead and coordinate community engagement activities), the challenge of establishing an institutional commitment (at the level of the publisher or owner) to sustain these types of community engagement and credibility initiatives beyond the efforts of any one champion, and a variety of resource and budgetary constraints.

Ultimately, it is the readers who determine whether a news organization is credible. It is important to remember that the types of institutional changes that this study was designed to explore are not ends in themselves, but strategies for engaging the community in ways that can ultimately increase reader trust and understanding. Additional research may be undertaken that can conclusively tie the institutional outcomes explored in this study to improved "credibility" as reflected in the impressions, opinions, and behavior of the reading public. This study demonstrates that the roundtable model of community engagement and journalist-community dialogue is an effective and replicable starting point for building linkages with the community and catalyzing concrete change at the sponsoring news organizations.

Study Methodology

APME selected independent consultant Alan Brickman and the University of Nebraska College of Journalism and Mass Communication to conduct an evaluation of the roundtable project. The evaluation was based on an in-depth interview of high-level representatives (usually editors) of the news organizations that participated in the Project starting in 2001 and running through the first half of 2003. The interview focused on basic information about the sponsoring news organization, various features of their roundtable event, and the editors' observations and impressions of the impact of the roundtable. The survey was designed by consultant Alan Brickman in concert with APME to reflect the goals and objectives of the Roundtable Project.

The University of Nebraska-Lincoln College of Journalism and Mass Communication was selected to conduct the survey. Fifteen students representing all four undergraduate grade levels and all three journalism tracks (news editorial, broadcast and advertising) were selected to review the files of existing documentation from the sites and conduct the interviews. In the course of a five-week period during January and February 2004, the students successfully completed telephone interviews with representatives of 94 of the 113 news organizations involved in the project.

It is important to understand that this evaluation was designed to ascertain the extent to which the roundtable events and related activities catalyzed specific types of changes in practice, in policies, and in personnel that are believed to increase reader confidence in the newspaper. The study was not an assessment of the impact of project activities on credibility per se, in that it did not include input from a broad cross-section of community members regarding their impressions of the paper with respect to a number of credibility-related issues.

The full evaluation report, whose principal author is Gloria Bucco of the University of Nebraska and which includes a detailed compilation and analysis of the interviews, is available upon request.

Implementation at the Local Level

In the following section, we have presented information about the sponsoring news organizations and various selected features of their implementation of the roundtable model. This is intended to present an overall picture of the project over the period covered by the evaluation. In addition, this information provides a framework for some aspects of the data analysis presented later in the report.

Circulation

More than half of the papers surveyed had circulations of 60,000 or less (53.2% or 50 papers). Of those remaining, 32.9% (31) fell between 60,000 and 200,000, and 8.5% (8) papers had circulations greater than 200,000. (Note: Five of those surveyed were Web sites or online news sources where circulation could not be calculated.)

Prior Experience with Public Engagement Activities

For the majority of those interviewed (52 of 94, or 55.3%), this credibility roundtable was not the first time they had sponsored a community dialogue or public engagement event with their readers or community. However, for the other 42 news organizations, this was the first such activity.

Roundtable Discussion Topics

Before a roundtable session, each news organization selected a discussion topic that they thought would best stimulate discussion and raise issues of news and newspaper credibility. For the purposes of this evaluation, a list of categories was created so we could group the wide range of topics developed by the sponsoring news organizations. The top three most common topic categories were: diversity and minority issues (18.1%), special issues of local concern (18.1%) and the general credibility of the media (17%). These three were followed by: crime, safety and gun issues (10.6%), education and youth (8.5%), and specific aspects of journalistic practice (8.5%). Remaining were politics and government (7.4%), growth and environment (6.4%) and other topics (5.3%).

The Use of a Facilitator

Realizing the importance of keeping the meeting focused and the comments and discussions on track, and ensuring all those present were given a chance to speak, all papers used some type of facilitator. By far, the majority of those surveyed (69.1%) chose to use an independent facilitator, i.e., one not affiliated with the sponsoring newspaper. Among the independent facilitators, approximately half were paid and the other half volunteered.

Recruitment Strategies and Attendance

A key part of building a successful roundtable is determining who would be invited and how the recruitment would be handled. The most popular recruitment strategy proved to be outreach to targeted community organizations, with 55.3% of respondents employing this method. Editors also sent invitations directly to selected news sources (52.1%). Other methods included: invitations sent to subscriber lists (24.5%), public announcements or invitations in the newspaper (23.4%), invitations to local journalism experts (20.2%), outreach to existing formally established reader groups (19.1%), general public relations or media announcements (4.3%) and use of a professional recruiter (3.2%).

The roundtables events were often designed to include specific active roles for selected attendees while others observed, and then reflected on the dialogue they had just witnessed. This provided opportunities to address not only the discussion topics that framed the roundtables, but also to highlight specific aspects of journalist-community communication . In the surveys, respondents were asked to indicate how many active participants, both journalists and community members, were at the roundtables, and how many observers were present.

Journalists, including both editors and reporters, attended all roundtables. The average number of journalists attending the 94 roundtable events for which we have data was 10, of which an average of approximately half had active roles in the discussion. Eighteen roundtable events (19.1%) had 15 or more journalists attending, and nine sites (9.6%) had 20 or more journalists in attendance. The maximum number of journalists attending a roundtable was 40. At 35 roundtables (37% of the sites), all the journalists in attendance had active roles in the discussion.

The average number of community members attending the roundtables was over 21.8, of which an average of 17 had active roles in the discussion. There were 16 roundtables (17%) that had over 30 community members attend, and only 4 sites (4.3%) that had over 50. Eleven sites (11.7%) had under 10 community members attending. The majority of the roundtable events were designed for maximum active participation by the community members. At 70.2% of roundtables, every community member in attendance had an active role in the discussion.

Additional Reflections on Implementing a Roundtable Event

The representatives of the participating news organizations were asked to reflect the choices or decisions they made in planning and structuring the event that contributed to a positive and productive session, as well as the missteps or aspects of the roundtable that did not go well or as planned. The most common responses were:

Positive choices or decisions:

• Using an independent or trained facilitator (by far the most common response);

• Recruitment efforts aimed at achieving a diverse audience;

• Logistics and participant-friendly site details such as signage, name tags, seating arrangements conducive to discussion, and a welcoming reception;

• Setting ground rules for the discussion;

• Selecting a neutral and easily accessible location;

• Providing food.

Missteps or aspects that didn't work well:

• Not enough time for the roundtable;

• A topic that was timely during the planning stages was old news on the night of the roundtable;

• The need for more communication between those planning the roundtable and the entire news staff so misconceptions about goals could be avoided;

• Problems with facilitation (dominant participants, "axe-grinders", facilitators who talked too much, …);

• Recruitment misjudgments (e.g., not enough regular readers and too many "sources”, under-attendance, …);

• Unexpected developments related to the date or the facility;

• Some logistical issues (seating arrangement, room too small or too big, …).

Impact of the Roundtables on the Sponsoring News Organizations

The APME survey included a number of questions about the impact of the roundtables on the sponsoring news organizations. These questions began with a yes or no answer related to a number of specific types of outcomes and then asked editors to elaborate. This section will present and analyze their responses to these questions.

NOTE: There are several important points to be made regarding how respondents dealt with these questions about impact. First, the survey questions regarding impact focused on changes that are attributable to the roundtable events and to the other activities of APME's project. In a number of cases, respondents answered "no" to particular questions because the specific activities or practices in question were already in place at the newspaper and not the result of the roundtable activities. In the following sections we will try clarify these types of responses whenever possible. Second, although the questions were developed to inquire about different types of impacts (such as policies, practice, coverage, etc.), many of the respondents did not fully understand the differences as expressed in the questions and that we sought to break out. This resulted in some degree of "block voting" across the different questions, which is important to understand in reviewing the compiled responses.

There was at least some impact in nearly all the sites. There were only two of the 94 sites that answered "no" to all the questions about impact. The percentages of respondents answering "yes" to the interview questions concerning the impact of the roundtables on the sponsoring news organizations are presented in the following chart, along with a summary of the issues raised in the discussions of the impact. (Note: In the final evaluation report, there is a detailed compilation of direct quotes from the respondents regarding many aspects of the process of change at their news organizations. In the following chart, we have attempted only to summarize the issues raised in these responses.)

Question from the Interview % of respondents answering "Yes" Summary of key issues raised in the responses

Policies: As a result of the roundtable, did the newspaper adopt any specific changes in its policies or standards of practice?

43% Of the changes that did occur as a direct result of the input received by newspapers at the roundtables, most had to do with training of new journalists, expansion of sources, more news coverage of minority issues, more judicious handling of corrections, and/or a renewed emphasis on accuracy. Many of those who answered "no" to this questions focused on existing policies and standards already in place.

The Newspaper Itself: As a result of the roundtable, were there any specific changes in the newspaper itself, such as new information regularly presented in the paper, new sections of the paper, etc.?

37.2% The structural changes that were made in the newspapers are consistent with the general trends in the survey responses regarding increased accuracy, added attention to minority readers, more "reader-friendly items," and greater emphasis on background and basic news. Many of those who answered “no” either said they did not receive enough input on this point or believed that changes based directly on the roundtable discussions were not warranted.

Journalistic Practice: As a result of the roundtable, has the everyday practice of journalism changed in a significant way across the newsroom (for example: new approach to privacy concerns, expanding and diversifying sources, greater internal collaboration or communications, an improved accuracy program...)?

73.1% The overwhelming majority of the respondents noted a new emphasis on accuracy and additional sources, and identifying these new sources from outside mainstream government and political networks. Again, editors highlighted efforts to include all community members in news coverage, and to be more responsive to the needs and sensitivities of diverse groups. Crime coverage was also mentioned in the context of this question, with one paper creating a database for tracking felonies as a result of the roundtable discussions.

Coverage: As a result of the roundtable, were there any specific changes in newspaper coverage such as what the paper now covers or how it covers certain stories?

63% Many editors believed they had, for the most part, answered this question in their previous responses. Even so, many made an effort to continue to characterize changes to their papers and in their newsrooms. Addressing minority coverage was again a top priority. In general, responses to this question seemed to highlight a variety of issue-specific measures taken by editors as a follow-up to the roundtables.

Newspaper Personnel: As a result of the roundtable, were any new staff positions created at the newspaper (ombudsman, reader advocate, outreach staff, etc.) or were new responsibilities related to liaison with the community added or shifted among existing staff positions?

11.7% Budgetary constraints prevented the predominant majority of newspapers from adding new staff, often despite their desire to. Some papers already had an ombudsman or reader advocate on staff, and a few others had editors or reporters who had added these types of responsibilities to their regular jobs.

New Internal Discussions: As a result of the roundtable, was there any new discussion of newspaper policies or practices, or the reconsideration of an existing policy or practice, that did NOT result in changed policies, but led to an affirmation or a more thorough understanding of the newspaper’s policies and practices?

75.5% This question did not work as we had anticipated due to its unusual phrasing and use of the negative. Many respondents indicated that they did NOT implement changes at the newspapers in response to specific issues raised, and in fact reaffirmed existing policies and practices. It is worth noting that the trends pertaining to accuracy, new sources, and diversity were still present.

Follow-up Public Engagement Events and Activities: As a follow-up to the roundtable, were there additional credibility sessions or other public engagement events scheduled?

36.2% Lack of time and energy, as well as staffing and budgetary constraints, were all cited as reasons for not hosting additional roundtables. Another common reason given was that the editor heading the roundtable effort left the paper, and no one was designated to continue organizing public engagement events.

Involving the Community in the Newsroom: As a result of the roundtable, have you involved community members in news meetings or other daily activities of the newsroom?

33.7% For this question, many "no" responses came from editors who claimed that they have had community members at news meetings for years, and that this was not a new activity that came about as a result of their roundtable. The respondents that indicated they did expand community involvement in these types of meetings did so as a very natural outgrowth of the roundtable event.

New Relationships: As a result of the roundtable, has the newspaper established new relationships with other community organizations or institutions?

40.9% The key word in this question is “new.” Editors were asked if any new relationships had been established, and although the majority answered “no,” many indicated that the roundtable activities strengthened these relationships. Responses from those who did establish new relationships as a result of their roundtable show great enthusiasm regarding their expanded community interaction.

A New Leadership Role: As a result of the roundtable, are individuals at the newspaper playing a new leadership role in the journalistic community with regard to promoting public engagement or credibility initiatives with other news organizations or associations?

26.9% The vast majority of newspapers did not take on a new leadership role in their journalistic communities, for a number of different reasons. Many are the only news outlet in their area and the notion of leadership in their "journalistic community" does not strictly apply. The editors that indicated that they were playing an increased leadership role often described partnering with statewide associations or other community journalism initiatives.

Other Changes or Impacts: Are there any other changes, impacts, or outcomes at the newspaper and/or regarding its relationship with the community that you can attribute to the activities related to the Roundtable?

45.6% Many of the editors took this opportunity to express their strong overall satisfaction with the roundtables and to reiterate other areas in which they perceive an impact. Some additional accomplishments that were described here include: increased goodwill, a closer relationship with the community, the positive experience of the roundtable for all participants, changes in attitude among the roundtable attendees, better understanding by both journalists and community members, and the enhanced reputation of the paper.

Additional Analysis of the Impact on the Sponsoring News Organizations

Because the overall number of project sites for this study is relatively small (N=94), there is a limited amount of analysis that can be done for various identifiable subgroups with a high degree of statistical significance. However, we wanted to extract some data of this type that may be useful to individuals or organizations seeking to replicate the roundtable activities in new sites. This analysis is offered with the caveat that the following conclusions have not been rigorously vetted in a statistical sense, but point to possible patterns and lessons that can be considered when newspapers plan the implementation of future public engagement activities.

a) The appearance of a more widespread impact among smaller papers: There are 50 news papers (53% of the total) that have circulations of 60,000 or less. We looked at the percentages of these newspapers who answered "yes" to the various questions about the impact of the roundtables, and they are, across the board, higher than for the total sites in this study. The reasons for this will have to be further explored, but it might relate to two factors: that a greater percentage of the larger newspapers already have the policies and practices I question already in place, and therefore answered "no" more often, or the smaller newspapers are more flexible and can change more easily than their larger and possible more bureaucratic counterparts. The following chart presents these percentages.

Percent indicating "Yes"

Potential impact Smaller newspapers

(N=50) Total

(N=94)

Changed policies 78.0% 42.6%

Changes in the newspaper itself 70.0% 37.2%

Changes in journalistic practice across the newsroom 92.0% 73.1%

Changes in newspaper coverage 84.0% 61.7%

New personnel or new assigned responsibilities 52.0% 11.7%

New internal discussions not resulting in major changes 90.0% 75.5%

Follow-up public engagement events and activities 68.0% 36.2%

Involving the community in the newsroom 62.0% 33.7%

New relationships with community organizations 78.0% 40.9%

New leadership role in journalistic community 64.0% 26.9%

b) Some apparent differences when the discussion topic was journalism per se: There were 24 sites that chose as their discussion topic the general credibility of media or various aspects of the practice of journalism, rather than coverage of a societal issue such as crime, education, the environment, etc. For most of the impacts explored in the survey there were no differences between the percentages of these sites and the sample as a whole. However, with regard to changes in policy, new personnel, and a new leadership role in the journalistic community, there were noteworthy differences in the prevalence of change, as shown in the chart below.

Percent indicating "Yes"

Potential impact Topic:

Media or journalism

(N=24) Total

(N=94)

Changed policies 70.8% 42.6%

New personnel or new assigned responsibilities 41.7% 11.7%

New leadership role in journalistic community 54.2% 26.9%

c) No apparent differences in impact related to other selected factors: There are a number of factors differentiating the roundtable sites that one might imagine would generate a greater prevalence of impact on the sponsoring news organizations. Such factors include: whether this was the first public engagement activity or event for the news organization, whether the roundtable involved an independent facilitator, and whether that facilitator was paid or volunteer. However, when you examine the percentages of these newspapers responding "yes" on the various types of impacts explored in the survey, there were no significant differences from those of the sample as a whole.

d) The impact of attendance: The roundtables were attended by an average of 10 journalists per site (with a range of 1 to 40) and an average of 21 community members (with a range of 3 to 202). There were 41 sites that had 10 or more journalists attending, and 37 with 21 or more community members attending. For the most part, the percentages of sites with greater journalist and community attendance at their roundtable events (as defined by the cut-offs mentioned above) responding "yes" on the various impact questions were the same as for the sample as a whole, with several noteworthy exceptions. When a roundtable event had 10 or more journalists attending, there was a greater chance that there would be a follow-up public engagement event (49% to 36% for the sample as a whole). The larger attendance of journalists may reflect the news organizations' commitment to public engagement, hence the increased possibility of follow-up. Also, when there are 10 or more journalists attending, there is a slightly lower percentage of respondents who answered "yes" to the questions regarding involving community members in newsroom meeting (22% to 34% for the sample as a whole). This may reflect the dynamic, on which we have already commented, that "no" often means that these practices are already in place and not the result of the roundtables. The institutional commitment reflected in the strong attendance by journalists would suggest this is the case. Finally, there was one type of impact that was significantly more prevalent among the sites where the roundtables were attended by more than 21 community members. In these sites, there was more evidence of changes in coverage by newspapers than in the sample as a whole (81% to 62%). Because respondents may have interpreted this impact to include diversifying news sources, the greater attendance by community members may have catalyzed this outcome.