eNEWSLINE - Business Education News from AACSB International

Creating Community: The Measurement of Research

By Eileen Peacock, Senior Vice President and Chief Officer, Asia AACSB International

For many years, AACSB has been defining research expectations as demonstrated by Standard 2 in the 2003 Eligibility Procedures and Business Standards:

"The mission incorporates a focus on the production of quality intellectual contribution that advance knowledge of business and management theory, practice, and /or learning/pedagogy. The school's portfolio of intellectual contributions is consistent with the mission and programs offered."

In 2006, AACSB created a task force to examine ways to increase the overall value and visibility of business school research. The task force's report, published in 2008, affirmed that commitment to scholarship is a crucial component to quality management education. However, it also acknowledged that schools often find it challenging to demonstrate the linkage between the resources used to support scholarship and research, and its subsequent impact. A major recommendation of the report was that the scholarship of the institution be measured in terms of impact, rather than simply a count of scholarly inputs (research papers, conference proceedings, text books, etc.) with respect to stakeholders such as the academic and professional organizations, the community where the school operates, the students and funding organizations.

Following the Impact of Research Report, further investigation and debate among the AACSB community resulted in a focus on research impact instead of pursuing a strategy that establishes "greater academic legitimacy to compete effectively within their institutions for resources." The rationale for this shift was that the current approach often results in too great an emphasis on "top" journals, too much attention devoted to what to count, and indeed the volume of such activity. Historically, there has been little effort to highlight the link between the business school, society, and even organizational performance.

While top journals provide visibility, there are other ways to enhance the value and visibility of a school's scholarship and research. Questions should be raised about the current popular model of an academic's workload being measured in terms of time allocation e.g. 40/40/20 (teaching, research, service). Is this optimal if one wants to start changing measure to those of impact? Time spent is, in essence, an input measure, whereas impact is definitely an output. The report suggests that schools evaluate scholarly output strategically—considering not only an individual's contribution—but that of the school as a whole.

Regarding the question of research classification, let's take an example of some of the research that is focused on the role of an internal auditor under some specified set of conditions. This could influence practicing internal auditors, the profession, or those teaching internal auditing would be the prime audience. So should the focus be discipline-based, or contributions to practice? An academic staff member might ask, "where are the incentives to publish in such journals?" Traditionally, more acceptable outlets are those of the disciplined based type, but the greater question is, where is the most value?

Questions regarding how research can influence teaching should be considered. Generally, it is held that the enquiring mind of a researcher enhances the teaching environment as it guides students into the inquiry mindset. What is learned from research enhances the richness of teaching. However, as the 2008 task force report states "these functions [teaching and research] are not always performed by the same people." So, how does that valuable information get transferred between these two groups?

Research findings should also be used to assist in practice. In some cases this does happen. For example, the development of the theory around activity-based costing was developed from case work done by Kaplan and Cooper. But in reality, much research on practice does not get that visibility, and therefore gets lost. How can business schools use that valuable work to assist the development of practice? These examples show how a change from looking at journals and journal quality to looking at the value of research and intended audiences can demonstrate impact.

In response to increasing levels of discussion among the AACSB community about the ability to measure research impact, AACSB designed an exploratory study to gather insight on the feasibility of this recommendation. Ten AACSB-accredited schools were selected to participate, and were asked over a two-year period to study how they can measure the impact of research while taking into account their school's mission, stakeholder needs etc. If schools had to demonstrate the impact of their scholarship, the first question asked was "to/for whom are we doing this demonstration: ourselves, AACSB, our stakeholders such as students or the community, or for funding sources? Where is the impact noted? Where should it be noted?"

It's important to acknowledge that each school was asked to define their expectations for research impact at the school level and not the level of the individual faculty member. Each faculty member conducts scholarly inquiry to stay current in their field, but the purpose of the study was for schools to look at their scholarship strategically and assess whether the school's scholarship (in aggregate) has an impact on the intended audiences. Admittedly, this was easier said than done, as there was no prescription for schools to define their research expectations. Defining and assessing research expectations was an interesting challenge for many schools involved in the study; it was most likely the first time a school had to come to terms with understanding their mission and its impact on their research focus.

Reading their reports is indeed illuminating, thanks to the diversity of issues raised and types of measure elicited. Schools discussed research that focused on their work in niche areas, and how this approach is not fully appreciated by mainstream researchers. Others discussed what role, if any, the enhancement of undergraduate research initiatives should play in discussing impact. One school focused on measures which helped them benchmark against their peers and aspirant schools while another talked of their efforts in working with local agencies to enhance regional environment and how these could impact that region. This clearly shows that schools can, after some debate, describe their scholarly activity in such a way as to help further define their missions.

Eventually, AACSB published another report: Impact of Research: A Guide for Business Schools. This documents the findings from the exploratory study. It states, "Quite simply, this report is intended as a resource to assist schools ... to better understand the connection between the research activities it supports and the school's mission."

It is recommended that business school leaders internalize the findings of this report and begin to engage their academic staff. It's interesting to show how the exploratory work opened up ideas on how to approach measuring impact as well as the multitude of possibilities for the measures. The movement from a focus on journal ranking measures to impact measures will demonstrate the potential utility of research in its widest sense to those funding institutions.

As the examples above demonstrate, the debate of how to measure research will be a continual one. The bottom line is that with respect to research and scholarship, a business school can create a story of why it paints a particular picture of itself, and how that links with the school's mission. A thorough reading of AACSB's Impact of Research: A Guide for Business Schools will prove to be enlightening to many business schools leaders and their staff and can be downloaded from AACSB's website.

Eileen Peacock
Senior Vice President and Chief Officer, Asia
AACSB International