Toward Objectivity in Faculty Evaluation

Here’s a proposal for a new comprehensive system to evaluate faculty performance.

By H. W. Elmore

The productivity of faculty members often figures prominently in annual evaluations, post-tenure reviews, and decisions about tenure, promotion, merit pay, release time, awards, and other kinds of recognition. Yet the procedures and instruments that institutions use to assess productivity and merit vary, leaving little that unifies the evaluation and rewards system in U.S. higher education. To date, no movement has emerged to standardize the process and maximize objectivity while linking productivity in an empirical fashion to rewards. Disputes and allegations of administrative bias and abuse of the faculty evaluation process often result in dissatisfaction among faculty, low morale, grievances, and litigation. Simplifying the evaluation process and making it more objective would obviously be worthwhile. But changing policies and practices that have evolved over many years on college and university campuses will require effort.

Evaluation Systems Now in Use

On any single campus, the faculty evaluation process may include one or more components with instruments developed to measure specific areas of productivity. For example, professors must compile substantial documentation for promotion and tenure even though the same information is included in annual reports. Similarly, teaching, research, and service awards require applications of various levels of complexity, letters of recommendation, and selection committees. And department chairs often confer reassigned time, especially at undergraduate institutions where teaching loads are heavy, to allow selected faculty to devote more time to research. Many colleges and universities also use merit pay as a financial incentive for exemplary faculty. Thus faculty may be evaluated repeatedly on the same performance standards using multiple instruments to determine rewards that are related but not linked, resulting in redundant effort and inefficient use of time.

Because humans are social animals who form alliances, make friendships, and develop biases, the evaluation process is often subjective and may be affected by extrinsic factors. Indeed, there are many junctures in the steps leading to recognition or reward where individual administrative perception and bias can affect an evaluation. The subjectivity of established evaluation practices may lead some faculty members to conclude that their evaluators are unfair and the process is flawed. If many faculty members feel this way, discontent, erosion of morale, and a decline in the quality of the work environment will result.

Moreover, it is ironic that systems designed to measure productivity can become so complex that they unintentionally lead to a decrease in performance. Writing evaluation reports is labor intensive and consumes large blocks of time among professors under evaluation, faculty committees, department chairs, and deans. Developing a sleek, simple, and objective evaluation system would thus increase productivity and greatly enhance faculty morale by minimizing the possibility of unfairness.

Defining Merit

A major problem in discussing comprehensive faculty evaluation is how to define merit. Although opinions vary, viewpoints tend to sort into two major categories. One school of thought measures merit qualitatively. Under this system, the quantity of work performed is not as important as its quality. Only the most outstanding work—for example, performance in highly respectable areas of academic endeavor, publications in prestigious journals, or books that receive critical acclaim—is regarded as meritorious. Routine tasks such as advising, assessment, outreach, recruitment, and service are seen as “normal duties” regardless of the amount of time devoted to them, diminishing their value as worthwhile efforts.

The second school of thought stresses quantity and relies on performance benchmarks, with accomplishments above a baseline representing increasing levels of merit. Work completed in an area considered essential to the success of a college or unit is viewed as an incremental component of productivity that contributes to the measure of merit. The magnitude of a faculty member’s productivity is assessed by tabulating overall performance across all areas of faculty work rather than evaluating only some types of work. So performing strongly in routine areas of the faculty workload contributes to a meritorious ranking.

Such a system encourages faculty to take all aspects of the faculty workload seriously, decreasing the tendency among those who perform well in highly visible areas to regard mundane aspects of faculty work as beneath them. Aptitude and interest determine the main thrust of a faculty member’s flexible work plan, and productivity in any area can result in rewards.

Some existing evaluation systems create an illusion of objectivity. For example, Developing a Comprehensive Faculty Evaluation System: A Guide to Designing, Building, and Operating Large-Scale Faculty Evaluation Systems, by Raoul Arreola, suggests precision and objectivity by generating a number—the overall composite rating—that is rounded to the second decimal place. Yet the values used to calculate the number are assigned by students, peers, and administrators, making them subjective. As a result, the rating itself is subjective. To eliminate subjectivity from the merit system, empiricism and objectivity must be maximized.

Doing Away with Subjectivity

The level of merit should be determined through quantitative and qualitative assessment of faculty work using the most empirical and objective means possible. It is, however, difficult to develop a definition of merit that is amenable to objective evaluation. The value of work has to be measured, and value has qualitative and quantitative components, which complicates the calculation.

The best way to evaluate quality objectively is to establish several rankings for different types of work and to give them consensus values, or numbers reflecting values assigned by faculty through consensus. For example, one might place three columns beside each work item representing accomplishments such as journal articles or books published, students supervised, community outreach activities, and so on. The first column would contain the number of items completed, the second would show the consensus value previously agreed upon by the faculty, and the third would contain their product as merit points earned. Journals or other outlets for distributing scholarly work could be sorted into categories based on criteria decided by faculty, such as reputation or stature. In a three-category system, each item published in the most prestigious category would be multiplied by a consensus value of three; in the second category, the multiplier would be two; and in the third, it would be one. This procedure would be carried out for each work item in a comprehensive list.

A spreadsheet could be used to calculate the total evaluation points. During the year, faculty members could enter information into it as they finish work, which would give them immediate feedback about the impact of each item on their evaluations. At the end of the year, the sheet containing the tabulated merit points could be submitted to the chair as an annual report.

The total points professors earn could be sorted into percentiles or quartiles to determine the level of performance that deserves reward. Decreasing levels of merit pay might be awarded to the top, second, and third quartiles with nothing to the fourth quartile. The evaluation system could be unified by coupling the quartile ranking to all rewards. For standing awards for accomplishments such as meritorious teaching, the subset of the work items dealing with teaching could be added together to identify the person with the highest total as the recipient. That way, nominating and review committees could be eliminated. Other rewards might include workload flexibility with a reduced teaching load, reassigned time, or awards for service or research.

This system would permit clear identification of exemplary professors, minimizing subjectivity. A biased or vindictive administrator would have no way to influence a process in which all work performed would contribute to a faculty member’s evaluation through a system set by faculty; there would be no cap on the total points that could be accumulated. Assigning consensus values to work would assure faculty members that performing certain tasks or achieving specific accomplishments would result in a definite reward. Moreover, the system is versatile in that it could be adapted according to local values, needs, and culture.

The listing of work items and their corresponding values must, however, be comprehensive. Once the components of merit are listed, work not deemed important enough to be itemized would probably be omitted by faculty when they build their work plans. If administrators were unwilling to consider routine tasks such as student advising, recruitment, outreach, and service as contributing to merit, they would have to shift budgetary resources to support these areas in some other way.

Hurdles

The system I’ve described here would probably have the greatest impact on the power of chairs and deans, who might object that it reduces their influence on campus governance and usurps their prerogatives. But the system’s ability to minimize the effect of philosophical or personal differences between faculty members and chairs or deans would seem to outweigh this concern.

Establishing a list of work items and the value of each through a voting process among faculty would undoubtedly involve lengthy debate and compromise. But faculty participation leading to a consensus means that, in the end, the system would be seen as equitable. Few grounds should exist to criticize the process, question the sum of evaluation points, justify feelings of disenfranchisement, or initiate grievances or litigation.

I am not aware of an institution that has adopted such a comprehensive, innovative approach, although many institutions practice fragments of the process. Evaluation procedures have evolved over a long time on most campuses, and many faculty and administrators like the existing systems.

A concerted effort would be necessary to replace established practices, but potential benefits abound. Faculty and administrators would have to spend less time on the evaluation process, which would lead to greater productivity. For administrators, the annual evaluation process and the award of merit pay and other perks would be reduced to a simple process of confirming the accuracy of claims made by faculty members about their achievements and tabulating points.

Faculty could plan their work in a deliberate and informed manner with a clear understanding of the value the university attaches to each task performed. Arbitrary decisions on the value of contributions could not be made after the fact resulting in unfavorable evaluations. Morale would increase and, perhaps most important, the faculty workplace and the student learning environment would be improved.

H. W. Elmore is professor of biological sciences and associate dean of science at Marshall University.

 Printer-Friendly Page