Nonprofit Program Evaluation
How it works and why you need to evaluate your program
Making evaluation a priority
What’s the one thing nonprofits most overlook in managing, planning and strategizing for the future of their organizations?
If you ask Chris Allers, executive vice president of the United Way of Metropolitan Atlanta, he’d probably tell you it’s program evaluation.
Often relegated to the bottom of the budget barrel—or even considered a non-essential line item on grant request applications—funds allocated for evaluating the effectiveness of nonprofit programs are not always a priority. And Allers, for one, believes they should be.
The United Way of Metropolitan Atlanta has, over the past 10 years, gone from funding 70 programs to more than 400. During that time, the organization’s method of measuring programmatic success has evolved. A system that was once output-based has grown into a full-fledged outcome model—a process most grantees now agree is a value-add.
Although there is a new breed of nonprofits that consider self-gauged evaluation part of their work from the get-go (Atlanta’s Whitefoord Community program is one example of a nonprofit that wove evaluation into its plans early-on; see http://www.whitefoord.org/evaluation.htm for details), Allers says nonprofits are often reluctant to take evaluation into their own hands (as opposed to conducting outside audits).
Organizations often find it hard to switch their focus from delivering programs to evaluating them, he says. “They feel impassioned to help, but overwhelmed by the unmet need,” Allers says. Evaluating programs in a scientific manner—with results-based outcomes—is more difficult than counting outputs and can seem overwhelming.
Understanding program evaluation
For starters, mastering the jargon surrounding the evaluation process is almost as intimidating as the thought of starting or revamping the evaluation process itself.
Dr. Carlyle Bruce, president and founder of Wellsys Corporation, an Atlanta-based consulting group that performs evaluation and research services for nonprofits, defines program evaluation as “a process for helping an organization understand what they do, who they do it for, and what the results are. In other words, helping the organization and their stakeholders answer questions with a process that systematically collects and analyzes the information that is needed.”
Bruce and Allers agree that understanding the difference between output and outcome is key to the success of any evaluation system. Misinterpreting or confusing output and outcome is not only a common mistake that nonprofits make during the evaluation process, but a costly one that can affect the validity of the evaluation and can even have a negative impact on the effectiveness of the programs being evaluated.
Output vs. Outcomes
In simplified terms, Bruce says “outputs” are a measure of effort— “the who did what, when, where, how, and how much.” Outcomes, on the other hand, show “what changed, and how have clients or others benefited from the program.”
Here are several key terms defined by the Nonprofit Good Practice Guide (http://www.npgoodpractice.org):
Output: The tangible products of a project or program activities.
Output Indicator: A measure showing the product or accomplishment of the activities of an organization or individual employee, over a specific period of time.
Outcomes: The measurable results of a project. The positive or negative changes that occur in conditions, people, and policies as a result of an organization’s or program’s inputs, activities, and outputs.
Outcome Evaluation: An evaluation used to identify the results of a program’s effort. This type of evaluation provides knowledge about
(1) the extent to which the problems and needs that gave rise to the program still exist,
(2) ways to ameliorate adverse impacts and enhance desirable impacts, and
(3) program design adjustments that may be indicated for the future.
At first glance
Whether your organization is a large nonprofit with an evaluation system in place or a small nonprofit in need of a new measurement system, the Georgia Center for Nonprofits is a good place to start, says Martha Talbott, who leads the organization’s consulting program.
“We can help people figure out which steps to take next,” she says. Sometimes that means hiring a consultant through the Center’s program, which matches expert consultants to member nonprofits. The Center also offers training on evaluation along with online resources related to topic: Q&A’s, published research, and links to other resources.
Talbott says that any nonprofit considering evaluation—or a new method thereof—shouldn’t require much encouragement. The reasons to do it are pointedly clear, she says.
“For one,” says Talbott, “many funders require it now. But more importantly, proper evaluation is the way to best determine if your programs are making a difference and if your nonprofit is delivering the best service.”
Field Guide to Nonprofit Program Design, Marketing and Evaluation
http://www.authenticityconsulting.com/pubs/PG_gdes/PG_pubs.htm
The Results Accountability Implementation Guide
http://raguide.org/
Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations
with Very Limited Resources
http://www.managementhelp.org/evaluatn/outcomes.htm
Program Evaluation Practice in the Nonprofit Sector
http://www.nonprofitresearch.org/publications1526/publications_show.htm?doc_id=56015
United Way resources
http://national.unitedway.org/outcomes/library/pgmomres.cfm