Thursday, May 6, 2010

Evaluation theory vs. Program design

I had a question in a presentation yesterday that had me somewhat stumped. The question was, essentially, what are we supposed to do when the planners tell us to take a results-based approach where we only plan and measure the key things, and the auditors tell us they want formal plans that are activities-based and have exhaustive lists of plans and measurements. Good question, and not one that I had the answer to immediately.

Then I got to thinking about a presentation that a friend of mine at Grant Thornton LLP forwarded me recently. In it, one of the panelists talked about the divergence that occurs between evaluation theory and program design. That, it occurred to me, was the problem.

This problem that seems to arise in public sector programs between results-based management and evaluation, comes down to the relationship between the two schools of thought. Evaluation theory is based on a rational approach: you perform a needs assessment; you develop a logic model; you allocate resources; then you monitor and evaluate as you continuously improve the quality of your program delivery.

Program design, however, can often go a bit different: a politician (likely the Minister) conceives of a program to respond to what he or she perceives as a public need, or is receiving public pressure over; and then the program design then begins to respond to internal and external factors (ie., lack of resources, so it looks for cost-sharing opportunities). So what emerges is not a program defined by needs assessments and logic models, but one that is defined more by external factors and political whims.

So how, then, do you resolve the two? Because, surely, the likelihood of eliminating program evaluations and audits is extremely low. (And I would never advocate such an idea, as these evaluations and audits can provide valuable information!) And neither is it likely that politicians will begin to make decisions solely based on rationality, and not public pressure.

It's not an answer I have, but would be interested in knowing what other people thought.

1 comment: