| Sumario: | Ex post impact assessment (epIA) is often promoted as a method of illustrating to funding
bodies that agricultural research investments are effectively contributing to the achievement
of development goals. In response, impact assessors have produced a great deal of evidence
that international agricultural research is an efficient and effective form of investment.
Despite these results, international agricultural research, including the activities of the Centres
of the CGIAR, has at times faced funding shortages. This may indicate that the evidence
produced has not been entirely persuasive.
To help foster a more demand-led approach to epIA, this analysis investigates the information
demands of CGIAR Members (donors), the primary audience for accountability–oriented
epIA studies. Two methods were applied to analyse information demands and uses. First, an
email survey of all CGIAR Members was conducted, so as to ascertain preferences for
different epIA approaches, methods, and metrics. Subsequently, a series of interviews of
donor representatives was used to explore in more detail how the results from epIA contribute
to funding decisions.
The 24 email survey responses received from 22 Member agencies indicate high demand for
impact related information. Analyses of impact “far down the impact pathway” with metrics
directly related to poverty and improved livelihoods are most frequently claimed to be most
useful. Similarly, a majority of respondents report that the “magnitude and distribution of
benefits” is the primary determinant of the use of epIA findings. EpIA conduct by an external
assessor is reported as important for credibility by most respondents.
Follow-up interviews of representatives of 26 CGIAR Members illustrate that agricultural
research funding decisions within Member agencies are complex, and the consideration of
patterns of past impact in these deliberations is often less than central. Other factors, such as
political priorities, perceptions of scientific quality, and desires for funding continuity also
play prominent roles in these decisions, and decision-makers face an onslaught of many forms
of information apart from epIA findings. In this context, brief summaries of epIA results are
critical to capture attention, and the influence of findings is often indirect and “conceptual.”
Despite the indirect nature of this influence, the interviews reiterated strong demands for
evidence of impact.
A number of Members indicated that the transparency of impact studies could be improved,
as certain analyses have been assumption-laden, and sufficient details about the assessed
research have not been provided in some cases. These audiences also appear to demand epIA
of a broader range of research activities, as “comprehensiveness of coverage” received
2
relatively low ratings. Finally, a greater focus on poverty-related metrics and the distribution
of assessed benefits was demanded by a large proportion of interviewees.
Responses are compared with patterns documented in the broader evaluation literature. The
observations of the present study are generally consistent with prior findings that indirect use
is predominant. In addition, the information preferences observed, when compared with
approaches advocated for internal feedback in the evaluation literature, suggest that the
information demands of donors may differ from those of internal audiences.
|
|---|