The evaluation literature reports three kinds of evaluators: external evaluators, internal evaluators, and internal evaluators with external consultants¹. External evaluators are contracted from an outside agency to conduct the evaluation. An internal evaluator is a staff member assigned to the evaluation or hired in. The third category combines the efforts of staff members and consultants’ assistance. The evaluation’s goals and the audiences (for whom the evaluation is intended) should determine the evaluator’s role. The choice between internal or external evaluation depends on the foundation’s field of activity and the foundation’s investment in evaluation.

As the W.K. Kellogg Foundation Evaluation toolkit reports, a common range for evaluation spending by foundations is 5 to 10 percent of total programme costs, since costs vary by programme complexity, number of sites, customisation and the work needed for data collection, analysis, and reporting, as well as scientific rigour.
In the framework of the Lloyds TSB Foundation for England and Wales’s 2002 evaluation, trustees recognised the resource implications of investment in evaluation. In retrospect, this pilot evaluation was judged to require a total of 1,396 hours², nearly equivalent to one full-time employee for a year.

External evaluators

As reported in the study carried out by Compagnia di San Paolo ³, most foundations engage an external adviser to evaluate projects, programmes and outcomes. External evaluators are trained in many fields and are based in many different types of institution: they can be academic researchers, consultants from independent practices, management support organisations, volunteers, sociologists, economists, psychologists and educators.

Examples

• The Fondation de France commissioned a specialist firm for its evaluation of programs in 2005 . The firm made a quantitative and qualitative study of supported projects and analysed recent laws which affect the French disability sector.
• Evaluation of the telephone advice centre in the framework of the OxCAB-Springhill Partnership (OSP) funded by the Esmée Fairbairn Foundation was carried out by a research team from Oxford University’s Centre for Criminological Research, and Cambridge University’s Institute of Criminology.

Internal Evaluators

When an organisation employs its own staff and skills to carry out an evaluation, it is usually termed ‘self-evaluation’. Foundations report that the main advantages of using an internal evaluator are the positive effects in terms of organisational development; the increase in communication between the foundation and its grantees, and the good grasp the evaluator has of the foundation’s operation, easing the interviewing process and making the answers easier to understand. However, the issue of subjectivity might also be raised as well as how open grantees feel they can be with their evaluator-funder, particularly if they may apply for future grants⁴.

Examples

• The major 2002 evaluation of the Lloyds TSB Foundation for England and Wales was built around a questionnaire-based survey and focused on challenging disadvantages and discrimination in the mental health sector. It was conducted by foundation staff drawn from the Central Office in London and from the ten regions involved in the projects. The foundation gave training to the staff involved and undertook pilot assessments before starting the whole survey.
• The French Savings Banks foundation, in its evaluation to assess financed projects’ contribution to welfare and the fight against exclusion, prioritised internal evaluation conducted by a local savings banks’ administrator.

Internal and External Evaluators

In some cases, foundations use both types of evaluators simultaneously.

Examples

• During the implementation and evaluation phase (2001-2005) of The Youth Empowerment Partnership Programme (YEPP), funded by a consortium of European and American foundations, internal evaluation was provided by the programme team, supported by the local researchers responsible for the programme’s development where it was implemented; while the external evaluation was provided by the respective countries’ OECD (Organisation for the Economic and Co-operation Development) agencies⁵.
• The King Baudouin Foundation commissioned an evaluation of the 2000-2 pilot project “L’Ecole en scène”, in partnership with the Ministry of the French Community (Belgium) and the Centre Dramatique de Wallonie pour l'Enfance et la Jeunesse (CDWEJ). The 2002-4 evaluation was carried out in two stages: the internal evaluation was conducted by an expert at the CDWEJ, and the second part entrusted to a professor and team of students at the Catholic University of Leuven’s Faculty of Psychology and Education Science.

Throughout the evaluation process, a reporting system is usually set up to establish consistent and interactive communications between the evaluation team and relevant stakeholders. This system helps refine the evaluation design, questions, methods and interpretations.

>Back to How do foundations evaluate?