Methodological approaches of evaluation are classified in two broad categories: qualitative (detecting and interpreting value judgments) and quantitative (detecting and interpreting quantities). Both approaches are strongly complementary.

Qualitative methods

These methods include three kinds of data collection: (1) in-depth, open-ended interviews; (2) direct observation; and (3) written document analysis. The data for qualitative evaluation typically come from field-work. The evaluator spends time in the setting under study, which can be a programme, organisation, or community where change efforts can be observed, people interviewed, and documents analysed. Qualitative data help explain how programmes work and why they reach particular outcomes, why programmes faced certain obstacles, and may even explain, and provide evidence of, hard-to-measure outcomes which cannot be defined quantitatively.
Example

• The Esmée Fairbairn Foundation commissioned an evaluation of the operation and impact of the telephone advice service run by trained Springhill Prison inmates from the Oxford Citizens’ Advice Bureau (OxCAB), in the framework of the OxCAB-Springhill Partnership (OSP)¹. The research methods included in-depth interviews, attending meetings, observing advice work and the selection process, surveying CAB client satisfaction, and two focus groups to assess public opinion. The evaluators adopted an action research approach which allows a review of findings following the evaluation’s first phase, which are included in the project’s subsequent development before the final evaluation.

Quantitative methods

These methods deal with numerical information that can be analysed. It usually answers the questions: Where? When? How many? and How often? Foundations might perform this type of evaluation by questionnaire, which should ensure uniformity and make certain that the evaluation process covers all outcomes. The results of the questionnaire can then be aggregated into indicators appropriate to assess and explain the findings against the project’s contextual factors.

Example

• The Lloyds TSB Foundation for England and Wales commissioned two evaluations in 2001 and 2002, built around a questionnaire for grantees coupled with face-to-face interviews. However, evaluators found that using scales to measure assessment and how interviewees interpret scales are subjective and can sometimes cause difficulties.

It is also frequently said that common sense and creativity are important principles in designing or conducting evaluation processes.

Example

• The teams of evaluators of The Effectiveness Initiative (EI), a Bernard van Leer Foundation programme on early childhood development, were encouraged to use whatever tools and methods they found most appropriate to reach all the programme stakeholders. Many were children, and illiterate, or unable to express themselves in readily understandable ways. Evaluators used traditional enquiry tools and innovative data collection methods. The key was that the tools used should aid participation and be relevant within the programmes’ contexts. Selected tools were borrowed from participatory learning and action techniques, and included analysis of photos, analogies and pictures, family albums, calendars, drama and songs, games and toys².

> Back to How do foundations evaluate

^[1] The summary of the full evaluation report is available at http://www.esmeefairbairn.org.uk/docs/OxCABsummary.pdf
^[2] For information, http://www.bernardvanleer.org/publication_store/publication_store_publications/stories_we_have_lived_stories_we_have_learned._about_early_childhood_development_programmes./file)