Hoverview

logframe · logic model · philanthropy · programme design · results frameworks

There is no best results framework

The frameworks in use are calibrated to different constraints. What each one trades away, and how to tell which constraint you are actually under.

Somebody in a planning meeting says the organisation should use theories of change rather than logframes. Somebody else says logic models are too vague to manage against. Both have a point and the argument has no end, because it is being conducted as though one of these tools is better than the others in general.

None of them is. They are different objects with different columns, and every column that one has and another lacks is a trade that somebody made deliberately. The useful question is not which framework is best. It is which set of trades matches the money, the timeframe and the reporting relationship in front of you.

Getting this wrong is not a matter of taste. A team that builds a full logframe for a fifty-thousand-dollar community grant has committed to a reporting burden that will consume a visible share of the grant. A team that submits a one-page logic model for a fifty-million-dollar initiative has given the funder no way to aggregate anything. Both teams picked a reasonable tool. Neither picked one that fits.

What the tools actually differ on

Strip away the vocabulary and the frameworks in common use differ on a small number of structural features. Each is worth naming precisely, because these are the things that vary.

The W.K. Kellogg Foundation's Logic Model Development Guide, published in 2004 and the standard reference on the philanthropic side, describes a chain from resources through activities and outputs to outcomes and impact. It carries an assumptions box and an external factors box. The assumptions are framed as if-then statements about the programme, and the guide is clear that reading a logic model means following that chain of reasoning. External factors are described as resources or barriers that enable or limit effectiveness: attitudes, lack of resources, policies, laws, regulations, geography.

The logframe, in the form set out in the UK guidance from 2011, carries the same causal chain and adds two things. Each row has a means-of-verification cell naming where the data for that indicator will come from. And assumptions attach per level rather than globally, so the outcome row and the output row each carry the external condition that has to hold for that specific step.

So the difference is not that one has assumptions and the other does not. Both do. The difference is that a logframe assumption belongs to a particular causal link and a logic model assumption belongs to the whole model, plus the presence or absence of a column recording where each number comes from.

The UN system uses a third arrangement. The UNDG Results-Based Management Handbook, published in 2011 to harmonise concepts across agencies, works from a results matrix with six columns rather than four, sitting alongside a separate M&E plan and a risk mitigation strategy framework. UNODC's application of that handbook sets out the sequence explicitly: situation analysis and problem tree first, then the theory of change, then the results framework, which it describes as the backbone of the theory of change, containing a logical model that sustains it and translates it into measurable products and changes. Here the two tools are not alternatives at all. One is derived from the other.

Two further axes vary and are less discussed. Format: Kellogg's guide explicitly permits a non-linear representation, saying a logic model may appear as a simple image or concept map, which no logframe template allows. And time: the same guide binds outcomes to bands, short-term at one to three years, long-term at four to six, impact at seven to ten, a temporal dimension a level hierarchy collapses.

BetterEvaluation, which catalogues methods without selling any of them, is careful on the distinction: do not confuse logframes with results frameworks, since the two share many similarities but the latter is more focused on articulating results. It also notes that logframes are best used early in design and are harder to apply to activities that were not designed using logframe principles in the first place, which is a statement about fit rather than quality.

Each difference is a trade with a beneficiary

Put those features next to the contexts they came from and the pattern is not arbitrary.

A means-of-verification column costs time at design and buys checkability. The European Commission makes this unusually explicit. In its guidelines for grant applicants the logical framework is Annex C, a mandatory Excel file transmitted to evaluators alongside the budget, and one of the scored evaluation criteria asks directly whether the logframe includes credible baseline, targets and sources of verification, and if not, whether a baseline study is foreseen and budgeted. Five points hang on that question. The column is not decoration in that system; it is assessed before the money moves.

On a small grant to an organisation that will collect the data itself and knows perfectly well where it is, the same column records something nobody needed written down.

Per-level assumptions cost more design time and buy diagnosis. When something goes wrong under a logframe, the assumptions column tells you which link is suspect. Under a global assumptions box, you learn that the theory may be wrong, which is true but not actionable. That diagnostic value scales with the number of causal steps and with how long the programme runs. On a one-year single-output grant there is not much to diagnose.

Non-linear format costs comparability and buys fidelity. A concept map can represent a programme whose logic genuinely is not a chain, and a great deal of community work is not a chain. The price is that no funder can put it side by side with another grantee's model, which is why funders who need portfolio views do not permit it.

Time bands cost simplicity and buy honesty about pace. A framework asserting an impact will land in seven to ten years is making a claim a four-level hierarchy cannot express, and where the timeframe is the contested question that is the most important thing on the page.

None of these trades is a mistake. Each one is right somewhere.

Aggregation is the variable that decides most of it

Across every tradition, the same requirement produces the same design response, which is the strongest evidence that context is doing the work rather than institutional taste.

Where results have to roll up into a portfolio view, indicators get specified centrally. The GEF's results measurement framework requires every project to map onto core indicators numbered one to eleven. The European Commission points applicants at Global Europe Results Framework indicators for use when preparing a logframe, explicitly for aggregation at programme level. The Gates Foundation's Guide to Actionable Measurement describes working with grantees to develop, and asking them to report on, a limited set of relevant common indicators that can be aggregated to advance learning at initiative level, with the Urban Reproductive Health Initiative developing core indicators and providing them to grantees, and the Empowering Effective Teachers initiative building benchmarks such as four-year cohort graduation rates into grantee reporting requirements.

An environmental fund, a European institution and a US foundation, arriving independently at central indicator specification because they each need results to sum. The reasoning is legible: indicators chosen independently by each grantee do not aggregate.

Where nothing needs to roll up, the constraint disappears. The Gates Foundation's grant applicant FAQ says the typical document set includes an investment document, a budget, and sometimes a results framework and tracker. The word doing the work is "sometimes". The same funder does not apply the same apparatus to every grant.

Funders also differ on where they stop. The Asian Development Bank's DMF guidelines require no indicators at impact level at all, because the ADB does not consider project-level impact measurable through a project's own monitoring. That is not a lighter framework. It is a different judgement about what a single project can honestly claim, and it produces a structural difference for that reason.

How to tell which constraint you are under

Four questions get most teams to the right tool, and none of them is about which framework is better.

Does anything need to aggregate? If the funder is summing results across grants, indicators have to be specified centrally and comparably, and the framework has to be one the funder can put next to others. If nothing aggregates, that constraint vanishes and the framework only has to be legible to the two parties.

How long is the causal chain, and how long does it run? Per-level assumptions earn their cost where there are several links and enough time for one of them to fail. A one-year grant with a single output does not have much to diagnose.

Who holds the data? If the implementing organisation collects its own data and knows where it lives, a verification column documents something already known. If the data comes from a government system, a partner, or a survey that does not yet exist, the column is the cheapest available check on whether the indicator is real, which is presumably why the European Commission scores it.

Is the logic actually a chain? Some programmes are sequential and some are not. A framework that forces a genuinely networked theory into four linear levels produces a document that is neat and wrong, and the cost shows up when somebody tries to report against it.

A team that can answer those four has enough to choose. A team that picks the framework it used last time has made the only choice in this whole area that is straightforwardly wrong, and it is by far the most common one.

Where the reasoning runs out

Two limits, and the second is the one that matters.

The first is that most teams do not get to choose. The funder specifies the template and the question becomes how to render the programme's logic under a frame somebody else picked, which is a different problem. Even then the analysis above is worth doing, because it tells you what the funder's frame is optimised for and therefore what it will not show you. A framework with no verification column will not surface a collection problem, so somebody has to check for that outside the document.

The second is that none of this rests on outcome evidence. The trades described here are inferred from what each framework contains and from the contexts each emerged in. Nobody has compared programme outcomes across framework choice while holding grant size, sector and organisational capacity constant. That study is possible in principle, since philanthropic, EU, UN and bilateral funding overlap in size in several sectors and all of them keep records, and as far as the published literature shows it has not been run.

So the argument that these tools are calibrated rather than ranked is a structural reading, not a finding. It is a better reading than the one it replaces, which holds that a single framework is correct everywhere, but the standard of proof is not the same as the standard those tools are supposed to hold programmes to.

Sources

Method catalogues and independent guidance

  • Logframe, BetterEvaluation. Source for the caution against confusing logframes with results frameworks, and for the advice that logframes suit early design and sit poorly on activities not designed with them

Philanthropic frameworks

  • W.K. Kellogg Foundation Logic Model Development Guide (2004). Source for the assumptions box and its if-then framing, the external factors box and its examples, the permission for non-linear formats, and the time bands on outcomes. NACCHO mirror; wkkf.org would not serve the PDF
  • Grant applicant FAQ, Bill and Melinda Gates Foundation. Source for a results framework and tracker being part of the typical document set on some grants rather than all
  • Guide to Actionable Measurement, Bill and Melinda Gates Foundation. Source for common indicators developed with grantees and aggregated to initiative level, the Urban Reproductive Health Initiative core indicators, and the Empowering Effective Teachers benchmarks incorporated into grantee reporting requirements

Bilateral and multilateral frameworks

European Union

United Nations system

  • Results-Based Management Handbook, United Nations Development Group (October 2011). Source for the six-column results matrix and its separation from the M&E plan and risk mitigation strategy framework
  • Handbook on Results-Based Management, UNODC. Source for the sequence running from situation analysis and problem tree through theory of change to results framework, and for the description of the results framework as the backbone of the theory of change