The wrong solution to the right problem

What counts as evidence in development?

I am a mathematician.

Numbers make sense to me.

Equations make sense to me.

Econometrics makes sense to me.

Models bring order to complexity.

They require assumptions, they test relationships, they reveal patterns that would otherwise remain hidden.

If the assumptions are sound, the mathematics follows.

If the mathematics holds, the conclusions can be defended.

Precision matters.

Measurement matters.

Evidence includes measurement.

What mathematics cannot do is recognise realities that have never entered the model.

Models are only as good as the realities they recognise.

The deeper question is never whether the mathematics is correct.

The deeper question is whether the mathematics begins with the right reality.

When evidence stops being evidence

My first encounter with that question came long before artificial intelligence.

It came through participatory research.

As an undergraduate, I was introduced to Rapid Rural Appraisal and later Participatory Rural Appraisal. Robert Chambers' work challenged many assumptions that conventional surveys rarely questioned.

The methods looked almost too simple.

Walk with people.

Observe.

Listen.

Map.

Rank.

Draw.

Ask communities to explain their own realities before asking them to answer ours.

Coming from mathematics, the methods initially felt almost uncomfortable.

Where were the variables?

Where was the sampling frame?

Where were the confidence intervals?

Yet something extraordinary kept happening.

Communities consistently revealed realities that structured questionnaires never uncovered.

The issue was never that questionnaires were wrong.

The issue was that questionnaires could only measure realities that had already been imagined.

Participatory inquiry often revealed realities nobody had thought to measure.

That was the first crack in my own understanding of evidence.

Evidence existed before questionnaires.

Questionnaires merely captured the evidence they had been designed to recognise.

"We do not use corridor rumours"

Some years later I participated in a qualitative study on poverty.

The findings challenged many accepted assumptions about how poor people experienced poverty, vulnerability and survival.

We presented the findings to senior government officials responsible for planning and resource allocation.

One very influential official listened carefully before responding.

"We do not use corridor rumours to make decisions."

I have never forgotten that sentence.

It was not a rejection of our work.

It was a definition of evidence.

Voices were not evidence.

Experiences were not evidence.

Stories were not evidence.

Only quantified data deserved to inform policy.

At the time, the distinction appeared obvious.

Qualitative inquiry generated insights.

Statistics generated evidence.

Planning required evidence.

The conversation ended there.

Or so it seemed.

When institutions learn to listen

Several years later the same government introduced Participatory Poverty Assessment as part of national planning and budgeting.

Communities were no longer treated simply as respondents.

They became sources of knowledge.

The same institutions that had dismissed qualitative evidence now depended upon it.

What had changed?

Not the communities.

Not poverty.

Not reality.

Only the definition of evidence.

That shift mattered far beyond research methodology.

It transformed the kinds of realities government became capable of recognising.

Listening was no longer viewed as anecdotal.

It became part of planning itself.

The lesson was profound.

Evidence had not changed.

Recognition had.

The priority was right. The solution did not fit.

One example remains vivid.

For several consecutive years government allocated funds to improve access in an island district.

Every year the money was returned unspent.

Officials concluded that the district lacked capacity.

Perhaps local government was inefficient.

Perhaps implementation was weak.

Perhaps leadership was poor.

All perfectly reasonable explanations.

Until participatory inquiry asked a different question.

People explained that they did indeed have an access problem.

Government had diagnosed that correctly.

But the proposed solution assumed access meant roads.

The district needed waterways.

Ferries.

Landing sites.

Water transport infrastructure.

The funds could only be spent on roads.

No roads were required.

So the money remained untouched.

Later, one senior official reflected on what had happened.

We had identified the right priority.

Poor access.

But we had designed the wrong solution.

The intervention failed not because the diagnosis was wrong.

It failed because the evidence describing reality was incomplete.

One road does not fit all.

Neither does one form of evidence.

Evidence does not speak for itself

We often describe evidence as though it exists independently of those who collect it.

It does not.

Evidence is always mediated by recognition.

Before evidence is collected, someone decides what deserves attention.

Before indicators are constructed, someone decides what counts.

Before data are analysed, someone decides what reality the analysis seeks to explain.

Numbers do not remove these earlier decisions.

They inherit them.

This is why evidence cannot be defined by its format.

A statistic is not automatically better evidence because it is numerical.

An interview is not automatically weaker evidence because it is qualitative.

Evidence becomes evidence because it reveals reality.

Not because it appears in a spreadsheet.

The question is therefore never simply:

How much evidence do we have?

The deeper question is:

Which reality does our evidence recognise?

How does something become evidence?

Evidence does not begin with data.

It begins much earlier.

Before anything is counted, observed, compared, modelled or analysed, a prior decision has already been made.

Something has first been recognised as worthy of attention.

Something else has not.

Only then do we ask questions.

Only then do we collect information.

Only then do we decide what can be documented, compared, analysed or monitored.

Evidence is therefore not simply collected.

It is constituted.

Not because reality changes.

But because our theories determine which parts of reality are allowed to enter the evidentiary record.

This is why evidence is never merely technical.

It is always epistemological.

Once evidence has been constituted, measurement becomes one possible way of describing it.

Some realities can be counted.

Others can be observed.

Others are recognised through experience.

Others emerge through conversation.

Others become visible only when people are allowed to explain their own lives.

The question is never whether one form of evidence is superior to another.

The question is whether the form of evidence matches the reality being investigated.

A carefully measured quantity cannot compensate for a reality that has first been misunderstood.

A survey can produce extraordinarily precise answers.

To questions that never should have been asked.

A statistical model can describe relationships with remarkable accuracy.

Within a reality that has already been defined too narrowly.

The mathematics is not the source of the error.

The error occurs earlier.

It occurs when the reality itself has been incompletely recognised.

Technical excellence cannot repair a conceptual failure.

It can only make it more persuasive.

Evidence is not collected. It is constituted.

AI is already learning from us

Artificial intelligence is no longer a future possibility.

It is already influencing research, policy, education, finance, health care, taxation and public administration.

The question is therefore no longer whether AI will shape development.

It already does.

The deeper question is:

What understanding of reality is it learning?

Most discussions focus on whether AI systems are accurate.

Accuracy matters.

But accuracy is never independent of evidence.

Accurate according to what?

Accurate according to whose reality?

AI does not begin with intelligence.

It begins with data.

Data inherit indicators.

Indicators inherit definitions.

Definitions inherit theories.

Theories determine what counts as evidence.

Artificial intelligence therefore inherits far more than datasets.

It inherits the way societies have learned to recognise reality.

Train an AI system on evidence that recognises markets more readily than households and markets become the economy.

Train it on evidence that recognises paid work more readily than care and care disappears.

Train it on evidence that recognises transactions more readily than reciprocity and reciprocity ceases to exist.

Not because these realities disappear.

But because they never entered the evidence from which the system learns.

Artificial intelligence is not creating a new blindness.

It is institutionalising an old one.

The question before evidence

Looking back, I no longer see my intellectual journey as a movement away from mathematics.

I see it as a movement towards a broader understanding of evidence.

Mathematics still matters.

Statistics still matter.

Models still matter.

They always will.

But they cannot determine what reality is.

They can only describe the reality that has first been recognised.

The first question in development is therefore not:

What does the evidence say?

The first question is:

How did this become evidence?

Everything else follows from the answer.

Because evidence does not become evidence because it is numerical.

It becomes evidence because it reveals reality.

Once evidence ceases to reveal reality, it no longer serves its purpose.

It begins to confirm assumptions that have quietly become invisible.

And no amount of methodological sophistication can compensate for beginning with the wrong reality.

Next:  What Counts as Knowledge? Evidence asks: What counts? Knowledge asks: Who decides what counts?