When baseline data carries hidden theories of change

The questionnaire was not the problem.

The problem was the theory of change it was carrying.

Early in my career, I was part of a team conducting a feasibility study for a rural road project.

Before fieldwork began, I designed a questionnaire to establish baseline conditions.

The purpose was straightforward.

Before the road was improved, we needed to understand existing conditions.

After the road was completed, we would return and measure change.

Had travel time reduced?

Had transport costs fallen?

Had access to markets improved?

Had agricultural production increased?

Had household incomes risen?

Had consumption patterns changed?

The questions were logical.

They reflected what development practice considered important to measure.

Consumption.

Expenditure.

Income.

Markets.

Services.

Transactions.

On paper, nothing was wrong.

Then the interviews began.

"How much sugar do you buy per month?"

"We buy sugar when a child is sick. Or when a woman has just given birth."

"Cooking oil?"

"We rarely use it."

"Beef?"

"Easter. Christmas."

"Electricity?"

"There is none."

"Water bill?"

"We fetch water from the well."

"Transport?"

"We walk."

The questionnaire kept waiting for a market that was not there.

The people kept describing a life the questionnaire had never imagined.

The problem was not that they had no answers.

The problem was that the questions assumed a different world.

The theory of change hidden inside the questionnaire

Every development intervention contains a theory of change.

Sometimes it is written explicitly in a project document.

Sometimes it is hidden inside indicators, models and baseline questions.

The road project carried a familiar development logic.

A better road would reduce isolation.

Reduced isolation would improve mobility.

Improved mobility would increase access to markets.

Market access would increase production.

Higher production would increase income.

Higher income would increase purchasing power.

Higher purchasing power would change consumption patterns.

Changed consumption patterns would indicate improved wellbeing.

The questionnaire was designed to capture movement along this pathway.

The questions were not random.

They represented the future the project expected to create.

If the road worked, households would buy more manufactured goods. They would sell more produce. They would purchase more inputs. They would travel further. They would earn more income.

The questionnaire was not simply measuring the present.

It was measuring whether the expected future was beginning to appear.

This was not an unreasonable theory.

Roads do create opportunities.

Market access does matter.

Infrastructure can transform lives.

But every theory of change contains assumptions.

The critical question is whether those assumptions fit the reality into which the intervention enters.

The assumptions behind the questionnaire

The assumptions were not random. They were the standard assumptions of transport economics.

A better road reduces travel time.

This is true for vehicles. A truck carrying goods may move faster on an improved road.

But travel time depends on who is travelling, where they are going and why they are moving.

A woman walking to collect water may still walk the same distance. A child carrying firewood may still carry the same burden. A family walking to the clinic may still walk the same hours.

The road does not make them walk faster. It does not reduce the distance their bodies must cover.

A road changes movement for some. It does not automatically reduce every form of mobility burden.

Reduced travel time improves market access.

This assumes the household is waiting to be connected to markets.

But for a family that produces mostly for their own consumption, a market is not central to their survival.

They are not waiting for a road to enter a market they already have little interaction with. Their surplus is sold at the farm gate. Their purchases are made locally. The road connects them to a market, but they were not waiting to be connected.

Market access lowers input prices and raises produce prices.

This assumes the household buys inputs and sells produce.

If they do not buy many inputs, price changes do not matter. If they do not sell much produce, price changes do not transform their lives.

They still grow what they eat. They still eat what they grow.

Higher production increases income.

Production depends on more than transport.

It depends on land, labour, climate, technology, finance and institutional support.

A farmer with one acre of land still has one acre after the road arrives. A farmer without affordable inputs still lacks inputs. A farmer without storage, extension services, processing facilities, or reliable buyers still faces those constraints.

The road may make movement easier. It cannot create productive assets that do not exist.

Higher income changes consumption patterns.

Income is not monthly. It is seasonal.

Income comes when crops are sold, when the harvest is in, when the rains are good. It is used strategically. School fees are paid when they fall due. Essential purchases are made when cash is available.

A road does not create employment. It does not produce a wage. It does not replace the irregularity of survival with the regularity of a salary.

The road may connect people to markets. It does not automatically put money in their pockets.

Changes can be measured through a questionnaire.

The questionnaire assumed patterns of consumption, income, and expenditure that did not exist.

It asked about markets when people lived outside markets. It asked about monthly purchases when income came in seasons. It asked about transport when people walked. It asked about expenditure when people grew, stored, exchanged and shared.

It was measuring the wrong thing.

The economy the questionnaire could not see

I had asked about monthly income.

They talked about seasons.

There were two harvest periods. One major. One smaller.

Income came when crops were sold.

It did not arrive every month. It was used strategically.

School fees were paid when they fell due. Essential purchases were made when cash was available. Grain was stored. Seeds were protected for the next planting season.

Food security was not secured primarily through monthly market purchases.

It was secured through what was grown, stored, exchanged and preserved.

Sweet potatoes remained in the garden. Cassava remained in the ground. Yams continued to provide food. Banana plantations provided a continuous source of sustenance.

The household economy was organised across seasons rather than months.

It was organised through production before consumption. Through storage before expenditure. Through knowledge before transactions.

The questionnaire was looking for cash flows.

The people were explaining livelihood systems.

Their economy was not absent.

It was organised differently.

When measurement becomes intervention design

This was the deeper lesson.

A baseline study appears to measure reality before an intervention begins.

But measurement does more than describe. It defines.

The indicators chosen determine what progress will later mean. The categories created determine what problems become visible. The questions asked determine which futures can be recognised.

A questionnaire designed around income, consumption and market participation makes certain pathways of change visible. It makes other pathways harder to see.

This is why development does not only fail when projects are poorly implemented.

It can fail earlier, when the reality requiring intervention has been incorrectly defined.

The intervention may be technically sound. The implementation may be effective. The indicators may be achieved.

Yet the original problem may have been understood incompletely.

The question is not only whether a project works.

The question is whether we correctly understood what needed to change.

The same challenge in artificial intelligence

This lesson did not stay with transport.

It followed me across sectors.

In taxation, it appeared through questions about who contributes and whose contribution is recognised.

In gender, it appeared through questions about whose labour counts as work.

In artificial intelligence, it appears through questions about data, bias and automated decision-making.

AI systems learn from data.

But data does not arrive without choices.

Someone decides what is collected. Someone decides what categories exist. Someone decides what counts as relevant.

An algorithm can process information efficiently. It cannot correct assumptions that entered before the data was created.

The questionnaire and the algorithm share the same vulnerability.

Both are built on classifications.

Both inherit the blindness of their creators.

Both can produce precise answers to the wrong questions.

The question is not whether AI produces accurate answers. The question is whether we have asked the right questions before building the system.

The question that remains

The road project ended.

But the questions it raised had only just begun.

They would follow me across transport, taxation, gender, and artificial intelligence—always asking the same thing.

What are we unable to see because of the questions we choose to ask?

If a questionnaire can carry a theory of change without anyone noticing, what else is development carrying without being examined?

Every discipline has its questionnaires. Some are printed on paper. Others exist as models, indicators, policies and algorithms. All illuminate certain realities. All leave others unseen.

The first decision in development is not the intervention.

It is the definition of reality that makes the intervention appear necessary.

Because the worlds we build are shaped by the realities we choose to recognise.

And sometimes the most important discovery in research is not finding the answer.

It is realising that we have been asking the wrong question.