Does Intervention Help?
Linking Research and Implementation
Case Studies or Surveys
Reduced O&M Costs
Changes in Agricultural Production
Fisheries and Other Impacts
Abstract: There is a shortage of information about the performance of intervention in small scale irrigation. It is not safe to assume that worthwhile results will appear simply because money is being spent. Benefits can be assessed quickly and efficiently using a mixture of methods including rapid appraisal, case studies and surveys. Care is needed to make sure information is representative and adequately comprehensive. Results of evaluation can correct mistaken assumptions, provide information about returns to different approaches to intervention, encourage better analysis of the cost effectiveness of investment and contribute to improving the process of intervention.
There is a serious lack of information on the impact of intervention in small scale irrigation in Indonesia (Varley 1989) and elsewhere. Feedback about the performance of past investments could help lead to more successful intervention by improving procedures and developing better criteria for choosing sites and designs. Such information would provide a better basis for deciding whether to increase or decrease the level of investment in small schemes. It would help in choosing between alternative strategies for investment.
Several factors hinder assessment. Sites are small and scattered. Individual projects are too small to justify the kind of analysis done of large irrigation projects. Little information is available, so assessment has to start from scratch. Levels of investment per hectare are lower than for larger schemes, so there is a tendency to accept more casual justifications for investment. Decisionmakers may feel threatened by assessment and its results and reject it if the potential gains from assessment are not clear.
... There is a serious lack of information on the impact of intervention in small scale irrigation. ...
What this paper would like to suggest is that assessment of the impact of intervention in small scale irrigation is feasible. It can be done relatively quickly and efficiently. The results can be useful for improving policies and procedures regarding intervention in small scale irrigation.
This paper draws on experience from carrying out two studies of the impact of intervention in small scale irrigation in northeast Thailand. The first was an assessment of small weirs and reservoirs built by the Khonkaen University-New Zealand (KKU-NZ) Small Scale Water Resources Development Project. It included case studies of three weirs and a survey of fifty sites. The second was a survey of small weirs and reservoirs built in Khonkaen Province by the Royal Irrigation Department. This involved a survey of a random sample of twenty-five sites, with household level information collected at each site. The discussion also draws on additional experience in Thailand and Indonesia and the literature discussing intervention in small scale irrigation.
The goal of the paper is to provide some suggestions about the ways in which assessment can be more productively used to improve intervention in small scale irrigation. The discussion primarily focuses on post-project assessment of impacts. The results of such assessment provide a basis for improving preliminary assessment for potential intervention sites, design of improvements to be constructed, assessment of the process of intervention, action research and other efforts to improve agency policies and procedures.
The first section of this paper discusses why assessment is needed. Different methods for collecting information are then compared and linkages between research and implementation discussed. Assessment must cover multiple benefits and costs which are not always easy to measure. Benefit-cost analysis provides one framework for comparing costs and benefits. The final section draws some conclusions about the role of assessment as a tool for improving the performance of intervention in irrigation.
Unfortunately it is not safe to assume that intervention will automatically yield benefits to farmers. Intervention may simply displace what farmers would have otherwise done themselves. Farmers might have gotten assistance from other sources. Intervention may create new problems, for example by disrupting existing local institutions for operation and maintenance. The benefits of intervention may be small, especially if compared with the benefits of other ways in which the same amount of money could have been spent. The results of intervention as part of a large, routine program may be very different from those of pilot projects.
The history of large scale irrigation projects has shown that results are often far below what was expected. There are arguments why investment in small scale irrigation may be more productive, but these are far from enough to ensure that all such investment will be worthwhile.
More general experience with evaluation of government programs also leads to pessimistic conclusions. Based on broad personal experience and an extensive review of the literature Peter Rossi formulated the Iron Law of evaluation (cited in Murray 1989). His conclusion drawing on a broad range of studies was that: The expected value of any net impact assessment of any large scale social program is zero.
After subsequent research and debate he supplemented this with the stainless steel law. "The better designed the impact assessment of a social program, the more likely is the resulting estimate of net impact to be zero." Many factors cause project results to diverge greatly from original intentions. These include limitations in agency capacity, flaws in project design, creation of incentives with unanticipated perverse impacts and political manipulation such as capture of benefits by elite groups. Rural credit is an example of an area in where empirical and theoretical analysis have demonstrated how greatly actual impacts can differ from intentions.
... Unfortunately it is not safe to assume that intervention will automatically yield benefits to farmers. ...
Benefits cannot be taken for granted. Empirical assessment is necessary to identify consequences of intervention. Analysis is needed to see what is the net impact of intervention, taking account not just of poor performance but of more subtle factors such as displacement of what might have occurred without the project, the generation of new problems and other factors which may offset direct benefits.
If there is extensive intervention by government so that production increases lead to relative declines in the cost of rice or other products then the benefits of investment accrue more to consumers than farmers. The question of the mobility of farmers and the resources they use into other investments then becomes crucial to assessing to what extent there are actually net benefits for farmers from government "assistance" to agriculture. However at a minimum if the government seeks to aid agriculture it should not just assist farmers served by large scale irrigation systems but also aid small scale irrigation and rainfed crops, if comparable investment opportunities are available. The question of the boundaries of the system to be assessed is important.
These findings emphasize the need to ask whether what the government does actually helps farmers. This may help us to learn how to best make use of scarce government resources to assist farmers.
Experience from evaluation research also makes it clear that assessment must look at the process of intervention, not just final impacts. Intervention cannot be usefully treated as a black box or scientific experiment. Later projects never perfectly replicate earlier projects and should learn from earlier mistakes.
Research focused only on net project impact, without a concern for process, provides little useful information for improving implementation. It may sometimes be useful for macro-policy decisions about whether to start or stop programs or change levels of funding. However major decisions of this kind are usually made at senior levels and are heavily influenced by political factors. For improving implementation additional information is needed.
Action research, process documentation (Veneracion 1989) and other methods have been developed for gaining a better understanding of the intervention process. These provide information about the actual implementation process and opportunities for improving it.
Approaches emphasizing social learning and the implementation process are based on the idea that changing projects is not trivial and cannot happen just as a result of decisions by senior policy-makers (Korten 1987, D. Korten 1989). The approaches are also based on the idea that within agencies there are often many intelligent, dedicated people who are aware of problems but lack adequate resources to create solutions. Changing implementation is not a matter of a few high level decisions but requires a thoroughgoing process involving people at many levels in the agency.
Assessment of impact should be linked to activities to improve implementation. Assessment as discussed here might be done by a university, the agency itself, a specialized government unit, an NGO or other institution. The concern is with information gathering and analysis that directly support improvement of agency activities, applied rather than pure research. This does not mean that the results of such studies may not also be useful for more academic analysis. One suggested solution has been that academics involved in research of this type should plan to write up their findings in two ways, one report directed at the applied concerns of the agency and one dealing with the implications of research results for more abstract disciplinary concerns.
... Research focused only on net project impact, without a concern for process, provides little useful information for improving implementation. ...
The results of assessment are more likely to be used if the agency is involved from the beginning in defining the issues to be studied and continues to be involved through the course of the research in discussing and analyzing interim findings.
One key technique is the circulation and discussion of draft reports. This has the effect of involving agency staff in the analysis. This may sometimes lead to some findings not being reported formally, or being rephrased in more diplomatic language. However if agency attention is drawn to important problems and ideas generated about possible solutions then the more fundamental applied goals may be achieved. Similarly some sensitive issues may only be discussed verbally, especially if they are beyond the official scope of the research. However as an applied research process this still functions to improve agency action. Such a participatory approach is likely to be much more productive than simply presenting a final report, which is easily ignored.
In the study of the KKU-NZ weirs the researchers provided information to Khonkaen University faculty who were in the process of preparing a project directed at institutionalizing the project's approach within a government agency. Two of the researchers were recent engineering graduates. After the evaluation was complete they went on to work with the new project. The results were also used as evidence in efforts to convince national level officials that the approach developed in the project should be considered for broader implementation.
The survey of RID weirs and reservoirs was carried out as part of a project directed at exploring ways to institutionalize the use of community organizers within RID. Before the SSIS project was designed rapid appraisal methods had been used to gain information about small scale irrigation development in northeast Thailand. Many staff members of the project participated in pretesting and carrying out the survey. Thus the study was part of a broader set of methods used to gain a better understanding of irrigation development as carried out by RID. It provided useful information on what the results of intervention had been when done without community organizers.
Action research, process documentation and related methods provide valuable information about process. This is essential to finding feasible ways to actually change agency procedures. However this should not mean that the question of the impact of intervention is neglected.
Several different methods are available for collecting information and the advantages and disadvantages of these need to be considered. The question of what methods will be most persuasive in influencing decisionmakers needs to be considered along with the goals of scientific objectivity.
A first answer to the question of whether to use survey or case study methods is that neither may be appropriate. Often much can be learned without the time and expense required by these methodologies. The first priority is often to get out into the field and find out what is going on, which can be done using relatively informal methods.
The quality of information gathered in informal ways can be improved using the concepts developed concerning rapid appraisal (Khonkaen 1987). They also help to avoid the biases common to "rural development tourismChambers 1983."
Several of these concepts have been summed up using the idea of triangulation, obtaining information from multiple sources. Information can be gained from interviews, secondary data such as maps and agency records and from direct observation. Talking to poor and average villagers helps avoid the biases which may come from relying solely on better off villagers who usually occupy leadership positions. Talking to villagers at the tail end of irrigation systems and those who are not involved with the system can check information provided by those in more privileged locations. Women and young people can give information which may not be forthcoming in interviews with more senior men speaking as household heads.
... The first priority is often to get out into the field and find out what is going on, which can be done using relatively informal methods. ...
Semi-structured interviews, open-ended questions, group interviews, discussion while walking through the irrigation system and other techniques are also very useful. At times the label of rapid appraisal may be useful for systematizing what is done and emphasizing the value and validity of information obtained in this way. In most cases where current information about the impact of government intervention is quite limited, the best first step will be to gather information using such quick, inexpensive methods.
Sometimes rapid appraisal alone may provide sufficient information. This may be particularly true if problems are widely recognized at the field level. In such cases rapid appraisal provides an inexpensive, systematic way of providing information to decisionmakers.
In other cases rapid appraisal can provide a basis for formulating more specific research questions. If these questions are primarily oriented toward issues of process, how things happen, then further rapid appraisal may be appropriate, or case studies may be useful.
Case studies offer the opportunity to gain more in depth information than can be obtained during a few hours or days per site of rapid appraisal or survey research. They also generate qualitative, process oriented information and examples which are often more credible to decisionmakers than are the results of survey questionnaires.
Case studies can provide strong evidence to disprove common myths and misconceptions. For example they can show why farmers reject improvements designed and built without local participation in planning. Such "improvements" frequently ignore local knowledge about flooding, landholding and other critical factors. Case studies allow an opportunity to explore alternative explanations and comprehensively look at what factors may have influenced events.
However case studies are relatively expensive in terms of time. Also their results tend to be more suggestive than conclusive. It is quite hard to firmly establish what changes are due to the intervention and what due to other factors. Thus they are more useful for generating ideas and providing examples than for creating a strong foundation for evaluation. This is particularly true for small scale irrigation where conditions are diverse, so that it is unlikely that any "typical" site can be found.
... Case studies can provide strong evidence to disprove common myths and misconceptions. ...
It is particularly hard to be sure how representative the results from any case study or group of case studies are. In northeast Thailand several case studies had been done on the economic impact of government construction of small weirs and reservoirs (Palanisami 1984, Tubpun 1983). The results of these studies suggested that government investment had provided significant returns. However when looked at collectively it became clear that these were almost all done at RID "model" sites, which had received substantial amounts of special attention. There was a systematic bias in the selection of sites for case studies, which meant the information from the studies was not representative of conditions in the majority of non-"model" sites.
Surveys can provide a broader, more representative source of information, but at a higher cost in time, money and professional expertise. They will be useful if specific questions have already been defined. Without such previous exploration and analysis they can easily generate an overabundance of data with little relevance. This is a major reason why survey results are often ignored.
The procedures for carrying out survey research are fairly standard and are discussed in many sources. However some aspects deserve particular attention in regard to small scale irrigation.
The survey should be developed to answer specific questions concerning the impact of intervention. Standard household socioeconomic surveys are likely to be of little use. Trying to disentangle the effects of a project on household income is an extremely difficult task. It is more likely that the intermediate impacts such as higher yields or increased area cultivated can be measured and linked to the intervention. Standard socioeconomic surveys usually only focus on households and neglect many system level issues which are far more important for understanding the impact of intervention.
... Without previous exploration and analysis surveys can easily generate an overabundance of data with little relevance. ...
Formulation of research questions should draw on earlier rapid appraisal or case studies as discussed above. Even then additional study will need to be done before a survey can be developed. In effect this is a form of rapid appraisal. It may be carried out in the context of survey pre-testing, but should go well beyond testing and refining specific questionnaire items.
Even with a good initial foundation thorough pre-testing is essential. This means trying out the survey at sites like those at which the actual survey will be done. It means repeated testing and refinement, not just a single round of pre-testing. If done well, pretesting usually reveals serious conceptual problems which go well beyond just simplifying language and making forms easier to fill out. These problems require the attention of the principal researchers in deciding how to reformulate questions, adjust the scope of the survey, change sampling strategies and otherwise make sure that the survey comes closer to being able to achieve its goals.
A key part of survey research is how to sample. There is often a temptation to avoid random sampling, using arguments of convenience, limited resources or other excuses. However this undermines the main advantage of survey research over other methods, which is its greater claim to representative information. Cluster sampling and stratified sampling are usually sufficient to avoid excessive travel costs, while still allowing results to be analyzed quantitatively. Use of these sampling methods will probably mean that some expert assistance will be needed in designing the research and analyzing results.
For the survey of RID weirs and reservoirs in Khonkaen a random sample of irrigation systems was selected. Within each system a list of irrigators was prepared. If possible all households were interviewed. If there were too may households to be interviewed during the one to two days the researchers spent at each site, then a sample of households was interviewed. Random sampling of sites generated broad range of sites. It included a few spectacularly bad sites which would not have been accepted as representative if chosen according to any other procedure. Random sampling did include some of the the agency's model sites.
A slight modification would have allowed inclusion of even more high performing sites, helping to reassure the agency that these sites would not be neglected. This would be to set up a separate stratum for these sites and sample a much higher proportion of them. The skewed distribution of irrigation system size and other key variables mean that there are statistical as well as pragmatic reasons for a higher sampling rate for high performance systems. In the Khonkaen case this could have been done by using a separate stratum for sites irrigating over 80 hectares. At the extreme such a special stratum could be sampled at a one hundred percent sampling rate. This would mean that all these sites would be included. In the analysis they could be appropriately weighted, to produce an accurate estimate of average conditions. This allows a compromise between the agency's desire to insure that certain sites are included and the need for representative information.
The survey of RID sites drew on experience from previous studies to develop focused questions and efficient methods for data collection. Based on this it was possible to collect information for systematic quantitative assessment during a one day visit to each site.
Relying on a single research method is likely to be crippling, producing results far weaker than can be obtained from a combination of methods. The different research methods can be fitted together in a sequence.
A sequence can start with exploratory research, using methods such as rapid appraisal, which is relatively cheap, flexible and open ended, which helps to correct misconceptions and identify important questions for more detailed analysis. Case studies stand in the middle of such a sequence, being more expensive in time and money but providing more depth than rapid appraisal. Survey methods tend to be more demanding in time, money and expertise. If applied correctly to well defined questions, surveys can provide fairly precise answers.
... Relying on a single research method is likely to be crippling, producing results far weaker than can be obtained from a combination of methods. ...
During research new questions emerge, so a simple progression through the methods above is not enough. New questions may require returning to rapid appraisal or focused case studies. Many of the questions raised about how to improve policy require trying out changes in agency procedures, whether in the context of routine activities or special action research.
Irrigation projects offer many different types of benefits. They involve costs, including land used for construction, In order to be successful, assessment has to identify what changes are due to the intervention and what the value of the changes is. More rigorously the comparison is of conditions with the intervention compared with what conditions would have been without the intervention.
For farmers the two most attractive benefits from government assistance are usually the prospect of reduced costs for building and repairing irrigation systems or increased agricultural production through reducing losses from drought or expanding irrigated area. If these benefits are not present then farmers may have little interest in operating and maintaining new structures. Therefore it seems worthwhile to try to assess the extent to which these benefits actually occur.
Assessing the reductions in operations and maintenance costs due to intervention is fairly straightforward. Farmers remember the time spent building and repairing systems. There may be written records. However it is important to see to what extent there is actually a reduction in O&M costs, as opposed to just a change.
In one system in northern Thailand a permanent weir was built. Previously the bamboo weir was damaged by floods each year and had to be rebuilt. With the new weir this was no longer necessary. However the new weir blocked flow in the stream so that sedimentation was greatly increased. Where before farmers had spent ten days each year repairing the weir they now had to spend ten days or more digging out sediment, meaning that there was no net gain (Uraiwan Tan Kim Yong, personal communication).
In most countries the government heavily subsidizes the cost of new construction, requiring little local contribution, except perhaps land. This means from the farmers' point of view a reduction in time spent on maintenance is a pure gain, since they do not have to pay for it. If this is the case they have little reason to reject or even carefully consider whether to accept government assistance. There may be little risk of loss and they may gain. This clearly influences their decisions and provides one argument for why farmers should pay part of the cost.
One of the principles of analysis is that it should look at the issue from both the individual and societal point of view. Without a balanced approach it is easy to fall into the trap of activities which benefit some individuals but may not be worthwhile when considered from a broader point of view, or conversely programs which make sense from a collective point of view but do not provide adequate individual incentives.
Estimating the influence of intervention on agricultural yields and area cultivated is a complex matter. Baseline information is rarely available for small schemes. Many factors influence yields and area cultivated besides water supply. Estimating yields itself is difficult. Crop cuts offer the appeal of precision, but are difficult to carry out, limited to a short harvest period and themselves subject to several forms of error.
For the studies in northeast Thailand the choice was made to rely on farmers' estimates of average yields before and after the project. Most of the sites had been in use for several years after the project, offering the hope that yields would not be biased by factors specific to a single season. The harvest is important to farmers, so it seemed reasonable that they would remember with a fair degree of accuracy. The idea of average yields was more difficult, but with some explanation it seemed possible to get useful answers.
The form of the answers usually was that before construction of the new weir a farmer might get three of four truckloads of rice, while afterwards they got four to five truckloads. Short of a very elaborate methodology this seemed the best that could be done. It clearly provided information about whether yields had changed. Asking about changes in fertilizer use, rice variety and pumping provided some control over other factors. This approach involves some significant compromises concerning data quality, but if these are accepted then useful information can be obtained.
One pitfall in assessments done without adequate field information is to ignore the amount of irrigation that already occurs, even if with relatively simple temporary structures. The largest gains from irrigation come from irrigating previously rainfed land. Sites for small irrigation schemes are usually in relatively good locations for benefiting from overland runoff and groundwater flows. Intervention usually takes place in sites where farmers have already been irrigating. This means that the change in yields due to an improved supply of irrigation water is much smaller than it would have been if fields which relied on rainfall alone had been irrigated.
... Intervention usually takes place in sites where farmers have already been irrigating. ...
Similarly care may be needed to distinguish the benefits of irrigation in general, from those due to a specific type of intervention. If the intention is to compare alternative approaches or suggest why a particular approach should be strengthened then the question concerns the additional benefits unique to that approach, not just the benefits which are likely to come from any irrigation project.
Irrigation systems do not only provide agricultural benefits. Assessment needs to take other benefits and costs in to account. This is an area where rapid appraisal and pretesting can be very useful. They enable additional benefits to be identified.
In northeast Thailand the biggest problem concerned fisheries benefits. No existing methodology seemed to be available for estimating these. In the first study they were estimated at KKU-NZ case study sites. Question were asked during the KKU-NZ survey but did not yield much useful information. In the RID survey a method was developed for gaining more useful estimates by asking key informants who could estimate the amount of fishing. This suggested that benefits from fishing were as large or larger than those from increased rice production. This showed the importance of trying to include information about other benefits, whether from fishing, milling or other sources.
Conversely some benefits may be frequently cited but actually provide quite small benefits. This was the case with livestock and domestic water supply. During the dry season fields are open to all for grazing, so if a water source is available it will be used. In most cases there are already alternative sites, so the marginal benefits for livestock are often quite small.
... Assessment needs to take other benefits and costs in to account. ...
Similarly for domestic water supplies, shallow groundwater could supply domestic water far more cheaply than investing in irrigation systems. Estimating the nature and value of such benefits provides an important corrective to the rhetoric used for project justification. It helps emphasize the importance of considering alternative solutions to water supply problems, so that inexpensive alternatives are not ignored.
In Thailand, land for small systems is supposed to be donated, without compensation. This creates a political problem. While the benefits are spread widely, the costs are often born by a few people who own the land next to the proposed site. In theory it should be a straightforward matter for local people to arrange compensation locally. In practice this rarely happens, leading to conflicts and opposition by landowners.
This is one area in which the use of community organizers to catalyze local action can clearly lead to more equitable sharing of the costs of development, rather than asking a few unluckly landowners to make a unilateral sacrifice. The value of the land alone is not sufficient basis to assess the costs involved in obtaining land and their influence on the economic value of the project.
Projects are often said to provide "social benefits." Most of these tend to be vague and may fade away under closer examination. Many specific benefits can be assessed, and shown to be relatively small. Even if they cannot be measured precisely it may be possible to put a ceiling on their value, for example the cost of improving domestic water supplies through other methods.
Other benefits, such as creating short term employment may be valuable, but are not necessarily linked to irrigation and can be provided through other rural public works. In Thailand, the Job Creation Program has provided funds to subdistrict councils. These councils then select what projects they think would be most useful. This allows more flexible use of funds than channeling it through a bureaucracy with a specific mandate such as irrigation or roads.
In sum, it is possible to assess the benefits and cost resulting from intervention in small irrigation schemes. Such measurements are often sufficient to estimate the level of benefits with reasonable accuracy. Even where precise estimates are difficult it may be possible to roughly estimate the boundaries within which they lie. Assessment can correct the myths and vague arguments often used to justify projects. It also provides a basis for more thorough comparison of costs and benefits.
The methods discussed above provide information about various costs and benefits of intervention in irrigation. By themselves they can help to correct some mistaken assumptions and more accurately identify what gains and losses occur. However a framework is needed to combine and compare costs and benefits.
The conventional approach is to use benefit-cost analysis to estimate a benefit-cost ratio or internal rate of return. The methods are standard and are presented in many sources (Gramlich 1981, Gittinger 1982. See also Schmid 1989 ). The weaknesses of the methods have also been widely discussed.
Some of the key problems include the fact that some important impacts cannot easily be assigned a value in monetary terms. The techniques discount the value of future benefits. For manufactured products and renewable resources this may be justifiable. Economic growth means that the capacity to build and improve irrigation systems is likely to be greater in the future. However for non-renewable resources and ecosystems it is less safe to assume that they can be recreated in the future or satisfactory substitutes found. So these cannot be entered directly into the calculation of economic costs and benefits, but instead must be included in additional broader assessment.
Nevertheless most intervention in irrigation is fundamentally justified on the basis of expected increases in agricultural production. The resources used to build weirs and canals could be used to build roads, clinics, schools, factories and other investments which would also yield benefits for rural people. Therefore it is fair to ask how such funds and other resources could best be used. Assessment of economic costs and benefits provides one important source of information for decisions about how to allocate limited government resources.
... The resources used to build weirs and canals could be used to build roads, clinics, schools, factories and other investments which would also yield benefits for rural people. ...
For large scale irrigation systems elaborate methods have been developed for economic analysis. Where the total amount of investment in small schemes is similar then it may be worthwhile to use the same methodologies in assessing small scale irrigation, to better estimate the opportunity cost of labor, the value of increased rice production, the cost of capital and other factors.
For small scale irrigation often simpler methods can be used and the controversies and complexities of shadow prices avoided. In this case the methods are fairly straightforward. Studies by Hayami, Kikuchi and collaborators in the Philippines and Indonesia provide good examples of how such analysis can be done (Kikuchi, Dozina and Hayami 1978, Hafid and Hayami 1979).
In the case of northeast Thailand it was possible to further simplify the analysis. Operation and maintenance costs were minor and could be neglected for purposes of analysis. By assuming a constant level of annual benefits it was possible to use the simple formula for estimating the value of an annuity to compare initial capital investment with the stream of annual benefits. The availability of computer spreadsheets means that calculations can be done quite easily. (For example in Lotus 123 @PMT (200000,0.12,15)=29364). Computer spreadsheets also make it easy to carry out sensitivity analysis and present conclusions about how much the results of analysis would be affected by changes in particular variables, such as yields, prices or capital costs.
Experience with evaluation research has shown that focusing single mindedly on estimating the net impact of a project is not productive. There are too may assumptions involved to generate unique answers. However economic analysis may help identify boundaries, floors or ceilings within which the actual level of return is likely to lie. This may show that economic returns are low and the project needs to be reconsidered. However they may also show that returns are high, supporting continuation and expansion. Since many sites are involved in such projects they can be compared, providing lessons about what factors may hinder or help intervention.
The previous sections have suggested that benefits and costs of intervention in small scale irrigation can be assessed, and that assessment can be done relatively quickly and efficiently. However for this to be done clear goals need to be set for evaluation.
Assessment can provide information for decisions about whether or not to increase investment in small scale irrigation. If economic returns are small then then current approaches probably need to be reconsidered and changed, or the level of funding reduced. If interventions in small scale irrigation provide high payoffs then this is a useful justification for maintaining or increasing funding.
Assessment can help identify where the returns to government assistance may be higher. It can show whether improved procedures seem to make enough of a difference to cover their costs. Evaluation of sites in the Philippines suggested that farmer participation provided sufficient benefits to cover the additional costs costs of a more participatory approach (de los Reyes and Jopillo 1989). However the question of whether total benefits exceed total costs also deserves to be addressed.
Assessment can contribute to projects aimed at improving the process through which government intervenes in irrigation. The diversity and large number of sites of intervention in small scale irrigation make detailed economic assessment of each site expensive and inappropriate. The results of assessment of a sample of existing projects can be used to develop rules for screening sites and potential construction. Based on assessment economic criteria can be built into project planning so it is not just based on abstract technical requirements or local wish lists.
Average levels of benefits per hectare irrigated can be compared with typical capital costs to suggest how much area needs to be irrigated. For the KKU-NZ weirs in northeast Thailand it was suggested that weirs irrigating more than thirty hectares would probably have enough benefits from increased rice production alone to justify their cost. However smaller weirs needed careful screening to check as to whether there were other benefits which would be sufficient to justify the investment.
Studies could be done to provide more specific criteria for improvements such as lining canals, something farmers frequently request. The implicit value of water saved can be analyzed. This value, in terms of additional production, could then be compared with the cost of lining. This would at least allow some limits or general criteria to be suggested as to when lining is clearly economically worthwhile or when benefits are likely to be worth much less than costs.
Similarly in small irrigation systems farmers frequently use temporary diversion structures which must be rebuilt or repaired each year. Farmers often request government aid to build more permanent structures. If building a more permanent diversion structure will reduce the amount of labor and materials used in repairs, this can be compared with the capital cost of a new more permanent headworks of gabion, mortar or concrete construction.
... Cost-effectiveness analysis is a very useful intermediate step between ignoring economic considerations and using the full apparatus of benefit-cost analysis. What is the least expensive way to achieve a specific goal, or how can a given amount of money be used most productively. ...
Such an analysis will often show that the benefits from reduced repair costs are far smaller than the cost of building a more permanent structure. This suggests that farmers have already developed an economically appropriate technology and that government funds would be more productively used elsewhere.
Comparison of different approaches to intervention in irrigation can show major differences in costs, even though project results may be similar. The KKU-NZ weirs provided benefits similar to the RID projects, but at less than a tenth the construction cost. So for many sites the cost of intervention could have been greatly reduced, while providing equal or higher benefits.
Cost-effectiveness analysis is a very useful intermediate step between ignoring economic considerations and using the full apparatus of benefit-cost analysis. What is the least expensive way to achieve a specific goal, or how can a given amount of money be used most productively. In may cases there are several ways to produce the same result, for example diverting water from a stream into a canal or reducing water losses. Alternatives should be identified and assessed in order to find the cheapest effective solution and make the best use of limited government resources. Perhaps the most beneficial result from evaluation is not grand conclusions about sectoral investment policy or net impacts, but rather showing how worthwhile it can be to develop local participation and other changes which strengthen cost-effectiveness analysis in planning interventions in small scale irrigation.
Paper presented at the Irrigation Studies Center at Andalas University, West Sumatra, October 24, 1990. Published in Indonesian as "Intervensi pada Irigasi Skala Kecil: Beberapa Prinsip Penilaian." Visi 5 (April 1991): 6-13.
Views expressed in this paper are the responsibility of the author and do not necessarily represent the official views of any institutions or projects which I am presently or have previously been affiliated.
The study of the Khonkaen University-new Zealand weirs was primarily funded by the New Zealand government. My role in it was as part of dissertation research supported by a Fulbright grant. The study of RID weirs and reservoirs was carried out as part of the Small Scale Irrigation Systems Project funded by the Ford Foundation. The studies discussed here are described in my dissertation and the study reports (Nalinee Tantuvani, Pithi Angsuwothai and Bryan Bruns 1986 and Bruns 1987.
Revised May 23, 1992. [html 97-07-13]
Comments and questions regarding this paper should be directed to Bryan Bruns at 39/1 Ban Daun Ngeun, A. Pong, Phayao 56140 Thailand, or at:
Adams, Dale W., Douglas H. Graham and J.D. Von Pischke. 1984 Undermining Rural Development with Cheap Credit. Boulder: Westview Press.
Bruns, Bryan 1991 The Stream the Tiger Leaped: A Study of Intervention and Innovation in Small Scale Irrigation Development in Northeast Thailand. Ph.D. Dissertation, Cornell University. Forthcoming.
Bruns, Bryan, with Taweesit Panyeung and Somkiat Khonchan. 1987 "Rice, Vegetables and Fish: 25 Small Scale Weirs and Reservoirs in Khonkaen." Water Resources and Environment Institute, Khonkaen University.
Chambers, Robert 1983. Rural Development: Putting the Last First. Harlow, UK: Longman.
Coward, E. Walter. n.d. "Action Experiments in Irrigation Development: Identifying Policy and Program Implications."
de los Reyes, Romana P. and Sylvia Ma. G. Jopillo. 1989. "The Impact of Participation: An Evaluation of the NIA's Communal Irrigation Program." pp. 90-116 in Korten and Siy 1989.
Gittinger, J. Price. 1982. Economic Analysis of Agricultural Projects. Baltimore: Johns Hopkins University Press.
Goldsmith, Edward and Nicholas Hildyard. n.d. The Social and Environmental Effects of Large Dams. A Report to the European Ecological Action Group. Bourdeaux, France.
Gramlich, Edward. 1981. Benefit-Cost Analysis of Government Programs. Englewood Cliffs, New Jersey: Prentice Hall.
Hafid, Anwar and Y. Hayami. 1979. "Mobilizing Local Resources for Irrigation Development: The subsidi desa Case in Indonesia. pp. 123-133 in Taylor and Wickham eds. Irrigation Policy and the Management of Irrigation Systems in Southeast Asia. Bangkok: Agricultural Development Council.
Khonkaen University. 1987. Proceedings of the 1985 International Conference on Rapid Rural Appraisal. Khonkaen, Thailand: Rural Systems Research and Farming Systems Research Projects.
Kikuchi, Masao, Geronimo Dozina and Yujiro Hayami. 1978. "Economics of Community Work Programs: A Communal Irrigation Project in the Philippines." Economic Development and Cultural Change 26(2):211-225.
Korten, Frances F. 1987. "Making Research Relevant to Action: A Social Learning Perspective." pp. 293-306, International Irrigation Management Institute and Water and Energy Commision Secretariat of the Ministry of Water Resources, Government of Nepal. 1987. Public Intervention in Farmer Managed Irrigation Systems. Sri Lanka.
Korten, Frances F. and Robert Y. Siy Jr., editors. 1989. Transforming a Bureaucracy: The Experience of the Philippine National Irrigation Administration. Hartford Connecticut: Kumarian Press.
Morgan, Arthur E. 1971. Dams and Other Disasters: A Century of the Army Corps of Engineers in Civil Works. Boston: Porter Sargent Publisher.
Murray, Charles A. 1988. In Pursuit of Happiness and Good Government. New York: Simon and Schuster.
Palanisami, K. in association with Adul Apintara. 1984. "Performance Evaluation and Investment Priorities in Irrigation Systems of Northeast Thailand." Khonkaen Thailand: Faculty of Agriculture, Khonkaen University.
Rossi, Peter H. and James D. Wright. 1987. "The Iron Law of Evaluation and Other Metallic Rules." in Research in Social Problems and Public Policy. vol. 4, ed. Joanne Miller and Michael Lewis (Greenwich Conn.: JAI Press.
____. 1984. Evaluation Research: An Assessment. Annual Reviews in Sociology 1984, pp. 331-352.
Schmid, A. Allan. 1989. Benefit-Cost Analysis: A Political Economy Approach. Boulder: Westview Press.
Tantuvanit, Nalinee, Bryan Bruns and Piti Angsuwotai. 1986. Lessons from the KKU-NZ Weirs: An Evaluation of the Khonkaen University-New Zealand Small Scale Water Resources Project. Khonkaen: Water Resources and Environment Institute, Khonkaen University.
Tubpun, Yuavares. 1983. "The Economics of Small Tank Irrigation in Northeastern Thailand." Paper Presented at Water Management and Policy Workshop, Khonkaen University September 13-15, 1983.
Varley, Robert C.G. 1989. Irrigation Issues and Policy in Indonesia: 1968-1988. Harvard Institute for International Development Development Discussion Paper No. 322. Published in Indonesian as: "Permasalahan dan Kebijaksanaan Irigasi di Indonesia: 1968-1988." Pusat Studi dan Pengembangan Irigasi, Lembaga Penelitian, Pendidikan dan Penerangan Sosial dan Ekonomi (LP3ES), Jakarta. 1990.
Veneracion, Cynthia C. 1989. A Decade of Process Documentation Research: Reflections and Synthesis. Manila: Institute of Philippine Culture, Ateneo de Manila University.
C-723 Lanna Condo, T. Pa Tan A. Muang Tel 66-53 872-225
Chiang Mai 50300 Thailand Fax 66-53 872-226