Previous Next

Learning from agri-environment schemes in Australia

22

Improving the performance of agri-environment programs: Reflections on best practice in design and implementation

David Pannell

Key lessons

  • We should be guided by experience. Agri-environment programs have been run over many years in many countries. They provide lessons of success factors (and barriers to success) that should inform how we design new programs and projects.
  • Key elements of best practice relate to:
    • the design of programs/institutions;
    • the design of projects/investments;
    • how investment options are ranked;
    • how uncertainty is managed;
    • how people’s biases, preconceptions, and self-interest are managed; and
    • how transaction costs are managed.
  • Delivering best practice requires expertise. Agencies with responsibility for agri-environment programs should foster the development of expertise in these issues amongst their staff.

Figure 22.1: A mixed agricultural landscape showing linear plantings, patches of remnant native vegetation, and plantings in the middle of the paddock.

Source: Photo by Dean Ansell.

Research and practical experience with agri-environment programs around the world provides many lessons on what leads to success or failure. New programs are often designed without sufficient awareness of these lessons, resulting in lost opportunities to achieve more valuable outcomes.

In this chapter, I outline key elements of what I believe can be identified as best practice in the design and implementation of agri-environment programs. The recommendations are derived from various reviews of programs (e.g. European Court of Auditors 2011; Pannell and Roberts 2010), published guidelines (e.g. OECD 2010) and 15 years’ experience working closely with a range of agencies and organisations responsible for management of natural resources and the environment (e.g. Seymour et al. 2008; Roberts and Pannell 2009). I divide these issues into six sections:

  1. The design of programs/institutions;
  2. The design of projects/investments;
  3. Ranking projects/investments;
  4. Managing uncertainty;
  5. Managing people’s biases, preconceptions, and self-interest; and
  6. Managing transaction costs.

Depending on the political and administrative context, some elements of best practice may be difficult or impossible to achieve. For example, I note below the importance of long-term funding for many (probably most) environmental projects to deliver their intended benefits. However, given the political and administrative realities of the Australian Government, long-term funding arrangements for environmental projects are exceptionally rare. Perhaps the only example, the Environmental Stewardship Program, was shut down after only four years because it clashed fundamentally with administrative arrangements and culture, and because its virtues were not recognised. (See Chapter 3 for a description and reflection on the Environmental Stewardship Program.)

I have chosen not to exclude recommendations that may be incompatible with some government contexts. However, I have noted those that are likely to face the greatest challenges.

Design of programs/institutions

Additionality: Agri-environmental programs should aim to avoid paying farmers for undertaking actions that they would have done in any case. In other words, managers need to evaluate whether the benefits generated by a program investment are additional.

Continuation after investment ends: Where a program is intended to provide only temporary support to farmers (e.g. in all Australian programs, but not in European programs), it is important to ensure that the actions being supported are attractive enough that farmers will continue to undertake them once funding ends. Otherwise the investment has no enduring benefit.

These first two principles combine to mean that, where support will be temporary, perhaps the only defensible role for agri-environmental payments is to encourage farmers to get experience in a new practice that they are likely to be keen to continue once funding ends. The practice might be something new of which farmers are currently unaware, or which becomes more attractive to farmers with experience.

Appropriate institutional delivery: In some agri-environmental programs, responsibility for overseeing some or all on-ground delivery of projects is devolved to regional organisations. This has been the case in all of Australia’s major programs since the late 1990s. In these cases, the program needs to be designed in a way that provides incentives for these regional organisations to respond appropriately. In particular, they should be incentivised to pursue sustained improvements in natural resource outcomes, rather than to support project activities without considering their resulting outcomes. There should be an emphasis on spending program resources well, rather than rapidly. Unfortunately, some of Australia’s major programs have generated incentives that directly go against these recommendations. Short time frames for programs and rules that funding will be withdrawn if not spent rapidly enough increase the difficulty of meeting this best practice requirement. (Chapter 5 on environmental NGOs discusses how these organisations can help here.)

Balancing small, moderate and large projects: In programs where the availability of funding is small relative to the amount needed to fund all attractive projects (i.e. in all Australian programs), there is often a temptation to share the available resources amongst a large number of small projects. Sometimes this results in good leverage of program resources, but often it means that almost all projects have inadequate resources and are unable to achieve worthwhile outcomes. This advice sometimes clashes with political preferences to support many projects rather than few. A compromise strategy could be to use a portion of funding (e.g. 25 per cent) to support many small projects to satisfy political needs, and use the remaining 75 per cent to support larger projects that are more likely to be effective.

On the other hand, achieving the most ambitious environmental targets is often disproportionately expensive, with costs increasing dramatically as targets become more ambitious. To maximise outcomes, it may be best to pursue a moderate number of moderate-sized projects, rather than many small or few large projects.

Sufficient time for planning: Program performance is often hampered by a tendency for agencies to delay planning new programs until a previous program has ended or is about the end. Good planning and design of programs and prioritisation of investments requires more time than is usually allowed. Ideally, organisations should commence planning and analysis to develop the next program years before the end of the current program. Even though the scope and parameters of the next program cannot be known in advance, these can be predicted, and sometimes influenced, by the agency to some extent. Having already-analysed investment options ready to put forward can be highly persuasive, and increases the likely environmental benefits generated.

Investment longevity: Finally, funding for agri-environmental programs in Australia tends to be temporary and short-term — typically five years. Environmental problems usually take much longer than this to resolve, so systems for providing long-term funding should be used where possible. If it is not possible to ensure long-term funding, then this should have a strong influence on which projects are selected for funding. In particular, projects that would require significant funding in the long term to maintain the benefits generated by an initial project should not be supported. For example, most projects for control of feral animals or plants would fall into this category because feral animals reinvade once control ends. Similarly, cases where farmers are likely to disadopt practices once funding ends should be excluded. Typically, programs are much too optimistic about ongoing adoption of practices post funding.

Design of projects/investments

SMART targets: A number of agri-environmental programs have been criticised for failing to establish appropriate targets (e.g. European Court of Auditors 2011; ANAO 2008; Park et al. 2013). Specifically, targets should be SMART (Specific, Measurable, Achievable, Relevant, and Time-Bound) in order to facilitate monitoring and evaluation of a program, and to ensure that the funded investments are focused onto suitable activities (see Chapter 4 on setting SMART targets).

Sufficiency: Many projects funded in agri-environmental programs are not designed in a logically consistent way. They are consistent with a project logic, but only in a qualitative sense. They fail when assessed against quantitative questions, such as ‘are the funded activities sufficient to achieve the intended land-use changes?’, or ‘are the intended land-use changes sufficient to achieve the desired natural resource outcomes?’ A good project logic is more than a description or diagram of connections between elements of the system being managed or influenced; it quantifies the connections and makes assumptions transparent.

Selecting the right policy tool: There is a tendency for little thought or analysis to be put into the selection of policy mechanisms to be used in a project or program, resulting in inappropriate choices in many cases. In Australia, there is too much reliance on extension in situations where it cannot deliver the desired outcomes. For example, Australia’s national salinity program between 2001 and 2007 relied mainly on extension to encourage farmers to change their practices, but the practices being promoted were not attractive to farmers on the required scale and so were adopted to a very limited extent — much too limited to achieve the program’s goals (Pannell and Roberts 2010). On the other hand, in Europe and the United States, financial payments are almost the only mechanisms used, often funding activities that are not additional. The framework of Pannell (2008) helps organisations to evaluate the type of mechanism that is most suitable for a particular project. (See also Chapter 18 on the choice of tools depending on public benefits and private benefits arising from an investment.)

Sometimes programs specify which policy mechanisms will be used by projects prior to identification of the projects, and then project investments are selected without considering whether they are suitable for the predetermined policy mechanism. Preferably, policy mechanisms should be selected to match the type of projects that will be necessary to achieve the desired program outcomes. They should be project specific, to some extent. As noted in Chapter 18, Australian programs tend to rely too much on extension and too little on the development of technology.

Ranking projects/investments

Prioritisation: Where funding is limited, prioritisation of investment options is essential. The quality of the prioritisation process can make a major difference to the natural resource outcomes delivered (e.g. Barry et al. 2014).

Rank projects: Programs should prioritise projects, not problems, regions, or issues. Some programs prioritise regions or issues without defining projects, which means that it is not possible to properly consider issues of project cost, project risks, project benefits, or time lags, all of which should be factored into the prioritisation process.

Rank according to value for money: Projects should be ranked according to their value for money — a measure of their benefit divided by their cost (see Chapter 15 on designing cost-effective agri-environment schemes). Failure to do this is one of the most serious mistakes that can be made when ranking projects, but unfortunately it is common. Some systems fail to consider costs entirely, some do include costs but fail to divide by them, and many include only some of the costs that should be considered. For example, it is important to factor in long-term maintenance costs, since they vary so much between different projects, but few Australian systems do so. If maintenance costs are needed but are not expected to be provided, then project benefits should be scaled down accordingly in the ranking process.

Measure the gain against a counterfactual: When ranking projects, benefits should be estimated from the predicted difference in natural-resource outcomes with those without the proposed investment (see Chapter 19 on counterfactuals). Although this seems like common sense, Maron et al. (2013) found that 15 out of 16 systems in use for ranking biodiversity projects failed to do this correctly.

Incorporate all the benefits and risks: There are many factors that could be considered when estimating the benefits of a proposed project. The essentials are the potential values generated, the likely level of adoption/compliance with the project by landholders (Pannell et al. 2006; chapters 12 and 13), various risks that might result in project failure (technical, social, financial, and managerial risks), and time lags until benefits are generated.

Use a sound metric: A commonly neglected issue is how to combine the variables that determine benefits and costs into a metric for ranking projects. Most metrics in use are theoretically unsound and provide poor rankings, even where project information is accurate. Indeed, as is discussed in Chapter 17, the level of benefit delivered is more sensitive to the quality of the metric than the quality of the information fed into the metric. Potential benefits from investment are very sensitive to the use of inferior ranking metrics. Chapter 17 on metrics and Pannell (2015) outline the requirements for a sound ranking metric.

Figure 22.2: INFFER team members discuss the protection of environmental assets with landholders in north central Victoria.

Source: Photo by Geoff Park.

Managing uncertainty

When decisions about project funding are being made, uncertainty about those projects is usually high. Common areas of major uncertainty include the technical feasibility or effectiveness of the proposed actions to be funded by the project, and the valuation of those environmental benefits that are generated. Uncertainty should be accounted for in several ways.

Identify key uncertainties: Project proponents should be required to identify key uncertainties, and to specify what will be done in the project to reduce them.

Carry out feasibility assessment and pilot studies: Projects above a certain scale should be subject to rigorous feasibility assessment before longer-term funding is committed. Funding to support information collection, perhaps in a pilot study, should be provided for six to 12 months, after which longer-term funding should be conditional on the results obtained.

Learn from early experience: Projects and programs should be managed in an adaptive way, with information collected during early stages of the program or project being used to inform changes in management, or even cessation in some cases. In practice, few programs operate with that degree of flexibility, resulting in the continuation of poorly designed investments after their faults are apparent.

Managing people’s biases, preconceptions, and self-interest

Acknowledge the values that people bring with them: Being human, the people involved with agri-environment programs are subject to biases, preconceptions, and self-interest (consider the discussion of the different ways restoration and conservation are valued in different places in Chapter 10), all of which can reduce program performance. If managers are aware of these human traits, they can introduce systems to limit their negative impacts.

Optimism versus realism: A pervasive problem is the tendency for people to be overly optimistic about proposed projects. It is common to see proposals in which the benefits are exaggerated, and the costs, risks, and time lags are underestimated. Several factors contribute to this, including vested interests, wishful thinking, and a failure to recognise all relevant difficulties and risks that are likely to affect a project. The ideal strategy to overcome this problem is serious independent expert review of project proposals, but this is only justifiable where projects are sufficiently large. This is another factor that tends to favour moderately large projects over small projects.

Self-blindness: People involved in allocating program funds commonly perceive their existing prioritisation processes as being of high quality. For example, in a survey we found that staff from most regional natural resource management (NRM) organisations believe that their processes are better than average — clearly an impossibility. In reality, the majority of prioritisation processes I have examined have had serious problems. The common belief that they are strong is an impediment to the improvements that are needed. Addressing these misperceptions required strong leadership and participation in appropriate training, and it may be assisted by appropriate signals and incentives built into the program.

Equity versus effectiveness: Proposals to target investment in high-priority projects sometimes meet resistance in the form of arguments that this is inequitable — that resources should be distributed widely amongst many projects on the grounds of fairness. If natural resource or environmental outcomes are desired, these arguments should be resisted, as they can have a serious adverse effect on the achievement of those outcomes. A case built on maximising environmental benefits can readily be built.

Managing transaction costs

When considering potential improvements to the design and implementation of agri-environmental programs, there is a balance to be struck between improving natural resource outcomes and increasing transaction costs (Pannell et al. 2013b; Chapter 16). The most detailed rigorous approaches are only worth the transaction costs involved for relatively large projects. To limit overall transaction costs I have two suggestions.

Beware many small projects: Avoid having the program being dominated by numerous small projects for which an investment in information and analysis cannot be justified. Such programs have little prospect of delivering and demonstrating genuine natural resource benefits.

Start broad, finish deep: When evaluating potential investments, adopt a strategy of starting broad and finishing deep. In the early stages of the process, you can consider numerous potential projects, but each is evaluated in a relatively simple way that requires low transaction costs. This simple procedure is used to eliminate most of the projects from consideration. In the final stages, consider a relatively small number of project proposals, but require them to be developed in a rigorous way to allow sound decision-making about them.

Final comments

Improved natural resource outcomes from agri-environment programs are needed and expected (Audit Office reports have pointed out a repeated failure by the Australian Government to demonstrate environmental outcomes from NRM investment, ANAO 2008). Generating these outcomes is readily achievable with sufficient will, leadership, and attention to the issues raised here.

If you are involved with the design or implementation of an agri-environment scheme or program, can you answer the questions set out in Box 22.1? If you can’t, have you considered what this might mean to the success of your project or program?

Box 22.1: Key elements of good design.

A summary checklist of the issues raised in this chapter. Can you answer yes to the following questions?

1. Designing programs

  • Would farmers have adopted the desired practices even without the program?
  • Will farmers continue their adoption of the new practices once program support ends?
  • Are the institutions that are responsible for program delivery incentivised to pursue outcomes?
  • Is the typical project size large enough without being too large?
  • Is there adequate time for planning?
  • Will the practices being promoted require ongoing funding that the program is unable to provide?

2. Designing projects

  • Does it have appropriate targets?
  • Are the project activities sufficient to achieve its targets?
  • Does it use the right policy tool?

3. Ranking projects

  • Are actions (projects, not problems, issues or regions) being ranked?
  • Is ranking based on value for money?
  • Are benefits being measured against a counterfactual?
  • Are all relevant benefits and risks being factored in?
  • Is a robust metric being used for the ranking?

4. Managing uncertainty

  • Have the key uncertainties been identified?
  • Have feasibility assessments been done?
  • Can we learn from the early stages of implementation?

5. Managing people’s interests

  • Has independent expert review been undertaken to balance over-optimistic expectations?
  • Have efforts been made to deal with self-blindness?
  • Have arguments for equity undermined the effectiveness of the program?

6. Managing transaction costs

  • Does the program support projects that are too small to justify the transaction costs needed to deliver and demonstrate benefits?
  • Does project selection start broad and finish deep?

Of course, success requires recognition that there is a body of expertise that needs to be mastered, as described above. Agencies with responsibility for agri-environment programs should foster the development of this expertise amongst their staff.

INFFER (the Investment Framework for Environmental Resources) has been designed to streamline the implementation of many of the recommendations presented here (Pannell et al. 2012, 2013a; and see Box 4.1).

The knowledge and experience is there. It is within our power to improve the performance of agri-environment programs.

Acknowledgements

Thanks to the ARC Centre of Excellence for Environmental Decisions and the Australian Government’s National Environmental Research Program Environmental Decisions hub for funding support.

References

ANAO (2008) Regional delivery model for the Natural Heritage Trust and the National Action Plan for Salinity and Water Quality, Australian National Audit Office Audit Report No. 21. Available at: www.anao.gov.au/Publications/Audit-Reports/2007-2008/Regional-Delivery-Model-for-the-Natural-Heritage-Trust-and-the-National-Action-Plan-for-Salinity-and-Water-Quality.

Barry, L.E., R.T. Yao, D.R. Harrison, U.H. Paragahawewa and D.J. Pannell (2014) ‘Enhancing ecosystem services through afforestation: How policy can help’, Land Use Policy 39: 135–45.

European Court of Auditors (2011) Is agri-environment support well designed and managed? Special Report No. 7, European Union, Luxembourg.

Maron, M., J.R. Rhodes and P. Gibbons (2013) ‘Calculating the benefit of conservation actions’, Conservation Letters 6: 359–67.

OECD (2010) Guidelines for cost-effective agri-environmental policy measures, OECD, Paris.

Pannell, D.J. (2008) ‘Public benefits, private benefits, and policy intervention for land-use change for environmental benefits’, Land Economics 84(2): 225–40. Available at: dpannell.fnas.uwa.edu.au/ppf.htm.

Pannell, D.J. (2015) Ranking environmental projects, Working Paper 1507, School of Agricultural and Resource Economics, University of Western Australia. Available at: ageconsearch.umn.edu/handle/204305.

Pannell, D.J., G.R. Marshall, N. Barr, et al. (2006) ‘Understanding and promoting adoption of conservation practices by rural landholders’, Australian Journal of Experimental Agriculture 46(11): 1407–24.

Pannell, D.J. and A.M. Roberts (2010) ‘The National Action Plan for Salinity and Water Quality: A retrospective assessment’, Australian Journal of Agricultural and Resource Economics 54(4): 437–56.

Pannell, D.J., A.M. Roberts, G. Park, et al. (2012) ‘Integrated assessment of public investment in land-use change to protect environmental assets in Australia’, Land Use Policy 29(2): 377–387.

Pannell, D.J., A.M. Roberts, G. Park and J. Alexander (2013a) ‘Designing a practical and rigorous framework for comprehensive evaluation and prioritisation of environmental projects’, Wildlife Research 40(2): 126–33.

Pannell, D.J., A.M. Roberts, G. Park and J. Alexander (2013b) ‘Improving environmental decisions: A transaction-costs story’, Ecological Economics 88: 244–52.

Park, G., A. Roberts, J. Alexander, L. McNamara and D. Pannell (2013) ‘The quality of resource condition targets in regional natural resource management in Australia’, Australasian Journal of Environmental Management 20(4): 285–301.

Roberts, A. and D. Pannell (2009) ‘Piloting a systematic framework for public investment in regional natural resource management: Dryland salinity in Australia’, Land Use Policy 26(4), 1001–10.

Seymour, E., D. Pannell, A. Roberts, S. Marsh and R. Wilkinson (2008) ‘Decision-making by regional bodies for natural resource management in Australia: Current processes and capacity gaps’, Australasian Journal of Environmental Management 15(4), 211–21.


Previous Next