Multi criteria analysis
2.7.1 Where possible, costs and benefits should be valued in money terms. However, it is not always cost-effective or practical to do so. In many assessments there are non-monetary impacts such as environmental, social or health effects that can not be valued cost-effectively. These non-monetary costs and benefits must be taken into account and should not be regarded as any less important than the monetary values. They may be crucial to the decisions needed.
2.7.2 The aim therefore is to find a suitable way to assess non-monetary factors and present them alongside the money values. In the simplest cases, it may be adequate just to list and describe them. However, it will often be appropriate to use a more sophisticated technique. The umbrella term Multi-Criteria Analysis (MCA) is frequently used to describe the range of techniques available. The Department for Communities and Local Government published Multi-Criteria Analysis: A Manual in January 2009 which provides a detailed explanation of MCA theory and practice.
2.7.3 MCA brings structure and transparency to judgement of how options compare regarding factors that are not expressed in money values. It should generally relate closely to the stated objectives of the project, and consists of comparative assessments, both quantitative and qualitative, of how well each option meets the objectives. Sometimes the stated project objectives are sufficient to serve as the relevant criteria for the MCA, in other cases they may need to be developed into a set of more detailed criteria.
2.7.4 The nature of the option assessment can vary from (in the simplest cases) qualitative description, or ticking a box to indicate that an option satisfies a particular constraint; or (in larger or more complex cases) measurement of impact in suitable non-monetary units, or the use of relative weights for each criterion and explicit scoring or ranking of each option.
2.7.5 MCA techniques include, for example:
- impact assessment: this method tabulates the impact of each option upon each non-monetary factor in an impact statement or performance matrix. This is a versatile approach that is generally recommended by DoF - see 2.7.9 below for explanation
- the weighted scoring method: this involves assigning numerical weights to each factor to reflect its comparative importance; scoring the performance of each option against each factor on a numerical scale; and calculating a 'weighted score' for each option. Detailed guidance on the use of this approach is given in the weighted scoring method
2.7.6 MCA can be used as a way to bring data expressed in units other than money values into the appraisal process. It can be used to rank options or choose a preferred option, and usually involves an explicit relative weighting system for the different criteria relevant to the decision. This often involves an implicit monetisation of different impacts - especially once the performance against the various criteria is compared to the costs that are deemed worth spending to secure or to avoid them.
2.7.7 The available techniques should be considered carefully before choosing the method most appropriate to the case in hand. It is good practice to cover all non-monetary factors by either the impact statement method or the weighted scoring method. It is not helpful to cover some factors in a weighted scoring calculation and others in an impact statement. This can cause confusion and invalidate the rankings emerging from weighted scores.
2.7.8 Whatever the technique adopted:
- it is important to make clear how the options compare regarding non-monetary factors. Information should be presented in a way that facilitates this e.g. by use of suitable tables or matrices
- costs and benefits should be quantified in suitable non-monetary units where possible. For example, quantitative measures or indicators should be used to differentiate the performance of options against the needs and objectives established at the outset of the appraisal. Research may be needed to determine the best units of measurement
- details of the methods and assumptions used should be recorded
2.7.9 The impact assessment approach lends itself to most situations and is the method generally recommended by DoF. It involves assessing the impact of each option upon each relevant objective or assessment criterion. The presentation is often in tabular form, with the cells of the table containing suitable quantitative impact measures or indicators; and/or qualitative impact analysis. An accompanying commentary summarising the main trade-offs and other features of the analysis should generally be provided. Departmental economists can advise on the design of suitable impact statements. Examples are available in the TEO Policy Toolkit. The HM Treasury Green Book supplementary guidance page lists several guides on the assessment of various specific types of impact, including health, environment and transport among others.
2.7.10 In some cases it is necessary to conduct specific types of impact assessments, for example; health, transport, environmental or regulatory impact assessments. The relevant departments can provide specific guidance on how to prepare these. General guidance on assessing impacts in terms of sustainability, lifetime opportunities and equality is given later in this section of NIGEAE.
2.7.11 Useful general guidance on the preparation of impact assessments has been developed by Policy Innovation Unit in TEO. Their guidance on completing impact assessments is found in Workbook 4 of Policy Toolkit.
2.7.12 If the weighted scoring method is used, DoF requires the figure work to be supplemented by explanations of:
- what the chosen assessment criteria (or attributes) mean
- why particular weightings have been chosen
- the reasoning behind the choice of each option score
Failure to explain weights and scores properly has been a feature of a number of past appraisals, and can delay approvals. DoF will not accept figures that are not fully explained.
2.7.13 In some cases the primary concern of an appraisal may be to determine the least-cost option for achieving a specified level of service provision. However, even in these cases there may be a need to appraise benefits as well as costs. It is rarely the case that options offer identical benefits; there will usually be some differences in performance that need to be appraised along with the cost information.
Sustainable development
2.7.14 Sustainable development can be defined as "development which meets the needs of the present without compromising the ability of future generations to meet their own needs" (The Brundtland Report: Our Common Future - the report of the 1987 World Commission on Environment and Development). Sustainability is not just about protecting the environment, it is about ensuring stable social and economic growth for all. For this reason sustainable development is often defined as being based on three main pillars:
- social (people)
- environmental (planet)
- economic (prosperity)
2.7.15 There is now a statutory duty on public authorities to promote sustainable development. This new duty is given legal force in the Northern Ireland (Miscellaneous Provisions) Act 2006, which was introduced following the publication of A Sustainable Development Strategy for Northern Ireland: First Steps towards Sustainability in March 2006.
2.7.16 TEO (then OFMDFM) updated the Sustainable Development Strategy in May 2010 with the publication of Everyone's Involved Sustainable Development Strategy. This document reinforces the Executive's commitment to sustainable development and sets out a number of new priority areas for action and a number of strategic objectives to achieve in the pursuit of sustainable development. The new strategy reiterates that all new strategies and policies should be subject to a sustainability scan as part of the impact assessment process (see 2.7.19). Moreover the strategy makes a commitment to ensure that policy, strategy, procurement and investment decisions taken by government are based on strategic consideration of their whole life consequences and to measure this against the achievement of long-term economic, social and environmental impacts.
2.7.17 DAO(DFP)05/08 introduced departments to specific guidance produced by the Central Procurement Directorate (CPD) and the Equality Commission for Northern Ireland on "Equality of Opportunity and Sustainable Development in Public Sector Procurement". The purpose of this guidance is to embed consideration of equality of opportunity and sustainable development into procurement practice. The guidance states that Northern Ireland procurement policy recognises that the primary objective should be to achieve best value for money, defined as 'the optimum combination of whole life cost and quality (or fitness of purpose) to meet the customer's requirements'. [Note: in November 2010 the NI Procurement Board revised this definition to 'the most advantageous combination of cost, quality and sustainability to meet customer requirements']. This definition was drawn up to stress that value for money does not mean lowest price. Any procurement must meet its requirements for equality of opportunity and sustainable development to be deemed good quality and fit for purpose. Sustainable development must be considered at the outset of any procurement process and then throughout the life of the project.
2.7.18 This guidance provides a step by step guide to integrating sustainable development and equality of opportunity into procurement projects from the strategic development stage right through to contract management. It applies equally to large and small procurements and irrespective of the form of procurement, but is to be applied with proportionate effort. The full text is available at the Equality of Opportunity and Sustainable Development in Public Procurement page of the CPD website.
Screening for sustainability
2.7.19 The initial planning stage in any project offers the widest scope for including sustainable development goals and the maximum potential for impact. The first step is to screen projects to identify whether they are likely to have a significant sustainability impact. All options should initially be screened against a number of potential social, economic and environmental impacts. Further details on these impacts and a framework for screening are set out in Workbook 4 of TEO's Policy Toolkit.
2.7.20 This toolkit was designed primarily for assessing policies but states that it "can also be applied to individual projects where this is appropriate". Under the guidance issued with DAO(DFP)05/08, it is now a requirement to assess sustainability for every procurement project, so screening by reference to the toolkit should be undertaken in every case. However, as with other aspects of appraisal, the principle of proportionate effort should be applied.
2.7.21 The DAO letter and associated guidance refer specifically to public procurement projects i.e. projects involving the acquisition, usually by means of a contractual arrangement after public competition, of goods, services, works and other supplies by the public service. It does not apply to other expenditures such as, for example, revenue spending proposals and grants to the non-government sectors (i.e. the private, voluntary and community sectors). However, such expenditures may also give rise to sustainability issues and these should be screened and assessed where they are considered relevant and significant.
Sustainability impact assessment
2.7.22 Screening of options should help to identify those potential impacts which are relevant and those which are significant enough to deserve fuller assessment. Impact assessment provides the framework for carrying out the fuller assessment. Workbook 4 of the Policy Toolkit supplies advice on how to carry out impact assessment.
2.7.23 The effort to be put into impact assessment should reflect the scale and importance of the proposal and the impacts under consideration. The toolkit recommends that a full impact assessment should be carried out for the preferred option when assessing policies. However, when dealing with individual projects, there is no general requirement to undertake a full impact assessment. Options should always be screened for sustainability impacts, but detailed impact assessment should only be carried out for any option where the effort is judged worthwhile by reference to the relevance and significance of the impacts in view. Where different options have different impacts on sustainability, the assessment should make the differences clear.
2.7.24 Further advice on the statutory duty of promoting sustainable development and the associated screening and impact assessment process can be obtained from the Sustainable Development Unit in DAERA.
Social clauses in public procurement contracts
2.7.25 The Programme for Government (PfG) includes a commitment to “include social clauses in all public procurement contracts for supplies, services and construction.” Arising from this, DoF’s Central Procurement Directorate (CPD) issued a procurement guidance note PGN 01/13 on integrating social considerations into contracts.
2.7.26 Where social considerations are to be incorporated into procurement, then these should be identified in the business case. NIGEAE explains how social considerations should be addressed in the course of the initial strategic option appraisal. The opportunity to consider social considerations arises at two main stages:
- at the earliest stages, when determining the strategic context, the need for the project, and the project objectives - that is, NIGEAE steps one to three. For example, social needs and objectives may be framed in terms of providing employment and training opportunities for people that are disadvantaged in the labour market, improving social inclusion, equality, health, transport, education and providing opportunities for small businesses including social enterprises
- when assessing the costs and benefits of alternative options. Social impacts will be considered at NIGEAE step seven when weighing up non-monetary costs and benefits. This will include assessing how the various options perform against the social objectives set at the beginning of the appraisal; and there is also specific provision for screening for sustainability and equal opportunity impact and, where appropriate, conducting sustainability or equality impact assessments
- as indicated in the Buy Social Toolkit, it should not be assumed that the inclusion of social considerations will in all cases add costs. This depends on the specific requirements and in some cases contractors can offset potential costs through use of external funding, for example, for training and targeted recruitment
2.7.27 Having identified the need which the procurement will address, completed the business case and obtained expenditure approval, departments should work with the appropriate Centre of Procurement Expertise (CoPE) to agree a specification for the procurement. This is the appropriate time to begin considering in detail what social clauses to include in the contract, bearing in mind the PfG commitment to “include social clauses in all public procurement contracts for supplies, services and construction”.
2.7.28 It is important to identify the approach to including social considerations in contracts at the earliest possible stage. The Buy Social Toolkit and the Buy Social Team in Strategic Investment Board (SIB) may assist with this. This will ensure that the business case and procurement strategy are developed to maximise the benefits delivered by contracts.
Assessing and documenting sustainable development in business cases for procurement projects
2.7.29 DoF wrote to NI departments in December 2014 reminding them that sustainable development must be considered at the outset of any procurement process and then throughout the life of the project; and that social clauses (or "community benefit clauses") must be included in all public procurement contracts. It also clarified how to record consideration of these factors in a proportionate manner in business cases and introduced a checklist to be included in all business cases for procurement projects requiring DoF approval at SOC and OBC stages. For details of what is expected at SOC, OBC and FBC stages, refer to the full text of letter FD(DFP)11/14.
Distributional effects (equality of opportunity and lifetime opportunities)
2.7.30 Policies, programmes and projects may give rise to distributional effects, for example, between people of different incomes, ages, genders, religions, ethnic groups, health states, skills, or locations. Expenditures or other policy proposals often lead to both gainers and losers, and information on how the costs and benefits are distributed among different individuals, organisations, or sectors of the economy can be very important. In general, proposals that deliver greater net benefits to lower income groups should be rated more favourably than those that benefit higher income groups.
2.7.31 Significant distributional effects should be identified and, as far as possible, quantified in appraisals and evaluations. How the options differ regarding these effects should be analysed in much the same way as for other non-monetary factors. For instance, where an impact statement is being used, the distributional impacts may be summarised in it, together with those of all the other non-monetary factors. Alternatively, they may be scored as attributes in a weighted scoring exercise.
2.7.32 In Northern Ireland, particular attention should be paid to local policies and legislation including, for example, the statutory equality obligations of the Northern Ireland Act 1998 and Government's Anti-Poverty and Social Inclusion Strategy for Northern Ireland : Lifetime Opportunities. Further details of how to assess equality impact and lifetime opportunities impact are given below.
Equality of opportunity
2.7.33 Section 75 and Schedule 9 of the Northern Ireland Act 1998 came into force on 1st January 2000. The legislation places a statutory obligation on public authorities (including some UK departments operating in Northern Ireland) in carrying out their functions, to have due regard to the need to promote equality of opportunity between the range of social categories (i.e. religious belief, political opinion, gender, marital status, having or not having dependants, ethnicity, disability, age, or sexual orientation); and, without prejudice to this obligation, to have regard to the desirability of promoting good relations between persons of different religious belief, political opinion or racial group.
2.7.34 The main vehicle by which a public authority fulfils its statutory obligations is through its equality scheme and the commitment to carry out equality impact assessments (EQIAs) on new policies which satisfy the screening criteria set out in the Equality Commission for Northern Ireland document A Guide for Public Authorities (April 2010). The Commission has also issued guidelines on carrying out EQIAs in Practical Guidance on Equality Impact Assessment (Feb 2005). These and other information on equality issues are available at the Equality Commission website.
2.7.35 The legislation only requires an EQIA for new policies that meet the Equality Commission's screening criteria. However, as indicated above, DAO(DFP)05/08 has introduced specific guidance that requires consideration of equality of opportunity in every public procurement. Equality must be considered at the outset of any procurement process and then throughout the life of the project.
2.7.36 Equality impact, where relevant, should be taken into consideration in broadly the same way as environmental impact, health impact, transport impact and so on. The impact on each option should be considered. As when considering sustainability, proportionate effort should be applied to equality assessment. In some cases, screening may reveal no significant equality issues. In others, there may be substantial equality impacts to take into account, in which event the principles of the EQIA guidance should be applied, with proportionate effort. The appropriate depth of analysis is a matter for judgement depending upon the importance of the equality issues in individual cases.
2.7.37 The Equality Commission has issued specific guidance on how to screen for equality and conduct an EQIA, and they can advise departments on their requirements under the statutory obligations. The Equality and Good Relations Directorate within TEO provides general advice on equality issues. In addition, all departments have equality branches, usually located within central management units.
Lifetime opportunities: the government's anti-poverty and social inclusion strategy for Northern Ireland
2.7.38 The Government's Anti-Poverty and Social Inclusion Strategy for Northern Ireland (herein referred to as lifetime opportunities), was published in November 2006 by the then Secretary of State and aims to tackle the issues of poverty and social exclusion by targeting efforts and available resources on people, groups and areas in greatest social need. The lifetime opportunities strategy was developed in response to a commitment by the 2002 Executive to review the New Targeting Social Need (New TSN) policy, which led in turn to evaluation, analysis and extensive consultation. The strategy builds on the progress made under New TSN, sets out the challenges facing government and in particular stresses the need to prioritise the issue of child poverty.
2.7.39 The strategy is structured into four key life stages - early years (0-4), children and young people (5-16), working age adults and older citizens. Each of the life stages is expresssed in terms of an overarching goal followed by a series of long-term targets that will work towards achieving the goal. The challenges and resulting goals and priorities outlined in lifetime opportunities are as follows:
- eliminating poverty
- eliminating social exclusion
- tackling area based deprivation
- eliminating poverty from rural areas
- shared future - shared challenges
- tackling inequality in the labour market
- tackling health inequalities
- tackling cycles of deprivation
2.7.40 Lifetime opportunities targets social need objectively, wherever it is found, and does not favour or discriminate against any section of the community. It is not a spending programme and it does not have a separate budget. It is a policy running through all relevant existing spending programmes, across all Northern Ireland departments, including next steps agencies, non-departmental public bodies and north-south implementation bodies. It means changing the way they target the money they have, so that more of it can be used to benefit those who are most disadvantaged.
2.7.41 Lifetime opportunities is one of the impacts listed for screening in policy toolkit number four. The approach to assessing it in economic appraisal and evaluations is broadly similar to that for sustainability and equality. The first step is to screen the project to determine its relevance to lifetime opportunities. Where a project is considered to have relevant and significant implications for lifetime opportunities, the next step is to assess their impact in suitable detail for each option under consideration.
2.7.42 General advice on lifetime opportunities is available from the Social Inclusion Unit in TEO.