|
The Pangani River Basin drains Mount Kilimanjaro and Mount Meru in northern Tanzania and flows to the Indian Ocean in the town of Pangani in Tanga Region. The Pangani Basin has an estimated population of 3.7 million people, most of whom rely, either directly or indirectly, on agriculture for their livelihoods. The River and its Basin are greatly important to Tanzania, in terms of hydro-power, irrigation, fisheries, livestock etc. Irrigated agriculture in the basin (estimated as 30,000 to 40,000 ha) forms a significant contribution to Tanzania?s food security and supports almost 3 million livelihoods.
Currently there is not enough water to meet the demand in the Pangani Basin, yet the Pangani Basin Water Board continues to receive requests for new water permits from local, municipal and industrial investors. Climate change and reducing water supplies had yet to be taken into consideration in the allocation process. Conflicts are also emerging between various water users, such as commercial farms, small farmers and livestock keepers. The natural environment has yet to be recognized as an important water user and at present receives hardly any consideration in an already over-allocated resource.
The Government of Tanzania, IUCN, through its Water & Nature Initiative (WANI), European Union (EU) and the Global Environmental Facility (GEF) through UNDP have been committed to prepare water users and water managers in the Pangani Basin for reducing water supplies through the Pangani River Basin Management Project. The Project aims at mainstreaming the negative impacts of climate change into Integrated Water Resources Management in the Pangani River Basin, in order to support the equitable provision of freshwater for the environment and for livelihoods for future generations.
The project started its implementation in July 2006 and is executed by the Pangani Basin Water Board. The Pangani Basin Water Office, with technical support from IUCN ESARO under a special agreement form an Implementation entity through the Project Management Unit - PMU based at the Basin Office. Other partners are SNV and local NGO Pamoja. The project is funded by the Government of Tanzania, IUCN, UNDP/GEF and EU.
Project objective
The objective of the project is to prepare water managers and users for changing climatic conditions (especially reduced flows) through the provision of technical data, planning, and improved allocation, capacity building and awareness-raising. The project focuses on the following three technical areas:
- Understanding current and future climatic vulnerability (in the broadest sense of the term): developing and using information for equitable water allocation in a changing hydrological regime
- Negotiated outcomes to minimize future climatic vulnerability and future climatic risk: Continuing dialogues towards sustainable water resources management
- Mainstreaming climate change adaptation in the water sector: national linkages and lessons learned.
The three technical outputs would lead to a single measurable outcome: Management and allocation of water in Pangani Basin includes climate change preparation and adaptation and environmental considerations in a sound Integrated Water Resource Management (IWRM) framework. It is one of the first field-based climate change preparation projects in Eastern Africa with strong links to basin and national planning and policy, and as such builds national and regional capacity, provide lessons and serve as a national and regional demonstration site.
The assessment of the project impacts and achievements during the implementation period and the extraction of lessons learned both in terms of financial and technical approaches, require a proper evaluation of the project achievements and to measure the improvements or changes in the designed indicators, as a result of the project implementation, compared to the base line parameters. A set of indicators were identified during development of Monitoring and Evaluation framework for this project.
The project is seeking the services of two qualified experts (an international and a national) to conduct the terminal evaluation of the project. The consultants will assess the project achievements and impacts, in consultation with the main project stakeholders. The consultants will work very closely with the Project Implementation Unit (PIU) and the Pangani Basin Water Office in Moshi, the Ministry of water and Irrigation, the UNDP Country Office, the UNDP/GEF Technical Regional Office in Pretoria, IUCN, SNV and Pamoja. The consultants will be contracted by UNDP Country Office and, the Team Leader will report to the Deputy Country Director (DCD) programme with technical support from Energy & Environment Unit. UNDP/GEF Technical Regional Office will be responsible for quality assurance. Any support required and relevant documents will be provided as necessary to the consultants.
The Monitoring and Evaluation (M&E) policy at the project level in UNDP/GEF has four objectives: i) to monitor and evaluate results and impacts; ii) to provide a basis for decision making on necessary amendments and improvements; iii) to promote accountability for resource use; and iii) to document, provide feedback on, and disseminate lessons learned. A mix of tools is used to ensure effective project M&E. These might be applied continuously throughout the lifetime of the project ? e.g. periodic monitoring of indicators -, or as specific time-bound exercises such as mid-term reviews, audit reports and final evaluations.
In accordance with UNDP/GEF M&E policies and procedures, all regular and medium-sized projects supported by the GEF should undergo a final evaluation upon completion of implementation. A final evaluation of a GEF-funded project (or previous phase) is required before a concept proposal for additional funding (or subsequent phases of the same project) can be considered for inclusion in a GEF work program. However, a final evaluation is not an appraisal of the follow-up phase.
Final evaluations are intended to assess the relevance, performance and success of the project. It looks at early signs of potential impact and sustainability of results, including the contribution to capacity development and the achievement of global environmental goals. It will also identify/document lessons learned and make recommendations that might improve design and implementation of other UNDP/GEF projects.
|
|
The main objective of this evaluation is to provide the project partners i.e. GEF, UNDP, GoT, IUCN, and the community with an independent assessment of the key achievements of the project as compared to the objectives and outcomes reflected in the project document for the three years implementation period of the project. Assess the project expected outcomes and their sustainability and suitability for policy related review inputs and best practices for community adaptation. The evaluation results are envisaged to identify and discuss the lessons learned, through measurements of the changes in the set indicators, summarize the experiences gained and technical achievements, and recommend future policy dialogues.
TASK AND SCOPE OF THE EVALUATION
The scope of the evaluation will cover the success in mainstreaming climate change adaptation options in the Pangani River Basin. It will assess the achievement of the project in generating information for water managers to enhance water allocation in the basin, community water resources management strategies and information sharing among key partners and stakeholders in the water sector.
The following represents the minimum coverage of points to be included, but the evaluation should be adapted to specific concerns and issues that may be raised:
The mission will assess the
- Relevance of the project in terms of current development priorities and needs.
- Clarity and realism of the project's development and immediate objectives, including specification of targets and identification of beneficiaries and prospects for sustainability.
Quality, clarity and adequacy of project design including
- Clarity and logical consistency between, inputs, activities, outputs and progress towards achievement of objectives (quality, quantity and time-frame);
- Realism and clarity in the specification of prior obligations and prerequisites (assumptions and risks);
- Realism and clarity of external institutional relationships, and in the managerial and institutional framework for implementation and the work plan;
- Likely cost-effectiveness of the project design.
Efficiency and adequacy of project implementation including
- Availability of funds as compared with budget for both GEF and nationally funded components (i.e. track co-financing);
- The quality and timeliness of input delivery by partners
- Managerial and work efficiency;
- Implementation difficulties;
- Adequacy of monitoring and reporting;
- The extent of national support and commitment and
- The quality and quantity of administrative and technical support.
Project results, including a full and systematic assessment of output produced to date (quantity and quality as compared with the work plan and progress towards achieving the objectives)
After the termination of the project. The mission should examine in particular:
- The degree to which project outputs have been defined based on adequate consultation with potential product users, and used and internalised by the national focal point institutions;
- The outcomes of the consultation processes used by the project;
- The scope for uptake by other related initiatives in the region.
Review the clarity of roles and responsibilities of the various agencies and institutions and the level of coordination and efficiency of partnership between relevant stakeholders. In particular look at the roles of the Project team, PBWB, PBWO, Pamoja, district authorities, SNV, IUCN, UNDP, EU, VPO-DOE and MOW.
Assess the level of stakeholder involvement in the project from community to higher Government levels and recommend on whether this involvement has been appropriate to the achieving goals of the project.
Describe and assess efforts of UNDP (CO and UNDP-GEF) in support of the implementation.
Review donor partnership processes, and the contribution of co-finance.
Examine the potential of scaling up and replication of good practices from the project outcomes, identifying how will it be financed and, who will be responsible for financing and implementation.
Based on the GEF evaluation procedure, provide rating for each project outcome indicating at what level has been achieved.
Based on the above analysis the mission will draw specific conclusions and make proposals for any necessary further action by Governments, GEF, and/or UNDP and other partners, including any need for additional assistance and activities by the project prior to closure. The mission will draw attention to any lessons of general interest.Any proposal for further assistance should include precise specification of objectives and the major suggested inputs, outputs and outcomes.
METHODOLOGY
The terminal evaluation will be conducted in a participatory manner through a combination of processes including a review of the key project documentation, interview with project stakeholders and site visit as deemed necessary. It will include visits to UNDP Country Office, Project Executing Office/PIU, Pangani Basin Water Office, Ministry of Water Irrigation, VPO-DoE, IUCN, EU, SNV, Pamoja as well as selected national partners and stakeholders, including interviews with key individuals both within the project, government staff, NGOs, private sector and project beneficiaries mainly communities and water user associations. The following documents are recommended to be reviewed by the team:
- Project Document
- Project implementation reports (APR/PIR's);
- Annual technical reports and work plans of the various implementation task teams;
- Mid Term Evaluation report, and
- Minutes of the project Steering Committee Meetings
- Reports of the studies undertaken by the project such as EFA report, Vulnerability Assessment report, etc.
EXPECTED OUTPUTS
The consultants shall provide the project partners and the PIU with a comprehensive report with the following:
- Impacts and key project achievements identified and documented according to the project indicators;
- Project achievements and sustainability in relation to the project design;
- Relevance of the project achievements and the national policy development agenda;
- Efficiency and effectiveness of the project in terms of financial and planned activities;
- Project shortcomings and lesson learned and policy review for integrating climate change adaptation in the water policy and IWRM;
- Clear and specific recommendations for future follow-up addressed to the stakeholders in the project.
Outline for the Final Report
Executive summary
- Brief description of project
- Context and purpose of the evaluation
- Overall rating of project performance against objective and outcome as well as project implementation, main conclusions, recommendations and lessons learned.
Introduction
- Purpose of the evaluation
- Key issues addressed
- Methodology of the evaluation
- Structure of the evaluation.
The project(s) and its development context
- Project start and its duration
- Problems that the project seeks to address
- Immediate and development objectives of the project
- Main stakeholders
- Results expected.
Findings and Conclusions
In addition to a descriptive assessment, all criteria marked with (R) should be ratedusing the following UNDP/GEF six-point rating scale: Highly Satisfactory (HS), Satisfactory (S), Marginally Satisfactory (MS), Marginally Unsatisfactory (MU), Unsatisfactory (U), Highly Unsatisfactory (HU).
Project Formulation
- Conceptualization/Design (R). This should assess the approach used in design and an appreciation of the appropriateness of problem conceptualization and whether the selected intervention strategy addressed the root causes and principal threats in the project area. It should also include an assessment of the logical framework and whether the different project components and activities proposed to achieve the objective were appropriate, viable and responded to contextual institutional, legal and regulatory settings of the project. It should also assess the indicators defined for guiding implementation and measurement of achievement and whether lessons from other relevant projects (e.g., same focal area) were incorporated into project design.
- Country-ownership/Driveness. Assess the extent to which the project idea/conceptualization had its origin within national, sectoral and development plans and focuses on national environment and development interests.
-
Stakeholder participation (R) Assess information dissemination, consultation, and ?stakeholder? participation in design stages.
-
Replication approach. Determine the ways in which lessons and experiences coming out of the project were/are to be replicated or scaled-up in the design and implementation of other projects (this is also related to actual practices undertaken during implementation).
-
Other aspects to assess in the review of Project formulation approaches would be UNDP comparative advantage as IA for this project; the consideration of linkages between projects and other interventions within the sector and the definition of clear and appropriate management arrangements at the design stag
Project Implementation
-
An overall rating of project implementation employing the six-point rating scale (HS, S, MS, MU, U, HU) should be provided by the review.
Implementation Approach (R). This should include assessments of the following aspects:
- The use of the logical framework as a management tool during implementation and any changes made to this as a response to changing conditions and/or feedback from M and E activities if required.
- Other elements that indicate adaptive management such as comprehensive and realistic work plans routinely developed that reflect adaptive management and/or; changes in management arrangements to enhance implementation.
- The project's use/establishment of electronic information technologies to support implementation, participation and monitoring, as well as other project activities.
- The general operational relationships between the institutions involved and others and how these relationships have contributed to effective implementation and achievement of project objectives.
- Technical capacities associated with the project and their role in project development, management and achievements.
Monitoring and evaluation (R). Including an assessment as to whether there has been adequate periodic oversight of activities during implementation to establish the extent to which inputs, work schedules, other required actions and outputs are proceeding according to plan; whether formal evaluations have been held and whether action has been taken on the results of this monitoring oversight and evaluation reports.
Stakeholder participation (R). This should include assessments of the mechanisms for information dissemination in project implementation and the extent of stakeholder participation in management, emphasizing the following:
-
The production and dissemination of information generated by the project.
-
Local resource users and NGOs participation in project implementation and decision making and an analysis of the strengths and weaknesses of the approach adopted by the project in this arena.
-
The establishment of partnerships and collaborative relationships developed by the project with local, national and international entities and the effects they have had on project implementation.
-
Involvement of governmental institutions in project implementation, the extent of governmental support of the project.
Financial Planning: Including an assessment of:
The actual project cost by objectives, outputs, activities
The cost-effectiveness of achievements
Financial management (including disbursement issues)
Co-financing
Sustainability. Extent to which the benefits of the project will continue, within or outside the project domain, after it has come to an end. Relevant factors include for example:?development of a sustainability strategy, establishment of financial and economic instruments and mechanisms, mainstreaming project objectives into the economy or community production activities.
Execution and implementation modalities. This should consider the effectiveness of the UNDP counterpart and Project Co-ordination Unit participation in selection, recruitment, assignment of experts, consultants and national counterpart staff members and in the definition of tasks and responsibilities; quantity, quality and timeliness of inputs for the project with respect to execution responsibilities, enactment of necessary legislation and budgetary provisions and extent to which these may have affected implementation and sustainability of the Project; quality and timeliness of inputs by UNDP and the government, and other parties responsible for providing inputs to the project, and the extent to which this may have affected the smooth implementation of the project.
Results
Attainment of Outcomes/ Achievement of objectives (R): Including a description and rating (employing the six-point rating scale) of the extent to which the project's objectives (environmental and developmental) were achieved using Highly Satisfactory (HS), Satisfactory (S), Marginally Satisfactory (MS), Marginally Satisfactory (MS), Unsatisfactory (U) and Highly Unsatisfactory (HU)ratings. If the project did not establish a baseline (initial conditions), the evaluators should seek to determine it through the use of special methodologies so that achievements, results and impacts can be properly established.
This section should also include reviews of the following:
- Sustainability: Including an appreciation of the extent to which benefits continue, within or outside the project domain after GEF assistance/external assistance in this phase has come to an end.
- Contribution to upgrading skills of the national staff
Recommendations:
- Corrective actions for the design, implementation, monitoring and evaluation
- Actions to follow up or reinforce initial benefits from the project
- Proposal for future directions underlining major objectives
Lessons learned
This should highlight the best practices and poor practices in addressing issues relating to relevance, performance and success.
Evaluation report Annexes
-
Evaluation TORs
-
Itinerary
-
List of persons interviewed
-
Summary of field visits
-
List of documents reviewed
-
Questionnaire used and summary of results
-
Comments by stakeholders (only in case of discrepancies with evaluation findings and conclusions)
EXECUTION ARRANGEMENTS
The two consultants should work together as a team towards producing the evaluation report. The national consultant will be responsible for providing any necessary background information and preparation of the agreed parts of the report. The international consultant will be accountable for the timely submission of required outputs and submission of the deliverables, including the final report.
The consultants will be contracted and supervised by the UNDP Country Office in consultation with the GEF Regional Office. The PIU shall arrange for the consultants all necessary site visits and meetings in the project site according to the TOR. UNDP country Office in coordination with the PIU shall arrange logistics for the mission including hotel reservation and transportation during the mission. The mission will maintain close liaison with UNDP CO, Pangani Basin Water Office, IUCN as well as the PIU.
TIME FRAME
The evaluation will be carried out through a period of 25 working days, which includes a 15-day mission to Tanzania, including field visit in the project site.
DELIVERABLES
Submit 5 hard and soft copies of draft report after two weeks
Submit 5 hard copies and 1 soft copy of the final report.
REPORT ANNEXES
- TORs
- Itinerary
- List of persons interviewed
- List of documents reviewed
|
