Consultant: Sustainable Land Management Project Terminal Evaluation

The Project on Building Capacity and Mainstreaming Sustainable Land Management in Timor-Leste is a Global Environment Facility (GEF)-funded Project through the United Nations Development Program (UNDP). The Project has been implemented by the Ministry of Agriculture and Fisheries through the National Directorate of Coffee and Forestry (NDCF), over a three year period (2007-2010). The Project has been supervised by the Project Steering Committee (PSC) and received technical guidance from a Project Working Committee (PWC).

Further details on the background and activities of the SLM Project are contained in the Project Document (2007), the Mid-Term Evaluation Report (2009) and the Project Manager?s Terminal Report in August 2010.

OBJECTIVES OF THE EVALUATION
In accordance with the UNDP/ GEF Monitoring & Evaluation (M&E) policies and procedures, all projects must undergo terminal evaluation at the end of the project. The four general aims are: i) to monitor and evaluate results and impacts; ii) to provide a basis for decision making on necessary amendments and improvement; iii) to promote accountability for resource use; and iv) to document, provide feedback on, and disseminate lessons learned.

Terminal Evaluations (TE) are intended to provide an objective and independent assessment of project implementation and impact, including achievement of global environmental benefits and lessons learned to guide future conservation efforts. Specifically, the TE will assess the extent to which the planned project outcomes and outputs have been achieved, as well as assess the relevance, effectiveness and efficiency of the project as defined in the guidelines for Terminal Evaluations . The evaluation will also evaluate the strengths and weaknesses of project design, implementation, monitoring and adaptive management and sustainability of project outcomes, including the project exit strategy. The evaluation covers the entire project including non-GEF financed components.

SCOPE OF THE EVALUATION

The TE should consider and report on the following evaluation issues and criteria:

1.Project relevance and consistency with country priorities and the GEF Land Degradation Focal Area (specifically GEF-4 Strategic Objective 1 & 2, to develop an enabling environment that will place Sustainable Land Management in the mainstream of development policy and practices at the regional, national, and local levels; and to scale-up SLM investments that generate mutual benefits for the global environment and local livelihoods).
2.Ownership of the project at the national and local levels; stakeholder participation at national and local levels.
3.Effectiveness in realizing project immediate objectives, planned outcomes and outputs; the effects of the project on target groups and institutions; the extent to which these have contributed towards strengthening the institutional, organizational and technical capability of the government in achieving its long-term sustainable development objectives (including environmental management goals).
4.Sustainability of project achievements and impacts, including financial and institutional sustainability, and an assessment of planned replication and exit strategies.
5.Management arrangements, including supervision, guidance, back-stopping, human resources, and the Implementing Agencys (UNDP) supervision and back-stopping; the quality and timeliness of inputs, activities, responsiveness of project management to changes in the project environment and other monitoring feedback.
6.Financial planning and sustainability, including the timely delivery and use of committed co-financing.
7.Efficiency or cost-effectiveness in the ways in which project outputs and outcomes were achieved.
8.Adaptive management, including effective use of logframe, UNDP risk management system, annual Project Implementation Reviews, and other parts of the M&E system, tools and mechanisms as appropriate; evaluate whether project design allowed for flexibility in responding to changes in the project environment. Review the recommendations of the MTR and assess how the MTR had helped adaptive management of the project.
9.Risk management, including the UNDP risk management system within ATLAS, which is also incorporated in the annual PIR. The evaluators are requested to determine how effectively the risk management system is being used as an adaptive management tool. Risks may be of a financial, socio-political, institutional, operational, environmental (or other) type.
10.Cross-cutting issues:

• Poverty reduction: How has the project contributed to poverty reduction through SLM initiatives in the pilot sites and enhanced sustainable livelihoods
• Governance: How has the project facilitated the participation of the local communities in natural resource management and decision making processes
• Promotion of gender equity: Has the project considered gender sensitivity or equal participation of man and women and boys and girls in decision making processes
• Capacity development of participants and target beneficiaries.

11.Lessons and Recommendations: The evaluator will present lessons and recommendations on all aspects of the project s/he considers relevant. with special attention given to analyzing lessons and proposing recommendations on aspects related to factors that contributed to or hindered attainment of project objectives, sustainability of project benefits, innovation, catalytic effect and replication, the role and effectiveness of M & E and adaptive management in project implementation.

Ratings of key evaluation criteria
The evaluator will provide ratings for the following main criteria, in accordance with GEF Guidelines for Terminal Evaluations:

1.Rate the Relevance, Efficiency and Effectiveness of each Project Outcome:

• Highly Satisfactory (HS)
• Satisfactory (S)
• Moderately Satisfactory (MS)
• Moderately Unsatisfactory (MU)
• Unsatisfactory (US)
• Highly Unsatisfactory (HU)

2.Rate the Sustainability of project outcomes along 4 key dimensions, Financial, Socio-Political, Institutional Framework & Governance; and Environmental, using the following scale:

• Likely (L)
• Moderately Likely (ML)
• Moderately Unlikely (MU)
• Unlikely (U)

Rating the Projects M&E system
The evaluator will give a rating of the Projects M&E system:

• Highly Satisfactory (HS)
• Satisfactory (S)
• Moderately Satisfactory (MS)
• Moderately Unsatisfactory (MU)
• Unsatisfactory (US)
• Highly Unsatisfactory (HU)

In addition, the evaluation will assess the Projects contribution to establishment of a long-term monitoring system. If it did not, should the project have included such a component If it did, what were the accomplishments and short comings in establishment of this system Is the system sustainable, i.e. is it embedded in a proper institutional structure and has financing Is the information generated by this M&E system being used as originally intended.

PRODUCTS EXPECTED FROM THE EVALUATION

The main products expected from the evaluation are:

• Interim draft terminal evaluation report;
• Final terminal evaluation report refer to ANNEX 1 for suggested minimum contents.
• Completed Tracking Tool for Strategic Priority 1 & 2 of GEF 4 (Tracking Tool will be provided to the consultant for completion).

The evaluator for the SLM Project TE will be an international consultant with in-depth understanding of UNDP and GEF projects including evaluation experience. S/He will be responsible for developing the evaluation methodology, conducting the evaluation and delivering the key products expected from the evaluation. ANNEX 2 specifies the requisite qualifications of the evaluator-consultant, which applicants should address in their proposals.

The evaluator will work with a small consultative group from MAF, MED and UNDP Timor-Leste . Because the SLM Project manager and implementation unit have been disbanded already, the evaluation exercise will be supported and facilitated by the UNDP Programme officer responsible for the SLM Project.

The consultant will sign an agreement with UNDP to undertake the SLM Project evaluation and will be bound by its terms and conditions set out in the agreement.

METHODOLOGY and CONDUCT OF THE EVALUATION

The evaluation will include

• desk review of documents;
• a series of consultations or interviews to gather information and views from key stakeholders;
• a short field trip to observe SLM issues and sites of interest to the project; and
• preparation and delivery of the evaluation report. The work will be undertaken both from the consultants home base, and in Timor-Leste.

The evaluation will be carried out in accordance with the requirements of GEF and UNDP as articulated in various guidelines, policies and manuals on the conduct of evaluations for GEF projects, as well as key project documents such as the approved GEF project brief, the final UNDP project document, the inception workshop report, the project log-frame and annual budgets and work plans, the annual Project Implementation Review, Project Board/ PSC and PWC meeting minutes, and other technical reports and documents as relevant. The evaluation methodology should be documented in the final evaluation report, including details of the following:

• Documents reviewed
• Interviews conducted
• Consultations held with stakeholders.
• Sites visited.
• Techniques and approaches used for data gathering, verification and analysis.

Tentative Schedule for the TE

The consultants input will be required for 18 days (12 days in-country and 6 days to work from home), with the terminal evaluation tentatively scheduled to begin on 13 December 2010. The tentative schedule is given below.

Action Consultant inputs Dates
Preparation, documents review 2 days13-14 December
In-country consultations, and document review 5 days16 - 20 December
Field trip 2 days 21- 22 December
Further consultations and document review; report drafting 4 days 23 - 26 December
Presentation of initial evaluation progress; departure from T-L 1 day 27 December
Compilation of complete draft TE report 3 days 03 - 07 January
Delivery of draft TE report to UNDP T-L for circulation and collection of comments and additional information 10 January

Completion of final draft TE report based on comments and information received 1 day 24 January
Delivery of final draft TE report to UNDP T-L for delivery to GoTL, UNDP and GEF 26 January
Totals 18 days 5-6 weeks

Focal Persons

UNDP Country Office
Poverty Reduction & Environment Unit Head, Lin Cao-lin.cao@undp.org

Ministry of Agirculture & Fisheries
National Director of Coffee & Forestry, Luis Godhino-alba_god@yahoo.com

ANNEX 1

Suggested contents of the Terminal Evaluation report and annexes

Executive Summary

• Brief description of the project and of how the evaluation was carried out; overview of the content of the report and its findings; summary table of the ratings given in the evaluation.

Introduction

Project Concept and Design

• Review and evaluation of the project concept and design; relevance and efficiency in addressing the problem; consistency and logic of the project strategy and the logical framework.

Project Implementation

• Review and evaluation of the efficacy and efficiency of the projects management and implementation; include inter alia: use of the logical framework and broad work plans; M&E activities and changes made; adaptive management; information management; communications; participation and operating relationships between agencies and stakeholders; technical capacities achieved.
• Project perfomance: were inputs, activities, results and outcomes achieved within budget and timetable with participation of institutions; information exchange and dissemination mechanisms; financial management, co-financing.
• Effectiveness of UNDP and the PMU in management and administration of the project, including staff, procurement, experts, consultants, counterpart staff, definition of roles and functions.

Project Results and Sustainability

• Review and evaluation of achievements - outcomes, objectives, impacts, results and outputs; successes and failures in reaching objectives and desired outcomes; use of indicators
• Assessment of prospects for sustainability of the outcomes; contribution to systemic capacity development.

Conclusions

• Main conclusions of the evaluation; comments on achievements of objectives and outcomes; synthesis of relevance, effectiveness, efficiency, results and sustainability of the project.

Lessons

Recommendations

Attachments

• Terms of reference, list of documents reviewed, itinerary, people interviewed
• Summary of results; project finances ? budget, co-financing, expenditure.

Corporate Competencies

• Demonstrates integrity by modelling the UNs values and ethical standards.
• Advocates and promotes the vision, mission, and strategic goals of UN.
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability
• Treats all people fairly without favouritism.

Functional Competencies

• Operational effectiveness;
• Solid knowledge of financial and human resources management, contract, asset and procurement, information and communication technology, general administration;
• Ability to lead business processes re-engineering, implementation of new systems (business Management and Leadership
• Builds strong relationships with clients, focuses on impact and result for the client and responds positively to feedback.
• Consistently approaches work with energy and a positive, constructive attitude.
• Demonstrates excellent oral and written communication skills.
• Demonstrates openness to change and ability to manage complexities.
• Shows mentoring as well as conflict resolution skills

• The candidates should have at least MSc or higher degree in Environment, Agriculture, Natural Resource Management or related fields, and should have adequate experience in the management, design and/ or evaluation of comparable projects.
• In-depth understanding of land and environment issues in tropical/ sub-tropical and island environments. A minimum of 10 years of relevant working experience is required.
• Prior experience in evaluation of international technical assistance projects with major donor agencies, including UNDP-GEF projects, is an advantage.
• Familiar with SLM approaches in Asia-Pacifc either through management and/or implementation or through consultancies in evaluation of land related projects. Understanding of local actions contributing to global benefits is crucial.
• Demonstrated ability to assess complex situations, succinctly distil critical issues, and draw forward-looking conclusions and recommendations.
• Excellent written and verbal communication skills in English. Good knowledge of Tetum, Portuguese and Bahasa Indonesia advantageous.
• Ability to deliver quality reports within the given time.