| Description |
MID-TERM PROJECT EVALUATION, INTERNATIONAL CONSULTANT
|
| Location : |
Maseru, LESOTHO |
| Application Deadline : |
03-Aug-12 |
| Additional Category |
Environment and Energy |
| Type of Contract : |
Individual Contract |
| Post Level : |
International Consultant |
Languages Required :
|
English |
Starting Date :
(date when the selected candidate is expected to start) |
20-Aug-2012 |
| Duration of Initial Contract : |
15 Days |
|
|
Background
|
The Global Environment Facility (GEF), through the UNDP Country Office and the Government of Lesotho, has a project on “Capacity Building and Knowledge Management for Sustainable Land Management”. The Project Document was signed by the Government of Lesotho and UNDP Lesotho in September 2009, and project implementation began in earnest in 2010. The project is presently planned for four years of implementation from 2010-2013.
The project is premised on the fact that the integral functioning of Lesotho’s mountainous ecosystems is vital not only to the livelihoods and welfare of its people, but for the delivery of ecosystem services and global environmental benefits to a large part of Southern Africa. The mountainous Kingdom is the source of rivers that reach the Atlantic Ocean in the west and supply an increasing proportion of the water consumed in South Africa’s industrial heartland. Sustainable Land Management (SLM) in Lesotho is therefore a vital ingredient of broader environmental wellbeing in the region.
The Project Document looks for Sustainable Land Management (SLM) to provide a strong base for sustainable development in Lesotho while providing a range of global benefits to the region. In order to overcome identified barriers and address the corresponding programmatic gaps, the specific objective of the project is that, supported by a knowledge management network, Lesotho is equipped at local and national levels with the techniques, approaches, capacity and strategy for up-scaling successful SLM in support of national biodiversity conservation, food security and poverty reduction strategies. Three project outcomes are intended to achieve this objective:
- Proven, strengthened, participatory, replicable models and techniques that successfully overcome current institutional and governance barriers to SLM are ready for national implementation.
- Adequate local and national capacity for adapting and scaling up proven SLM models and techniques in place.
- SLM Policy Enabling Environment - Enhanced awareness, dialogue, understanding and analysis of SLM best practice at resource user, community, local government, NGO and national government levels across the country, reflected in the relevant policies, strategies and programmes.
The project is nationally implemented by the Ministry of Forestry and Land Reclamation (MFLR) as the Implementing Partner (IP). The IP is accountable to the Government of Lesotho and the UNDP for the quality of project outcomes and the appropriate use of project resources, both when directly implementing project activities and when delegating others to do so. The pilot project area is in mountain area of Maseru District.
|
Duties and Responsibilities
|
Standard GEF M&E Requirements:
The Mid Term Evaluation (MTE) is a requirement for full size and medium size GEF projects and it aims to provide the project management team and other stakeholders with strategy and policy options for more effectively and efficiently achieving the project’s outcomes and for replicating its results. It also acts as a basis for learning and accountability for managers and stakeholders. In addition to providing an independent in-depth review of implementation progress, this type of evaluation is responsive to GEF Council decisions on transparency and better access of information during implementation.
The Monitoring and Evaluation (M&E) policy at the project level in UNDP/GEF has four objectives:
- To monitor and evaluate results and impacts;
- To provide a basis for decision making on necessary amendments and improvements;
- To promote accountability for resource use; and
- To document, provide feedback on, and disseminate lessons learned. A mix of tools is used to ensure effective project M&E. These might be applied continuously throughout the lifetime of the project – e.g. periodic monitoring of indicators, or as specific time-bound exercises such as mid-term reviews, audit reports and independent evaluations.
Mid-term evaluations are intended to identify potential project design problems, assess progress towards the achievement of objectives, identify and document lessons learned (including lessons that might improve design and implementation of this and other UNDP/GEF projects), and to make recommendations regarding specific actions that might be taken to improve project implementation in the remaining period. It is expected to serve as a means of validating or filling the gaps in the initial assessment of relevance, effectiveness and efficiency obtained from monitoring. The mid-term evaluation provides the opportunity to assess early signs of project success or failure and prompt necessary adjustments.
Scope of the Evaluation:
Project concept and design: ??
- The evaluation will assess the relevance of the project concept and design. It should review the problem addressed by the project and the project strategy, encompassing an assessment of the appropriateness and cost-effectiveness of the stated objectives, planned outputs, activities and inputs as compared to alternatives. The implementation approach and managerial arrangements should also be judged; as well as country ownership, stakeholder participation, and UNDP’s comparative advantage in carrying out the project. The evaluation should assess progress in achieving the agreed indicators, review the annual workplans, and assess whether the agreed objectives are likely to be achieved given the planned duration and budget of the project.?? Implementation performance??The evaluation will assess the implementation of the project in terms of the quality of the outputs produced and the timeliness, efficiency and effectiveness with which activities have been carried out. Also, the effectiveness of management, timeliness of monitoring and the quality of technical backstopping provided by all parties to the project should be evaluated. The following are key aspects of implementation that should be addressed:??a) Project Review and Revision Process.
- By the end of the first year of project implementation, it had become clear that the indicators set in the original project document and logframe were not realistic or grounded in a rational assessment of what the project could be expected to achieve with the available time and resources. This led the project management unit to drive a review process leading to a revised set of project indicators (i.e. a revised project logframe and monitoring schedule) and coverage metrics. The evaluation should examine how this process was conducted and validate or otherwise the findings and recommendations it makes.
Monitoring Systems:
- Do the monitoring systems being used provide adequate information to assess project progress?
- Do they involve key partners?
- Are they efficient and cost-effective?
- Are additional tools required?
- Ensure the monitoring system, including performance indicators, at least meets GEF minimum requirements. Apply SMART indicators as necessary.
Risk Management:
- Validate whether the risks identified in the project document and PIRs are the most important and whether the risk ratings applied are appropriate. If not, explain why. Describe any additional risks identified and suggest risk ratings and possible risk management strategies to be adopted.
Work Planning:
- Assess the use of the logical framework as a management tool during implementation and note any changes made to it, examining their justifications;
- Ensure the logical framework meets UNDP/GEF requirements in terms of format and content;
- Assess the use of routinely updated workplans;
- Are work-planning processes result-based? If not, suggest ways to re-orientate work planning;
- Consider the financial management of the project, with specific reference to the cost-effectiveness of interventions. Any irregularities must be noted.
Reporting:
- Assess how adaptive management changes have been reported by the project management;
- Assess how lessons derived from the adaptive management process have been documented, shared with key partners and internalized by partners.
UNDP Contribution:
- Are field monitoring visits undertaken and do these visits provide useful input for adaptive project management
- Steering Committee/TOR follow-up and analysis
- PIR preparation and follow-up
- Assess the contribution to the project from UNDP “soft” assistance (i.e. policy advice & dialogue, advocacy, and coordination). Suggest measures to strengthen UNDP’s soft assistance to the project management.
Partnership Strategy:
- Assess how partners are involved in the project’s adaptive management framework
- Involving partners and stakeholders in the selection of indicators and other measures of performance
- Analyzing progress towards results and determining project strategies.
- Identify opportunities for stronger substantive partnerships;
- Assess how local stakeholders participate in project management and decision-making; Include an analysis of the strengths and weaknesses of the approach adopted by the project and suggestions for improvement if necessary;
- Consider the dissemination of project information to partners and stakeholders and if necessary suggest more appropriate mechanisms.
Progress Towards Results:
- Measurement of change: Progress towards results should be based on a comparison of indicators before and after (so far) the project intervention. Progress can also be assessed by comparing conditions in the project site to conditions in similar unmanaged sites.
- Project strategy: how and why outcomes (listed as outputs in the project document) and strategies contribute to the achievement of the expected results. Examine their relevance and whether they provide the most effective route towards results.?
- Sustainability: Extent to which the benefits of the project will continue, within or outside the project domain, after it has come to an end. Relevant factors include for example: development of a sustainability strategy, establishment of financial and economic instruments and mechanisms, mainstreaming project objectives into local government systems, etc.
Product Expected from the Evaluation:??
The key product expected from this MTE is a comprehensive analytical report in English that should, at least, include the following contents:??
- Executive summary
- Brief description of the project
- Context and purpose of the evaluation
- Main conclusions, progress rating for project implementation and towards project objectives and outcomes (using the six-point UNDP/GEF rating scale), recommendations and lessons learned
- Introduction
- Project background
- Purpose of the evaluation
- Key issues addressed
- The outputs of the evaluation and how will they be used
- Methodology of the evaluation
- Structure of the evaluation
- The Project and its development context
- Project start and its duration
- Implementation status
- Problems that the project seek to address
- Immediate and development objectives of the project
- Main stakeholders
- Results expected
- An analysis of the situation with regard to the outcomes, the outputs and the partnership strategy;
- Key findings (including best practice and lessons learned, assessment of performance)??Project concept and design
- Implementation approach
- Country ownership/Driveness
- Stakeholder participation
- Replication approach
- Cost-effectiveness
- UNDP comparative advantage
- Linkages between project and other interventions within the sector
- Management arrangements
- Implementation
- Monitoring systems
- Risk management
- Project management (work planning, reporting)
- UNDP country office contribution
- Partnership strategies
- Results
- Measurement of results
- Project Strategy
- Prospects of sustainability
- Rating of project implementation (employing six-point UNDP/GEF Rating Scale)
- Rating of Progress towards project objectives and outcomes (employing the six-point UNDP/GEF rating scale.
- Tracking and verification of project co-financing.
- Conclusions and recommendations
- Corrective actions for the design, implementation, monitoring and evaluation of the project
- Actions to strengthen or reinforce benefits from the project
- Proposals for future directions underlining main objectives
- Suggestions for strengthening ownership, management of potential risks
- Lessons learned
- Good practices and lessons learned in addressing issues relating to effectiveness, efficiency and relevance.
- Methodology or Evaluation Approach
- Annexes: TOR, itinerary, field visits, people interviewed, documents reviewed, etc.
The length of the mid-term evaluation report shall not exceed 30 pages in total (not including annexes). An outline of an evaluation approach is provided below; however it should be made clear that the evaluation team is responsible for revising the approach as necessary. Any changes should be in-line with international criteria and professional norms and standards. They must be also cleared by UNDP before being applied by the evaluation team. The evaluation must provide evidence-based information that is credible, reliable and useful. It must be easily understood by project partners and applicable to the remaining period of project duration.??The methodology to be used by the evaluation team should be presented in the report in detail. It shall include information on:
- Documentation review (desk study) - the list of documentation to be reviewed is included as an appendix to this Terms of Reference;
- Interviews will be held with the following organizations and individuals at minimum: UNDP Lesotho, UNDP/GEF RTA from Pretoria, SLM project team, the Department Range Resources Management, Project Steering Committee members, Project Manager, Community Council members, chiefs, natural resource association members, other communities/households
- Field visits;
- Questionnaires;
- Participatory techniques and other approaches for the gathering and analysis of data.
|
Competencies
|
Evaluation Team??:
A team of independent experts will conduct the evaluation. The evaluators selected should not have participated in the project preparation and/or implementation and should not have conflict of interest with project related activities.??The evaluation team will be composed of one International Consultant or team leader and one National Consultant. The International Consultant will have overall responsibility for the final evaluation report and will be expected to provide substantial analytical input into the process of framing the report’s conclusions. The local consultant is also expected to assist with analytical findings and to lead the process of document capture and the logistical arrangements for the mission. The consultants shall have prior experience in evaluating similar projects. Former cooperation with GEF is an advantage.
Team Qualities:??
- Recent experience with result-based management evaluation methodologies;
- Experience applying participatory monitoring approaches;
- Experience applying SMART indicators and reconstructing or validating baseline scenarios;
- Recognized expertise in renewable energy technologies;
- Familiarity with RE policies and management structures in Lesotho;
- Demonstrable analytical skills;
- Work experience in relevant areas for at least 10 years;
- Experience with multilateral or bilateral supported projects;
- Project evaluation experiences within United Nations system will be considered an asset;
- Excellent English communication skills.??The National Consultant will provide input in reviewing all project documentation and will provide the International Consultant with a compilation of information prior to the evaluation mission.
Specifically, the national expert will perform tasks with a focus on:
- Review documents;
- Prepare a list of the outputs achieved under project;
- Organize the mission programme and provide translation/interpretation when necessary;
- Participate in the design of the evaluation methodology;
- Conduct an analysis of the outcome, outputs and partnership strategy (as per the scope of the evaluation described above);
- Assist Team leader in finalizing document through incorporating suggestions received on draft related to his/her assigned sections.
|
Required Skills and Experience
|
International consultant?:
Education:?
- Masters or Ph.D. degree in environmental sciences or in another area related to international development;
Experience:
- Extensive (at least 10-year) experience and proven track record with land management, policy advice, project development and project implementation;
- Highly knowledgeable of participatory monitoring and evaluation processes, and experience in evaluation of at least 3 projects with a major donor agencies;
- Familiar with the issues pertaining to sustainable land management in Africa either through management and/or implementation or through analytical consultancies;
- Demonstrated ability to assess complex situations, succinctly distills critical issues, and draw forward-looking conclusions and recommendations;
- Ability and experience to lead multi disciplinary and national teams, and deliver quality reports within the given time;
- Writing and communication will be in English, and must have excellent communication skills in English. The consultant must bring his/her own computing equipment.
|
|
