Consultancy for Evaluation of Climate and Environment Program

The Institute for Sustainable Communities (ISC) seeks the services of a Consultant to conduct an external evaluation of ISC’s Climate and Environment program, with particular emphasis on the Climate Leadership Academy (CLA) in the United States. ISC would like the evaluation to help answer the following questions:

• What has been the impact of the CLA program and the four modules that have been offered to date?
• How does that impact compare with the impact model or “theory of change” (described below)?
• What evidence or indicators can be used to show that participation in the Climate Leadership Academy leads participants to take more effective local action on climate change?
• Has the CLA feature on the ISC website (especially the resource guides and case studies) contributed to the CLA’s overall impact?
• What was the impact of the CLA Network for Green Boot Camp participants (i.e. ISC’s attempt, to date, to create an on-going peer-learning program and network for CLA participants)?
• How can the Climate and Environment program team cost-effectively monitor and evaluate the CLA program and its impacts going forward?

ISC plans to scale up the CLA—transforming it from a pilot project to a robust, on-going program—by delivering 6 to 8 modules each year. As we add the new elements to our C&E Program described above, we are committed to strengthening and improving our program.

A final report, including recommendations for near- and long-term improvements, will be the foundation upon which ISC will build a more robust program for measuring and reporting on the impact of the CLA Program going forward. It also will inform ISC’s overall approach to monitoring and evaluation.
 

Organization Description

The Institute for Sustainable Communities is an independent nonprofit organization that has worked since 1991 with communities around the world to give passionate, committed people the tools, skills and resources to make their communities better places to live. Headquartered in Montpelier, Vermont, we now have offices in Washington DC, China, Serbia and Kosovo. We are currently a staff of ninety, of which seven are working on the US-based Climate and Environment Programs.

The Institute for Sustainable Communities’ US-based Climate & Environment (C&E) Program is growing rapidly. The flagship project, called the Climate Leadership Academy, is emerging from a highly successful 1.5-year pilot phase during which ISC designed and launched four modules. A module focuses on a specific topic related to climate mitigation or adaptation and consists of an intensive two and a half day peer-learning workshop designed to help US communities accelerate local solutions to global climate disruption. Each CLA employs 5-10 leading experts, called “CLA Faculty,” to lead sessions and serve as speakers. Each module also has a companion Resource Guide featuring case studies and references on the sustainability topic addressed at the workshop. The modules offered to date have been: green job creation; adaptation and resilience; transportation efficiency and building energy efficiency. Each CLA module is attended by teams of 3-6 practitioners representing 13-16 US communities (total number of participants is approximately 75 plus faculty).

In addition, this year ISC is launching two new elements of its C&E Program – the Urban Sustainability Leadership Academy (peer-learning services for sustainability directors in towns, cities and counties throughout North America) and the Gulf Coast Sustainable Economies Project (a capacity building program to help Gulf Coast shape recovery efforts, develop diverse, sustainable economies and strengthen resilience to the local impacts of global climate disruption).

ISC’s efforts to date to monitor and evaluate the impact of its US-based C&E Program, notably the Climate Leadership Academy, has been severely limited by time and resource constraints. ISC did develop an “impact model” (see below) describing the “theory of change” underlying the project design, and describing high-level desired outcomes/metrics.

However, efforts to gauge the extent to which this theory is working and these desired outcomes are occurring have been limited. Thus far, ISC’s monitoring and evaluation of the C&E Program have been limited to two mechanisms: 1) review and analysis of feedback forms submitted by CLA customers immediately after participating in CLA workshops; and 2) extensive (but sporadic) follow-up surveys with some CLA participants, in particular those attending the original Green Boot Camp (aka CLA module on building energy efficiency) and the CLA on Transportation Efficiency. The data collected via these two mechanisms to date—including analysis of feedback forms from all five CLA workshops delivered so far, as well as follow-up consultations—will be shared with the selected consultant.
 

Responsibilities

ISC C&E Program staff members will provide guidance and feedback throughout the process and will participate actively in decisions affecting the final report’s design.

The Consultant will be responsible for the project’s substantive work, including taking the lead in interviews and writing the final report text. The Consultant will provide feedback to ISC during the process and will be responsive to ISC requirements, priorities, sensitivities, or other concerns throughout the process.
 

Objectives

The goals of this project are threefold:

• To better understand the impact of the Climate Leadership Academy so far, both to guide refinements of the project as well as to support business development efforts;
• To develop a more robust and systematic approach to tracking and reporting on the impact of the CLA and other elements of the C&E Program; and
• To guide refinements to ISC’s overall approach to monitoring and evaluation, in particular of projects funded by corporations and/or foundations (i.e. non-US Government funded projects)

The Consultant will provide ISC with a final report evaluating of the impact of the Climate Leadership Academy thus far, assessing the program’s monitoring and evaluation needs, and recommending a roadmap for improvement. Emphasis will be placed on making recommendations as to how ISC can measure changes in CLA participants’ capacity to address local climate disruption, determine the program’s contributions to these changes, identify critical areas where the Climate and Environment Program can improve M&E practices and suggest changes to ISC’s monitoring and evaluation of the program.
 

Methodology

The Consultant is welcome to propose any methods or approaches to data collection and evaluation. The final methodology will be decided upon by the selected Consultant as a part of the project’s deliverables. ISC will make the following resources available to the Consultant:

• Program final reports, including the reports to the private CLA donors;
• Program documents including the initial proposals (that include goals and objectives for programmatic activities) and final work plans;
• Resource Guides;
• CLA participant surveys;
• Documentation of interviews with a diverse cross-section of CLA participants to date, conducted prior to the CLA workshops;
• Program success stories and communication materials; and
• Other referential documents as identified by consultant and/or by ISC and as appropriate/relevant.

Deliverables

The primary deliverable will be a final report that evaluates the impact of the Climate Leadership Academy, assesses the program’s monitoring and evaluation needs, and recommends an M&E roadmap for improvement, including tools and systems. The scope of this work includes three parts:
 

• Develop an evaluation methodology. Based on input from ISC and the available resources described above, the consultant will develop a proposed methodology.
• Design and conduct an assessment of the impact of the CLA so far. At a minimum, this assessment should address the validity of the program’s Theory of Change and what the data says about how well this Theory of Change explains the CLA approach and impact.
• Develop recommendations for future monitoring and evaluation of ISC’s C&E Program (including but not limited to the CLA). This should include:
• Recommended metrics for all three C&E Program components;
• Recommended mechanisms for assessing progress toward those metrics; and
• Recommendations regarding the scope and frequency of monitoring and evaluation.

The final product will be due to ISC by COB on October 14, 2011.

To ensure that the consultancy is progressing in a satisfactory manner, the following interim deliverables will also be required:

• Final methodology and work plan including timeline, interview questions and preliminary interview list, due within two weeks of commencement of work;
• Regular check-ins between the Consultant and ISC’s project manager, to be scheduled after the consultant’s selection;
• Oral presentation (may be by phone or Skype) to selected ISC C&E staff members to convey initial findings and share a proposed report outline; and,
• Draft report, due at least three weeks before the final report is to be submitted.

Level of Effort and Schedule

A mutually agreeable timetable will be sought, but ISC’s preferred schedule is:

• Start Date—July 1, 2011
• Final Methodology and Work Plan due—July 15, 2011
• Draft report due—September 23, 2011
• Final report due—October 14, 2011

Qualifications

Key qualifications to successfully complete this consultancy include:

• Solid experience in conceptualizing and conducting evaluations;
• Prior experience designing monitoring and reporting systems for nonprofits or a government entities;
• Familiarity and experience with organizations that provide capacity and technical assistance to US city and state governments, particularly on topics related to sustainability and climate change;
• Experience with evaluating complex capacity-building programs;
• Knowledge of evaluation tools and techniques relevant to evaluating topics such as network effectiveness, distance learning, adult learning, and activities to foster peer learning
• Strong written and oral communication skills

Consultants will be selected based on:

• Quality and completeness of the proposal
• Qualifications and experience with similar projects
• Proven ability to complete projects on time and within budget
• Strength of proposed methodology
• Cost-effectiveness

To Apply

Questions concerning this scope can be addressed to Debra Perry, dperry@iscvt.org. Questions will be responded to until June 3, 2011 at 4:00 PM (EST). Interested and qualified candidates should submit the following information to dperry@iscvt.org no later than close of business on June 10, 2011:

• Resume or CV with references;
• A draft of your proposed evaluation approach/methodology, including a description of each work phase, a justification of your proposed design, timeline, and any support required;
• Estimated budget between $15-$25K

Please address proposals to Steve Nicholas, VP of Climate and Environment Programs, Institute for Sustainable Communities, 535 Stone Cutters Way, Montpelier VT 05602.

ISC is an equal opportunity employer.