Evaluator

Project Evaluation Terms of Reference:

Date: (Draft 1)

Partner Name: LAMP Pakistan

Project to be evaluated:

Flood Relief and Recovery Response In Sindh and Balochistan Provinces – Pakistan (DEC) March 2023 to August 2024

1. Overvie

Introduction to Partner: LAMP Pakistan 

LAMP Pakistan has been working on a non-political, non-commercial and non-religious basis. LAMP Pakistan is Child & Environment Friendly, Gender sensitive, not for profit Organization. LAMP is working in Sindh Pakistan for poverty reduction, through capacity building of the local communities through, Child Focused Community Development Programs, Livelihood, Food Security & Environment, WASH and Emergency Relief & Rehabilitation programs. Since 2007, LAMP has been registered under societies act 1860 in 2007-08. LAMP is committed to join efforts of civil society organizations, government institutions and local communities to bring sustainable changes in the lives of poor and disadvantaged communities through integrated community based approaches. LAMP Pakistan strengthens communities through Capacity Building, Group Formation and Experience Based Learning.

LAMP Pakistan has a dedicated and committed team with expertise in Emergencies & Early Recovery (Shelter), Livelihood & Food Security, WASH, Child (Education and Basic Health) and Women focused activities supporting thousands of families throughout Sindh province. LAMP has trained staff who can supervise, build capacities, have a local workforce, and volunteers who are the asset to the organization. LAMP PAKISTAN has its own internal controls for monitoring and implementation of projects.

LAMP has a Head office in Hyderabad Sindh and other offices are in Jamshoro and Sanghar. LAMP was registered as society/trust/company on 3rd June, 2008 in society act 1860, Registration # 5370. LAMP was registered with EAD on 15th March 2021.

The vision of LAMP Pakistan is “Society with improved lifestyle and equal livelihood opportunities”. With the mission of “Humanitarian development through integrated approach of capacity building, group formation & experience based learning techniques”.

The geographical areas where LAMP is actively working are Jamshoro, Tando Allahyar, Mirpurkhas, Umerkot, Sanghar, Tharparkar.

THEMATIC AREAS:

The main thematic areas are:

•             Capacity Building of local communities

•             Disaster Risk Management Shelter

•             Livelihood & Food Security (Sustainable/Organic Agriculture & Livelihood) & Environment

•             Women and Children Empowerment (Primary health, education and protection)

•             Relief, Rehabilitation and Development.

•             WASH

1. Background of the Project: 

Project Title: Flood Relief and Recovery Response In Sindh Provinces – Pakistan:

LAMP Pakistan implemented the recovery phase project in District Jamshoro Sindh for the communities affected by the flood of 2022. The project is funded by the Disaster Emergency Committee (DEC) through Tearfund Pakistan. The project implementation started in March 2023 and is intended to end by August 2024. The project targeted 8 UCs which include UC-Lakha, UC-Shalmani, UC-UnarPur, UC-Amari, UC-Shah Awais, UC-Petaro, UC-Khan Pur, and UC-Bholari.   The project aimed to achieve 3 specific outcomes through 10 respective outputs to achieve project goal. The outcomes are:

Outcome # 1: Vulnerable households in Sindh are equipped and enabled to achieve sustainable livelihoods and improved food security.

Outcome # 2:  Vulnerable communities in Sindh are equipped and enabled to have improved lives with disaster management techniques.

Outcome # 3: Vulnerable households in Sindh experience increased well-being through improved access to WASH (Water Sanitation and Hygiene) awareness, services, and facilities.

The project was able to reach approximately 1793 direct beneficiaries of 35 villages or 4 UC of district Jamshoroo Sindh through different activities including capacity building of local communities, distribution of livestock, vegetable seeds, agricultural tools, solar submersible pumps, and construction of water storage tanks, constructions of washrooms.

1. Purpose & Objective of Evaluation

The main purpose of the evaluation is to assess progress towards the achievements of the project outputs/outcomes as per the project log frame.  Additionally, it will assess the project design and evaluate the capacity to attain the results. This evaluation will generate/provide evidence-based knowledge by identifying lessons  learned, best  practices, and substantive recommendations that may be used for future programming, community action, stakeholder engagement, and project sustainability initiative.

The evaluation will focus on five key evaluation criteria as per OECD-DAC that:

1.      Relevance,

2.      Coherence,

3.      Efficiency,

4.      Effectiveness,

5.      Impact & sustainability.

Some additional criteria should also be included as it was an emergency and relief project so the connectedness and coverage will be included also.

Some key questions and focus areas under OECD-DAC evaluation criteria are annexed as Annexure-A. Please note the key focus areas are indicative and may be developed further upon scrutiny of relevant primary and secondary documents provided by LAMP.

1. Methodology

The individual/firm selected for this assignment is expected to use methodologies and techniques which are appropriate and sensitive to the context and nature of the implemented project. The individual/firm is expected to use and analyse all information sources and materials provided and referenced by the LAMP Pakistan to develop relevant tools which will be described in detail in the Inception Report.  

The individual/firm is recommended to consider the principles of Voice and Inclusion, Appropriateness, Triangulation, and Contribution and Transparency when devising suitable methodology.

The proposed methodology is expected to include the following: 

●       Quantitative component: The evaluation must follow the project baseline/initial assessment to the extent that a meaningful comparative analysis can be conducted. A household/Beneficiary survey of the project’s target communities using a sample size of 317 households/Beneficiary from the total of 1793 households/Beneficiary is required (indicating 95% confidence and 5% margin of error); this component will identify quantifiable changes or lack thereof over the project period;

●       Qualitative component: Key informant interviews, in-depth interviews  (with Tearfund staff and relevant stakeholders) and focus group discussions to highlight perceptions and experiences of the project;     

It must be noted that the selected approaches and developed tools must focus on the project’s outcomes and impacts and the final evaluation must articulate the same with appropriate evidence.

A preliminary list of documents has been annexed as Annex B with TORs for reference. All interviews and focus groups are expected to be in in-person for stakeholders within the country. The evaluation team may facilitate remote focus group discussions (FGDs) and/or Key Informant Interviews (KII) if required and agreed with LAMP Pakistan. An initial stakeholder and potential interviewee list is annexed as Annex C which includes a list of potential internal and external stakeholders.

 4.1    Ethics

The evaluator is advised to adhere to Tearfund\\\\\\\'s approach to research ethics, outlined in this guidance:******************************************** work with the LAMP Pakistan and Tearfund team to complete an ethics risk assessment ahead of data collection. The proposal should outline key ethical considerations and how these will be addressed.

Deliverables and Output

Deliverable

Specification/Expectations

Inception Report

Outline of Methodology and tools to be used during the evaluation

Final Report

• A stand-alone Executive Summary (maximum length 5 pages)
• A detailed evaluation report (maximum length 35 pages exc. annexes) with the following sections:

Section 1 – Executive Summary (no more than two A4 sides)

Section 2 – Introduction

Section 3 – Methodology

Section 4 – Context Analysis

Section 5 – Project Overview

Section 6 – Key Findings (Divided as per OECD DAC Criteria)

Section 7 – Conclusions

Section 8 – Key Insights

Section 9 – Specific Actionable and Prioritised Recommendations (maximum 10) - this may be supplemented by a list of learning points

Section 10 – Annexes (indicative)

1. 1. Terms of Reference for the Evaluation
   2. Profile of the Review Team
   3. Evaluation Schedule
   4. Documents consulted during the Evaluation
   5. Tools used for qualitative and quantitative data collection
   6. Persons participating in the Evaluation
   7. Field data used during the Evaluation, including baselines
   8. Bibliography

MAJOR ACTIVITIES THROUGH THE PROJECT:

The activities of the project over the last one year, intended to meet the goals are or have included:

• Conducting Training, workshops, seminars for the marginalized communities in the field focusing on Basic Health, Livestock management, Agriculture best practices.
• Capacity building sessions on agriculture , encouraging and imparting best practices such as reduction of chemical fertilizers and use of organic fertilizer, provision of Seeds & Fertilizers and Seed Storage Containers along with basic tool kits
• Promotion of Kitchen Gardening practices at household level along with Provision of vegetable seeds.
• Ensuring the sustainability of Project, formation of Village Development Committees (VDCs) to ensure the timely maintenance of hard components and encouraging and introducing networking strategies
• Establishing close contact with the Government and non-governmental organizations
• Provision of goats to the most deserving women to improve their livelihood and empower them for future sustainability
• Provision of WASH facilities including Water Facility and construction of Toilets along with triggering communities to adopt improved hygiene practices. 
• Sensitization of communities for DRR to be well equipped to cope with disasters   and plantation of 10,000 trees in District Jamshoro.

1. Background of the Project: 

Flood Relief and Recovery Response In Sindh Provinces – Pakistan:

In flood 2022, some locations of Sindh province were covered under water for several months, prolonging the dire humanitarian situation for people in these areas. Flood water remains stagnant in one of the districts in Jamshoro. Many families were dispersed and had no shelter. They left their houses and were not able to take anything with them. They managed to save themselves only. The effect of floods was massive as it destroyed everything and left them under open skies. Many were displaced and were compelled to stay on the roads. They needed temporary shelter, food and medicines as they were affected by numerous diseases and hunger along with the safety and security of women, young girls and children. Food security and livelihood components were major needs for the communities which were targeted by the implementation of intervention. It was a great challenge for local Government as they lack economical, physical and social resources. The team of LAMP along with Rural Education & Economic Development Society (REEDS) and SSEWA PAK took this challenge. The rapid need assessment was conducted during the month of August 2022. The team conducted initial surveys to most affected and vulnerable villages.

1. Current Focus of the Project:

Flood Relief and Recovery Response In Sindh– Pakistan

In 2022 the heavy rains and floods hit the communities badly and created a devastating impact on the lives of local communities of rural Sindh. The communities migrated to safe places without any food availability, living under open skies with severe unhygienic conditions facing challenges of safety and security for girls especially. With the help of Dutch Relief Alliance the need assessment was conducted from 24th to 31st of August 2022 and 8 UCs in Jamshoro district were identified in consultation with local communities and local leaders and focal persons. The selected UCs were Lakha, Shalmani, UnarPur ,Amari, Shah Awais, Petaro, Khan Pur and Bholari. LAMP Pakistan was able to reach 1135 direct beneficiaries by providing Multi Purpose Cash (MPC) and Food Security & Livelihood (FSL) support in Jamshoro district. LAMP provided cash support to help the communities to increase their purchasing power with the support of cash grants. Communities were able to purchase food items & medicine for their survival. The project was sensitive to the needs of the communities including women, men, boys and girls at all age levels. Further support was provided to 800 farmers for Agriculture Inputs as they lost their crops and their tools. LAMP also included special people (disabled) in the list of beneficiaries so they should not be left out. The main objective was to support communities to survive in this crucial time but also to equip them to become sustainable. Not all the needs of communities were fulfilled as it was an emergency phase. Communities require more support to face the future. 

In March 2023 LAMP was supported by DEC as the proportion of flood affected communities was huge. The project aims to not only support through hard components but also planned to build the capacities of local communities. The staff managed to conduct planned training sessions and awareness sessions. The team conducted distributions also.

 The following are the specified outcomes and outputs along with activities conducted in the villages.  

Outcome # 1:

1. Vulnerable households in Sindh are equipped and enabled to achieve sustainable livelihoods and improved food security.

Output 1.17: People receive 70 climate smart agriculture awareness sessions in phase 2 (736 beneficiaries/participants, women 471, Men 265) to improve the agricultural practices such as use of organic fertilizer, crop rotation and strategies for the future.

Activities: 200 flood affected farmers received 200 Agriculture Inputs, Humic Acid and agricultural tool kits (7 tools in each bag that includes: Head pan, Grape Hoe, sickle, hand Rake, Pickaxe, Garden fork, Showel and wheat seeds of 50 kg 1 bag per acre) of land.

Output 1.4: People receive livestock (Goats)

Activities: 600 HH (total 1200 goats, 2 female goats to each selected beneficiary) assisted through livelihood restoration activities (farmers received goats)

Output 1.7:  People receive seed banks to store their seeds

Activities: Provision of 600 seed banks to 475 women and 125 men (household and community)

Output 1.8: People receive agriculture inputs

 Activities:       Distribution of vegetable seeds to promote kitchen gardening to 300 families of selected 30 villages.

 Output 1.9: People receive agriculture inputs (seeds and tool kits)

Activities: Distribution of Agriculture Inputs  to 200 farmers (women 146, male 54) (seeds and took kits)(Wheat seeds  50kg 1 bag (fakhar e Bakhar) Humic acid (Jalender) Fertilizer urea Granular 50kg 2 bags). (Headpan, Grape Hue, Sickle, Hand Rake, Pickaxe, Garden fork, Showel).

Outcome # 2.1:  Vulnerable communities in Sindh are equipped and enabled to have improved lives with disaster management techniques.

Output 2.1: People receive DRR training

Activities:  70 DRR trainings; (736 beneficiaries/Participants, 471 women, 265 men)

Output 2.8: Communities equipped for climate change adoption

 Activities: Distribution of 10,000 trees sapling in 30 selected villages

Outcome # 3: Vulnerable households in Sindh experience increased well-being through improved access to WASH (Water Sanitation and Hygiene) awareness, services and facilities.

 Activities: 70 Hygiene promotion sessions in communities, sessions with school children, hygiene clubs for children and street theatre)

 Output 3.1: WASH (Water Sanitation and Hygiene) awareness services and facilities

 Activities: Installation solar submersible pumps in 30 selected villages

Output 3.2: Communities equipped with WASH infrastructure

 Activities: Construction/rehabilitate of water storage tanks in 30 selected villages.

 Output: 3.3: People receive 54 hygiene awareness Trainings. The training attended by 1034, women 901 and men 133)

Activities: Livestock Management Training Participants 945 women 680, male 255

60 Washroom were constructed.

1. PURPOSE OF THIS EVALUATION:

This evaluation will cover the period of March 2023 to August 2024. It is a “Social Economic Evaluations or Closing Evaluations”, which will inform the LAMP team about:

•  Changes in social and economic relationships that have been evident in 35 communities
•  Changes in terms of knowledge, attitudes and practices in 35 communities
•  Behaviours and social changes that took place and how has this project contributed to them?

The evaluator will be required to use

     4.1. Evaluation Focus 1:  Review the current activities and assess the Relevance/Appropriateness of provided support

4.2. Evaluation Focus 2: Review current activities and outcomes of the project and their effectiveness at contributing towards the project goals.

Specific, evidence-based comments are requested on

Overall

1.         The influence and observable changes as a result of the project in marginalized communities in Jamshoro.

2.         The capacities of communities to advocate and link with government and other groups to access resources that will assist their communities.

3.         To what extent have communities and people with disabilities been included in project activities

4.         Are there any other outcomes – positive or negative - observable upon which the project has had influence?

5.         Understand the experience of women in being part of the project, and the extent to which the project has contributed to them becoming ‘change agents’ in their communities.

6.         Comment on the extent to which the project contributed to (taking into account differences due to gender / disability / ethnicity):

a.         Increased percentage of women participation in meetings and decisions.

b.         Participation of adults with disability in the project activities.

7.         To what extent have activities assisted communities towards the overall program goal?

Specific, evidence-based comments are requested on:

4.3       Evaluation focus 3: Sustainability and strategy.

Specific, evidence-based comments are requested on

Sustainability and linkages

1.      How LAMP are seeing the linkages with Govt department helping in future sustainability

2.      How are communities managing their responsibilities linked to ownership of provided resources? What does LAMP need to learn from this existing project implementation?

3.         Comment on the Village Development Committees in 35 villages– specifically:

a.         on-going operation,

b.         active engagement,

c.         capacity to lead change in their community,

d.         linkages with other Offices

4.         What strategies have been successful for LAMP and communities to increase sustainability?

5.         What is being learnt about the sustainability of the other aspects of the project at the completion of project specifically: community ownership to the given items, livestock and tools, women’s empowerment groups, capacity building and so on.

6.         What activities have LAMP initiated to improve sustainability, and did the communities adopt those changes to strengthen themselves?

Effectiveness

7.         Are the project inputs and activities contributing effectively and efficiently towards project goals? Comment on ineffective or inefficient use of resources.

8.         Comment on the progress and effectiveness of the staff served in the project? 

4.4. Evaluation Focus 4: The evaluation will also ensure that the Core Humanitarian Standards have been followed and implemented throughout the delivery and completion of the project.

1. appropriateness and relevance of provided support
2. Effectiveness of support and timely response
3. capacities of Stakeholders have been enhanced and strengthen
4. Project Staff and communities developments strategies to diffuse the negative effects
5. Communities are effectively communicated, participated at every level of project and able to share their feedback
6. Complaints were heard and addressed
7. Complementary and Coordinated response was provided