Consultant for Project End Line Evaluation 2023-78 - Tirah Valley

Terms of Reference / Request for Proposal (RFP) of the Project End Line Evaluation 2023-78 For Rehabilitation Support for Returnees in Tirah Valley of Khyber District (Khyber Pakhtunkhwa Province) Muslim Aid Pakistan (MAP)

Introduction

a. Proposals are invited for End Line Evaluation of the Rehabilitation Support for Returnees in Tirah Valley of Khyber District (Khyber Pakhtunkhwa Province) according to the conditions as prescribed in the ToRs.

b. Interested consultant/consultancy firms may get ToRs free of cost from Muslim Aid Pakistan, Country office, Islamabad, during office timings i.e., 9.00 am to 5.00 pm from Monday to Friday, or download from the Muslim Aid Pakistan website.

c. MAP may conduct reference and background checks for selected consultants/Firms.

d. MAP is an equal opportunity employer and requires Consultants to sign its Code of Conduct and Child Safeguarding Policy.

e. Proposal must be submitted within 5 days from the date of advertisement at Muslim Aid Pakistan Islamabad Office or submit online.

Background

Muslim Aid is an international charity organization registered in the UK (Registered Charity Number 1176462). Working since 1985, Muslim Aid has been serving humanity around the globe as one of the largest Muslim development and humanitarian organisations in the U.K. 

In Pakistan, Muslim Aid started working in 2005, under the Trust Act with registration no. 24340. Our strategic thematic working areas are improving integrated water resource management, access to education, sustainable livelihoods, health, emergency response and resilience building. Our country programme comprising of dedicated experts is always been striving to alleviate the sufferings of the most marginalised and vulnerable segments of society affected either because of natural calamities or man-induced disasters. 

The global organisation currently has presence in seven countries including Bangladesh, Bosnia, Pakistan, Somalia, Sri Lanka, Sudan and Myanmar and eight partner organisations. 

Muslim Aid has commitments to:

• Sustainable Development Goals
• Core Humanitarian Standards 
• Accountability to affected populations 
• Principles of Partnership

MAP recently implemented an emergency project for Rehabilitation Support for Returnees in Tirah Valley of Khyber District (Khyber Pakhtunkhwa Province), targeting the most vulnerable approximately 750 HHs flood-affected and their family members as indirect beneficiaries which are 4,500 families in 10 villages, 7 main areas Tirah valley, Tehsil Bara, District Khyber Pakhtunkhwa (KP) province. The humanitarian action was designed to address with the aims to support crisis affected populations in Tirah valley of Khyber District. In line with the needs assessment conducted by Pakistan Armed Forces (focal District for development and humanitarian efforts in the district), MAP and its Implementing Partner (IP) Action for Society Welfare Awareness & Development (ASWAD) adopted an integrated approach to livelihoods and agricultural support for the returnees to the Tirah valley. In response to the needs identified; these actions contributed to longer-longer-term development by providing livelihoods support to beneficiaries through different interventions.

The needs identified by the assessment aligned to government priorities i.e., districts Khyber severely insecure, communities especially children and women needing immediate support to tackle severe food insecurity (through nutritional support) and contributing to resilience building for in capacitating communities to better prepared and adapt to future conflicts. The action has been aligned with the government of Pakistan’s priorities for sustainable livelihood initiatives and hence the ownership is increased by co-creation and engaged relevant stakeholders such as Agriculture and Livestock Departments which has been instrumental in scaled up and scaling up these interventions further in the future.

In total, 750 individuals have been directly benefited through livelihoods components of project. However, the entire population of the targeted area of Tirah valley 4,500 individuals of 750 HHs is benefitted indirectly as part of the provision to support public facilities including:

Forming 10 Village organizations in the targeted villages and each village organization has 20 members and total 200 members.

Project Summary

Office/Organization Muslim Aid Pakistan (MAP)

Project Title Rehabilitation Support for Returnees in Tirah Valley of Khyber District (Khyber Pakhtunkhwa Province)

Theme ☐ Education ☐ Healthcare   ☐ Livelihoods ☒ Emergency ☐ Seasonal

Crosscutting Themes ☐DRR ☐WASH ☒Gender ☒Protection ☒Older Persons ☒Disability ☐Environment ☐Other: [please specify]

Duration in Months Six Months Proposed Project Dates 15 October 2022 – 14 May 2023

Direct Beneficiaries Approx 750 HHs

Indirect Beneficiaries 4,500 Individuals of 750 HHs

Location of Project Tirah Valley, District Khyber, of Khyber Pakhtunkhwa (KP) province

Project Goal and Outcomes

Goal/Aim: To improve food security resilience for 750 vulnerable returnee families through the revival and restoration of livelihoods in Tirah Valley, Khyber District (Khyber Pakhtunkhwa)

Outcome 1: Improved food security and income through livelihood interventions for 750 vulnerable returnee families at Tirah Valley, District Khyber.

Outcome 2: Communities are empowered to take ownership of the rehabilitation programme and secure their rights for livelihood, especially for women, girls and PWD

Purpose and Scope of work

The purpose of the End Line Evaluation is to assess whether the project aim to achieve its objective and outcomes described in the proposal/LFA. And finally, to document of the best practices, lessons learned and recommendations during the process and will guide decisions regarding Relief, Response and Recovery Services in the future with a focus on quick response. The end-line project evaluation seeks to determine whether the objectives of the project have been achieved and to provide a learning tool for MAP to enhance decision-making at all levels of future project implementation for the disaster affected communities. It will also identify any emerging issues within project implementation and provide a guideline to effectively address these issues.   

It is expected that the work of the consultant/firm shall include, but not be limited to, the following;

Assessment of the Project in terms of the followings:

1. Relevance

2. Effectiveness

3. Efficiency

4. Quick Impact

5. Lessons learnt

Program relevance

How well the project addressed the key issues or problems that affected the communities over the project period.

Program efficiency

The measure of the extent to which the project has used the least possible resources to achieve the required quality of outputs/outcomes.

Program effectiveness

An assessment of the effectiveness of the various processes involved (assessment, selection of beneficiaries, registration, training, distribution of appropriate items/ tools, delivery of inputs, grievance, and redressal mechanism) in terms of whether the intervention has achieved what it set out to achieve in its plans.

Quick Impact

The analysis of Swift Impact, the project brought about in the lives of crisis affected communities in the District Tirah-Khyber. 

Evaluation Major Objectives

1) To assess the relevancy, efficiency, effectiveness, timeliness and immediate impact of the project’s goal and outcomes.

2) To undertake an evaluation on how the project improve food security resilience and income through the revival and restoration of livelihoods of the returnees’ families. 

3) Undertake an analysis of the identification and selection of the families that were part of the project.

4) To evaluate the impact of the Village Organizations (VOs) in terms of inclusion and empowerment for women, girls and PWD and the strengthen of community’s relationships.

5) To assess the efficient use of financial resources to achieve its outputs, specifically to determine the optimal use of resources regarding the quality of outputs and project delivery, and the value for money generated by the project.

6) To assess the longer-term impact and sustainability of the actions in terms of Response and Relief Initiatives implemented during the period; with a view to document lessons learned, best practices, and provide recommendations (for replication, modification, and further effectiveness) that will be relevant to stakeholders.

Key Evaluation Questions

The key questions against each Evaluation Major Objective for which we need answers are the following:

1) To assess the relevancy, efficiency, effectiveness, timeliness and immediate impact of the project’s goal and outcomes.

Q1: How effectively project\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'s goals and outcomes were aligned with the identified needs and priorities of the target beneficiaries?

Q2: Did the interventions effectively achieve its intended outcomes? What aspects work well? If so, for whom, to what extent and in what circumstances? 

Q3: Were the project activities completed within the planned timeframe?

Q4: What was the immediate impact of these interventions in the overall livelihoods of households?

Q5: How did the beneficiaries perceive the sustainability and long-term impact of the project on their communities? 

2) To undertake an evaluation on how the project improve food security resilience and income through the revival and restoration of livelihoods of the returnees’ families. 

Q1: How effective was the utilization of through agricultural, and poultry inputs to enhanced food security resilience?

Q2: What was the impact of the of different trainings provided such as Orchard Management and Pre & Post Harvest Fruit Management for men, Kitchen Gardening and Poultry Management for women?

Q3: To what extent did these interventions (agricultural and poultry inputs) impact the economic empowerment of women within households and how much families improved their income through all the project interventions?

Q4: What was the impact of these interventions on the productivity and sustainability of agricultural and poultry initiatives? 

3) Undertake an analysis of the identification and selection of the families that were part of the project.

Q1: How effectively were households identified and selected in the project, and what criteria were used to determine eligibility?

Q2: Were there any challenges or discrepancies in the identification and selection process?

Q3: What challenges were faced in the inclusion of women beneficiaries?

4) To evaluate the impact of the Village Organizations (VOs) in terms of inclusion and empowerment for women, girls and PWD and the strengthen of community’s relationships.

Q1: How do VOs collaborate with other community stakeholders to achieve their goals (inclusion of women, girls & PWDs), and what impact have VOs had on community relationships and cohesion?

Q2: What specific initiatives were taken by VOs to promote the inclusion and empowerment of girls, women, and PWD?

Q3: What challenges were faced by VOs in the inclusion and empowerment of girls, women, and PWD and how they addressed these challenges?

Q4: What challenges were faced by VOs in the identification of project most vulnerable beneficiaries? 

5) To assess the efficient use of financial resources to achieve its outputs, specifically to determine the optimal use of resources regarding the quality of outputs and project delivery, and the value for money generated by the project.

Q1: Were financial resources allocated and utilized optimally to achieve the desired outputs and project deliverables? 

Q2: Was the project cost-effective and efficient in terms of the value for money?

Q3: Was the utilization of resources efficient throughout the project\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'s implementation? 

6) To assess the longer-term impact and sustainability of the actions in terms of livelihoods support implemented during the period; with a view to document lessons learned, best practices, and provide recommendations (for replication, modification, and further effectiveness) that will be relevant to stakeholders.

Q1: What longer-term impacts have been observed as a result of the livelihoods support implemented during the project period?

Q2: How sustainable are the initiatives implemented by the project?

Q3: What challenges were faced in obtaining the NOC from District Authorities and Law Enforcement Agencies to access the target communities?

Q4: What lessons learned, and best practices were documented throughout the evaluation process?

Q5: What recommendations can be provided for replication, modification, and further effectiveness of similar initiatives, and how relevant are they to stakeholders?

Q6: What were the specific aspects of the project that were most beneficial or impactful?

Q7: Were there any challenges or gaps in addressing the needs and expectations of beneficiaries through the project?

General Questions: 

1) Do the beneficiaries/communities have information regarding how to contact, provide feedback or make complaints through MA Complaint Response Feedback Mechanism (CRFM)?

2) Were the beneficiaries/communities were satisfied with the project implemented activities?

3) What unintended results – positive and negative – did the intervention produce? How did these occur?

4) How efficient was the collaboration/coordination with stakeholders such as relevant line departments, law enforcement agencies and implementing partner (ASWAD)?

5) How far ASWAD has been successful in the implementation of and achieving project envisaged objectives. Consultant may suggest/add more questions keeping in view of the deliverables of the project?

6) Are benefiting communities satisfied with the project interventions particularly which directly positive impact on them.

Methodology 

The overall End Line Evaluation process will be based on participatory research techniques involving the beneficiaries, their family members, Influential, Elected Representatives, Govt. Representatives from line departments, MAP staff, IP-ASWAD and Other Organizations worked/working in that area.  Both qualitative and quantitative aspects of the project should be brought under consideration. 

The following general steps are proposed for conducting the project End Line Evaluation:

1. Review the proposal and related project documents.

2. Prepare and initiate a report with methodology, checklist, and research tools with input from MAP

3. Conduct interviews with key stakeholders involved

• Project proposed 10% Target HHs, Groups and Beneficiaries as well as Community Organizations representatives. Assuming 95% confidence level, 5% margin of error and 50% response distribution with a gender distribution of 50% males and 50% females, with a 10% flexibility

• Government line departments (DCO, Law Enforcement Authorities, Livestock, Forrest, Agriculture Department 

• Elected and appointed district government representatives

• Concerned MAP staff at the field and at the Country Office.

• Concerned IP (ASWAD) staff

4. Evaluate the project in the field by applying quantitative and qualitative research tools

5. Prepare a draft report using standard reporting protocols and organize in consultation with MAP.

6. Get feedback from MAP

7. Final report

a. Executive summary

b. Context of End Line Evaluation and methodology

c. Sampling

d. Gender profile

e. Key findings

f. Recommendations 

g. Lessons learned

h. Strengths and Limitations

i. Conclusion

Please note that the above guidelines for the Key evaluations questions and methodology are intended to assist with analysis and report preparation. They do not limit the responsibility of evaluators to exercise their best judgment, to avoid redundancies and repetition, and to focus attention on significant issues.

Timeframe and Deliverables

1. Evaluation report with methodology and research tools (05 days)

2. Evaluation at the field (07 days)

3. Submission of Draft Report after 07 days of the finalization of field evaluation shall cover both qualitative and quantitative aspects arising from the above examinations

4. Dissemination and consultation with key stakeholders (relevant staff, partners, Government officials, etc) (3 days)

5. A copy of the final report to be submitted in spiral binding (along with a softcopy of the database and the report in Word or database format and not a PDF file). All filled-up questionnaires should be submitted and do add pictures from the field.

6. An oral presentation of the findings 

7. 03 case studies with photos explaining the impact of the project – one male, one female, and one PWD

The suggested time frame is 30 days with a possible start in the end of May 2024. The final report must be submitted by the end of June 2024. However, the consultant will submit a proposed work plan with key milestones within a week of signing the contract, which will be reviewed and approved by MAP. It is anticipated that the first draft of the report will be produced within 20 days of the signing of the contract. The final reports should be submitted within 7 days on receiving feedback.  During the whole period, follow-up meetings will be held between the evaluators and MAP.  

Required Competencies of the Consultant/ Consultancy Firms

The eligibility criteria for participating individuals and organizations are as under:

• Proven track record of conducting participatory reviews/research and End Line Evaluation for similar programs and must have successfully completed at least 5 studies/projects of the same nature

• Proven Expertise in quantitative and qualitative study methodology and instruments design and successful experience of data collection through designed tools

• Must have professional expertise of the relevant field

• Must have the infrastructure to undertake the assignment (Office, Team, people’s time and expertise, equipment, and Financial Resources)

• Must have valid NTN number

Note: Failing to qualify on any of the above criteria will not be considered for further evaluation 

Content of the Proposal

The consultant/ Consultancy Firm will be required to submit a detailed proposal for evaluation. The proposal should be divided into two parts, i.e. technical and financial. The technical part will contain the following sections

• Experience in data collection/ End Line Evaluation/Impact evaluations
• Experience in relevant field
• Experience in developing of methodologies and design quantitative and qualitative questioners/ scales of similar kind of studies
• Qualification and experience of the staff along with CVs
• Documents of available infrastructure
• Analysis reports prepared in past

 Detailed methodology designed for the evaluation

• Sampling
• Data Collection Methods & Tools
• Field Work
• Data analysis and Management

Detailed timeframe/ Work plan of evaluation (including dates for submission of the first draft and final report)

The financial part will describe the estimated cost for the evaluation in detail.

Technical and Financial Evaluation

Detailed technical evaluation will be done for Individual Consultant/firms who qualify the preliminary evaluation mentioned above (These Consultants/ firms will be called for interview and also asked to them give presentations on their proposals. 

Costing

All currency in the proposal shall be quoted in Pakistan Rupees (PKR).

Terms of payment for the service:

• Payment for the services will be made through crossed cheque.
• Payment will be made as per following deliverables:

o Acceptance of contract of award and signing of Contract/ Agreement 

Evaluation design and analysis plan                           30%

o Presentation and acceptance of draft report30%

o Presentation and acceptance of final evaluation report 40%

 Income tax, sales tax and other government duties:

All taxes, duties, Sales Taxes and other Govt. charges are responsibilities of the individual/institution. Proposal costing should be inclusive of all taxes.

Acceptance of Proposals

MAP reserves the right not to accept the lowest or any proposal and to annul the bidding process without assigning any reasons whatsoever.

General Terms and Conditions 

1. The proposal and cost shall remain valid for a period of not less than 60 days from the closing date of the submission of the proposal. 

2. The technical and financial proposals should be delivered in separate sealed envelopes. At the top left of the envelopes, it should be clearly stated “Proposal for end evaluation of emergency response project in Tirah valley, district Khyber”, and it should be clearly stated on the envelope, whether it contains the technical or the financial proposal. 

3. Obtaining NOC and other permissions from the government/LEAS/Armed forces will be responsibility of the consultant.

4. RFP must be accompanied by bid security/earnest money (refundable) for an amount of 2% of the bid value in the shape of a pay order/ bank draft in Favor of MAP. The earnest money should be included in the sealed financial proposal. RFP without earnest money or less than 2% of the bid value will not be entertained and rejected straightaway. 

5. The MAP reserves the right to accept or reject any or all RFP at any stage without assigning any reason thereof. 

6. At any time prior to the deadline for submission of bids, MAP may, for any reason, amend this RFP, whether at its own initiative or in response to a clarification requested by a prospective bidder. 

7. In case of delay in the execution of the contract, MAP reserves the right to impose a penalty not exceeding 10 % of the total amount of the contract. 

8. If the progress of work is not to the satisfaction of the MAP, the work will be awarded to another party at the risk and cost of the bidder. In such an eventuality, if any excess amount is to be paid by MAP, it will be recovered from the bidder. 

9. MAP has the rights to add, enhance or remove any functionality not disturbing the major scope of work. 

10. MAP will not bear any expense incurred in the preparation of proposals in response to this RFP. 

11. All responses to this RFP shall become the property of MAP. 

12. Proposals sent to MAP by Fax or Email will not be accepted. 

13. Proposals submitted after due date and time will be rejected. 

14. An effort by any individual/firm(s) to influence MAP, “directly or indirectly through unfair means”, in MAP proposal evaluation, proposal comparison or contract award decisions, to meet or discuss with any MAP official unless desired by the MAP may result in the rejection of bidder’s proposal. 

15. The consultant/firm will abide by MAP’s vision, mission, values in its entire period of work. 

16. The consultant/firm will sign abide by MAP’s safeguarding policy in its entire period of work. 

17. Submit no COI on MAP standard template.

18. The consultant will be hired subject to antiterrorism and money laundering check as well as reference check.