AYA girl
AYA
  BOTSWANA GHANA TANZANIA UGANDA
 
Overview
Strategies
The Toolbox
Photo Gallery
Related Links
Contact Us
AYA Home
 
AYA Logo
   
MONITORING & EVALUATION
Photo
Youth celebrate the AYA program in Uganda.

Background
In adolescent sexual and reproductive health (ASRH) programs, a clear need exists — particularly in developing countries — for well-documented evaluations (process and impact) linking program intervention, changes in youth behavior and health outcomes. Evidence or program impact is mixed in literature reviews of HIV/AIDS programs, with evaluations suffering from “inadequacies in study design, analytic techniques, outcome indicators and reporting of statistics” (UNAIDS, 1997). Commonly, programs lack a focused conceptual framework showing intersecting processes through which program strategies aim to achieve impact. Without this program theory, the choice of intermediate and impact indicators (the basis of evaluation) is haphazard (Shepard, 2001).

For the ASRH field, with little evidence of best practices, the current focus should be the testing of innovations; yet few projects include evaluation and those that do are too new to allow assessment of quantitative outcomes (Senderowitz, 1994). Scaled-up youth programs, widely acknowledged as important to reach youth in need, are still rare. Other research identifies two basic reasons evaluations
have only demonstrated limited results: many have been short term and therefore unable to show results, and most programs that have been evaluated apply a limited set of strategies that are insufficient to change behavior (Adamchak, et al, 2000). In this regard, evaluation of AYA — its design, approach, strategies and impact — is critical to

  • Build evidence to support the relationship between program intervention, changes in youth behavior and health outcomes
  • Evaluate and document the innovation of partnerships and the “alliance” model
  • Identify components of successful scaled up interventions
  • Better understand through process evaluations why programs do/do not achieve desired outcomes

The AYA Monitoring and Evaluation Mandate
The African Youth Alliance (AYA) program seeks to contribute to the improved sexual and reproductive health of young people aged 10-24 years old, with particular emphasis on those 10-19 years old. Specifically, AYA’s goals include reduction in HIV and other STIs, unwanted pregnancy, unsafe abortion, and forced/coerced sex; and increased condom use, age of first sex and contraceptive
prevalence rate. Program objectives, individually and collectively, contribute to achievement of the program goal. Intermediate results identify essential steps to achieving program objectives. Indicators will be used in the assessment of progress (monitoring) and successes (evaluation) at the goal and program objective level.

AYA’s underlying logic is based on —

  • the well established relationship of condom use, age of first sex and number of sexual partners on HIV/AIDS, sexually transmitted infections, pregnancy (AGI, 2002, UNAIDS, 2000, 2002) and by deduction, abortions. AYA seeks to achieve ASRH heath outcomes by promoting selected behavioral outcomes: increased condom use, abstinence, use of modern contraceptives, reductions in the number of sexual partners, and elimination of first sex that is forced/coerced.
  • the integration of six program areas to achieve the objectives:
Policy and Advocacy An improved, enabling and supportive environment
for ASRH
Behavior Change Communication Increased knowledge, skills, norms and positive
attitudes towards adoption of safer sexual practices
Youth-Friendly Services Increased access to and use of quality, youth-friendly
ASRH services
Livelihood Development Programs Integration of ASRH into livelihood programs
Coordination and Dissemination Established and/or strengthened coordination and dissemination mechanisms for improved ASRH partnerships
Institutional Capacity Building Strengthened implementing partner institutional
capacity to sustain ASRH outcomes
  • The use of participatory and sustainable program design and implementation approaches in all program areas, to achieve the following cross-cutting objectives:

Partnership Establish internal and external partnerships for effective
and integrated program design and implementation
Youth Participation Maintain active participation of youth in program
design and implementation
Gender and Sexuality Design AYA programs to reflect and address issues of
gender equity and sexuality (including ASRH rights)
Sustainability Increase sustainability of ASRH programs
Scaling Up Increase number of youth reached in a broader geographic
area by institutionalizing effective programs

AYA will aggressively seek these results (goals and program objectives) and practice results-oriented
management of its initiatives and activities. Monitoring and evaluation will be an on-going process to regularly assess and report on progress towards achievement of these results. Successes, best practices and lessons learned, as identified by this process, will be disseminated through various
knowledge sharing mechanisms. AYA works to ensure processes of monitoring, evaluation, documentation and dissemination occur sequentially. Dissemination serves both a knowledge sharing and partnership strengthening function with various stakeholders, including other donors, implementing partners, NGOs, host communities, host governments and youth.

Program Design
AYA recognizes that its ability to monitor and document the program in a high quality and meaningful way is inextricably linked to the design and implementation of the program itself. In recognition of this, AYA has identified and defined “what” the program aims to do (goal and program objectives) and “how” AYA will accomplish its aim (strategies and activities). AYA will also “show” the progress of these strategies and activities towards the goal and program objectives (indicators). Program design is evidence-based, drawing on documented successes and lessons learned and applying state-of-the-art approaches, to the extent possible, from ASRH evaluations.

Monitoring and Evaluation Challenges
Partnerships like AYA play an important role in the achievement of positive sexual and reproductive health outcomes. But such partnerships also face unique challenges that have an effect on — and may even shape — the results-oriented programming approach —

  1. The involvement and management of multiple partners at different operating levels.
  2. As an approach, partnerships are an innovation for which little documented evidence exists to show that they work (evaluations, lessons learned, success stories, best practices. Can partnerships achieve stated outcomes? Can outcomes be sustained? Can partnerships reduce the cost of initiatives? Are they the best choice for scaling up?
  3. Differences in implementing partners’ capacity to implement, monitor and evaluate ASRH activities — with the result that the intensity and quality of program implementation, monitoring and evaluation will vary.
  4. Significant reliance on implementing partners to conduct monitoring and evaluation activities.
  5. Differences in partners’ geographic coverage. Some implementing partners will be national organizations, while others will be community-based — with the result that the scope of program implementation will vary significantly.
  6. Comparability issues when implementing and evaluating a program that is regional and spans countries with different characteristics.
  7. Attribution issues due to the high level of involvement of other donors in AYA countries who are implementing similar programs.
  8. Varying degrees of political commitment of host countries, whose participation is critical to implementation, monitoring and evaluation.

Monitoring and Evaluation Approach
Characteristics of the AYA monitoring and evaluation process include —

Design
• Baseline and endline surveys, a mid-term assessment, annual school surveys, as well as qualitative and formative research to inform program design and to support and enrich population-based quantitative data.
• A dose response evaluation of AYA implementation and control sites (as part of the endline evaluation), based on variances in time, intensity, quality, and levels of integration and partnership attained, and their subsequent effect on the program goal and objectives.

Responsibility
Specific partner responsibility for monitoring and evaluation of respective program areas according to program objectives.

Monitoring
On-going monitoring to ensure that all activities in all sites are implemented as designed and according to joint workplans.

Documentation
Process evaluation and documentation of the partnership/alliance management process to show how a multi-site, multi-approach program is coordinated to go to scale.

Research
Operations research studies that will be determined based on maximizing opportunities in which AYA is best suited to contribute to the body of ASRH knowledge. Primarily, AYA is not an operations research project and therefore, to the extent possible, will leverage external resources to achieve
monitoring and evaluation objectives through external partnerships and technical assistance.

Structure
• Technical advisory committees at the global and country level to advise AYA on development and implementation of a monitoring and evaluation plan.
• An internal advisory and technical group made up of partner agency monitoring and evaluation staff, to provide technical assistance for practical implementation and support.

Process
• Participatory, consultative and inclusive approaches that ensure the involvement of stakeholders at the different operating levels.
• On-going capacity building for AYA implementing partners and staff. As appropriate, maximize opportunities to use and/or strengthen existing monitoring and evaluation systems (e.g. research and data collection systems).

Dissemination
• Dissemination of evaluation findings, identification of best practices and lessons learned and knowledge/information sharing using existing systems, with ASRH stakeholders. Promotion of the dissemination effort through various fora at global, national and district levels.