The MTN Awards for Social Change: Rewarding good M&E
The MTN Foundation, in partnership with Trialogue, launched the MTN Awards for Social Change in 2019, to encourage and reward good monitoring and evaluation (M&E) practice in the non-profit sector. A total of R1 million of prize money was awarded to winning non-profit organisations (NPOs) in each of the three categories, as well as a fourth bonus award winner.
MTN awarded R300 000 to each of the winning NPOs in the following categories:
Seventy NPOs across the three categories entered these Awards. Entries were shortlisted by Trialogue, with the 20 shortlisted applicants reviewed and scored by three judges. Overall, the judges found that applicants have an appreciation for the value of M&E, building
in processes and checks to monitor and track progress over time. However, such efforts are often limited to the monitoring of outputs, with significantly less emphasis placed on the measurement of changes at outcome level. One of the contributing factors to this is that the pathways to change (theory of change) for interventions are often unclear, such that organisations are unable to clearly articulate how their inputs and activities contribute to the achievement of their stated objectives. Where M&E was done well, organisations shared the findings internally and externally, and used these to improve both project design and implementation.
Registered NPOs and/or public benefit organisations were required to enter a project or programme that is creating positive impact, has been running for at least two years and has some level of associated M&E practice. Submissions were judged against project design, outputs, outcomes and advanced M&E practices as follows:
The design of the project, including the needs that are being addressed, are clearly defined and illustrate extensive stakeholder consultation. There is an evidence-based design with strong objectives and activities. Quality control and external factors or risks that could affect the project have been considered.
The outputs are comprehensive and use appropriate data collection methods. The targets expressed are realistic, with coherent and truthful reporting on actual performance in terms of volume and scale, as well as whether there was a deviation of project outcomes.
The outcomes are understood, correctly identified and have appropriate data collection methods, realistic targets and actual performance data. Deviations from targets are explained and outcome information is backed up with qualitative case studies and evaluation reports.
Advanced M&E practices (for bonus award)
A sound theory of change or logic model with an understanding of systemic changes and an exploration of causality must be illustrated. The consideration of the unintended consequences and ethical practice of the M&E on the project should also be included.
About the judges
Zulaikha Brey is a developmental economist who joined Trialogue in 2019 after seven years at DNA Economics. In this role, she analysed and evaluated development programmes, and the design and implementation of M&E frameworks for a range of public and private sector clients. She is passionate about helping organisations design and implement programmes that are impactful, and ensuring that their M&E accurately reflects their achievements and positively contributes to ongoing programme management and improvement.
Asgar Bhikoo is a South African Monitoring and Evaluation Association board member and participates in international M&E associations.
He has served in M&E roles that are internal, intermediary and external to the organisations he has worked with in his eight years in the CSI sector. His passion for helping organisations develop their M&E to improve their sustainability and impact has led him to initiatives such as the Bridge Education Community of Practice, Man Impact Accelerator programme mentor and consulting with various NPOs and social enterprises.
Kenneth Thlaka is a development practitioner with more than 18 years of experience in the non- profit sector. He has extensive knowledge on fundraising, funders and stakeholder management. He is currently the executive director at the Southern African NGO Network, SANGONeT. He completed an advanced certificate in organisational development with UNISA, an Honours in Public Development and Management at Wits Public Development Management School and a Masters degree in Public Development and Management with Regenesys Business School.
TEARS Foundation, for Help-at-your- fingertips®
Winner in the small NPO category (R300 000 prize)
TEARS Foundation supports survivors of rape and sexual abuse and has an annual turnover of R1 million. The TEARS’ Help-at-your- fingertips® programme, which began in 2012, provides support to survivors of domestic violence, sexual assault and child abuse using prompt-based mobile technology. When someone sends a distress message, using *134*7355#, they are geolocated. The person is then sent the locations of the nearest partner facilities that can assist them. In the last financial year, TEARS Foundation spent R1 million on the programme.
TEARS began developing Help-at-your-fingertips® in 2005, based on gender-based violence (GBV) data from the Department of Justice and Constitutional Development (DoJ), the South African Police Service (SAPS), Statistics South Africa, the United Nations and the University of Pretoria’s Student Health Services. Input was also included from NPOs such as the Teddy Bear Foundation, Thuthuzela Care Centres, Shukumisa, Matla A Bana, Lifeline, the South African Depression and Anxiety Group, Families South Africa, Saartjie Baartman Centre, the Christian Social Council and Moshate.
To date, 57 318 survivors have received speedy responses and access to medical assistance, counselling and criminal justice support. Through the programme, 150 SAPS officers have been trained on how to treat GBV survivors with dignity, empathy and confidentiality. TEARS also drives education campaigns across the country and collaborates with various healthcare stakeholders.
Daily activity data is collected and analysed for quality assurance and internal improvements. TEARS is a member of Shukumisa,
a coalition of 80 NPOs that contributes to a national database to inform service delivery in the country. The data also contributes to GBV trend reports produced by the DoJ.
“TEARS Foundation’s Help-at-your-fingertips® programme has an in-depth understanding of the problem areas that it is addressing, and the programme is designed to serve a training, advocacy, intervention and citizen monitoring role. Monitoring within the organisation is strong, with activities and outputs actively tracked and the evidence used for programme strengthening. The organisation is also able to track the lower- level outcomes (improved access to services), but the higher- level outcomes around systemic changes to the justice system are yet to be evaluated in full. As the programme is attempting to address systemic issues, the new thinking around outcome harvesting (as an evaluation method) could be used to determine its contribution to change.”
gold Youth Development Agency, for gold Youth Development South Africa
Winner in the medium NPO category (R300 000 prize)
The gold Youth Peer Education Model was established in 2004 to train peer mentors, in order to improve school results and social behaviour among grade nine to 12 learners in 21 disenfranchised schools in Mpumalanga, KwaZulu-Natal and the Western Cape. The programme also aims to create internships, employment and income-generating opportunities for youth from the disenfranchised communities in which the programme is run. In the last financial year, gold Youth Development Agency, which has an annual turnover of R8.9 million, spent R8.5 million on the programme.
Peer educator training sessions are hosted weekly for 1 164 participants, community upliftment projects aligned to the Sustainable Development Goals are run in the 21 partner schools, 315 work shadow experiences for peer educators and alumni are facilitated, and one ‘emerging market engagement’ workshop is hosted per province, annually.
Peer education methodology, delivering quality education under difficult circumstances and HIV-prevention were researched in great depth, the learnings of which informed the programme design, curriculum and programme indicators. The design of the model draws on global best practice and is constantly refined with evidence-based learnings from 153 communities in southern Africa. All gold stakeholders, including interns, peer educators, programme graduates, social franchisees, youth from target communities and government representatives are engaged in curriculum revisions.
M&E findings are shared with all team and board members using the balanced scorecard approach. Quarterly reviews measure progress against indicators and all programme activity targets are assessed. Findings are used to inform programme and strategic refinement. M&E findings are also shared with external stakeholders, including directors of social franchisees, and decisions are taken to improve the programme as needed. Evaluation findings are shared with all stakeholders, including funding partners.
“The gold model uses peer education and mentoring to effect change. The programme is well designed, informed by academic research and was piloted before extended roll out. gold Youth has well-established M&E practices. M&E evidence is continuously used to assess the needs of communities and has been used to scale the operations of the organisation. It is also used to hold the organisation accountable to its stakeholders.”
Teach A Man To Fish, for EEESAY
Winner in the large NPO category (R300 000 prize)
Teach A Man To Fish helps schools create fully functional student-led businesses. The Entrepreneurial and Environmental Empowerment for South African Youth (EEESAY) programme started in 2016 to teach learners life skills that empower them to run environmentally responsible businesses. In the last financial year, Teach A Man To Fish which has an annual turnover of R19 million, spent R4 million on the programme.
Currently, Teach A Man To Fish trains teachers from schools in KwaZulu-Natal and the Eastern Cape. The schools are provided with multimedia educational resources, ongoing coaching, WhatsApp communication and school visits. To further support the schools, youth advisory boards have been set up and Teach A Man To Fish hosts annual provincial events to showcase learners’ progress.
To date, EEESAY has worked with 3 967 young people in 35 schools. A total of 117 teachers have been trained since 2016. While the aim was 80%, only 45% of the learner-led businesses met the requirements of the environmental impact assessment. An unforeseen benefit is that several school businesses use part of their profits for social causes in their communities and to support disadvantaged learners.
M&E data is analysed at project, organisation and country level annually, and is used to determine impact and sustainability. Internally, annual organisational M&E reviews are used to refine the strategy, approach and content for training. Externally, Teach A Man To Fish shares data with district and provincial Departments of Education for participatory planning and problem-solving. Each school receives a report summarising learners’ progress. Additionally, data is presented to national education working groups.
“This organisation has a good grasp of the objectives of its EEESAY programme and can clearly articulate how it perceives the change that needs to happen. The problem statement is clear, with the design of the programme informed by government and academic literature, as well as consultations with key stakeholders to ensure the appropriateness of the intervention. Internal M&E practices appear strong and there is evidence of good use of data and the sharing of findings with stakeholders. The programme is yet to be externally evaluated to test the theory of change and assumptions but, overall, a strong application.”
Ubunye Foundation Trust, for the Savings and Livelihoods Programme
Bonus winner (R100 000 prize)
Established as a rural development trust, Ubunye aims to address chronic economic poverty and marginalisation through an asset- based community development approach and has an annual turnover of R4.6 million. The Savings and Livelihoods Programme is a self-sustainable, community-led and owned financial savings and enterprise initiative started by the Trust in 2013. In 2018 a total of R1.5 million was spent on the programme.
Ubunye worked in partnership with SaveAct, an NPO that specialises in the formation of savings and credit groups in rural communities. Currently, SaveAct has over 80 000 members in five provinces, and a savings and credit model that incorporates financial education and enterprise development training. In 2013 Ubunye joined SaveAct in a joint five-year programme and, through this, contributed to a national savings and livelihoods M&E system which is still in effect.
Ubunye conducts two community engagement sessions, three financial health checks and six training sessions for savings group members, annually. The programme also includes training and monthly mentoring sessions. Seven savings events are hosted annually to evaluate the programme.
In 2018, 38 new savings groups with 720 members were formed and 30 groups successfully completed the programme, with the remaining eight completing their training in 2019. Twenty-six groups with 520 members moved onto the livelihoods module of the programme. Members reported increased savings, improved business and financial management skills, social capital and business competence and confidence.
Members have moved away from soliciting money from loan sharks to healthier ways of accessing loans, which has ultimately saved them money. The wider communities have come together to share saving experiences, stimulate business, collaborate and even introduce new ideas.
“This organisation has done very well to work with the community to shape its programme and ensure the relevance and appropriateness of design. The pathways to change for both programmes are clear, with well-articulated outputs, outcomes and objectives.
The programme demonstrates an alternative way of collecting M&E data to show accountability and learning which complements their style of programme delivery, and the findings are clearly communicated to internal and external stakeholders, and to inform programme change.”