Wednesday, April 02, 2014

Getting authorisation to do Research and Evaluation in Schools

A colleague working in an educational NGO asked this question, about working in schools in South Africa:

I just wanted to ask a quick question. Do I need to get permission from the relevant Provincial Department of Education to carry out research in schools if the schools are part of a project we’re running? In other words, the district is aware of us and probably interacting with us?
My answer: 
I've only done research or evaluations in a few Provinces, not all of them, but in all of those Provinces the Education Departments have guidelines for researchers that require you to fill in forms, submit your research proposal (and sometimes evaluation instruments) for review, and also binds you to some promises about the use of your research or evaluation findings. (E.g. the Province may require copies of reports, may require you to present your findings, etc.) Check any of the Provinces' annual reports to see which Director in the Provincial office is in charge of Research, and lodge your enquiry about requirements there, if you can't find details on the Provincial Education Website.

The officials in Education Districts are often not aware of the Provincial requirements, so one might be able to get away without Provincial authorization, but this is a bad idea for at least two reasons: 

* It helps if the Research Directorate in the Provincial Education Department have your details on their database because it promotes use and coordination of research, and
*It can solve a lot of headaches for you should someone complain about your research going forward. 

Since Education in schools is a Provincial competence, I have been unable to get blanket approval from National Education to work in multiple Provinces - so that meant filling in the different forms and providing the different details to the different Provinces, and following up on the outcome of each of these processes.

Besides Provincial approval, some clients might also require that any human subject research gets vetted by a research ethics approval board, like the ones attached to universities, or science councils. I've only dealt with a few of these, but they mostly require you to prove that you have authorization to conduct the research, so the two goes hand in hand.

Of course approval by the Province and Research Ethics Boards are still not all that you need to do to ensure that you conduct your work ethically - Some fields (E.g. Marketing Research - see the ESOMAR guidelines),  have guidelines about ethics... so it would be good to study these and make sure your practice remains above board.

And then this, of course, is also true:

Live one day at a time emphasizing ethics rather than rules.
Wayne Dyer


Thursday, March 27, 2014

I am because you are

In a previous blogpost I reflected on how African values shape my practice of Evaluation.

This week I attended a seminar during which Gertjan Van Stam shared some provocative views on development in Africa. I started reading his book 'Placemark'. I love the way he gives voice to rural Africa. I find it interesting that this Dutch Engineer manages to give voice to Africa in a way that I can relate with.

His beautifully written take on Ubuntu:

I am, because You are

Is it possible that people in rural areas of Africa can connect with people in urban areas around the world?

That one can walk into a scene and meet someone who walks into the same scene, even if it is geographically separated?

That we explore and connect rural and urban worlds worldwide without anyone being forced into cultural suicide?

That we meet around the globe and relate, embrace, love, and build meaningful relationships?

That we find ways to be of significance and support to each other and together shuffle poverty and disease into the abyss?

That we encourage each other to withstand drunkenness and drugs, bullying, self harm, and greed?

That we share spiritual nutrition to deal with wealth, loss, alienation and pain in this generation?

That we unite through social networks, overcoming divides and separations?

That we share ancient, tested, and new resources, opportunities, visions, and dreams that lead to knowledge, understanding and wisdom?

That we collaborate to discuss, and engineer tools, taking into account the integral health of all systems?

That together, South and North, build capacity, mutual accountability, and progress, for justice and fairness?

That I am, because You are?

Monday, March 17, 2014

21st Century Skills of Rural African Teachers and Learners

I’m evaluating a project that aims to build the 21st century skills of rural African teachers and learners. Until recently I did not even know what people meant when they used the phrase 21st Century skills, but I have been enlightened and must now find a way to measure it for our evaluation. 

It seems I’m not the only one struggling with the problem of having to measure something very broad - There are a range of resources available that wrangle with the idea of defining and measuring 21st Century Skills – Some of the resources I found particularly useful include:

Everything I read, however, seems to have the focus on a context that is not rural and not African. Perhaps there is scope for our project to contribute to the general discussion on 21st Century Skills by adapting the definitions and measures specifically for our context? Perhaps this is an opportunity to develop an example of African Made, African Owned Evaluation?

Wednesday, February 26, 2014

My plans for AfrEA 2014 conference

I'm off to Cameroon on Sunday for a week of networking, learning and sharing at the 2014 AfrEA conference in Yaondé. I love seeing bits of my continent. If internet access is available I'll try to tweet from @benitaW.

I am facilitating a workshop on Tuesday together with the wise Jim Rugh and the efficient Marie Gervais to share a bit about a VOPE toolkit EvalPartners is developing. ( A VOPE is an evaluation association or society... voluntary organization for professional evaluation)

Workshop title:Establishing and strengthening VOPEs: testing and applying the EvalPartners Institutional Capacity Toolkit

Abstract: One of the EvalPartners initiatives, responding to requests received from leaders of many VOPEs (Voluntary Organizations for Professional Evaluation), is to develop a toolkit which provides guidance to those who wish to form even informal VOPEs, and leaders of existing VOPEs who seek guidance on strengthening their organization’s capacities.  During this workshop participants will be introduced to the many subjects addressed in the VOPE Institutional Capacity Toolkit, and asked to test the tools as they determine how they could help them apply such resources in strengthening their own VOPEs.

The workshop will be very interactive with lots of exploring, engaging, and evaluating of the toolkit resources. Participants should not come to this workshop expecting that they will sit still for more than 30 minutes at a time. We'll use a combination of learning stations and fishbowls as the workshop methodology.  I'm really looking forward to it!

Eventually the toolkit will be made available online. Follow @vopetoolkit on twitter for more news about developments.

I served on the boards of both AfrEA and SAMEA so I hope that the resources that the Toolkit task force and their marvellous team of collaborators put together in the toolkit will be of use to colleagues across the continent who are still founding or strengthening their VOPEs. It is hard and sometimes thankless work to serve on a VOPE board, and if this toolkit can make someone's life a little easier with examples, tools and advice, I would count this as a worthy effort.

I expect that the workshop will be a good opportunity to get some Feedback to guide us in the completion of the work.

Monday, February 17, 2014

Working Rigorously with Stories - Impact Story Tool

I've had some people email me about a paper I presented at the 2013 SAMEA conference. This paper introduces a tool for collecting and rigorously analysing impact stories that could be used as part of an evaluation. The full paper with the tool can be accessed here. The abstract is presented below:

 Beneficiary stories are an easily collected data source, but without specific information in the story, it may be impossible to attribute the mentioned changes to an intervention or to verify that the change actually occurred. Approaches such as Appreciative Inquiry and the Most Significant Change Technique have been developed in response to the need to work more rigorously with this potentially rich form of data. The “Impact Story Tool” is yet another attempt to make the most of rich qualitative data and was developed and tested in the context of a few programme evaluations conducted by Feedback RA.
The tool consists of a story collection template and an evaluation rubric that allows for the story to be captured, verified and analysed. Project participants are encouraged to share examples of changes in skills, knowledge, attitudes, motivations, individual behaviours or organizational practice. The tool encourages respondents to think about the degree to which the evaluated programme contributed towards the mentioned change, and also asks for the details of another person that may be able to verify the reported change. The analyst collects the story, verifies the story and then codes the story using a rubric. When a number of stories are collected in this way, they are then analysed together with other evaluation data. It may illuminate which parts of a specific intervention are most frequently credited with contributing towards a change.
Besides introducing the tool as it was used in three different evaluations, the usefulness of this tool and possible drawbacks are discussed.
 (The picture above is of a character known as "Benny Bookworm" from a South African TV show called "Wielie Walie" which I watched as a child)