Reinventing the Rules
Discover the Latest Innovations and Lessons Learned in Rule of Law and Legal Empowerment Projects
Impact Evaluation on Training Traditional Leaders in Zimbabwe
Recently, USAID released a report titled “Impact Evaluation of Supporting Traditional Leaders and Local Structures to Mitigate Community-Level Conflict in Zimbabwe.” The publication discusses results from a survey conducted in two rural districts where traditional leaders were trained in human rights and mediation skills (with a special emphasis on gender rights). Interestingly, the report also points out that while major donors have trained traditional leaders in a variety of countries, this is the first time an impact evaluation was conducted to systematically study the program. Check out excerpts below!
Credit: Roberto Huner
Purpose of the Training Program
The project was motivated by pervasive tensions and violence at the community level, which many observers have attributed to the growing politicization and partisan behaviors of traditional leaders as well as their inability to deliver justice impartially. This project targets traditional leaders because they are recognized as strategic agents of change at the community level. Per Zimbabwean law, they hold responsibility for good governance, land issues, and the resolution of civil disputes.
Theory of Change
The premise behind the project is that many traditional leaders have been susceptible to coercion due to gaps in knowledge and skills. According to the IRC’s inception document, “If traditional leaders clearly understand their roles and responsibilities under the law and possess knowledge and skills in conflict dynamics and mitigation, then they will be more likely to perform their roles impartially and resolve conflicts peacefully in the community.”
The IRC’s programming was initially hypothesized to have four broad effects:
- Training for traditional leaders will improve their governance (specifically, their knowledge of the law, attitudes toward rights, impartiality in decision-making, and legitimacy in the eyes of community members).
- Training for traditional leaders will allow them to resolve conflicts more effectively and fairly, reducing conflict within their villages.
- Training for traditional leaders will allow them to resolve conflicts more effectively and fairly, improving social cohesion within their villages.
- Training will reduce barriers to participation within communities, increasing participation in local and national affairs.
Incorporating Social Pressure for Traditional Leaders to Change
In addition to studying the effectiveness of training village heads, the study examines whether training is more effective if structured in a way that creates social pressure on traditional leaders to change their behavior. It is possible that providing information on the proper roles and responsibilities of traditional leaders to these leaders alone would not be sufficient to change their behaviors. However, providing this information to village heads and other community members simultaneously would create an expectation that traditional leaders put into practice the information learned and produce a degree of accountability.
Past Impact Evaluations on Capacity Building
One recent impact evaluation in Liberia suggests that training workshops may not be very effective in initiating behavioral change. This study will contribute to this nascent literature, providing evidence on the extent to which capacity building in the area of traditional justice is effective, and whether the effectiveness of the capacity building depends on who is being trained.
Impact of Training on Good Governance & the Law
Our overall index of good governance equally weights four sub-components of good governance: the village head’s knowledge of the law, the village head’s attitudes toward rights, the village head’s impartiality, and the village head’s legitimacy. The results showed that training the village head by himself (or, in rare instances, herself) does not have a positive effect on any of the indices of good governance. The village head only training does not even appear to have increased the knowledge of village heads. The effects of training only the village head are consistently negative but small and statistically insignificant.
Training Village Heads & Community Leaders: In contrast, the results shows the effect of the “training plus horizontal pressure” variant in which both village heads and community leaders were trained. The total effect of training both leaders is consistently positive, and the effect on the village head’s knowledge is statistically significant at the 95 percent confidence level.
Training Attended by Community Leaders: The results show that training sessions also attended by community leaders had a significantly more positive effect on good governance by the village head than training sessions attended only by village heads. The additional effect of training a community leader on the overall index of good governance is positive and statistically significant at the 90 percent confidence level. The effect of training community leaders on the village head’s knowledge is also statistically significant at the 95 percent level, suggesting village heads learn better when other community leaders are also trained.
Impact of Training on Conflict & Social Trust
Next, we consider the effects of the programming on the amount of conflict in the village and social trust. We examine the effects of the programming first on the total amount of conflict in the village and then on five sub-classes of conflict, since the effects of the training could presumably be different for different classes of conflict. The five sub-classes of conflict were domestic conflict, physical assault, burglaries and theft, land and livestock conflicts, and witchcraft. In particular, the training focused particularly on gender rights, and so we might expect to find the training sessions resulted in more acknowledged domestic conflict even while it reduced other forms of conflict in the village.
Training Village Heads & Community Leaders: The results show that training the village head by itself also does not reduce total reported conflict. The estimated effect of training the village head is tiny and statistically insignificant. In contrast, training community leaders and village heads had a larger negative effect on total incidents of conflict, but the effect is also not statistically significant. Overall, the training appears to have had little effect on incidents of conflict in villages. In addition, neither the village head training nor the community leader training had a statistically significant effect on any of the sub-indices of specific types of conflict.
Reducing Conflict Surrounding Elections: The IRC’s programming was also particularly concerned with reducing conflict surrounding elections. However, it was not possible to ask a direct question about this on the survey. Instead, the survey included a survey experiment to measure cases of threats of violence against supporters of particular political parties. Interestingly, although training the village head by him or herself has little effect, training community leaders and village heads has a moderate positive effect on the proportion of the population that is aware of threats of political violence; this effect is statistically significant at the 95 percent confidence level. Furthermore, the difference in the effects of the two training variants on political intimidation is statistically significant at the 95 percent confidence level.
Political Intimidation: Training village heads and community leaders appears to correspond with higher levels of political intimidation (statistically significant at the 95 percent level) and might have slight negative effect on social trust.
Impact of Training on Decisions by the Village Head
First, we consider the effect of the different training variants on the procedures by which village head’s make decisions. The results confirm that training village heads alone had little effects on good governance – this treatment increased consultation with resource management committees, but its effect on the other procedural outcomes were statistically insignificant and even negative in some instances. However, when community leaders were trained alongside village heads, the training sessions results in changes in decision-making procedures and, in particular, greater consultation of women and other community organizations. Thus, this additional analysis confirms that the “training plus horizontal pressure” training variant had procedural effects.
Did these procedural changes make village heads more or less effective in exerting authority within his (or her) community?
Specifically, village heads exposed to this training variant were more likely to strongly agree with the statement that most people in the village were influenced by their opinions. However, they were also less likely to think that all groups in the village respected their authority and all groups in the village took their disputes to their dare. The training made them believe their influence had increased among the village majority, but also drew attention to divisions within their community and the existence of minorities within the community who did not respect their authority.
Independence from the Government: The results show that neither training variant had a significant effect on the village head’s perceived independence from the government, and neither training variant increased or decreased their appetite for further training. This suggests that the village heads did not feel their power to have been excessively curbed by the training sessions.
Impact on Community’s Understanding of the Laws
The survey also considered the effects of the training sessions on community member’s knowledge of the laws surrounding traditional governance, and their perceptions of their village head’s authority. Interestingly, when community leaders were included in the training sessions, the knowledge provided also permeated down to the community more generally; In contrast, when community leaders were not included, the training sessions did not increase the knowledge of community members.
However, the “training plus horizontal pressure” variant did not, on average, change community member’s likelihood of taking their disputes to the village head, their perceptions of the village head’s influence on the majority of the community, or their perceptions of the village head’s independence from the government. Thus, on average, the village head’s authority was not perceived to decrease as a result of the training sessions.
Receiving Public Assistance: In particular, the survey collected information on the receipt of food and agricultural aid by groups against which village heads are often thought to discriminate — people outside the village head’s family and people with different political views from the village head who received aid. We find that the training of the village head has a positive effect on the proportion of non-family members who are assisted, and this effect is statistically significant at the 90 percent confidence level; however, the training of the village head does not affect the proportion of people with different political views who receive aid. The additional effect of training a community leader on the distribution of aid to both non-family members and people with opposing political views is small and statistically indistinguishable from zero in both cases. These results slightly complicate the results presented on the effects of training village heads alone and training village heads alongside community leaders.
Qualitative Analysis
In order to better understand the degree of change in these different outcomes and the mechanism by which community leaders affected the impact of the training sessions, we conducted qualitative research in ten villages in May and June 2014.
Sustaining Legal and Procedural Knowledge in the Community
Reassuringly, we find the legal and procedural improvements measured in the follow-up survey have persisted in this small sample of villages. Village heads still answered the legal knowledge question we posed to them correctly. The increase in the number of women on the village head’s dare caused by the community leader variant of the training sessions has also persisted. Finally, the fees charged by the village head remained lower in villages where a community leader had been trained.
Depth of the Procedural Changes
But how deep are the effects of these procedural changes? Has the inclusion of women on the village dare made the village head consult them more often? Is there really greater transparency in these villages? In order to assess this, we constructed behavioral measures of consultation and transparency based on direct observation in meetings and on a comparison of responses given to the same question by leaders and citizens in different settings.
Including Women & Minorities: The behavioral measures suggest that village heads exposed to the village head plus community leader training variant are not more consultative or inclusive of women and minority viewpoints. To see this, we asked the village head to bring some of his closest advisors to his meeting with us and then looked to see who he invited and how much they spoke in the subsequent discussion. In cases where a community leader was trained alongside the village head, the village head was no more likely to invite women to this meeting, and he invited fewer community members from outside his immediate family.
Women were no more likely to participate in the ensuing discussion, and people were no more likely to contradict the village head. Furthermore, in the focus group discussions organized by the village head in these villages, respondents were less likely to express critical opinions; this is even though the focus groups with a random sample of people in these villages were as critical as their counterparts in the villages exposed to the other variant of the treatment, suggesting there was not greater satisfaction with the system of governance in these communities. This suggests village heads trained alongside community leaders have not become deeply committed to inclusive and consultative governance; in fact, village heads exposed to this treatment may have become savvier about surrounding themselves with people of similar views, choosing family members and people who do not express critical views to attend meetings.
Expressing Critical Views:
Interestingly, in communities in which a community leader was trained, we found that were more willing to express dissenting and critical views in public in our focus groups, even though they were not more dissenting or critical in private during the follow-up survey. This suggests the higher levels of social distrust in these villages is, at least in part, a function of respondents’ greater willingness to express critical and minority views on contentious topics. There may not be greater latent social divisions in these villages.
Overall Impact of Training Community Leaders
The interviews suggest community leaders were able to do two things to improve the effectiveness of training. First, they were able to act as a check on abuses of power by the village head after the training session. Second, they were able to disseminate information about the legal framework governing the village head’s leadership to other community members.
Legal Framework: The main points emphasized by the village heads were that the community leader helped “remind” them of the law, thereby checking their powers, and the community leader effectively disseminated information on the legal framework, especially to groups – such as youth — over which the village head had limited influence. The focus group respondents also emphasized both the ability of the community leader to act as a check on the village head and to help disseminate information on laws and procedures to community members.
Building the Capacity of the Community Leader: The community leader who was trained alongside the village head became a more powerful local leader in their own right after the training sessions. In almost all instances, the village head subsequently included them in the group of close advisors they invited to meet with the research team. They all reported being delegated tasks by the village head, including the resolution of cases, and making presentations to community meetings. They all reported that the training sessions had made them closer working partners of the village head. The village heads trained alongside community leaders also universally agreed that this had made the program more effective.
Reinventing the Rules 