Reinventing the Rules
Discover the Latest Innovations and Lessons Learned in Rule of Law and Legal Empowerment Projects
A Review of 23 Access to Justice Assessments in Asia
In 2012, UNDP published a report titled “Access to Justice Assessments in the Asia-Pacific: A review of Experiences and Tools in the Region.” For anyone looking for lessons learned, the report analyzes the approaches, strategies, methodologies and tools used in over 23 access to justice assessments conducted over the past decade (2000–2010) in 15 countries (Bangladesh, Cambodia, India, Indonesia, Lao People’s Democratic Republic, Maldives, Mongolia, Nepal, Pakistan, the Philippines, Sri Lanka, Thailand, Timor-Leste, the Republic of Vanuatu and Viet Nam). Check out excerpts below and the appendices at the end which offer tools on creating a solid assessment.
Credit: Giri Wijayanto
Assessment design
Using a Multidisciplinary Team to Design Assessment
In the Philippines, the study recommended using multidisciplinary teams both in designing and undertaking the surveys and in analysing results.
In Timor-Leste, the team was comprised of national legal advisers who were familiar with the legal culture and the legal institutions of the country, social scientists with socio-anthropological and historical background familiar with tools for doing ethnographic research and an understanding of cultural issues behind customary law, and linguistics or ethno-linguistics with an understanding of the legal culture expressed in the different languages used in the country. Furthermore, the study improved the quality of the process by employing translators, documenting the whole assessment process, and providing training workshops on the methodological framework for all participants in the study.
Determining Which Marginalized Groups to Assess
Many access to justice assessments target marginalized and vulnerable groups. It is important for assessments to further identify target or sample groups or to pinpoint and explore in-depth issues that are group-specific. Selecting target and sample groups is often based on existing information about groups that are facing or have faced difficulties accessing justice. If criteria to measure the disadvantaged and marginalized people are established on the denial of a multitude of rights, selection criteria are based on qualitative data gathered through socio-economic mappings of sites.
In Cambodia, the researchers learned that local workshops should be organized first for similar groups, such as women, indigenous peoples, minorities, etc., in their own languages. Later, the local level workshops could include a mix of people and should include local authorities. Then these workshops should be taken to the regional and/or national level. The survey also faced difficulties due to mixing indigenous peoples and minorities as one group. This is also not recommended as each group has its own authorities and procedures for conflict resolution within their communities. In terms of institutions, the survey recommends not mixing institutions of ADR established by law with local authorities as the sources of their legitimacy, their behaviour, kinds of matters they manage, and the results they obtain can be very different.
In the UNDP-led study in Indonesia entitled Justice for All – An Assessment of Access to Justice in Five Provinces of Indonesia, poor and disadvantaged status was the focus for target groups selection. The criteria used to measure the poor and disadvantaged groups was the non-fulfillment of basic rights to food, health care, education and other government services; discriminatory treatment by government or other community members; or the inability to participate in decision-making affecting their futures. From a rights-based approach, this type of mapping provides a useful method of identifying and selecting the sample population for an access to justice assessment.
The target groups were categorized by geography, type or mode of employment, gender or ethnicity; and one disadvantaged group was selected in each village to be the primary respondent in the qualitative phase of the assessment. The group selection process was based on the level of disadvantage compared to other groups in the village. In some cases, the group was selected because it constituted a type of group that had not been chosen in other villages.
Focus group discussions were conducted with these groups to find out their opinions about the most pressing justice-related issues they face on an everyday basis, and their experiences of resolving grievances through both formal and informal justice systems.
The focus group discussions were supplemented by approximately 700 in-depth interviews, primarily with ordinary villagers, but also with village-level community and religious leaders, sub-district officials, police, prosecutors, lawyers, judges and other duty-bearers.
Assessment tools
Reviews show that assessments use a mix of quantitative and qualitative information-gathering methods to collect data. Assessments rely on primary and secondary data. Primary data is gathered as firsthand empirical evidence through surveys, focus group discussions and case studies. Secondary data are gathered from literature/desk reviews of government reports, statistical data, published and unpublished reports, books, magazines and other established sources.
Access to justice assessments can use a range of tools for data and information gathering and analysis. These include:
1) Surveys can be carried out to understand needs, existing systems, actors and means of improving access to justice. Specific tools include household surveys, needs assessments, analysis of formal and informal institutions, justice needs surveys, attitudes and perception surveys. The tools that can support surveys are sampling techniques, questionnaires, manual and computer-based databases to produce data tables, graphs and charts and analysis systems.
2) Focus group discussions are structured discussions with selected sources of information, which are a good means to collect qualitative information. Tools for focus group discussions would be guiding questions, which are open-ended. Analysis is made easy where information is coded and can be done manually or with computer-based systems.
3) In-depth interviews with key informants (or those who know best) provide detailed information on the specifics of access to justice. The main tool for in-depth interviews is a list of guiding questions. The information gathered should be organized through a coding system (manual or computer based) for easy analysis.
4) Expert interviews provide a depth of information that is important to understating formal and informal systems of justice delivery as well as gaps, constraints and challenges. As in any in-depth interview the main tool for in depth interviews is a list of guiding questions and the information gathered organized through a coding system (manual or computer based) for easy analysis.
5) Geographic analysis provides quantitative data on the environmental issues that impact access to justice. A structured questionnaire for information gathering and content analysis of existing data are the ideal tools for a geographic analysis.
6) Capacity assessments provide in-depth information on structures of justice delivery. Capacity assessments involve a range of tools such as surveys, in depth interviews and infrastructural and financial analysis.
7) Institutional monitoring are an effective means of assessing the needs of justice is through monitoring of formal and semi-formal institutions of justice. A tool to be used is a well-structured monitoring guide.
8) Key informant interviews are an ideal method of testing or corroborating previously collected information. The tools for this has to be specially designed with a series of specific questions administered uniformly.
9) Verification process means the study should include a detailed verification processes. This involves feedback of information to sources via structured workshops and meetings, key informant interviews, expert interviews. This also involves the appointing of a special team member for monitoring of process.
Gaining Credibility for an Assessment
In Sri Lanka, the main lesson learned was the need to select partners by placing the access to justice assessment in a country-specific context where partners would ensure acceptance and credibility, and facilitate the assessment. In other words, selecting partners for technical capacity alone would not be sufficient in particularly difficult situations; partners would also need to bring in qualities that would ensure acceptance, ease of implementation and credibility of the whole process.
Developing a strategy and engaging with partners at the outset is valuable for conducting an effective access to justice assessment. An engagement strategy could map: (1) all potential actors in the process; (2) the ways of engaging with the different partners in shaping methodology, implementation and arriving at policy- and programme-related recommendations and initiatives; and (3) the roles each partner could play in the monitoring and evaluation of access to justice interventions following the assessment.
Consulting Official Stakeholders
Another lesson learned is that consultation with official stakeholders from the beginning helps ensure that the questions and methodology are seen as relevant and appropriate, and assessment findings are utilized in terms of policy recommendations and conclusions. Without this, survey findings might be underappreciated or altogether ignored. Further, the study experience finds that official stakeholders must be engaged in discussion and dissemination of findings if the assessment findings are to be implemented. Failure to do this puts the assessment reports at risk of not being disseminated, which would mean that findings would neither be known nor able to shape government responses to access to justice needs.
Training & Deployment of Survey Teams
In the LAO PDR, the access to justice assessment provided comprehensive trainings for the survey teams. The team received training on key concepts concerning access to justice, assessment methodology, target areas and tools, and the justice systems in the country.
In Cambodia, some of the positive lessons from the survey also show that including broad questions related to the main problems people suffer is better than focusing exclusively on conflicts or disputes as having a narrow vision of access to justice through the study of conflicts lead to missing out on important aspects of social demands for justice such as corruption or violations of individual and social rights. The broad questions posed permitted a wider comprehension of social needs.
In Indonesia, one of the key factors in the study was the focus on training researchers and personnel involved in administering the study. A clear and strategic plan was made to train personnel at different stages and for different tasks. The study produced valuable and replicable training material as a result of these trainings. The trainings included the following subject areas:
- Access to justice assessment framework;
- Access to justice concepts and the principles of a human rights-based approach;
- Interview and case study guidelines for disadvantaged groups, formal justice actors and informal justice actors;
- Surveys for disadvantaged groups, formal justice actors, informal justice actors, and prisoners;
- Data collection guides, reporting guidelines and a field working guide.
The study also developed a ‘Do No Harm’ note, which set the guiding ethical principles and standards for the research.
Lessons Learned Training Surveyors in Indonesia
- Roles and responsibilities of partners should be clearly outlined. Clearly setting out participants’ entitlements and responsibilities at the beginning of a training programme enables them to focus on the substantive issues and limits the grounds for complaint at the end of the training that their participation was based on false expectations.
- In the training exercises, it was necessary to have a consistent message given to those involved in the research. It was also necessary to exercise sufficient control over the presentations of invited speakers in order to ensure the researchers’ first impression and understanding of the project was correct; changing incorrect initial impressions is much more difficult than forming the impressions in the first place.
- Too much of the training was spent in the classroom with minimal practical hands-on training in how to approach the task of interviewing and working with disadvantaged groups.
Challenges Conducting Surveys in Urban & Rural Areas
In the four target areas, the teams faced more challenges while working in the urban setting than they did in rural communities. This is in part because the rural setting usually has fewer justice-related issues, and in urban settings, an increasing portion of the population is unavailable, as they are busy trying to earn money.
Disseminating Findings to the Community
In Vanuatu, development aid and academic programmes are often criticized for ‘exporting’ information and local knowledge from communities without any real benefits in return. Following its research on the islands of Epi and Tanna, [the World Bank] held meetings with participating communities where research was undertaken to discuss findings and stimulate community dialogues on how current problems with land leasing may be addressed at both the local and national level. The innovative dissemination of research findings was combined with a legal awareness and legal aid programme, thus providing a way for the World Bank to give something back to the participating communities and solidify the networks and relationships developed during field work. It further helped communities to strategize about ways to challenge inequitable land leases and potentially negotiate more equitable deals in the future.
In Indonesia, an incredible amount of qualitative data and real life stories emerged during the research processes of the access to justice assessments. But these were merely reduced to a paper form final report, which faced the inevitable constraints of editing and simplification. Had there been alternative forms of capturing the knowledge and data (e.g. documentary, picture-story books, journalistic articles, topical publications.) the assessment findings could have been better used for public education on access to justice issues. One consideration that is valuable in conducting access to justice assessments is to identify creative approaches and greater efforts to capture and utilize the real life stories and rich data from the research processes in forms beyond a printed final report.
Results of an Access to Justice Assessment in Vietnam
Strengthening people’s access to justice is crucial and this includes increasing the level of legal awareness, as well as the level of access to and confidence in legal institutions in place. The level of confidence in the legal institutions is relatively low. It also shows that the level of confidence among those who have accessed legal institutions and those who have not is quite similar, suggesting that people’s perceptions correspond with the real situation;
People who have accessed grass-roots mediation groups and legal aid centres appear to have a significantly higher level of confidence in these institutions than those who have not,
The impact of lawyers and legal aid activities does not appear in meeting the needs of people. Access to justice, particularly for the poor, has not improved over the period, despite a continuing growth in the number of lawyers and the increase in provision of legal aid services. The percentage of poor people who have accessed legal aid services, or the services of lawyers, remains very small and has not increased.
Press, mass media, and civil society organizations have been playing an increasing role in protecting people’s rights and interests, with more than 90 percent of respondents considering that media play an important role in protecting people’s rights. Compared to 2003, the percentage of people putting their confidence in laws as well as current legal reforms increased significantly, even though information about legal reforms does not appear to have been more effectively disseminated.
Results of an Access to Justice Assessment in Bangladesh
Certain ADR approaches are better positioned to effect social change at the local level, and these same changes in values will make it more likely that community members will continue to conduct ADR on their own, accept ADR settlements, and/or enforce compliance with these settlements after donor-funded NGO assistance ends.
In the Appendices, you can find the following guides:
- Access to Justice Questionnaire (Viet Nam)
- Access to Justice Survey Questionnaire (Viet Nam)
- Applying HRBA to Access to Justice Assessments (Indonesia)
- Case Study Format (Indonesia)
- Consultative workshop agenda (Timor-Leste)
- Court User Questionnaire (The Maldives)
- Data Recording Format (Indonesia)
- Do No Harm Note (Indonesia)
- Field and Training Guide (Indonesia)
- Guide Questions to interview authorities and institutions at field level (Timor-Leste)
- Guide to Focus Group Discussions with Selected Disadvantaged Groups (Indonesia)
- In depth Interview Guide (Indonesia)
- Justice Survey Questionnaire (Sri Lanka)
- Legal Knowledge, Attitudes and Perceptions Survey
- Note on Methodology (The Maldives)
- Note on Participation and Empowerment (Indonesia)
- Participatory Research Guide (Timor-Leste)
- Prioritizing Disadvantaged Groups (Indonesia)
- Public Perception Questionnaire (The Maldives)
- Questionnaire to interview Prisoners (The Maldives)
- Questionnaire to interview Professionals (The Maldives)
- Socio Economic Baseline Survey Questionnaire (Sri Lanka)
- Survey Questionnaire for ADR Minorities and Indigenous People (Cambodia)
- Survey Questionnaire for ADR Operators and Clients (Cambodia)
- Survey Questionnaire for Communal Authorities (Cambodia)
- Survey Questionnaire for Justice Sector (Cambodia)
Reinventing the Rules 