Accountability of a dispute resolution scheme

Deciding how and what

Dispute resolution schemes need to collect data and information that can be used to analyse the effectiveness of services and improve performance of both dispute resolution schemes and the regulatory systems in which they operate.

When developing a scheme, you need to consider how the monitoring, evaluation, and reporting will be done, including what data to collect and how, who is responsible, timeframes, and who receives reports.

If these decisions are taken by the scheme, your agency will need to think about how it will influence those decisions. Due to the sensitive nature of disputes, and confidentiality and privacy issues in data collection, anonymising data and compliance with the Privacy Act 2020 will be important.

Benefits of monitoring and data stewardship

Better use of data can improve the effectiveness of a dispute resolution scheme and generate positive outcomes across the dispute resolution sector. The benefits of monitoring a scheme and analysing dispute data include:

• identifying whether the scheme is doing the right thing and doing it correctly
• identifying any improvements required to the scheme, in addition to those that may have been identified through the Maturity assessment tool in the Aotearoa best practice dispute resolution framework (the Framework)
• ensuring accountability of the scheme and providers providing data to report to stakeholders, including through case studies
• ensuring a mechanism to promote open partnerships with tangata whenua and the public to ensure partnership and participation providing early warning of emerging issues and helping to guard against future regulatory failure.

Monitoring using a database

Monitoring involves collecting and recording the schemes data on an ongoing basis and is best done using a database.

The types of data collected will depend on the nature of the complaints and disputes, what insights you want to get from the scheme and why you need them. The following data is a suggested minimum:

• who is using the scheme and their demographics the number and nature of complaints and disputes
• the timeframes for processing complaints and disputes
• the outcomes of complaints and disputes
• any escalation of disputes (eg, unresolved disputes going to court).

You should set clear targets (eg, timeframes) for relevant data and monitor performance against these targets. However, you should balance achieving targets against the need to ensure appropriate resolution of disputes, particularly early in a new scheme’s operation.

Assessing a dispute resolution scheme

The Framework is a tool that allows you to assess your scheme’s level of organisational maturity. This should be conducted annually to track progress, and to inform improvement and strategic planning.

Many schemes are required to have an independent review within a set timeframe (eg, 5 years after set up). Have a review plan outlining the purpose of the assessment, main questions, and methodology.

Best practice reviews should include examination of the:

• scheme against the principles of the Framework, and the satisfaction of parties with the process and the outcomes
• providers' compliance with the standards of the Framework and any service standards along with the appropriateness of those standards
• the scheme’s maturity through using the Framework’s maturity assessment tool
• compliance with any established measures, such as timeframes
• feedback from staff, practitioners, users, and other stakeholders, on their experiences of the scheme.

It is preferable that the evaluation of your scheme is independent (ie, undertaken by an external party). 

Reporting the results

Reporting involves communicating scheme results to your stakeholders, regularly and systematically. It should include:

• the scheme’s performance against objectives and the principles and standards in the Framework

• the scheme’s assessment results using the maturity assessment tool

• targets

• outcome trends

• issues arising.

Maturity assessment tool [XLSX, 507 KB]

Make it clear who the reporting is for and who is responsible for taking any actions in response to reported issues. Otherwise, emerging risks may go unidentified and opportunities for management or improvement may be lost.

The policy team responsible for the scheme should  determine what action is required, particularly to resolve issues with policy settings or legislation. It may be necessary to bring together different sector participants to discuss these issues and agree on the right course of action to address them.

Complaints mechanism

There needs to be an impartial process to deal with complaints made about the scheme itself. This is covered by Standard 3 of the Framework. At the most basic level, the person complained about should not be the person investigating the complaint about them.

The process should be clearly documented, including who is responsible and the timeframes for each part of the process.

If the scheme has particularly vulnerable users, or there is a significant power imbalance (eg, residents at a retirement home) or lack of trust in the relationship between the parties (eg, history of lying), it will be necessary to support access to relevant external avenues of complaint (eg, the Ombudsman’s Office).

Aotearoa best practice dispute resolution Framework (the Framework)