Review of Digital Records
This section provides an overview of key findings from the expert review of a sample of digital records created using the Artisan tool. These digital records were reviewed in isolation. In contrast, when reviewing these records to make a determination of compliance with the building code, BCA officials will have a range of information sources about the plumbing work and broader building work.
The Artisan tool provides an effective platform to review digital records, but these records are only as good as those using the platform
The expert reviewer found that Artisan is a useful tool to enable the creation and review of digital records. The reviewer also noted that the platform was intuitive and can be picked up with only a short training session.
The quality of digital records varied considerably between records. Some of this variation was a result of different requirements or points of clarification sought between the two BCAs. However, most of this variation was due to differences between the practices of individuals lodging the records. Both BCAs had stand-out examples of practitioners whose cases were consistently completed to a high standard – with clear photos and supporting information provided in commentary. Likewise, both BCAs had records which lacked necessary clarity or detail.
There may be value in documenting the practices of high-performers and including these ‘tips and tricks’ in future training sessions. For example, if a digital level is not being used in photographs the fall/angle of a pipe should always be included in notation for each photo. Requirements such as this can be stipulated in the Artisan shot list at the discretion of the local BCA.
The majority of digital records reviewed had room for improvement, though many of these issues were minor
Around 80 percent of records reviewed as part of this evaluation were found to have aspects that could be improved (see Table 1). The expert reviewer also noted that, while widespread, the issues with digital records were typically minor and easily addressed.
Most of the issues identified with the digital records related to missing information: 32 of 93 records were assessed as not covering the full scope of work, and 67 of 93 records were assessed as not having sufficient notes or comments. While a large proportion of records were assessed as having missing information, the missing information in question was typically either minor or likely to be included in other consenting documentation such as detailed plans.
The review found only minor issues associated with visual clarity, the inclusion of sufficient information to consider contextual risks, or sufficient information to understand what each photo refers to.
The following points provide an overview of issues identified for the criteria assessed by the review:
- Visual clarity: The review found the quality of almost all photos to be of a high standard. Where there were issues related to the visual clarity of photos included in the digital record, it typically related to the photo being taken on a poor angle or was a photo of a photo - making it difficult to read measuring tape or determine the incline of a pipe.
- Scope of work: Around one-third of digital records were assessed as not including photos that cover the full scope of work being assessed. Typically, this information had been requested in Artisan but was not provided by the individual lodging the record.
- Contextual elements: The reviewer identified only a single case where the record did not include sufficient information to identify any environmental, structural, or safety issues associated with the work.
- Identifiable subject: The reviewer identified two cases where it was not possible to identify the subject of the photo. In both cases, the lack of notes or markings on pipes made it impossible to determine whether pipes included in the digital record were for sewer or stormwater.
- Supporting information: The most frequently identified issue was the lack of sufficient supporting notes or comments in digital records. Most of these cases were the result of information requested by Artisan but not provided by individuals lodging the record. For example, the percentage of fall/angle on pipes. In a smaller number of cases the instructions were framed in a way that did not elicit sufficient information. For example, a prompt for hot water pipework asked for the size of pipe but failed to ask for details necessary to establish the volume and energy efficiency of the system.
Many of the issues identified above could be addressed by inclusion of appropriate prompts or stipulations in the Artisan shot lists. Local BCAs can customise the requirements included in Artisan shot lists to meet their needs.
Table 1: Summary of expert reviewer ratings
| Assessment criteria | Yes | No |
|---|---|---|
| Visual clarity: Is the visual clarity of all photos sufficient to assess the work? | 87 | 6 |
| Scope of work: Do the photos cover the full scope of what is being assessed? | 61 | 32 |
| Contextual elements: Is there sufficient information to identify any environmental, structural or safety risks associated with the work? | 92 | 1 |
| Identifiable subject: Is there sufficient information to understand what each photo refers to? | 91 | 2 |
| Supporting information: Are there sufficient notes/comments where needed to support demonstrating compliance? | 26 | 67 |
| Overall: Is there room to improve the quality of the digital record to show the work complies with the building code?[3] | 18 | 75 |
While no examples were found in this review, high trust practices like using digital records for remote inspection could create opportunities for bad actors
Using digital records of low-risk plumbing work is a high trust practice, relying on the honesty of the individuals lodging the records. The expert reviewer noted that in considering a digital record, the assumption must be made that the photos are accurate and representative of the site in question. While there was no evidence of dishonesty in the records reviewed, the reviewer felt it was important to note this potential risk with adopting a digital compliance process. Note that Artisan captures GPS coordinates and user device information to help identify any fraudulent behaviour.
In discussing this possibility with BCAs, they were clear that while this is something that has been considered, they are confident that it will not create significant issues. The use of digital records for remote inspection is not something that has been opened to all tradespeople. Instead, it has been offered as an alternative to those with a consistently high level of performance. One BCA noted that if there were instances of dishonesty uncovered, those providers would lose the ability to use digital records in the future.
This could indicate that if this process is adopted for all plumbers, rather than at the discretion of the BCA, sufficient checks should be in place to detect dishonest behaviour.
Footnotes
[3] Note that this assessment is varies considerably from how a BCA would determine compliance, as: 1) this assessment was done solely based on the information contained in the Artisan record, while Building Consent Officials have access to additional information, and 2) this assessment adopts a strict pass/fail criteria, while BCAs will likely adopt a risk-based assessment.