Survey quality for the International Visitor Survey

Quality assurance metrics

The tables below contain results against quality assurance metrics that are monitored to reduce sampling error in the International Visitor Survey (IVS). These include:

Relative Margin of Error (RME)
Sample size
Response rate

See Survey error for more information about types of errors and how they apply to the IVS.

Survey error

Relative Margin of Error (RME)

The Relative Margin of Error represents the extent to which sampling error influences the mean value calculated. Given that we calculate RME at the 95% confidence level, if we ran the IVS 20 times, we would expect the true mean to be within one RME of the sample mean 19 times and outside one RME of the sample mean once.

The International Visitor Survey is designed to have an RME of:

• 5% of the total spending
• 10% for the top 6 markets: Australia, China, the USA, the UK, Germany, and Japan
• The RME for year ending September 2024 was 3.8%. 

For year ending September 2024, the larger sample size for annual data means results are available for Total (all countries), Australia, UK, USA, Rest of Europe, Asia, Germany, Canada, Rest of World, and China.

Visitor arrivals increased in the September 2024 quarter compared with June 2024 quarter. But the total sample size for the September quarter decreased from June 2024 quarter due to the decline in the overall response rate. Results for this quarter are for Australia, Rest of World and Total.

RMEs are estimates and should be used as an indicator of accuracy when interpreting results, rather than definitive figures. 

For the September 2024 quarter, the RMEs are as follows:

• Total: 7.9%
• Rest of world: 9.6%
• Australia: 11.8%

For year ending September 2024, the RMEs are as follows:

• Total 3.8%
• Australia: 6.6%
• Asia: 9.1%
• USA: 9.1%
• UK: 9.1%
• Rest of Europe: 10.1%
• Germany: 13.3%
• Canada: 13.5%
• China: 14.4%
• Rest of World: 14.6%
   

Sample size for September quarter (July to September 2024)

The annual target sample size for 2024 financial year of 8,900 was exceeded with a total of 9,132. 

The sample size for the September 2024 quarter was 1,772, a decrease from the June 2024 quarter as the overall response rate decreased for this quarter.

Non-qualifier rates for the quarter (people approached for the survey who did not qualify for participation, primarily New Zealand residents and passengers in transit) increased from the June quarter 2024. 

Quarterly level statistics have a higher margin of error than annual level statistics, we therefore advise caution when using and interpreting quarterly level results. The September 2022 quarter data, as the first after borders re-opened in July 2022, is an outlier. We advise caution when using September 2022 quarter data in either quarter to quarter or rolling annual data analysis.

The target overall sample size for the September 2024 quarter was designed to be the same as pre-COVID-19 (see Table 1). No country targets were put in place for the 2023/2024 year due to the uncertainty of flight schedules and consumer travel preferences. Country targets for the IVS will be reviewed once travel patterns are deemed stable.

More information on IVS survey methodology can be found in the Methodology Paper, which outlines the impact on the IVS due to changes in travel patterns following COVID-19.

International Visitor Survey methodology

Table 1: Achieved, and target, sample size for 2023/2024 and pre COVID-19

Response rate for the quarter ended September 2024

The response rate represents the percentage of people who complete the survey as a proportion of the total surveys emailed out. For the quarter ended September 2024 the response rate was 17%, down from 23% for the same quarter in 2023. This decrease can be attributed to the decrease (6 percentage points) in response rate for visitors from the USA, and the increased number of visitors from China (compared to same quarter in 2023) who are low-responding group to the survey.  

Table 2 presents the response rates by country for the quarter ended September 2024. 

Refusal and response rates remain broadly similar to pre-COVID ones for most countries. An exception to this is China where the response rate has increased – MBIE has made improvements to fieldwork documents. When the IVS resumed in mid-2022, the majority of qualified visitors surveyed were from Australia, however representation of Australian residents is returning to pre-pandemic levels of around 20% of visitors in the sample.

Table 2: Response rates for top 8 international markets