Change to methodology for estimating visitor expenditure and experiences in New Zealand for the September 2022 quarter

Summary

The International Visitor Survey (IVS) re-commenced in July 2022 after a break of 2 years. Ministry of Business, Innovation and Employment (MBIE) will publish data for the September quarter on 2 December 2022. Topline information will be published on the Tourism Evidence and Insights Centre (TEIC), and microdata published on MBIE’s website. Stats NZ will also publish its December International trade release, which uses IVS data, on the same date.

Planning for the relaunch of the IVS began in 2021. Modifications were made to the survey questionnaire and methodology due to changes in travel patterns following the pandemic and the changing needs of key users of the IVS. The break in the historical series, and the low number of visitors to New Zealand compared to pre-pandemic levels, will result in some changes to published outputs for the September Quarter from what was published previously.

Previous methods for estimating visitor expenditure and experience

Up until the March 2020 quarter, international visitor expenditure in New Zealand was measured through the IVS which was published quarterly by MBIE. This survey asked a sample of departing visitors for their total expenditure in New Zealand. This was then weighted up to a total spend for all international visitors using international travel statistics produced by Stats NZ. In addition to expenditure, the IVS determined the following:

• activities international visitors participate in
• transport and accommodation types used
• places visited
• demographic information
• motivation for visiting New Zealand
• satisfaction with their visit to New Zealand

Background and redevelopment of the International Visitor Survey

The IVS ceased collection near the end of March 2020 due to border closures caused by the COVID-19 pandemic. Stats NZ moved to using a new method of estimating visitor expenditure, initially using an experimental series, which was then adopted as the semi-permanent method while the travel restrictions due to COVID-19 continued.

Estimating visitor expenditure in New Zealand during the June 2020 quarter(external link) — Stats NZ

After monitoring the performance of this model over a number of quarters, Stats NZ determined it was generating increasingly unrealistic estimates of daily spend per visitor from some countries. Therefore, from the June 2021 quarter onward, Stats NZ changed to a new model, based on pre-COVID data on daily spend taken from the IVS, combined with up-to-date data on visitor numbers. This approach was always intended to be temporary until the return of the IVS. For more information on this approach, and its known limitations, see the page below.;

Change to methodology for estimating visitor expenditure in New Zealand for the June 2021 quarter(external link) — Stats NZ

Other variables published as part of the IVS release (such as travel methods, visitor satisfaction, and locations travelled to) were not available between Q 3 2020 and Q 4 2022 as a sufficient sample was not able to be obtained. This means that there will be a break in the historical series, as there was no practical way of collecting this information.

Re-opening of New Zealand’s border and re-launch of the International Visitor Survey

Preparation began in 2021 for re-starting the IVS. The date of the restart was not fixed, due to uncertainty around the opening of the New Zealand border, mandatory isolation periods, and travel restrictions in other countries. Data needs of key users of the IVS, including Stats NZ also changed, for example in relation to the impacts of COVID-19 and increasing interest in the relationship between tourism and the environment. This presented an opportunity for collaboration between MBIE, Stats NZ, and other stakeholders, and for a review of the survey method and questionnaire, which had not been meaningfully updated since 2018.

In early February 2022, the New Zealand government announced the planned reopening of the border, in stages, with the border not scheduled to be fully open for all travellers until October 2022. The IVS project team decided on a provisional date of 1 July 2022 for the recommencement of surveying for the IVS. In May 2022, the full reopening of the border was brought forward to July 2022, while most travellers from the key market of Australia were able to freely travel from June 2022.

Planned reopening of the border(external link) — Beehive website

NZ border fully reopening(external link) — Immigration website

The IVS collection commenced in July 2022, and MBIE will publish data for the September quarter on 2 December 2022, with topline information published on the Tourism Evidence and Insights Centre (TEIC), and microdata published on MBIE’s website. Stats NZ will also publish its December International trade release on the same date. This will incorporate data collected until 30 September 2022 in its estimates of travel credits (exports) (Travel credits can be defined as estimates of expenditure by visitors to New Zealand.).

Travel credits will then be reproduced in the 'Balance of payments and international investment position' release on 14 December 2022 and used in calculating quarterly Gross Domestic Product in the 15 December release.

Changes to the International Visitor Survey methodology

The overall method of collection, stratification, and allocation of sample to strata for the relaunch of the IVS has been designed to be as similar to the pre-COVID design as possible, to maintain continuity and achieve the same accuracy requirements. However, certain modifications have been implemented to account for changes to travel patterns.

The IVS uses a 2-step collection process:

• The first step involves screening departing visitors for eligibility and collecting email addresses. This is performed at Auckland, Wellington, Christchurch and Queenstown international airports.
• The second step, where the bulk of the information is captured, is via an online questionnaire. A link is sent to those who are eligible and have agreed to participate.

We define international visitors as all overseas residents departing New Zealand after a stay of less than 12 months.

Given this definition, the population excludes:

• individuals who reside in New Zealand (New Zealand was their last place of residence for 12 months or more)
• individuals who have been in New Zealand for longer than 12 months
• those in transit who haven't formally entered New Zealand (they haven't filled out an arrival card and handed it into Customs)
• individuals who visit New Zealand to serve in the armed forces, for diplomatic or consular business, or aircrew on duty or between flights
• individuals whose next country of permanent residence is New Zealand

The target population also excludes:

• those aged under 15
• those whose primary purpose of visit was to attend a recognised educational institute. The spending of this group is estimated separately as their spending patterns are significantly different from other international visitors.

More information on sample collection in International Visitor Survey methodology

The sampling methodology is designed by Stats NZ. It is a stratified sample design with the strata defined by sets of destination airports. This allows the sample to be balanced across visitor country of origin given the large difference in the total number of visitors from each country and manage the sampling for countries without direct flights from New Zealand. Stats NZ sends the number of Primary Sampling Units (PSUs) to be sampled in each strata for each quarter to the contracted survey collector two weeks ahead of the beginning each quarter. Each PSU is a flight, and thus the number of PSUs is the number of flights to be sampled from each set of airports to overseas locations. The number of PSUs is given on a monthly basis.

The sample frame is international visitors to New Zealand who have been through immigration and security and are waiting to board international flights departing from the four airports.

At the annual level, the sample is designed to achieve a 5% relative sample error (at the 95% confidence level) for total expenditure and <10% for the top 6 countries of origin for visitors historically (Australia, United Kingdom, United States, China, Japan, and Germany).

More information on the target measures of survey quality

The target overall sample size for the September 2022 quarter was designed to be the same as pre-COVID. Certain modifications needed to be made to the sample allocation process because the methodology is based on pre-COVID travel patterns. This means that for some strata the target PSU numbers were close to or even greater than the total number of flights. These modifications included:

• Considering the U.S. as a transiting country due to the current closure of Shanghai and Hong Kong to transiting passengers.
• Sampling the flights in strata to meet targets, but where the flight allocation is less than the total PSU (due to number of available flights being less than target PSU numbers), increasing the targets in the strata containing transiting airports.

Achieved survey sample for the September 2022 quarter

The September 2022 quarter saw a reduced sample size compared to pre-COVID. The primary reason is the high percentage of non-qualifiers; people approached for the survey who did not qualify for participation, primarily New Zealand residents, and passengers in transit. Refusal and response rates remain similar to pre-COVID. The majority of qualified visitors surveyed were from Australia. Previously, Australian residents have typically represented between 20-25% of visitors of the sample.

The IVS was designed to create annual statistics of international expenditure. Data quality targets, margins of error of 5% at the national level and 10% and the country level, were also calculated at the annual level. Due to the gap in the time series caused by COVID-19 in the first release of post-COVID-19 data, it is only possible to calculate margins of error at the quarterly level. Quarterly level statistics will have a higher margin of error than annual level statistics due to the lower sample size. The sample size for Q3 2022 is also slightly lower than usual due to the high rates of non-qualifiers in the airports. Therefore, we advise more caution when using and interpreting quarterly level results, especially for Q3 2022. More information on data quality, including sample sizes, response rates and margin of errors will be published by MBIE on 2 December 2022.

Changes to the International Visitor Survey questionnaire

Expenditure Section

The major change made to the IVS questionnaire that affects the estimation of visitor expenditure is the section about the cost of the visit. Previously, in addition to being asked the total cost, respondents were given the chance (but which was not compulsory) to report these costs in dollar figures by payment type (e.g., credit or debit card, cash, bank transfer). The new version of the questionnaire asks respondents to report costs by expenditure category (e.g., transport, accommodation, food and drinks), and then aggregates these estimates to derive a total. Respondents are then prompted with this total and asked to break it down by payment type as a proportion of the total cost. New Zealand related spending by cruise visitors before, during and after the cruise, was also added to the questionnaire.

These changes are intended to improve the quality of response to the questionnaire through assisting respondents in their recall of visitor expenditure and improved alignment of the questionnaire wording with the concepts being measured. Additionally, the reporting of expenditure by category will assist Stats NZ in the allocation of visitor spending to commodity level where it can be confronted with other data sources through the supply use balancing framework. Including the proportionate expenditure by payment type will help measure changes in the way visitors pay for goods and services in New Zealand in future and help to establish the robustness of other potential administrative data streams in future; for example, electronic transaction data.

Changes to other sections (Itinerary, Activities, Transport and Accommodation, Satisfaction, Environment and Travel)

Changes to the questionnaire were made during consultation with key stakeholders. Reasons for additions, deletions or revisions were due to increasing relevance of matters such as COVID-19 and the relationship between tourism and the environment, low or no current relevance, or previous low response from survey respondents. The length of the questionnaire was also a factor in prioritising changes.

The following changes were made:

• updated the location map and list in the Itinerary section
• updated the Activities section
• revised questions on experience of Māori culture and activities
• revised the Transport, Accommodation and Environment sections
• revised the Visitor Satisfaction section, retaining the core questions on satisfaction, expectations and recommendations
• reduced the length of the:
  • Decision-making section
  • iSite and Qualmark sections

View the updated IVS questionnaire in full

Other, more minor changes to the questionnaire included tweaks to the opening questions, establishing eligibility for the survey and purpose of visit, and instructions for respondents about how to answer the expenditure questions.

All changes were tested with trial respondents prior to the survey commencing, both through in-person cognitive testing, and through a pilot survey via both online collection and in person. Trial respondents were broadly representative of the expected respondent profile of the IVS.

Changes to IVS outputs from what was published previously by MBIE

The reduced sample size compared to the pre-COVID sample, and the break in the historical series resulting in 1 data point for the release, will result in some changes to published outputs for the September quarter from what was published previously. More information on the changes to outputs will be provided in the 2 December release.

Planned future work

International travel remains a highly dynamic environment due to ongoing changes to global travel regulations, flight schedules, and consumer travel preferences. We are continuously monitoring this fluid situation and the implications any changes may have on the sampling strategy for the IVS.

Further methodology changes such as outlier and calibration review will be implemented when the IVS data is seen as returning to stability. We are implementing a process to monitor for this stability, which is expected in the next 18-24 months. These changes will be processed and communicated as part of MBIE’s normal revisions practice.