Before School Check data in the IDI

This post describes the B4 School Check (B4SC) dataset and its usefulness as an analytic dataset in the IDI. We will discuss the many advantages of the dataset and the potential pitfalls to avoid when using the dataset.

Background

The B4SC is a national programme that monitors the health and development of four-year-old children. It was established in September 2008 by the Ministry of Health(1) and is the final and most comprehensive in a series of eight free Well Child Tamariki Ora visits that children receive, and is the only one with data available in the Integrated Data Infrastructure (IDI). Eight developmental areas are assessed including: vision, hearing, oral health, general health, growth measurement, strengths and difficulties (SDQ) as reported by parents and teachers, and a parental evaluation of development status (PEDS). The checks are carried out by registered nurses or nurse practitioners, with assistance from vision and hearing technicians.(1) The SDQ is completed by a child’s early childhood education (ECE) teacher, who is mailed a SDQ to complete and return to the B4SC provider.(3)

The B4SC is undertaken in different locations including preschools, kōhanga reo, doctors’ clinics and other venues such as churches and marae. In theory, all four-year-old children in New Zealand are eligible to receive a B4SC. District Health Boards send invitations to the parents of all four-year-old children enrolled with a Primary Health Organisation (PHO; around 95% of all four year-olds).(2)

What Before School Check data are in the IDI?

B4SC data from January 2011 to June 2017 are available in the IDI as a single table in the ‘MOH’ schema (moh_clean.b4sc). The Ministry of Health supply B4SC data (and other health data) to the IDI for use by researchers under Rule 11(2)(c)(iii) of the Health Information Privacy Code (1994) (see also the Integrated Data Infrastructure extension: Privacy Impact Assessment ).(7)

Table 2 describes the measures taken for each developmental area assessed as part of the B4SC (a StatsNZ summary of available variables can be found here). Referral status for each developmental area is also recorded (not referred / referred / referral declined / advice given / under care) along with the date at which the measures were taken.A Ministry of Health handbook describing the measures in more detail can be found here.

Advantages

Most other datasets in the IDI only provide information for people using services. However, B4SC data provides information on all four-year-olds, not just those using services. This means that there are, for example, data on vision and hearing for the whole population of four-year-olds, not just those who saw a specialist for a vision or hearing test. As such, the service-use biases affecting most administrative collections are less severe with the B4SC. Moreover, B4SC provides the only nationwide collection of information on vision, hearing, oral health, obesity, behavior, and developmental problemsin childhood.

Other advantages of B4SC data include: the large sample size (>50,000 children per year, >90% coverage); near-whole population measures of important outcomes, such as obesity, oral health, and behaviour; and the ability (in theory) to use the B4SC as baseline measures against which to assess later functioning. However, as the earliest available B4SC data is from 2011, the oldest children that can be followed up are now only 12 years old, which limits the range of education, health, and mental health outcomes that can be assessed. This will improve as time goes on and the B4SC time series gets longer.

Coverage and quality

Disadvantages include the incomplete coverage for some groups and some B4SC components (see below), and also some quality issues. For example, there is evidence that the B4SC obesity measures may overestimate obesity, possibly through incomplete adherence to measurement protocols regarding clothing.(8) There is also evidence that the SDQ may have data quality problems and problems with stakeholder buy-in around the SDQ administration.(9)

The Ministry of Health estimates of nationwide coverage, shown in Table 1, suggest poorer coverage in the early years (67% in the first fiscal year, 2009/10), with large improvements over time (coverage exceeded 90% since 2013/14).(4) Work conducted as part of the Better Start National Science Challenge suggests that coverage rates were slighter lower once children not on the PHO register were accounted for.(5)

Year	Coverage
2009/10	67%
2010/11	72%
2011/12	79%
2012/13	80%
2013/14	91%
2014/15	92%
2015/16	92%

Table 1. Coverage of the B4SC, MoH

Coverage is lower for some components. For example, while 92% completed the vision and hearing components in 2014/15, 87% completed the other in-clinic components, and only 62% of children’s ECE teachers completed an SDQ.(5) Coverage is also lower for some population groups: e.g., Māori coverage rates are around 4% lower, Pacific coverage rates 5% lower, and high deprivation group coverage rates 4% lower than that of the general population.(5) Those in poorer health are also more likely to miss out on a B4SC.(5) Coverage was also low for the Auckland region in 2010/11.(6)

Challenges

Those who do not attend a B4SC have characteristics suggesting that they are more likely than other children to need the services that accessed through the referral process.(5) This bias needs to be considered when producing descriptive reports, as there may be an underestimation of health and development issues. This will also need to be considered when evaluating the impact of being referred to, and accessing services, on later outcomes.

While some work has started to test the validity of data in the B4SC, much more work is needed to understand how accurate these data are. With the exception of vision and hearing checks, many of the measures are taken by practitioners who are not specialists in that area. While training and detailed protocols are provided to the practitioners undertaking the B4SC assessments, there will be instances where an expert’s clinical judgement and the trained practitioner’s judgement will differ. For example, no study has yet directly compared findings of the “lift the lip” screen of visible dental decay carried out during the B4SC to a clinical examination conducted by a dentist or dental therapist.(10) There is also no way to gauge adherence to the measurement protocols, and this likely varies between centres providing the B4SC and individual practitioners. What this means in practice is that we are likely to be underestimating the number of children in need of referral for further services.

Recommendations

The B4SC is an excellent dataset for understanding the population distribution of the developmental areas measured as part of the B4SC, and predictors of these measures. For example, analysis of the B4SC has shown that pre-school obesity is trending downward in New Zealand,(6) and that these trends are similar across territorial authorities in New Zealand.(11) Analyses have also demonstrated marked socio-economic inequalities in pre-school oral health.(10) The B4SC is less useful (currently) to determine predictors for later functioning. For example, the B4SC was shown to be a relatively poor predictor of the receipt of literacy interventions in the first year of schooling.(12) In part this is due to the short time series available for B4SC data, and the value of the B4SC for predicting adolescent and later outcomes may improve as the time series lengthens.

Summary

The B4SC is a national programme monitoring the health and development of four-year-old children, with data from 2011 onwards now available for analysis in the IDI. It is the only nationwide collection of information on vision, hearing, oral health, obesity, behavior, and developmental problems at any age in childhood. Currently >90% of four-year olds enrolled with a PHO receive the B4SC, but coverage is slightly lower for Māori, Pacific, and children living in high deprivation areas. B4SC data in the IDI are currently useful for investigating trends in, and predictors of, vision, hearing, oral health, and obesity, but the real value of the B4SC may be realized once the data have a long enough time series to assess associations between B4SC measures and outcomes in adolescence and adulthood.

References and further information

Ministry of Health. Well Child / Tamariki Ora Programme Practitioner Handbook: Supporting families and whānau to promote their child’s health and development – Revised 2014. Wellington: Ministry of Health; 2013.
Ministry of Health. Enrolment in a primary health organisation. 2017; http://www.health.govt.nz/our-work/primary-health-care/about-primary-health-organisations/enrolment-primary-health-organisation. Accessed 15th February, 2017.
Ministry of Health. B4 School Check information for early learning services 2015 [Available from: https://www.health.govt.nz/our-work/life-stages/child-health/b4-school-check/b4-school-check-information-early-learning-services.]
Ministry of Health. B4 School Check information for the health sector. 2016; https://www.health.govt.nz/our-work/life-stages/child-health/b4-school-check/b4-school-check-information-health-sector. Accessed 24th February, 2018.
Gibb S, Barry Milne B, Shackleton N, Taylor B, Audas R. (in press). How Universal are Universal Pre-School Health Checks? An observational study using routine data from New Zealand’s B4 School Check. BMJ Open
Shackleton N, Milne BJ, Audas R et al. Improving rates of overweight, obesity, and extreme obesity in New Zealand 4-year-old children in 2010-2016. Pediatr Obes 2018;13:766–77.
Statistics New Zealand (2015). Integrated Data Infrastructure extension: Privacy Impact Assessment (3rd ed) Wellington: Statistics New Zealand.
Hatch B, Gray AR, Taylor RW, Hanna M, Heath AL, Lawrence J, Sayers R, Taylor B. (2019). Examining the accuracy of the New Zealand B4 School Check universal health service anthropometric measurements of children. N Z Med J.; 132(1489): 89-101.
Kersten P, Vandal A, McPherson K, Elder H, Nayar S, Dudley M. (2014). A validation and norming study of the strengths and difficulties questionnaire in the New Zealand context. Auckland; Auckland University of Technology. https://www.health.govt.nz/publication/validation-and-norming-study-strengths-and-difficulties-questionnaire-new-zealand-context. Accessed 24th February, 2018.
Shackleton, N, Broadbent, JM, Thornley, S, Milne, BJ, Crengle, S, Exeter, DJ. (2018). Inequalities in dental caries experience among 4-year-old New Zealand children. Community Dentistry and Oral Epidemiology,46(3): 288-296.
Gibb, S, Shackleton, N, Audas, R, Taylor, B, Swinburn, B, Zhu, T, Taylor R, Derraik, J, Cutfield, W, Milne, B. (2019) Child obesity prevalence across communities in New Zealand: 2010-2016. Australian and New Zealand Journal of Public Health. https://doi.org/10.1111/1753-6405.12881
Schluter, PJ, Audas, R, Kokaua, J, McNeill, B, Taylor, B, Milne, BJ, Gillon, G. (in press). The efficacy of preschool developmental indicators as a screen for early primary school-based literacy interventions. Child Development.