Predictive risk modelling trialled for child protection intake decision-making

In October, The Ministry for Vulnerable Children, Oranga Tamariki (MVCOT) released documents relating to the Enhancing Intake Decision-Making Project under the Official Information Act (OIA). The project was commissioned by the Minister for Social Development in 2014. It was a collaboration between Insights MSD, Child Youth and Family (CYF), and the CYF National Contact Centre.

The project looked how social workers could use predictive risk modelling during intake, where a concern has been raised regarding a child or young person and a recommendation for a service response needs to be made. Specifically, the project explored whether a statistical risk model based on linked administrative data could improve the decisions made by social workers related to children notified to CYF. Intake decisions are assessments about whether the level of concern meets the threshold for involvement with CYF and, if so, how quickly CYF may need to respond.

The Report of the Enhancing Intake Decision-Making Project (October 2017) states that a statistical risk tool "can use administrative data to identify factors most frequently associated with cases that require CYF help. A tool can combine measures of the child, young person and their family’s characteristics in a systematic fashion to produce a single measure of overall risk." (p.14)

The project involved the development of a model, pre-testing of trial materials, further model development, and a trial in a non-operational context. The trial used CYF social workers within a simulated Contact Centre environment to assess the impact of the statistical risk model information (in the form of a 'Background Risk Indicator') on decision-making.

The report raises a number of areas that would need addressing or further exploring. It states that moral, ethical, and practical issues need to be considered. It also notes that the use of statistical risk tools within a care and protection context is a "developing field" and Florida is the only example where a tool has been successfully implemented. The report concludes that using the model could improve decision-making.

The report notes:

"The increase in accuracy provided by the statistical risk model was broadly comparable across different ethnic groups, although the model refers a higher number of Māori children and young people than under the status quo. The reason for this higher referral rate is currently unknown, and further work will be required to better understand this finding." (p.10)

It also states:

"Implementation of statistical risk modelling could impact on referral rates for Māori children and young people. Early engagement with Māori academics, service providers, and other stakeholders ahead of any implementation would be highly valuable, particularly given long-standing concerns about the high proportion of Māori children and young people already within the care and protection system. This work would also need to ensure the availability of appropriate services, which could meet the needs of Māori children and young people, and their whānau." (p.12)

The Overview and Next Steps for Enhancing Intake Decision-Making Project (October 2016) recommended pursuing this work further under MVCOT. The OIA letter and Aide Memoire (October 2017) state:

"Statistical risk modelling is one of a range of options that the Change Programme is interested in investigating further, as the services to support the future operating model are designed. Risk modelling may be able to provide another tool for social workers to access that can enhance decision making. At this early stage no decisions about the use of predictive risk modelling have been made."

Minister for Social Development Carmel Sepuloni has said the current Government is reviewing the previous Government's "social investment" strategy.

The use of predictive risk modelling is controversial and has attracted criticism. 

For further information see our previous news stories:

• Updates, events and resources on "social investment" approach, August 2017
• MSD trials Predictive Risk Modelling, July 2015
• 'Predictive risk modelling' for child maltreatment, October 2012
• Stories previous proposal to require NGOs to provide Individual Client Level Data

Research, reports and articles on predictive risk modelling are available from the Clearinghouse library.