MSD trials Predictive Risk Modelling

Wed 29 Jul 2015

The Ministry of Social Development (MSD) is trialling the use of Predictive Risk Modelling (PRM) in relation to child maltreatment. The model ...

The Ministry of Social Development (MSD) is trialling the use of Predictive Risk Modelling (PRM) in relation to child maltreatment.

The model was developed by Rhema Vaithianathan, currently Professor of Economics at Auckland University of Technology. The model seeks to predict the risk of a newborn baby experiencing substantiated child abuse by the time the child turns five. If a child is found to have a high risk score, the model’s authors propose that this trigger a voluntary, targeted response with the aim of preventing child maltreatment.

The model was created from a linked data set of information from agencies such as WINZ and Child, Youth and Family. The modelling was based on 132 variables. 45% of the variables relate to the demographics, socio-economic status and history of the primary caregiver (including the caregiver having a history of being abused themselves). 37% relate to the primary caregiver’s partner (if present). 18% relate to children (including prior substantiations of abuse).

PRM was flagged by the Government in the White Paper for Vulnerable Children which noted "preliminary results [were] highly encouraging in terms of the model's ability to predict maltreatment" (Vol 2, p.76). The White Paper also noted that an extensive literature review found "there are no jurisdictions currently making use of, or that had made use of, automated predictive risk modelling for this purpose" (Vol 2, p.76).

Following development of the model, MSD commissioned a Privacy Impact Assessment (2013), a review of ethical issues for Māori (2013), an ethical review (2014) and a feasibility study (2014), available on the MSD website.

Concerns have been raised about the use of PRM, by advocates, other academics and the authors of ethical reviews, including: its individualised conception of risk; the stigmatisation of people identified as having high risk scores; the stigmatisation of groups such as Māori and people receiving benefits; the variable quality of the administrative data the model is based on; the model's predictive ability (including the number of "false positives" and "false negatives"); questions about resource allocation; questions about the actions and obligations of agencies in relation to high risk scores; privacy and the use of data without consent; and potential impacts on people’s interactions with services and government agencies. See the media stories below for further detail.

In the United Kingdom, a database was set up in 2004 seeking to enhance child protection by improving the way information about children was shared between services. The database, ContactPoint, was disestablished in 2010 amidst concerns about its security, privacy, cost and effectiveness.

MSD states "while the application of Predictive Modelling to child maltreatment raises some significant ethical concerns, those concerns can either be significantly mitigated by appropriate implementation strategies or are plausibly outweighed by the potential benefits of such modelling."

MSD is currently testing whether Predictive Risk Modelling can enhance social workers' decision-making at intake for children who are reported to Child Youth and Family because of concerns about abuse or neglect. (This is a different application of the model to predicting risk of maltreatment at a child’s birth). MSD states that the aim is to support rather than replace professional judgment.

Updates:

The Ministry of Social Development had proposed to include 60,000 children born this year in an "observational study" to test the accuracy of its new predictive risk modelling tool. Stuff reported that the study would have involved risk-rating a group of newborns and not intervening in high-risk cases, to check whether their predictions came true. The proposal went to Social Development Minister Anne Tolley who called for an end to it, saying "Not on my watch! These are children not lab rats". The ethics proposal has now been withdrawn.

UNICEF New Zealand issued a media release urging caution with the model as a result. Further media is listed below.

Stuff also reported Tolley said it was "unlikely" the model would be used on children that had not already been notified to Child Youth and Family (CYF). Tolley said, "Where it goes from there is another big ethical question. Because God knows, do we really want people with clipboards knocking on people's doors and saying 'hello, I'm from the Government, I'm here to help because your children are going to end up in prison?' I just can't see that happening."

Finance Minister Bill English and Statistics Minister Craig Foss announced the formation of the New Zealand Data Futures Partnership on 6 August 2015. The Partnership, an extension of the New Zealand Data Futures Forum will be headed by Dame Diane Robertson and tasked with the safe collection, use and sharing of government and business data. Mr English said "The Partnership will complement other data initiatives which enable the Government to deliver better services to the public and ensure investment is made where it’s needed most."

Resources available from the Clearinghouse library

Understanding overrepresentation of indigenous children in child welfare data: An application of the Drake risk and bias models Fiona Cram, Pauline Gulliver, Rissa Ota and Moria Wilson (2015)

The ethics of predictive risk modelling in the Aotearoa/New Zealand child welfare context: Child abuse prevention or neo-liberal tool? Emily Keddell (2015)

Predictive modeling: Potential application in prevention services Moria Wilson, Sarah Tumen, Anthony Simmens (2015)

A Minority Report for Social Work? The Predictive Risk Model (PRM) and the Tuituia Assessment Framework in addressing the needs of New Zealand's Vulnerable Children (Eileen Oak, 2015)

Another Pandora's box? Some pros and cons of predictive risk modeling Irene de Haan and Marie Connolly (2014)

Addressing child maltreatment in New Zealand: Is poverty reduction enough? Tim Dare, Rhema Vaithianathan and Irene de Haan (2014)

Ethical issues for Māori in predictive risk modelling to identify new-born children who are at high risk of future maltreatment Anton Blank and others (2014) (see also reviews, on the MSD website)

Predictive risk modelling and child maltreatment: An ethical review Tim Dare (2014) (see also reviews, on the MSD website)

The feasibility of using predictive risk modelling to identify new-born children who are high priority for preventive services (and companion technical report) Insights MSD, Ministry of Social Development (2014) (see also reviews, on the MSD website)

Children in the Public Benefit System at Risk of Maltreatment: Identification via Predictive Modeling Rhema Vaithianathan, Tim Maloney, Emily Putnam-Hornstein and Nan Jiang (2013)

Vulnerable children: Can administrative data be used to identify children at risk of adverse outcomes? Rhema Vaithianathan (2012)