In May last year, in a blog about how the voluntary sector can demonstrate its impact in the context of Transforming Rehabilitation, I talked about the launch of the Ministry of Justice Data Lab pilot.

The Justice Data Lab allows organisations access to reoffending data for their service users, and compares them with a “matched control group” of offenders with similar characteristics who did not receive the organisation’s service. This provides a rigorous method for understanding the impact of an organisation’s service or intervention on reoffending . The pilot has been running since April 2013 and will run to the end of March 2014.

The latest set of results was published last week, which seems a good point to look back over the results from over the last 9 months and ask what lessons they have for us.

The findings

To date there have been 73 requests to the Data Lab from a range of organisations in the voluntary, statutory and private sectors. Forty Six services or interventions have been analysed, 11 requests could not be answered because the minimum criteria for analysis was not met, 1 was withdrawn and 15 are outstanding.

Thirteen of the complete analyses have found a statistically significant reduction in reoffending by the client group. These include offenders who received a grant through the Prisoners Education Trust (for a range of educational courses and for arts and hobby materials), offenders in 10 areas of the country who received support through the NOMS CFO service (which supports offenders to access mainstream services in the community), offenders who received short-term employment with Blue Sky and offenders targeted by Brighton and Hove City Council’s POAL project which provides homelessness interventions.

In the vast majority of the other services analysed, there was insufficient evidence to say whether an intervention had resulted in a significant change in reoffending.

There have been some difficulties in matching service users with official data, for example because of incomplete data (either from the organisations submitting data or from the official records themselves). In some cases the data submitted has included service users who were released after 2010 and at the moment reoffending data is only avilable until the end of 2010. This means that many organisations have ended up with a reduced sample that may not represent their complete service user group and as such is insufficient to identify statistically significant results.

You can read more detailed information on the results here.

Limitations and future developments

From the results so far it is clear that the larger the sample size submitted to the Data Lab, the more likely the results will be statistically significant (although Blue Sky and POAL did have smaller sample sizes). In addition, back in May I highlighted that the high threshold of data needed could present a challenge for the sector, particularly for small organisations, and this definitely seems to be an issue - the more complete the data is, the less likely it is that there will be unmatched cases.

Promisingly, reoffending data for 2011 will soon be available, meaning that new requests to the Data Lab should be able to match a greater number of service users. It’s also good news for organisations whose service users were largely still in prison after 2010, resulting in insufficient evidence, as they will be able to resubmit their data. Organisations are also more likely to have detailed information in the format required by the Data Lab on more recent service users, meaning that they are less likely to submit incomplete data in future.

Is the Data Lab for you?

The Data Lab provides organisations with a way to understand their impact on reoffending through using a quasi random control group. This is generally viewed as a high standard of evidence and can therefore provide you with a strong case that your service contributes to reduced reoffending. (You can read more on standards of evidence here). However, you cannot predict the results and for transparency the Ministry of Justice will publish all reports, so it is important to consider this before you decide to submit a request

If you do choose to use the Data Lab, you need to be able to provide the Ministry of Justice with each individual client/service-user’s forename; surname; date of birth; gender and, if possible, Police National Computer Identifier (PNCID) and/or Prison Number. You will also need to provide your intervention start and end dates and whether you worked with them in the community or in custody.

Some organisations have expressed concern about confidentiality/data protection in providing personal information about their service users. MoJ have considered this and are satisfied that the Data Lab meets the necessary standards. However, some organisations have chosen to seek legal advice on whether their existing consent forms give them sufficent permission to use the data in this way.

The Data Lab is currently being evaluated by the Ministry of Justice  in order to inform a decision as to whether it will continue and in what form beyond March. This means that there may only be three months left for organisations considering using it to submit their requests.

Of course, reducing reoffending is just one of many outcomes that the VCSE Sector working with offenders aims to achieve. It may be that in the future it is able to also provide information on other outcomes such as frequency and severity of offence. This, along with other evalution data on the attitudes and behaviour of service users, and factors like their mental health, wellbeing, family relations and employment, could provide us with a comprehensive picture of an organisations’ impact, which is a good thing - but we are at an early stage in using official data as part of that.

You can read Clinks’ guidance on submitting to the Data Lab here and the Ministry of Justice information here.

You can also find guidance and support on evaluation and research more broadly from Clinks and NPC’s Improving your evidence project here.