Evaluation of a Random Breath Testing Initiative in Victoria 1990 & 1991. Summary Report

Monash University Accident Research Centre - Report #39 - 1992

Authors: A Cavallo & M. Cameron

Abstract:

This report provides a summary of the findings of two evaluations, with differing methodologies, to determine the impact of a random breath testing (RBT) initiative (including publicity) in Victoria on severe crashes in high alcohol times of the week during 1990 and 1991. Bus-based RBT stations using highly visible "Booze Buses" largely replaced car-based stations, and a multi-million dollar, Statewide anti-drink driving publicity campaign through all mass media, was launched in mid December 1989, and reinforced throughout 1990 and 199 1.

It was found the initiative reduced fatal crashes (in high alcohol times of the week) in Melbourne during 1990 by around 19-24% relative to what was expected. Serious casualty crashes (in high alcohol times) in the rural areas around Melbourne were reduced by 15% relative to other rural areas of Victoria where the PBT initiative was expected to have had minimal effect. It was also estimated that there was a 13% reduction in serious casualty crashes (during high alcohol times) in the whole of rural Victoria. The effect on Melbourne serious casualty crashes (during high alcohol times) was more difficult to estimate, given that a new decreasing trend in these crashes emerged shortly before the intervention. The choice of the appropriate model to represent the expected post-intervention trend, and hence estimated change, differed between the two studies. The first study, using an estimate which emphasised the new road safety trend prior to the intervention, found no statistically significant evidence that the RBT initiative had an effect over and above that which would have occurred if the pre-intervention trend had continued downward. In contrast the second study, using a model which basically placed equal weight on the trend for the previous twelve month pre-intervention period, suggested that RBT reduced serious casualty crashes during high alcohol times in Melbourne by between 8 and 27%.

The second evaluation also attempted to assess the effect of the initiative on these crashes in 1991. The results suggested that there was a statistically significant reduction in serious casualty crashes (in high alcohol hours) in rural Victoria only, and no change in fatal crashes (in high alcohol hours) in either Melbourne or the rural area. The confidence limits were very wide for fatal crash estimates in this period however, reducing the chance of showing a significant effect. The findings for the 1991 period are also less conclusive because other (new or changing) factors may mask or weaken the effects further away in time from the beginning of the intervention.

Executive Summary

Background

This report provides a summary of the findings of two evaluations, with differing methodologies, to determine the impact of a random breath testing (RBT) initiative (including publicity) in Victoria. The first evaluation estimated the effect of the initiative on severe crashes in high alcohol times of the week during 1990 whilst the second study also estimated its effect during 1991.

The initiative, introduced late in 1989, involved a substantially different method of RBT enforcement compared with past operations. Bus-based RBT stations using highly visible "Booze Buses" largely replaced car-based stations, and a multi-million dollar, Statewide anti-drink driving publicity campaign through all mass media, was launched in mid December 1989, and reinforced throughout 1990 and 1991. This campaign was the cornerstone of public perceptions of the program, designed to both heighten perceptions of extended enforcement and sensitize the public to the consequences of drink driving. In brief the key aspects of the initiative were:

A major multi-million dollar, multi-media publicity campaign
Thirteen new, high profile 'Booze Buses', progressively replacing car-based testing, especially in Melbourne
A strike force using 'Probationary Constables In Training' (PCIT's) on monthly roster to operate buses
More than doubling the number of drivers tested, mostly in Melbourne, gradual but smaller increases in session hours and no change in the number of sessions conducted in Melbourne.

Evaluation Studies

Two different evaluation methods and statistical methodologies were used to evaluate the impact of RBT on severe road crashes in high alcohol hours of the week during 1990. The second evaluation also estimated effects during 1991.

The first study (Drummond, Sullivan & Cavallo, 1992) used a quasi-experimental time series analysis. A forecasting model was used to estimate the changes in two treated areas, Melbourne and a part of rural Victoria (Rural 1), taking into account changes in the same crash types in two respective comparison areas, Sydney and Rural 2 (a minimally affected area in country Victoria). Accounting for changes at these comparison areas (quasi-experimental control) increases the likelihood that measured effects are validly ascribed to the initiative, rather than the effect of any other factors. This quasi-experimental design was used in the absence of a full experimental design (which provides the most rigorous assessment), and in preference to the use of statistical control.

The second study (Cameron, Cavallo & Sullivan, 1992) used a multivariate time series analysis. An Intervention Analysis (assessing a step reduction with the introduction of the initiative) was used to estimate the changes in crashes in treated areas (Melbourne and rural Victoria) relative to comparison areas (Sydney and rural NSW). However, respective unemployment rates were also directly taken into account because it was considered that statistically controlling for this factor was appropriate.

Results

The quasi-experimental time series analysis found:

a 19% reduction in fatal crashes (in high alcohol times of the week) in Melbourne in 1990 due to the initiative, but no change in serious casualty crashes
a 15% reduction in serious casualty crashes (in high alcohol hours) in the Rural 1 area (mainly the rural area around Melbourne) in 1990, but no change in fatal crashes; a more tentative attribution of this effect to the initiative was made given the pattern of results.

The multivariate time series analysis found:

a 24% reduction in fatal crashes and an 18% reduction in serious casualty crashes (in high alcohol hours) in Melbourne in 1990 due to the initiative
a 13% reduction in serious casualty crashes (in high alcohol hours) in rural Victoria in 1990 ascribed to the initiative, but no change in fatal crashes.

Interpretation

The most significant finding from these studies is that the initiative reduced fatal crashes (in high alcohol hours) in Melbourne by around 19-24% relative to what was expected. This significant drop in high alcohol hour fatal crashes as measured by both studies is the strongest evidence of the program's effectiveness, given that:

high alcohol hour fatal crashes are more likely to be alcohol related, and
large statistically significant reductions were found even though the smaller number and greater fluctuation of fatal crashes make it more difficult to estimate changes.

The first study found a 15% reduction in serious casualty crashes in the rural areas around Melbourne relative to the other rural areas in Victoria (where the RBT initiative was expected to have had minimal effect). The second study found a 13% reduction in serious casualty crashes in the whole of rural Victoria relative to rural NSW (where no substantive change in RBT activity occurred during 1990). Both studies found no change in rural fatal crashes however, which are considered to be more alcohol related (than serious casualty crashes) and thus likely to be affected by the RBT initiative. It is conceivable that reduced statistical sensitivity for measuring changes in rural fatal crashes (given smaller numbers) means that an effect is difficult to detect. The absence of evidence for an effect on fatal crashes but evidence for a significant effect on serious casualty crashes provides tentative evidence of an effect of the RBT initiative in rural Victoria.

The effect on Melbourne serious casualty crashes was more difficult to estimate, given that a new decreasing trend in these crashes emerged shortly before the intervention. The choice of the appropriate model to represent the expected post-intervention trend, and hence estimated change, differed between the studies. The first study, using an estimate which emphasised the new road safety trend prior to the intervention, finds no statistically significant evidence that the RBT initiative had an effect over and above that which would have occurred if the pre-intervention trend had continued downward. It should be noted however, that this finding is also in part due to the wide confidence interval (-18.7% to 25.9%) in which the estimate lies. In contrast, the second study, using a model which basically placed equal weight on the trend throughout the previous twelve month pre-intervention period, suggested that RBT reduced serious casualty crashes in Melbourne by between 8 and 27%.

Findings for 1991

The second evaluation also attempted to assess the effect of the initiative on these crashes in 1991. The results suggested that there was a statistically significant reduction in serious casualty crashes (in high alcohol hours) in rural Victoria only, and no change in fatal crashes (in high alcohol hours) in either Melbourne or the rural area. It must be noted however, that the confidence limits were very wide for fatal crash estimates in this period which reduces the chance of showing a significant effect and providing precise estimates. The findings for the 1991 period are also less conclusive because other (new or changing) factors may mask or weaken the effects further away in time from the beginning of the intervention.

CONCLUSION

Evidence was found from the two evaluation studies for the effectiveness of the RBT initiative in reducing fatal (high alcohol hour) crashes in Melbourne in 1990.

Some evidence was also found to indicate (more tentatively) that the RBT initiative may have reduced serious casualty (high alcohol hour) crashes in Melbourne in 1990, and in rural Victoria both in 1990 and 1991.

Sponsor: Transport Accident Commission