Evaluating the effect of Australian policies on health and health care outcomes using panel data

Policy evaluations identify the causal effects of policies and help ascertain if desired objectives have been achieved, or if there have been unintended effects. Rigorous evaluation assists both thoughtful policy design, accountability and outcome improvement. Conversely, policies not informed by a strong evidence base are more susceptible to unintended consequences and costs. 

Data from randomised experiments are relatively rare in health and social policy evaluation, due to the high costs of running such experiments, difficulty of implementation and political concerns (Smith and Todd, 2005). Hence, the majority of data for such evaluations are nonexperimental and observational. Both experimental and observational data may be subject to a ‘missing data’ problem. However, with observational data, this occurs since it is not possible to observe the actions of the group affected by a policy (the ‘treatment group’) in the absence of an intervention. Under the potential outcomes framework, the task of rigorous policy evaluation is to construct a valid counterfactual through the choice or design of an appropriate comparator (‘control group’) and estimation technique (Blundell and Costa-Dias, 2000).

In Australia, most policymaking occurs at the Commonwealth government level, with policies often affecting the whole nation (Cobb-Clark, 2013). This creates challenges for evaluation and the construction of valid counterfactuals, as stronger assumptions are needed for identification than if geographic or time-based variations were available (Cobb-Clark, 2013; Cobb-Clark and Crossley 2003; Meyer 1995). This thesis tackles the challenges of policy evaluation using observational data in the Australian context, by evaluating the effects of three policies on three selected health and health care system outcomes, which were stated objectives of these policies. The analyses in this thesis explore the choice of the most suitable approach to evaluation for each specific policy context by considering differences in implementation, policy type, the probability of identification assumptions holding and differences in variable availability. 

Using the longest running panel dataset for Australia, the Household, Income, and Labour Dynamics in Australia survey, this thesis takes advantage of individual level data before and after policy implementation. This makes conditional independence more plausible in a nonrandomised context through the inclusion of more information than cross-sectional data. Additionally, panel estimation can remove potential heterogeneity bias from individual-specific constant unobservables.

The thesis is divided into five chapters. The introductory chapter provides an overview of the aims of this research and discusses challenges associated with policy evaluation. Chapter 2 analyses the impact of the harmonisation of workplace health and safety laws on workplace injury and disease by estimating effects on the probability of receiving workers compensation in the past year. Chapter 3 analyses the impact of the introduction of Australia’s national 2011 Paid Parental Leave scheme and complementary Dad and Partner Pay on maternal mental health. Chapter 4 evaluates the impact of means-testing a premium rebate and increasing an income tax penalty rate on private health insurance hospital coverage, under the Fairer Private Health Insurance Incentives reform. The final chapter of the thesis concludes with a discussion of findings, strengths, limitations and future directions for each study, and learnings for conducting policy evaluation in the Australian context.