Using Sequential Mixed Methods in Enterprise Policy Evaluation: A Pragmatic Design Choice?

Abstract

Abstract: How might policy instruments contribute to indigenous firm growth and how can the effects of these instruments be evaluated at both firm and policy level? This paper illustrates how a mixed methods research design and data analysis strategy can pragmatically address the research questions outlined above. The advantages and challenges of employing quantitative research methods (what happened?) followed by confirmatory qualitative research methods (how and why did it happen?) in a multiphase sequential explanatory design is explored. The data analysis strategy is firstly to analyse the data generated from a before and after quasi‑experiment (with statistical controls), then data from the confirmatory qualitative techniques (indepth descriptive case studies) and cross‑case analysis are added. The proposed research design and analysis approach is applicable to complex research settings where a study is unable, for a variety of reasons, to meet the exacting requirements of a true experimental design e.g. random assignment, establishment of counterfactuals, valid control groups etc. This sequential multiphase approach can deliver findings on the relative contribution of the myriad factors influencing a result showing whether the policy intervention in this study made a contribution to an observed result and in what way? The findings from the Phase 1: Quasiexperiment, Phase 2: Case studies and Phase 3: Cross‑case analysis collectively demonstrates that the policy instrument evaluated in this study made a marginal contribution at best to individual firm performance. Overall the state received a negative return on its investment (despite selecting the cohort of firms to invest in). The study concludes that, in the analysis period, the salient factors influencing value creation in the firms (and conversely the barriers to firm growth) were internal to the firm.