SEM for Experimental Designs: An Information Systems Example

Abstract

Abstract: IS research has matured significantly over the last three decades, leading to increasingly complex research designs as well as complex analytical techniques to analyze the data collected. Similar advances have happened in the experimental and quasi‑experimental designs. Some key characteristics of these advances are: 1) use of latent variables approaches to operationalize key variables; 2) the need to understand the causal relationship between elements of the study; 3) the need to study the effects of technology as an addition to existing methods of working; and, 4) recognition that some conditions create greater change in outcomes over time. In spite of these advances in data collection and design, researchers are still confirming data collected via experiments to use ANOVA for analysis. This paper outlines an analytical technique that moves Information Systems experimental research beyond ANOVA. By combining and extending three advances in Structural Equation Modeling techniques, namely Mean and Covariance Structure analysis, Stacked Group modeling and Latent Growth modeling, the paper outlines a robust analysis technique that accommodates the above‑mentioned advances in experimental design. The technique provides for an in‑depth test of all model assumptions, as well as the flexibility to accommodate an increasing variety of experimental designs. A detailed example is provided to illustrate the usage of the technique in an Information Systems context. The example shows now only the accommodations needed in an information systems context, but also how this technique can be used to extract results from existing research methods that was not possible with ANOVA. The arguments presented in the paper as well as the example on how to use should provide future researchers with a guideline on how to use these techniques as well as provide a platform for how they can extend these techniques to accommodate more research method advances.