Data → Theory or Theory → Data?

The ordering between data and theory in theory development deserves attention. What drives the process initially? Is it data → theory → data (repeating) or theory → data → theory (repeating)? What is the foundation of theory development? Direct and indirect commentary are presented in several management theory-oriented articles. Sutton and Staw comment on an example of data driving hypothesis development (and eventually theory) as “brute empiricism, where hypotheses are motivated by prior data rather than theory” (1995, p. 374). The connotation is that data driving theory is negative.

Related to the idea of whether data should drive theory or vice versa is the idea that interesting theories should receive more tolerance in regard to empirical support. Sutton and Staw conclude their article with a question of “whether the evidence provided by people such as Freud, Marx, or Darwin would meet… [current] empirical standards…” (1995, p. 383)? How far should the tolerance surrounding empirical support go? Beyond the confines of falsifiability?

Freud, Marx, and Darwin were all brilliant, prolific scholars who introduced fascinating theories of gargantuan scale. Yet while philosophically spell-binding and worthy of significant scientific inquiry, some of their theories challenge the call for falsifiability in theories because they are untestable at various, fundamental points. Using Darwin for example—his theory of natural selection is observable, testable and thereby able to be falsified—an important and relevant theory. But the theory of macro-evolution, purporting that planetary life evolved from a single-cell organism or organisms, is inherently unobservable, untestable and thereby not falsifiable. Karl Popper said as much in claiming “…Darwinism is not a testable scientific theory, but a metaphysical research programme—a possible framework for testable scientific theories” (1976, p. 168) [1] . As a program of research, the ideas purported by evolution are helpful to scientific activity, prompting much study and new ideas. Yet calling an idea beyond falsification a theory seems inconsistent [2] . Perhaps we should join Popper and call these ideas “metaphysical research programs” or simply “pseudo-theories.” Attempting to define theory as inherently falsifiable, yet providing tolerance for especially interesting ideas to “fudge” in this regard, especially beyond the confines of testability, is not helpful and only serves to further confound the definition of theory. Otherwise, as a minor case-in-point, the idea that theories must be driven by observable data itself could be broadly defined as a theory. Yet, this should not be so in management theory—the call for falsifiability as a threshold for the establishment of theory precludes it (Bacharach, 1989, p. 501). “Theory should be driven by observables” is simply an idea, it cannot be falsified and should not be titled a theory. Neither should more far-reaching ideas.

Now revisiting my beginning question of what should drive theory development—theory or data? I believe the answer to be data, a further criticism to Sutton and Staw’s remarks against “brute empiricism.” In the beginning, theory driving data is dangerous as it is more susceptible to individual influence and assumptions than objective theorizing from data followed by retesting with data and further theorizing (and on and on). Bacharach helps this critique in explaining theories as bound by “assumptions about values, time, and space” (1989, p. 499). Further support comes from a response to the Sutton and Staw article by Karl Weick. Weick calls attention to Bill Starbuck’s argument, summarizing that “just as the best medical doctors treat symptoms directly without relying on diagnosis to determine treatments, the best theorists may make prescriptions based on data alone without introducing theory between data and prescriptions” (1995, p. 387).

In conclusion, data → theory → data (repeating) promotes the most believable and objective science. Perhaps we could label such practices as “safe science.” Further, theorizing that does not allow for falsifiability should not be called theory regardless of how interesting it is, or even its likelihood of being true. Incidentally, such ideas cannot even be testable hypotheses, since they cannot be tested. Instead, they should be called what they are: ideas, systems-of-thought or ideologies. Grand and involved as they might be, they are not scientific theories and literature on theory building should clearly distinguish itself from them, not embrace them.

References

Bacharach, S. B. (1989). Organizational Theories: Some Criteria for Evaluation. The Academy of Management Review, 14(4), 496–515. doi:10.2307/258555

Popper, K. R. (1976). Unended quest: an intellectual autobiography (Revised.). London: Fontana/Collins.

Sutton, R. I., & Staw, B. M. (1995). What theory is not. Administrative Science Quarterly, 40, 371–384. doi:10.2307/2393788

Weick, K. E. (1995). What theory is not, theorizing is. Discussion of R. Sutton and B. Staw’s Article, 40, 385–390. doi:10.2307/2393789