Using Polynomial Regression (PR) and Response Surface Methodology (RSM) to Determine Fit/Congruence

A prominent research interest of mine is assessing person-vocation fit and its relationship to work outcomes, such as job performance.  Polynomial regression (PR) and response surface methodology (RSM) are ideal methods for measuring person-vocation.  Reasons for the superiority of PR-RSM are numerous, but not the point of this post.  The interested reader should see Jeff Edward’s writings here and here for rationale.  The point of this post is to offer high-level, practical guidance on how to apply PR-RSM to fit questions.  So, without further ado:

PR-RSM Guide for Evaluating PE Fit

1. Have  a useable data set. Finding higher-order effects (i.e. effects for squared and interaction terms) generally requires larger samples for adequate power.
2. Have an interesting research question–have (theoretically-grounded) fun with this.  The reality is that polynomial regression allows you to do much more than assess congruence and incongruence broadly.  Rather, you can hypothesize about incongruence at low, mid, or high levels of scale strength.  For example, at mid-levels, you can think about incongruence when one component (person or environment) is higher or lower than the other (evaluated by the line of incongruence).  You can hypothesize about changes along the line of congruence as well (e.g., is an outcome higher when congruence between components is high-high compared to low-low?).
   1. More specifically, I recommend considering the following areas of the plot for hypothesis development:
      1. First, consider the congruence line.
         1. Should the outcome of interest change along the congruence line?
      2. Next, consider where the outcome should be highest.
         1. Is it somewhere along the congruence line (i.e. supporting P-E Fit), or is it somewhere representing a degree of incongruence?
         2. If the outcome is highest where incongruence exists, develop theoretical reasons for this.
      3. Next, discuss incongruence when your referent variable (e.g. the “person-side” in person-vocation fit) is low (vocation high), medium (vocation high and low), and high (vocation low).
3. Add relevant controls and fix them at their average when plotting the response surface plot.
4. Use F-tests to see if your addition of fit-related variables from linear to quadratic (including the interaction term) is significant.  You will ultimately interpret the response surface, so you are most interested in whether the additional terms add a significant change to your existing model and not whether the coefficients are significant individually.  SAS and Systat will both compute these F-tests for you when doing their respective PR functions (Proc RSREG and RSM respectively). I also have syntax to do this in R.  Notably, Systat and SAS both separate the addition of squared terms from the addition of an interaction term, so to compare these added together at the same time requires additional model comparisons.
5. Plot your results. SAS’s RSREG procedure allows this as does Systat’s RSM.  Systat can plot a linear or quadratic surface while the RSREG procedure in SAS will not plot linear graphs! In R, I highly recommend the RSM package for plotting and analysis.  Also note, that while you can easily rotate the plot in SAS’s RSREG procedure via a “rotate=xx” option (a similar option exists in the R RSM syntax), SYSTAT and the RSM package both offer a “mouse click/hold and rotate” option which is really helpful for quickly viewing the plot from multiple angles. Whichever program you use, be consistent as it will be hard to format plots to look the same using the two different programs.  Jeff Edwards also provides an Excel spreadsheet to create surface plots (for SPSS users, you can directly enter output into the spreadsheet).
6. Now interpret the results in line with your hypothesized relationships (developed in point “2” above).  It may be helpful to create a table of hypothesized conditions and note when each is or is not met across all analyses (as in Edwards & Cable, 2009).  You should also show regression coefficients and the results of your significance tests for adding both the linear and quadratic terms of interest for each surface plot.  Show the relevant surface plot and report hypothesized aspects of the plot for significance (i.e. the slope and curvature of the line of congruence).  See Edwards (2002) and Edwards and Parry (1993) for direction on significance tests of response surface characteristics.

I hope this helps motivate those with interesting fit questions to think about using PR and RSM.  This has largely been aimed at examining hypothesized relationships, but you are free to do exploratory analysis as well.  There are challenges to accomplishing the necessary analysis and interpreting 3D plots, but the method is well worth the effort!