Friday, June 1, 2007

A Bizarre Sample

I have drawn two conclusions from the Lifeway Report:
  1. The percentage of Southern Baptists tempted by the Charismatic Movement is much larger than I had guessed.
  2. The same percentage is likely much smaller than reported by Lifeway Research.
Does anything seem asymmetrical to you about the research sample?
 PastorsLaity
Protestant Non-SBC6021,004
SBC403???
Where are the Southern Baptist laity? I raised this question earlier at Marty Duren's blog, and here is his response, presumably from Lifeway Research, unless Marty is otherwise somehow personally aware of the particulars of the research methodology for this survey:
The entire sampling was of 1,000 or so lay people (all Protestant) and 1,000 or so pastors (all Protestant, but with enough SBC pastors [405] to form an oversampling). The purpose of the survey was not to determine what Southern Baptist laypersons believe versus other Protestant layperson regarding the PPL issue, but what Protestants believe, with an emphasis on Southern Baptist pastors (who were oversampled to provide a large enough pool for statistical accuracy).
So, either (and I'm not sure which is the case) some number of Southern Baptist laypeople actually form a portion of the "Protestant laity" group, but have not been broken out to give us their views statistically, or all 1,004 of the laypeople surveyed were non-SBC. Either way, from what I've told you at this point the refusal to give statistics for SBC laity is merely a curiosity. But when you look at the data, you realize that it is not merely a curiosity. In the Protestant category, the laypeople broke decidedly away from a belief in PPL. Indeed, in the podcast, one of the commentators (Stetzer, I think) remarked at how strongly and surprisingly Protestant laity differed from pastors in rejecting PPL. A full 15 points (or 13, there appears to be an error in the slide I have in front of me) separates Protestant pastors and laypeople on this question. So, the very category that moves the statistics dramatically away from support of PPL is the category excluded from consideration as regards Southern Baptists. I note in this regard:
  1. Overall, Southern Baptists were far less likely to support PPL than were Protestants at large.
  2. If there were SBC laity in the Protestant laity sample, they may well be the explanation for this dramatic separation between Protestant pastors and "Protestant" laity. In which case, if Southern Baptist laity were considered separately (as they should have been), the resulting SBC number would be significantly lower than 50%.
  3. If there were no SBC laity among the 1,004 Protestant laity, then we might expect SBC laity to be as different from Protestant laity as SBC pastors are from Protestant pastors. In which case the inclusion of SBC laity would still result in an overall number far below 50%.
Of course, I'm speculating. Of course, I am left in a situation where I have to speculate because the data have not been given to us...because of a strangely sculpted categorization of the sample. I cannot imagine how the categorization could have been accomplished in a fashion more favorable to the pro-PPL group, even if one were trying. If there were SBC laity in the survey, I call upon Lifeway Research to publish the percentages for SBC laity responses to the various questions.

17 comments:

  1. Since LifeWay was conducting the survey, it seems strange that they did not plan to include a report of Southern Baptist laity.

    ReplyDelete
  2. R. L.,

    First, I note that I owe you responses on several threads. You always ask such substantive questions that they take TIME to respond! And I've been at a shortage of that lately. After San Antonio we need to get together finally and share some East Texas cuisine.

    ReplyDelete
  3. Bart-
    Thanks for admitting that you are speculating. You've sure speculated about me;^)

    I thought that I was clear in stating that, yes, the 1,000 lay people included Southern Baptists, though, apparently, they were not oversampled as were the pastors. Forgive me for lacking clarity there; I was actually trying to provide some :^)

    It would also be a very one sided comparison of people who had been "tempted by the charismatic movement" barely registering versus those who have been "tempted by fundamentalist legalism."

    "Sculpting"? The complete questions and answers are listed in the article. People can sculpt all they want.

    ReplyDelete
  4. Bart: given all the dialogues and arguements on this subject during this past year, I would like to know why Lifeway would release a survey that had so many questions attached to it. It's very problematic in helping folks determine what the statistics really are, what they really mean and how they were gathered. selahV

    ReplyDelete
  5. Marty,

    Thanks for the talking points. They are duly noted.

    Now, would you like to interact with the fact that the laity sample broke strongly away from the pastor sample against PPL—and the fact that this data has been presented in such a way as to prevent anyone from tying it to the question at hand?

    ReplyDelete
  6. SelahV,

    That is a great question.

    ReplyDelete
  7. Bart,

    Interesting analysis. I would like to add that statistics 'say' nothing. Statistics simply provide a framework whereby data can be analyzed and subsequently interpreted. Interpretation comes in the form of CONCLUSIONS by the researcher. Responsible research then dictates that further study be done to validate.

    Of course, no data is above scrutiny and we must remember that flawed survey/research design or implementation can lead to misleading results.

    With all of that said, polls can neither prove/disprove accurate theology. I appreciate the fact that you consistently bring the discussion back to Biblical theology and practice.

    This report and the flurry of blogging about it merely solidifies what everyone has known:

    1) There are Southern Baptist in both camps.

    2) Both sides are passionately engaged in the debate

    3) Either camp could find themselves in the minority.

    4) Both sides want to have their voice heard.

    We must find a healthy way to deal with this and other theological difference as a Convention or . . .???

    ReplyDelete
  8. Bart,

    Is there any way that you could contact Lifeway Research and get the information you desire without speculating or casting aspersion on their methodology? Did Dr. Yarnell attempt this? It seems that we should give one of our SBC entities the benefit of the doubt when it comes to their leadership and work before we attack them. If there is a problem with Lifeway Research leadership, I am sure that the Lifeway BoT will address it. If they do not address it to your satisfaction, I would recommend that you and your friends in this debate try and change the make-up of the trustees. If that takes 7-10 years, well, that's just the way that Southern Baptist polity works. I know that you are a big fan of our polity, so I am sure that you won't see any problems there.

    Obviously, I am just having fun with you, Bart. Take a deep breath. :)

    ReplyDelete
  9. Bart, when you slow down shoot me an e-mail and we'll work something out for some ET cuisine.

    If you could come to our church/cemetery homecoming you could try some of my 92 yr old mother's chicken and dressing! (BTW, I'm the baby!)

    ReplyDelete
  10. Bart-
    Thanks for the encouragement to engage your speculation.

    First, if Lifeway did a survey to determine what "Protestants" currently believe about PPL, then the lack of oversampling of SBC laity is moot. If it wasn't in the purpose, then not doing it is not a mistake. The point, obviously, was a subset of SBC pastors.

    Second, if at least 405 SBC laity were not surveyed, then the result would be outside the margin of error and therefore could not be used accurately. I'm sure, with your awareness of the need for accuracy, you would not want a bias to develop due to a less than minimal sampling.

    Third, your chart implies that no SBC laity were survey which is bogus. They were included in the big set of protestant laity as we know.

    Fourth, as to your immediate question, "Why did the laity break away from the pastors in the PPL belief?" I have no idea. But your assertion that this somehow means something is baseless, other than what the survey actually shows.

    Suppose that SBC laity follow the lay trend and, knowing that SBC pastors are at 50% belief, then that only demonstrates division between pastors and lay people. What else would be knew?

    I'll be willing to bet your retirement that if a survey was done on eschatology, the percentage of pastors holding to post- or a- mil positions would be substantially higher than the laity, who would lean much more heavily toward pre-.

    I'm sorry you didn't get the results that you wanted. It doesn't change the facts.

    ReplyDelete
  11. Marty,

    Sure, you would bet MY retirement.

    But even your comments seem to allow this: Considering the responses of SBC laity, the pro-PPL percentage is highly likely to have dropped well below 50%.

    ReplyDelete
  12. Bart,

    I have survyed 5 people who prefer Pepsi over Coke. All five are SBCers. Therefore, Pepsi is the beverage of choice amongst Southern Baptist. :)

    Just having a little fun with all this. I hope no one minds.

    DWMIII

    ReplyDelete
  13. dwm,

    put me in your survey as liking pepsi over coke as well. will you be bringing this report to the floor of the convention? i will stand at a mic to speak for it.
    :)

    david

    ReplyDelete
  14. David, can a good "Southern" Baptist prefer Pepsi over Coke? I mean, come on. Coca-Cola is headquartered in Atlanta, GA and Pepsi is headquartered in Purchase, N.Y.!! New York
    :-O

    Of course, there is Dr. Pepper, developed in Waco....

    ReplyDelete
  15. rl,

    i'm not sure, but i think the official drink of the sbc is royal crown cola, or better known as r.c.

    rl, have you ever eaten a moon pie by any chance?

    david

    ReplyDelete
  16. David, you'll never get me to confess to having an R.C. and a moon pie!

    ReplyDelete

Note: Only a member of this blog may post a comment.