Comments on: “Peer-review” of Marzano’s IWB Study Report, Part III

By: Educational Insanity » Blog Archive » “Peer-review” of Marzano’s IWB Study, Part IV

Thu, 24 Jun 2010 16:59:17 +0000

[...] [NOTE: this is the fourth in a series of posts about a report recently issued based on a study done by Marzano Research Laboratory. Part I is here, Part II is here, and Part III is here.] [...]

By: Brendan Murphy

Brendan Murphy — Wed, 10 Jun 2009 12:56:52 +0000

I can’t imagine being able to design two similar units on the same topic to be taught during the same year not having some effect on each other.
I suppose pre and post tests are suppose to mitigate the effect of teaching the same topic twice in one year, but honestly I doubt it.

By: Mike Scott

Mike Scott — Thu, 04 Jun 2009 10:12:50 +0000

John

You are my new hero. When I first saw this this so called “research”, the red flags went up in huge way. Marzano has made a huge living off of this kind of research and is possibly one of the most influential speakers on effective instruction touring the talk circuit today. Clearly the Promethean “research” has little value as research… it has much value to Promethean who has plastered Marzano’s face on the website. My problems with Marzano and meta analysis go back to his well known work on classroom instruction. After reading his paper on that work, was very skeptical about his claims of what was effective because, like the Promethean study, there are real problems with all aspects of his research. I can find no peer review, no critical comments or analysis. In fact, this is the first look at his work in any academic arena that I can find. What has bothered me more than anything, is a nearly universal acceptance of his assertions by instructional supervisor colleagues who have never read the research and accept that he’s provided reliable and valid information that they then tout as the solution to instructional problems in schools.

I look forward to reading the rest of your review and forwarding it into the realm of instructional technology.

By: Steve Ransom

Steve Ransom — Thu, 04 Jun 2009 01:38:27 +0000

Nice analysis, Jon. For me, the failure to control for reliability and validity of assessment measures undermines the entire study. And, as you point out so well, conducting a meta-analysis of such problematic “studies” makes the results even more problematic. The claim of action research done by the teachers by simply administering pre- and post-tests is beyond a stretch.

I look forward to the remaining posts on this.

By: Linda704

Linda704 — Thu, 04 Jun 2009 01:10:50 +0000

Jon, great series. I have to jump in on a couple of points. With all of the content pressures, I cannot imagine an elementary teacher taking time they already don’t have to teach an additional “similar” unit. (Did they get an IWB to participate? Could be enough motivation…but what content had to be sacrificed?) As far as tests being “similar” Stiggins asserts (I agree) that teachers do not have enough instruction in assessment design–wonder how “similar” the assessments were? Finally, I nearly choked about converting rubric score to a percent. What?!? Teachers sometimes convert rubric scores to grades. (I concede it because they have to have something to report to parents.) So on a 6-point rubric, 6=A, 5=B, 4=C and so on down. Using Marzano’s directions, converting 3 to 50% is failing, where it would (only) be a D (60%)in the points-to-letter-grade conversion. And I’m not sure that on a 4-point rubric “2″ is “half-way there.” Thanks for the opportunity to add to the dialogue.