What the Researchers Got Wrong About Their ‘Sesame Street’ Education Study

Last week, Melissa Kearney of the University of Maryland & Phil Levine of Wellesley College received a great deal of media attention for their in-process paper Early Childhood Education by MOOC:  Lessons from Sesame Street.  Asserting that research on Sesame Street & educational efficacy is lacking and has failed to engage beyond the immediate or short-term results, Kearney & Levine designed an apparatus in an attempt to find a correlation between exposure to Sesame Street and longitudinal outcomes such as high school graduation or post-school labor gains.  While their instrument did show statistically significant outcomes in the immediate and short-term for those with better access to Sesame Street, the instrument failed to note any significance beyond (the researchers note this as inconclusive, though the only inconclusive aspect is whether the failure was on the part of the instrument or if the findings are in fact insignificant).

What does this have to do with MOOCs?  Not a whole lot as per the research.  But the invocation of MOOCs is indicative of an ahistoricism that permeates this work-under-review. Saying Sesame Street is ostensibly the first MOOC shows a fundamental flaw in Kearney & Levine’s historical literature review on the subject, a flaw Audrey Watters notes and critiques in an excellent response to the paper and subsequent media furor.  There has been extensive research on Sesame Street across all sorts of disciplines in an effort to identify the show’s educational efficacy.  While Kearney & Levine cite Joan Ganz Cooney’s seminal writings on television and preschool, these documents are much more an externalization of thinking behind creating a specific educational program rather than a synthesis of the literature on broadcast transmission and education.  Cooney’s 1966 and 1967 papers heavily cite transmission education scholars such as Wilbur Schramm because doing so grounds her thoughts in a context and shows where the uniqueness of Sesame Street was in 1969:  a show designed for preschool-age children (3-5) with a budget that can support high-quality production values and cinematic elements out of the reach of the sizable number local educational broadcasters in American and beyond.  Kearney & Levine do not cite the history of broadcast education (radio, tv, correspondence or otherwise), which explains how they could assert that Sesame Street was the first MOOC but does not make such an assertion true.

Using MOOC in the title of their paper provided an avenue for hype and clickbait; many news outlets and organizations weighed in on the topic of the paper, sold by the Brookings Institute as equating Sesame Street to a MOOC and lessons to be gleamed from that lens.  However, with no factual or theoretical grounding of the term MOOC or the history of their subject, the paper-in-process’ effect once outside the Kearney & Levine sphere became akin to a game of telephone, the result being a conversation which questions the value of preschool in comparison to just having preschoolers watch Sesame Street (examples at Business Insider, Edsurge and SFGate).

Kearney & Levine have since written a response at The New Republic entitled What the Media Got Wrong About Our ‘Sesame Street’ Education Study, in which they wish to dispel the notion that we should abandon preschool and instead “shift the conversation toward the encouraging policy implications coming out of our work.”  Some of those encouraging policy implications include

  • TV can be a powerful tool to improve educational outcomes
  • TV/Electronic Media can be leveraged for real social good
  • We can think of Sesame Street as “Khan Pre-K”
  • TV (and I assume Electronic Media) has the potential to close educational performance gaps between rich and poor

The problem here is with Kearney & Levine’s research: the results of their data & instrument, and now what they are saying their instrument provided and how we might have a broader conversation in the wake of this work.  The main question Kearney & Levine asked (can we track the impact of Sesame Street beyond the immediate and short-term) was inconclusive, a term they use in place of saying their results were insignificant but that could be the fault of the research instrument.  The question Kearney & Levine answered (does Sesame Street have an impact on pre-K educational efficacy) was proven statistically significant, but this is not a new question; there is ample research proving this question.  There is nothing wrong with doing research that seeks to ensure the validity of prior research; this sort of research should happen more often.  But when that is the locus of the research, the call to inherency should address this from such a framework rather than developing a narrative around a lack of existing research and a need for longitudinal studies.  Of course, there is not nearly the same public relations energy around a research study that cemented the same results as the copious amount of existing research on the positive effect of Sesame Street.  There is a great deal of public relations energy around tying a new study to the premiere EduHype buzzword of our time, ahistoricism or not.

The belief in augmenting instruction/contents with educational media dates back nearly 100 years; in Don’t Whistle in School:  The History of America’s Public Schools, Ruth Tenzer Feldman notes:

John Tigert, a U.S. commissioner of education…spoke of technology in the classroom.  “Within the celluloid film lies the most powerful weapon for the attack on ignorance the world has ever seen.”  Speaking in 1922, Tigert was referring to educational film strips, by then in use in dozens of school districts around the country.

How we incorporate these broadcast materials for younger audiences is a complex question, one Wilbur Schramm was thinking about as early as 1962 in his book Television in the Lives of Our Children:

The relationship between a child and a picture tube is a complex one in which a number of elements are important.  It is always a kind of television content, sought by a kind of child, in a kind of situation.  Television usage, in other words, must be considered against message content, personality structure and the entire social and cultural complex which surrounds the event (emphasis original).

Before Sesame Street existed, the scholastic field of broadcast education understood the contextual nature of media contents:  the materials in and of themselves were nothing but opportunity, and without addressing localized environmental factors the opportunity would be wasted. To that end, the Children’s Television Workshop has dedicated substantial resources to providing opportunities to link their broadcasts to the work of schools as well as families.  So when Kearney & Levine say “Preschools and daycare centers should think about ways to effectively incorporate ‘Sesame Street’ into their day, or encourage children and their parents to watch it when they go home,” the only novel part of their advice is their belief that the advice is novel.

I do want to take a moment to note that I do not believe Kearney & Levine are charlatans.  It is clear in their paper that they do not believe that Sesame Street should replace preschool, and touch on this in the paper-in-progress, with a footnote that delivers their strongest statement on the idea of replacement:

Edward Zigler, Director of the Office of Child Development (which oversaw Head Start) in the Nixon Administration responded to the White House’s comparison of Sesame Street and Head Start by stating: “how long would a poor child have to watch Sesame Street to get his or her teeth filled? When nobody could answer, that was the end of the meeting.” (Zigler and Muenchow, 1994). The point was that Head Start did much more than just try to impact educational skills.

While Kearney & Levine see Sesame Street as supplement rather than replacement, it is ironic that their paper-in-progress’ addition to the conversation around educational contents was to renew a debate settled in the early 1970s Nixon administration.  Also, this quotation (and the paper in whole) never attempts to distinguish education as anything more than educational skills; rather, the paper conflates education with domain-specific cognitive development such as reading acumen.

But that this conversation has devolved is not surprising at all.  Starting with the research’s stated goal to see if there is a link between access to Sesame Street and “labor market outcomes,” we went to an inconclusive conclusion, then to the addition of MOOC in the title of a subject that has no connection whatsoever to MOOCs, to a PR push of the paper centered around MOOCs, and ended at the mass media cutting & pasting their “Will MOOCs replace higher education” boilerplate with “Will Sesame Street replace preschools?”  The media is not to blame; they drove attention to the piece and got eyeballs reading in the same way the Brookings Institute drove media eyeballs by focusing on MOOCs, in the same way the Maryland PR push focused on the immediate outcomes of Sesame Street rather than the paper’s intended look at longitudinal results, in the same way the paper’s title invokes the MOOC despite the paper having no relationship to the MOOC.

Rather, I would be surprised if this had not happened.  Kearney & Levine willfully engaged their research around an existing media trope of MOOCs and solutionism.  That their paper fails to engage the subject around its historical, political and theoretical backgrounds leads me to believe their first run of media engagement (see Brookings) was by design.  But the history of broadcast education, MOOCs and EdTech is vastly larger than hyper focused Sesame Street creation and history-as-technical-transmission.  Without being able to link their work to a foundational base, the media chose their dominant ahistorical base, and now when I search Sesame Street and MOOCs I have a screen full of conjecture, hype and gaps in the histories of each domain. That is the landscape because this is emblematic of the present history of EdTech as rhetoric:  why can’t transmission x replace superstructure y?  The difference today is that, rather than the record needle stuck in the groove or the CD player skipping and stalling, today it’s a website in a reload loop.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s