The Power(ful Perception) of Learning Analytics – Excerpts from MOOC Research

Note: I will use this space over the next month to share excerpts from my dissertation The Evolution & Impact of the Massive Open Online Course. The research was a Delphi study bringing together 20 MOOC experts to discuss the MOOC in educational, political, and sociocultural terms (slides from the oral presentation can be seen here). Upon library clearance, the entire document will be available through a Creative Commons license. The following is from Chapter 4, the results of the research study. This excerpt tackles the only prompt of 12 to gain consensus in Round 1 of the study, an agreement with the following:  The data we gather from students utilizing MOOCs will help us solve student struggles in learning through redesigning the learning system and content modules.

Round 1: Consensus. Only one prompt in Round 1 reached the consensus level of 75%, Prompt #3 data, experts agreeing with the contention that back-end data gathered from MOOCs would help solve learning struggles.

The positive view of data from a consensus majority of the expert panel potentially comes due to the panel’s make-up. Although panelists were chosen from five distinct disciplines, it was the congruence to MOOCs and educational technology that forecast expertise within the phenomenon. Panelists were bullish on back-end data in part because panelists were bullish on the overall confluence of education and technology.  Participant E8 stated, “Computer based learning generally, and the whole innovation mindset as brought to teaching and learning, will transform the possibilities for learning research and teaching practice.” Participant E12 added, “The analytics provided by MOOCs (and other online learning) can provide a window into actual student performance – missing in most F2F and online learning today.”

Datapalooza = Yelp.gov?

This is a response to President Obama’s recent initiative regarding higher education costs, value and affordability, but I will start by talking about Yelp.

I don’t like Yelp.  I’m not as disestablishmentarian as Jaron Lanier (defining Wisdom of the Crowd as Mob Mentality), but I understand that multiple variables color the aggregation system on a company’s reviews, expertise perhaps one variable, perhaps not.  This concerns me.  The LA Times discussed Yelp’s business backlash yesterday.  Yelp continues to deny that failure to advertise with their website results in the site’s algorithm casting less favor on a business, but business owners are convinced that Yelp runs like the mafia, and see advertising as a necessary evil to cull favor with the site.

The most obvious solution to this standoff would be for Yelp to publish their algorithm, but that will not happen.  I imagine Yelp would say publishing such sensitive data would destroy their business, equating their success to the algorithm and not their developed branding and affiliation.  What publishing the algorithm would certainly do is show the numerous variables that dictate a company’s ranking and placement within a community of businesses, and those variables could spark discussions about the efficacy of crowdsourced commentary (for example, prolific users’ comments hold more weight than irregular users, meaning certain quantity is valued over potential quality).  

MOOCs: Where’s the Lit Review?

One of the purposes of research is to establish a foundation of prior knowledge for future experiments to engage and extrapolate before proposing a new design that will further the field.  This is important; without an understanding of what came before, research runs the risk of reinventing the wheel, or even (worse yet) coming up with something more rudimentary than the wheel.

In my days of teaching creative writing, it used to be quite the stressor to get smart, motivated teenagers to take notes of their plots and characters.  These were students used to doing everything right and being able to beat the system just with what was stored in their heads.  I explained that creative writing was not about beating a system, and the more complex a story and a group of characters became, the more important it was to create a system where you could record those complexities so you could return to it as the story developed.  Some listened right away and got to work.  Some needed trial and error before coming to me so we could devise strategies.  Some never listened and became increasingly frustrated.  In the end, it was more likely for someone from the first or second group to have a coherent, rich story than someone from the third group.

I think about this as I read more literature on the history of MOOCs as described by the MOOC creators.  

MOOCs, Public Policy and Historical Ignorance

The due course of education in America is linked to public policy. This has existed on the state and federal level for over 100 years (well over); however, it is only in the last 30+ that there has been a federal department dedicated to education. Too often it seems that people within their own disciplines ignore societal factors and stressors when debating the merits of their discipline. This happens in education, an enterprise subsidized by governmental monies (to a shrinking degree, however). We cannot debate movements in education without looking at politics.

At the same time, politicians and policy hawks need a firm understanding of education if they are going to pitch for a model or debate a movement. Rhetoric and hyperbole only go so far, and ignorance of the theories, pedagogies and history of learning can cause great harm.

My Twitter network shot out an education article published by the Cato Institute, a libertarian think tank in Washington, D.C. Written by Alex Tabbarok, it’s title, Why Online Education Works, foreshadows a lack of historical perspective of both online and education (#cfhe12). The artifact is an important representation of existing thought in the political world, and with that I will dive in:

Reviewing Christensen’s Disruptive Technologies (Harvard Business Review, 1995) in MOOC Terms

One of the common citations in xMOOC artifacts and discussion is the idea of xMOOC as a disruptive technology.  The concept, developed by Harvard Business professor Clayton Christensen, is tossed into discussion as if it’s vital reading I should already know…none of the authors do more than give a cursory definition to the concept in abstract fashion rather than concrete, and in all of the articles I have read, I don’t see consensus on the definition.  This makes me think several possibilities:  1) this is a concept so integral to this field that I should know all about it and am an idiot for not having a foundational knowledge, or 2) this is a concept not fully understood but thrown out there in a way that sounds erudite but lacks foundation.  I think it’s a mix of both.  As a learner struggling to grasp a topic (my background is in both media and social sciences, not business), the best way is to personally dive in rather than rely on the previews of others.  At the same time, it took those previews to get here, so perhaps this review can help others start to nail out a more complete definition on the topic.  

MOOCs – Sliced Bread, or the Ron Popeil Bread Slicer?

There’s a lot of hype about MOOCs (and when I put hype and MOOC together, I mean xMOOC), and with the hype comes a resistance from ed tech folks.  The arguments go something like this:  hype machine says MOOCs are the next big thing and the best thing to happen in ages, and resistance says MOOCs aren’t great, aren’t new, and aren’t making things better.  A prime example comes from some hype dished up by the MIT Technology Review entitled The Most Important Education Technology in 200 Years, countered by D’Arcy Norman’s terse reply whose tag line involves fertilizer.  What we forget when we enter a point-counterpoint frame of mind is that both points of view come from ideologies and histories that result in the digital artifacts I have linked to.  Studying those artifacts to find the encampment inferences and foundations can help us see the positives and negatives of both sides rather than following one full throttle.

Ethiopian Tablet Hoopla – Take a Deep Breath

The OLPC initiative in Ethiopia is getting some traction in ed tech circles, and I came across a blog response by Ben Grey, a tech CIO for a K-12 district, professor, teacher and optimist in the field.  He was thrilled about the potential coming from this initial look at data.  I find his passion inspiring, but was weary of the article’s elements Grey was ignoring while extolling other virtues.  I responded to his blog via comment, and will mirror that below:

I appreciate your enthusiasm in this field…such energy is needed in the edu world. This article came across my computer today as well, and my reaction was slightly different.

I saw the Ethiopian children’s unintended use of the devices in the same light as you, but I don’t see anything new here. People in your comment stream have already talked about “more them, less us” in the student/teacher dynamic, and any constructivist worth their salt would look at these results and say Duh. Kids want to learn, want to explore, and want to create, and they want to take ownership. Jailbreaking the camera and the desktop design is part of a learning process, something folks like Piaget and Papert would have expected in any supportive environment.

MOOC Pedagogy – Waiting for Big Data?

I’m in a constant search to find documented discussion of the pedagogy behind the various MOOC iterations. cMOOCs are not so hard; they are borne of distance education and most adhere to connectivism, which is a pedagogical approach in my opinion, a learning theory in the opinions of others. xMOOCs are much more difficult; due to their newness there is little scholarly data on the model, and the creators have not shown as much interest in describing existing pedagogy as in stamping this model as the future of higher education (#cfhe12). Nicholas Carr notes that part of the reason for gathering learning analytics from the courses is to improve pedagogy: (xMOOCs) hope to build large behavioral data bases that can then be mined for pedagogical insights.

More from Week 1 of #CFHE12 – The Invisible College, Eroding Teachers, etc.

Continuing through my lit review log (which I plan to follow later in the week with some attempt at synthesis) of the first week of Current/Future State of Higher Education cMOOC:

Three Reasons Why India Will Lead EdTech in the 21st Century – Dr. Joshua Kim flags India as the prime sector for educational technology growth in the 21st Century, due to demand, culture and mobility (of devices and use).  Reading his thoughts, I wonder what his definition of educational technology is.  Is it apps?  Software?  Data collection and learning analytics?