In the readings and discussion so far this week I have found some interesting information about learning analytics.  The basic concepts seem to be very familiar as they are something I use every day in trying to determine student success, predict failure, and what I may be able to do in my own lesson planning to influence either objective.  In my readings, I found this interesting exchange between two of our reading’s authors, George Siemens and Mike Sharkey.  Very generally, the discussion forum was focused on the variable definition of learning analytics and how whatever your chosen definition could be applied.  I have included almost all of the discussion between Sharkey and Siemen, only editing out what I determined (if I am allowed to do so for this exercise) to be irrelevant at this point.

The reason I have used a large part of their discussion was that I wanted a record in the same place of the context of what Sharkey and Siemen were talking about.  I found it so applicable to how myself and others in my field approach academics and how we define success or failure in the classroom.  This is a struggle I deal with throughout each school year as I move with the ebb and flow of students’ accomplishments during their assignments and assessments.

The following discussion took place in Learning Analytics Google Group Discussion, in August of 2010 (the exchange below is conveyed verbatim and has not been edited in terms of grammar, syntax or emoticon use):

Mike Sharkey:

I wanted to add a dimension to the discussion, specifically around 
defining success.  In the descriptions of learning analytics we talk 
about using data to “predict success”.  I’ve struggled with that as I 
pore over our databases.  I’ve come to realize there are different 
views/levels of success:

Academic: 
In its simplest form, academic success means getting a good/passing 
grade.  That works for a 15-week course since you can use the first 
few weeks of data to predict the remainder of the course.  However, I 
work in an environment where courses are 5, 6, or 9 weeks long (we 
teach courses one- or two-at-a-time in serial).  That prevents me from 
using data within a course to predict the outcome for that student. 
There’s a second part to this argument about whether good grades = 
success.  That’s a discussion we need to have over a beer so I’ll pass 
for now. 😉 

Another academic metric is learning outcomes.  Look at assessment 
data and use mastery of outcomes as a gauge of success.  If the 
institution does a good job measuring learning outcomes, this is a 
possibility. 

Progression: 
If we can’t measure success within a course, we might look at it 
across the student’s program.  From an academic standpoint, that means 
GPA.  That will just lead us to the same discussion about whether or 
not grades are a good measure of success.  From a practical 
standpoint, success may mean “is the student still attending”.  Are 
they progressing through the program in a timely fashion?  This isn’t 
a good qualitative measure, but the argument can be made that if the 
student is still attending, there’s a better chance they will succeed 
in the program (especially when you compare that to students who have 
stopped attending and have zero chance of graduating). 

“Are they attending” is aligned with engagement.  Is the student 
actively engaged in the course?  We can measure this by attendance 
(did they show up) or by some alternate engagement metric (e.g. number 
of actions in the course LMS).  We can even get more detailed on the 
progression metric and look at two dimensions: 
– Persistence (when is the last time we heard from the student) 
– Density (over the last x weeks, what percent of the time has the 
student been engaged) 

I have started to model metrics and I haven’t come to any solid 
conclusions yet.  It really boils down to who you are and how you 
define success.  Different parts of the institution will have 
different definitions. 
I hope to chat more with you at the conference in February. 

Mike 
Director of Academic Analytics 
University of Phoenix 

 George Siemens:

Hi Mike – thanks for contribution. Last year, I met someone from U of Phoenix (can’t remember how it was!) and they mentioned some of the current – and planned future – use of analytics at UoP. It was quite advanced from what I’ve seen at other institutions. Analytics require explication. Online courses, programs, and institutions are uniquely placed to be early trail-blazers of analytics.

Good question about success. Success has come up a few times already and, as you note, will be different in different situations and institutions. Or learners, for that matter. For some learners, simply passing a course could be defined as success. For others, only top grades would be seen as success. 

Your points about persistence and density form part of the research that needs to be done around analytics. What learners characteristics contribute to success (however it is defined)? Which signals or deviation from those characteristics can we observe early enough through analytics to intervene to ensure success? Some great areas of research and exploration!

Mike (and others from the perspective of their institutions) – would you mind sharing a bit more about how you use analytics at UoP? What is working well? How are learners responding? What technology are you using for data collection and analytics? What role does visualization play?

George

In conclusion, sort of, when Siemens mentioned the types of analytics being discussed would work well for online course, etc., it reminded me of the evaluations we completed in the Course Design for Digital Environments course at the University of Edinburgh just last Fall.  We had to consider various analytical frameworks to create operable and meaningful learning outcomes for the courses we designed.  Of course, these outcomes were both dependent and determinant of the curriculum and activities we included in the course structure.  It is very easy to see, from my perspective, how difficult it is to create and implement a solid strand of outcomes yet try to address as many of the different facets of learning and teaching that each teacher and student face each day.

#mscedc

I recenly sent an email to Professor Jared Leising, at Cascadia College, telling him I had participated in his Innovative Poetery of Cascadia MOOC, and that I had completed an ethnography on his course.  I included a poem I wrote for the course as well.  I have attached below a copy of my letter as well as the response I received from Professor Leising.  It is not much a reply in length but he does express more than nominal appreciate for my participation in his group; enough to share my contat with his colleagues.  I thought it quite nice and a solid capstone for my effort.

Professor Leising:

I am a student at the University of Edinburgh, and am completing requirements for the Master of Science in Digital Education program.  I am also a Social Studies and Life Science teacher at a high school in Southern California.

As part of the coursework for my Education and Digital Culture course, we had to find a MOOC, enroll, and complete an Ethnographical study of the course.  I chose The Innovative Poetry of Cascadia, as I used to live in Oregon and the topic was, honestly, outside my field of expertise.

​I must say I thoroughly enjoyed working through the modules although I was not able to participate in the course during real-time.  What I did do was read many of the poems and thoughts of the other participants as well as exploring information about the Cascadia Poetry Festivals.

I have included with this email a link to the study I completed.  It is not presented in a typical research format but on a platform I believe allowed me to more fully express the color and spirit of the course and themes.  Lastly, I thought I would try my hand at poetry as if I was an active participant in the course.  I hope you enjoy what I have included.

Please feel free to repond if you like, and if so, I look forward to hearing from you.  Thank you.

​http://bit.ly/2nb1l7i

Philip Downey

(From Professor Leising):

Thank you, Philip.

I shared this with my co-teachers.  It was a lovely surprise!

Really thoughtful and interesting to see it presented in this way.

Jared

#mscedc