In the readings and discussion so far this week I have found some interesting information about learning analytics. The basic concepts seem to be very familiar as they are something I use every day in trying to determine student success, predict failure, and what I may be able to do in my own lesson planning to influence either objective. In my readings, I found this interesting exchange between two of our reading’s authors, George Siemens and Mike Sharkey. Very generally, the discussion forum was focused on the variable definition of learning analytics and how whatever your chosen definition could be applied. I have included almost all of the discussion between Sharkey and Siemen, only editing out what I determined (if I am allowed to do so for this exercise) to be irrelevant at this point.
The reason I have used a large part of their discussion was that I wanted a record in the same place of the context of what Sharkey and Siemen were talking about. I found it so applicable to how myself and others in my field approach academics and how we define success or failure in the classroom. This is a struggle I deal with throughout each school year as I move with the ebb and flow of students’ accomplishments during their assignments and assessments.
The following discussion took place in Learning Analytics Google Group Discussion, in August of 2010 (the exchange below is conveyed verbatim and has not been edited in terms of grammar, syntax or emoticon use):
I wanted to add a dimension to the discussion, specifically around
defining success. In the descriptions of learning analytics we talk
about using data to “predict success”. I’ve struggled with that as I
pore over our databases. I’ve come to realize there are different
views/levels of success:
In its simplest form, academic success means getting a good/passing
grade. That works for a 15-week course since you can use the first
few weeks of data to predict the remainder of the course. However, I
work in an environment where courses are 5, 6, or 9 weeks long (we
teach courses one- or two-at-a-time in serial). That prevents me from
using data within a course to predict the outcome for that student.
There’s a second part to this argument about whether good grades =
success. That’s a discussion we need to have over a beer so I’ll pass
for now. 😉
Another academic metric is learning outcomes. Look at assessment
data and use mastery of outcomes as a gauge of success. If the
institution does a good job measuring learning outcomes, this is a
If we can’t measure success within a course, we might look at it
across the student’s program. From an academic standpoint, that means
GPA. That will just lead us to the same discussion about whether or
not grades are a good measure of success. From a practical
standpoint, success may mean “is the student still attending”. Are
they progressing through the program in a timely fashion? This isn’t
a good qualitative measure, but the argument can be made that if the
student is still attending, there’s a better chance they will succeed
in the program (especially when you compare that to students who have
stopped attending and have zero chance of graduating).
“Are they attending” is aligned with engagement. Is the student
actively engaged in the course? We can measure this by attendance
(did they show up) or by some alternate engagement metric (e.g. number
of actions in the course LMS). We can even get more detailed on the
progression metric and look at two dimensions:
– Persistence (when is the last time we heard from the student)
– Density (over the last x weeks, what percent of the time has the
student been engaged)
I have started to model metrics and I haven’t come to any solid
conclusions yet. It really boils down to who you are and how you
define success. Different parts of the institution will have
I hope to chat more with you at the conference in February.
Director of Academic Analytics
University of Phoenix
Hi Mike – thanks for contribution. Last year, I met someone from U of Phoenix (can’t remember how it was!) and they mentioned some of the current – and planned future – use of analytics at UoP. It was quite advanced from what I’ve seen at other institutions. Analytics require explication. Online courses, programs, and institutions are uniquely placed to be early trail-blazers of analytics.
Good question about success. Success has come up a few times already and, as you note, will be different in different situations and institutions. Or learners, for that matter. For some learners, simply passing a course could be defined as success. For others, only top grades would be seen as success.
Your points about persistence and density form part of the research that needs to be done around analytics. What learners characteristics contribute to success (however it is defined)? Which signals or deviation from those characteristics can we observe early enough through analytics to intervene to ensure success? Some great areas of research and exploration!
Mike (and others from the perspective of their institutions) – would you mind sharing a bit more about how you use analytics at UoP? What is working well? How are learners responding? What technology are you using for data collection and analytics? What role does visualization play?
In conclusion, sort of, when Siemens mentioned the types of analytics being discussed would work well for online course, etc., it reminded me of the evaluations we completed in the Course Design for Digital Environments course at the University of Edinburgh just last Fall. We had to consider various analytical frameworks to create operable and meaningful learning outcomes for the courses we designed. Of course, these outcomes were both dependent and determinant of the curriculum and activities we included in the course structure. It is very easy to see, from my perspective, how difficult it is to create and implement a solid strand of outcomes yet try to address as many of the different facets of learning and teaching that each teacher and student face each day.
Hello Myles, thanks for this review of your study of algorithms over the last week. And good to see you experimenting with another medium to convey your ideas, this time using Thinglink.
By coincidence, I’m writing this reply while in the background my son is watching his preferred dinosaur cartoon on Netflix. Even though we make use of the option for different profiles for each member of the household, I’m still amused by the some of the films that are recommended for me: the algorithm is sophisticated but not flawless. Unless of course someone else is using my profile to watch comedy-actions films…
Whilst accepting that it might be irritating for you, I was nevertheless amused by your mention that Futurelearn is now spamming you on account of your work around the micro-ethnography. An unintended consequence of the microethnography (combined with other influences) beyond the intention or control of those who designed the EDC course. It would be really interesting to see whether the subject of the advertised courses picked up on other of your online activity?
Within my own research something I’m interested in is how the experience of the marker might be affected by the algorithm. I’ve been thinking for instance how the experience of watching the same video assignment – and perhaps their interpretation – will alter depending on whether the student uploads their work to YouTube, Vimeo or MediaHopper? To apply this to my experience of your own artefact here, when I first looked at your Thinglink assignment my eye was temporarily drawn to the related images beneath: Chelsea Football Club (perhaps because earlier today I glanced at the sports news on the BBC website using this computer?); a crest for the city of Downey in California (because earlier this evening I had a Twitter exchange with Philip Downey from our EDC class?); suffragists and women pioneers (possibly because I had recently followed up your post about Ada Lovelace?). This would seem to be a really nice link into week 9 where we’re looking in particular at how algorithmic culture and learning analytics affect education: in this instance, my experience of viewing your work has been shaped by influences beyond what you intended as the author, and beyond my immediate control as the author. Fascinating stuff.
from Comments for Myles’s EDC blog http://ift.tt/2mKmOqj
I recenly sent an email to Professor Jared Leising, at Cascadia College, telling him I had participated in his Innovative Poetery of Cascadia MOOC, and that I had completed an ethnography on his course. I included a poem I wrote for the course as well. I have attached below a copy of my letter as well as the response I received from Professor Leising. It is not much a reply in length but he does express more than nominal appreciate for my participation in his group; enough to share my contat with his colleagues. I thought it quite nice and a solid capstone for my effort.
I am a student at the University of Edinburgh, and am completing requirements for the Master of Science in Digital Education program. I am also a Social Studies and Life Science teacher at a high school in Southern California.As part of the coursework for my Education and Digital Culture course, we had to find a MOOC, enroll, and complete an Ethnographical study of the course. I chose The Innovative Poetry of Cascadia, as I used to live in Oregon and the topic was, honestly, outside my field of expertise.I must say I thoroughly enjoyed working through the modules although I was not able to participate in the course during real-time. What I did do was read many of the poems and thoughts of the other participants as well as exploring information about the Cascadia Poetry Festivals.I have included with this email a link to the study I completed. It is not presented in a typical research format but on a platform I believe allowed me to more fully express the color and spirit of the course and themes. Lastly, I thought I would try my hand at poetry as if I was an active participant in the course. I hope you enjoy what I have included.Please feel free to repond if you like, and if so, I look forward to hearing from you. Thank you.Philip Downey
(From Professor Leising):
Thank you, Philip.
I shared this with my co-teachers. It was a lovely surprise!
Really thoughtful and interesting to see it presented in this way.