My micro netnography
I have completed the first course of the five making up the “Cameras, exposure and photography!” specialisation.
My focus is on one aspect, the peer review
Tags: #mscedc
March 01, 2017 at 06:19AM
Open in Evernote
Linked form evernote: My micro netnography
Eli, what a brilliant use of Spark: I really like how we ‘zoom in’ as we progress through your write up. As Helen has commented (https://twitter.com/lemurph/status/836823636761849858), comparing your RL experience with your xMOOC one is a great idea.
Your findings suggest that the peer review process in the xMOOC is superficial, offering a gesture towards community interaction and support without having the structures (such as a guiding teacher presence) to deliver meaningful and helpful outcomes.
This MOOC adoption of an offline practice (which you demonstrate works at a small scale) into the online delivery of a massive course, without consideration as to whether it is appropriate, reminded me of Tony Bates’ warning which is cited at the end of Baggaley’s article: ‘It was as if 45 years of work was for nothing. All the research and study I and many others had done on what makes for successful learning online were totally ignored, with truly disastrous consequences in terms of effective learning for the vast majority of participants who took MOOCs from the Ivy League universities.’
Thanks for sharing your experiences in such a clear and engaging way Eli.
Hey Helen,
yeah as I mentioned to Dirk, there was two streams of things going on, the MOOC (free version) which was pretty much as you’d expect support and community wise (except I didn’t get the sense of massive at all), and the paid for students who get all sorts of off MOOC support with facebook and flicker.
There is a definite feeling that this set of courses is important to the course instructors, but that they are restrained by the MOOC and so are trying to create the course they had intended to using off the record tools 🙂
I have to admire them for their commitment to creating good learning in the face of bland, mass market MOOCs.
The Tony bates comment is one I came across last semester and it made me grin fro ear to ear. It is in many senses true. Some educators as treating MOOCs as nothing more than some sort videos. They are not putting the same time and effort in ensuring they are good learning as they would for a standard classroom-based course. This was, in fact, one of the reasons I chose to enrol on the digital Education fun ride, hoping it would help me to help my colleagues in this situation. I also read a brilliant quote which I now can’t remember where, but one academic has referred to the use of small videos in MOOCs as being lectures treated as soap operas. I thought that was both funny and insightful.
As someone who works much better when there are small video lectures to boost reading etc, I do like this element of MOOC, but I also have to ask, given that we are now going down a path where we are saying that the traditional lecture isn’t actually the best way for students to learn, why aren’t there more people trying out alternative ideas for MOOCs? Is it just because educators don’t want to or can’t spare the time for something which is not considered an important part of their institution strategy?
I came across this today but haven’t read it yet, so sharing may be pointless after all, but I’m hoping to find some interesting thoughts in here: http://www.educationdive.com/news/5-professors-on-youtube-who-nailed-teaching-online/181504/
Eli
Beautiful work, Eli! Not is it beatifully visualized, your writing style is very good and comprehensive, and the length of your piece is in my eyes perfect without lacking depth. I as a reader enjoyed it!
Two minor suggestions:
(1) You have produced such great photos during your MOOC, why not use these as pics? They are great and it would be nice to see your work on the MOOC interlaced with your work on the MOOC.
(2) Maybe next time try to work with an academic reference or two. Not because I personally need it, but because tutors might be happy to see it.
Hey Dirk,
yeah there was soooo much more I could have gone into about how the MOOC has been created and it’s community, and I actually did. This is my second attempt at this but my wife pointed out that I had gone completely overboard for what the task actually was and it would have been far too much for such a small participation piece for the blog. Maybe save that for an assignment.
There is next to no community going on in the actual MOOC but there is a really strong community on facebook, which the course organisers created and manage. Almost like they knew there were going to be a limitation on the coursera platform (on a side note, the mooc has two separate communities, paid for and free…).
Yeah, I agree, I haven’t been doing much in the way of referencing of course readings on the blog, I have been treating our lifestream blog as a lifestream blog and having it purely as a record of my digital presence for this course. I’ve found this really difficult as I feel it goes against the ethos of being on an educational course so I’m a bit conflicted, I have however had a chat with Jeremy about it and I’m going to start having separate posts just for my thoughts on readings etc. That way I still feel like I’m being true to my lifestream ideals but also gives me a way of bringing my thoughts into my blog more.
Your work is beautifully presented and sparknote is now definitely on my things to try out list now. But I would second what Dirk says about adding academic references. Any of the blog content could count towards your final mark so you may as well use it as a chance to show your knowledge and understanding of the concepts in the course.
For example, there are different economic forces at work in the IRL lesson, the free MOOC and the paid MOOC. Can this be related back to the Lister reading “network, users and economics”?
Hi Eli,
The was a really interesting read and fascinating to compare with my own experience on an xMOOC.
If xMOOCs are to use peec-orrientated communications and process-focussed generation successfully do you feel that they could create the perfect balance of a MOOC?
Linzi
Sorry Linzi,
I’m not sure I understand your question
Eli
*peer-orientated
Stewart (2013) states that learners on xMOOCs are exposed to a ‘fledgling’ network. You highlight important issues regarding peer-review, where as I have experienced a very isolated xMOOC. After reading Stewart’s paper I considered peer-oriented communication (particularly feedback ) along with a strong course structure which provides in-depth content and resources as a successful MOOC. However, I have not experienced this so it would be premature to consider and I wondered if you thought it would be successful?
I’m guessing in a large scale environment peer feedback would need parameters and tutor surveillance?
Linzi
Thanks Linzi,
I think we have two things to consider when we are creating a plan for a MOOC: distribution and scale.
Open education and in particular the MOOC is often described as a disruptive force which questions some of the basic assumptions of education because it is seen (I think falsely) as open due to its availability online. There’s a great paper I came across “Open education and critical pedagogy” by Robert Farrow and he argues that much of this talk of disruption can be linked to the digitisation of learning materials, allowing for new collaborative and flexible models of learning and the ability to post digital learning materials online, anywhere, at a marginal cost but this in itself does not need a radical change to pedagogy. Teaching is still teaching and educational courses do not need to be created differently just because the medium is different. Video lectures are a great example of the good and bad in this. You can still have a lecture, as we do in almost all uni courses, the teacher is just no in the same room as the student. Peer review is still peer review but again, the teacher is just not in the same room.
Difficulties arise when we talk about scale, though, the Massive in MOOCs is a problem not because the class sizes can be in the tens of thousands, but because we have upscaled the number of students but not the number of teaching staff, so how does one or two teachers and a handful of tutors provide the same level of support (guidance and scaffolding) to these massive sized courses? More importantly, to keep with the insinuation of open, how do we do this, without increasing cost? I hate to say it, but even when universities are on board with the opportunity MOOCs provide in scaling up the delivery of their wares, they rarely want to do this using the same model for on-campus, they want to do it cheaply.
So in summary for what turned out to be a huge waffle, the internet has provided the opportunity to distribute educational material quickly and cheaply, but the opportunity of scaling up class sizes (which we all hate when we do this in schools), is the area that needs some thought and possibly a new pedagogy. Knox (2014)
Farrow, R., 2015. Open education and critical pedagogy. Learning, media and technology, pp.1–17. Available at: http://dx.doi.org/10.1080/17439884.2016.1113991.
Knox, J., 2014. Digital culture clash: “massive” education in the E-learning and Digital Cultures MOOC. Distance Education, 35(2), pp.164–177. Available at: http://dx.doi.org/10.1080/01587919.2014.917704.
Hi Eli
Your ethnography was really focused and tight and absolutely beautifully presented. My experience with peer review is that it works best when the students themselves help formulate the criteria when the assignment is set – that way they kind of internalise the requirements and helps them focus when creating their own work as well as when reviewing the work of their peers.
However, as I can demonstrate only too depressingly often, it is one thing to understand the criteria but another to actually follow it 🙁
Cathy
Cathy
Thanks Cathy,
I find that this is true of all assessment.
It is brilliant to involve the students in assessment and help them understand how it works, how they will be assessed, what a good example is and the reverse.
Peer assessment is a great tool for doing that. We use WebPA quite a lot and it works brilliantly, could do with a visual update though, still looks like it’s from the 90s.
Eli
Interesting focus on peer feedback here Eli – it seems to be a way of structuring-in a form of community, one that is developed around assessment. That seems to be quite a different kind of community from one defines around participant ‘roles’ (as in Kozinets, for example)?
In peer review, community interaction is structured, defined to a large extent by ‘teachers’, and also rather ‘pressured’, given that success or failure (even if of others) is at stake.
So, it was interesting to see you discuss the limitations with the MOOC peer feedback, providing a much more rigid kind of interaction.
‘there was no way to find out more about that student, their hopes for their learning or aspirations in photography’
The ‘hidden’ aspects are interesting here aren’t they: on the one hand perhaps allowing people to provide feedback ‘honestly’, but on the other, leaving little room for dialogue. I think you highlight this really well in relation to language issues, and the inability to follow up with assessment decisions.
‘It can also be seen that the tutor plays a vital role in guiding and leading in the review and no matter how carefully written the guidance materials, without the presence of a guide to interject where necessary, misunderstandings can go uncaught and risk devaluing the process for students.’
Indeed, and important to relate this to the underlying rationale behind peer feedback in MOOCs: to facilitate qualitative assessment at scale. However, as you conclude, this is not easily achieved!
Gosh, Eli, this was not only such a clever idea, but your micro-ethnography is so stylishly presented. It’s beautiful!
One of the points you made that I’m really interested in is the idea of taking into account a peer’s intentions and goals in the feedback offered, and how that might reframe the feedback offered. There’s little point, I suppose, in commending or castigating someone’s ability to take photos of seascapes if their absolute dream is to photograph faces – I know nothing, but imagine they’re different techniques. The scale of MOOCs would seem to rule out the possibilities of a more intimate community, but do you think there was any way that the MOOC organisers might have encouraged more discussion around hopes and dreams?
-Helen
A wonderfully creative and visually absorbing ethnography, Eli. I really like your style and the way your findings revel themselves as you move through it. Also, the fact that it combines much of the imagery around the courses too is a great inclusion. Im an amateur photographer myself so I probably appreciate the subject even more.
Its interesting to see that the submitted work review process was problematic. I would be interested to know how much this differs with subjects that are more scientific in their assessment and that have very defined outcomes. Did the course developers have any sort of disclaimers or guidance to courses users that opinions should always be respectful and helpful?
Hey Myles,
yes there was actually a piece of guidance before every review asking people to be courteous and respectful and also to remember that the feedback is important and to give explanations for your thoughts. I think however that guidance isn’t always great, especially with students who are not confident or perhaps over-confident, sometimes guidance needs a little extra support which is difficult in a MOOC when the design is that it is self directed study.
I think what made it seem worse was that there was no way to discuss feedback that as received so whatever a student did, good or bad, was the final act.
This actually became an issue on the forums after I submitted my netnography and another participant complained on the forum about how they ad received poor feedback which was not helpful and had been marked down incorrectly and there was no tutor intervening.
I think this in itself could be a much bigger study if we had time.
Eli, amazingly presented work and what an interesting area to focus on! You work has elicited lots of discussion and I find the ‘peer feedback’ you’ve received on this post very entertaining. 🙂
I think peer feedback serves a very good purpose both in online and face-to-face learning and it allows students to co-construct meaning together, but I wonder if it would ever be accepted in face-to-face classes to the extent it is being used in MOOCs. Would students who physically attended a course, even if it was free, be content with accepting final feedback from their peers? I don’t think so. I think, judging by the ethnography we’ve seen in MSCEDC, MOOCs are no longer as massive as they once were and if organisers are going to be continuing with the model and expecting people to engage in a meaningful way, the feedback methods will need to change.
the peer feedback thing seems to have really hit the mark with our classmates 🙂
Hi Eli
I loved your beautifully stylised ethnography. It would have been a bonus to see some of your photographs.
The peer review debate that it kicked off is really interesting. This is the third module in Mscde that has involved peer review for me and is in stark contrast to my MOOC experiences. I am wondering if it is due to small course size, formal qualification, masters level, longer duration or a combination of all or some of these elements? It is a skill in its own right and sometimes I find it difficult to get meaning across correctly.
Hey Clare,
thanks 🙂
I did consider using some of my photos but I haven’t done anything other than random yet so I didn’t have enough that were of the same theme to hold together the whole piece. However, I did a post previously that had some of my photos if you want to have a nosey and there’s a link there to my Instagram account as well 🙂
http://edc17.education.ed.ac.uk/eappleby-donald/2017/02/20/thought-i-should-share-some-mooc-creativity-mscedc/
The peer review thing is really interesting isn’t it? The MOOC was the first time I’ve been on the end of peer review and I found it quite hard as I wasn’t always happy that the peers opinions were accurate or fair, but unlike with a course like this, in the MOOC the peer review was final, there was no taking it to the tutor to discuss. We do a lot of peer review stuff at work and I’m always very positive about it, was really interesting to see things from the other perspective and to experience just how hard it is to give proper feedback and responses. I have a new found respect for all the guys marking assignments.
Someone else mentioned about whether or not we’d be happy with peer review if it was how our final grades were decided and I think that is a big question. Would we be happy if our grades were decided by our peers (we actually do this at work) or do we feel that only the official course tutor should be deciding grades? I know just from my interaction son the various MSCDE courses so far, I have been a bit harsher in my opinions of classmates work than the actual markers have been. It’s a scary thought that anyone would allow me to have an opinion on things 🙂
Eli
Thanks Eli
Your photos are fantastic (hard to choose between the tea and the bikes as my favourite). It really is a bit unexpected to be learning about one thing by learning about another!
Thinking about your comment “whether or not we’d be happy with peer review if it was how our final grades were decided and I think that is a big question” – in #deuloe we did anonymous peer review (the reviewer knew the writer, but the writer didn’t know who reviewed) with tight guidelines. However, this feedback could only IMPROVE the final mark given by the tutors. Here I think the anonymity and guidance were key. Also, we did three each so your peer review did not sit in isolation – it seemed a good balance and a good learning experience. Really not sure how I would feel if the final mark was significantly influenced by peer grading.
Clare
The bikes are my current favourite, I even have a massive print out on my wall at work 🙂
Yeah I enjoyed watching you guys going through the peer review thing but I’m not sure how I would have managed. It’s a lot of responsibility.
Eli