Seven Keys to Effective Feedback

This article is very helpful in defining the difference between true feedback, advice, and evaluation. The specific characteristics of feedback provide a means of evaluating your feedback to students prior to providing it. The could be adapted to a rubric for student peer assessment very easily, with fields of goal-oriented, timely, actionable, tangible, and user friendly. It also could easily be the basis for a PD exercise or brown bag.

http://www.ascd.org/publications/educational-leadership/sept12/vol70/num01/Seven-Keys-to-Effective-Feedback.aspx

Teaching Adolescents How to Evaluate the Quality of Online Information | Edutopia

This is an excellent and very accessible article on helping middle school (and I daresay any other level) students evaluate online resources for relevance, bias, accuracy, and reliability. If you are experienced in this area then you will find little in this article that is not familiar, but Julie Coiro's approach was useful to me in thinking about how to structure a lesson plan on this topic.



Teaching Adolescents How to Evaluate the Quality of Online Information | Edutopia

Reading a lot of iconoclastic writing about instructional design this week.

Today was sort of a watershed for me in terms of coming to grips with instructional models. I'm out actively looking for employment again, having been a "house Dad" for a year and a half. Earlier today. Many, if not most of the positions out there in distance education are citing the need for deep understanding of a model of one sort or another. ADDIE comes up frequently, as does ARCS and some variation on Merrill's component display theory (or at least some portion of it) in the advertisements. I'm familiar with, and have used these. My problem with them in the past is that they are fairly rigid and do not necessarily account for the organic nature of the teaching and learning processes, or with the individual idiosyncracies of the instructor or the content? So I have been pondering whether or not I am the only person finding these inadequate for the way I work.

So I am reading the materials in BLENDKIT2014 and I run across this statement:

"Brent Wilson (1995), a pioneer in e-learning, has been cautioning online course designers about the  downside of a systems approach for the past decade: An environment that is good for learning cannot be fully prepackaged and defined A more  flexible approach will open the doors to more possibilities based on learner goals and needs. However, as pointed out by Bates and Poole (2003), “a flexible approach requires a high level of skill to be effective”."

Yes! That's it. A model is simply a set of guidelines, and a set that may not be completely applicable in every situation. So what can we do to make the model more closely fit the organic nature of the educational process?

Canned feedback in online courses. What educational theory does that?

I just ran across an example of a means of feedback that I found a bit disturbing, but perhaps necessary in some cases.

In the past when I have needed to give feedback on assignments in classes I am am teaching, I have individually crafted responses. I will admit to having reused phrases when running into the same issues in essays over and over. I tend to treat these opportunities as "teachable moments", and so there is a requirement on my part to address the issues in a personal manner reacting to the individual and what I know of them instead of just the question or issue.

My experience is based upon class sizes that were never in excess of 50 individuals. That is my caveat.

But what do you do when you are dealing with the work of say, 150 individuals and you only have so many hours in the day?

Apparently a popular answer to this is to create a document with canned responses to assignments or problems. It is simple enough to copy and paste from the document into the feedback section of the assignment or the discussion and then customize with student name and any other information that is pertinent. In addition some instructors use macros to insert specific feedback into documents. These are interesting ideas, though I am hoping to never be in a situation where I need them. In an online course that is built on a basic constructivist model the primary role of the instructor is that of guide and/or curator and/or facilitator and/or concierge, which demands individually tailored interaction with the students.

Given that the ideal class size (the last time I looked) in an online class was something in the late 20s to low 30s, I wonder if these larger class sizes can meet the definition of constructivist, or are they constructionist, or something else?

Using Audio and video in the online course, some recent observations.

I am currently taking four MOOCs at the moment and finding some very interesting applications of video in these classes. To start off, video introductions and lectures seem to be quite common in the MOOCs that I have been associated with. That is not my mode when teaching, but that is partially due to my own preferences in mixing content. I'd prefer to use video by professional speakers and of ideas and concepts introduced by something other than a talking head. However I do understand how this might be valuable based upon your teaching and learning styles. It is particularly valuable when illustrating something highly visual, such as math or science.

After having spent several weeks with a variety of different approaches to video I have a few observations for what I consider to be best practices in the area. The suggestions below are equally applicable to regular online courses. Here is the list:

1. Chunk. A 40 minute video followed by a 15 question quiz is not a good way to avoid cognitive overload. I can understand this approach if the video cannot be edited for some reason, but at that, there should be some attempt at reinforcement prior to trying to assess. My approach would be to either break up the video into  minimum of half a dozen segments, each reinforcing  a concept or related objectives, followed by some sort of exercise to reinforce, and then followed by a short quiz for each segment. Give students an opportunity to interact with the concepts introduced and deep process them prior to the quiz. A single long video forces students to take notes in order to make certain that they record each point. While this is similar to what happens in the classroom, it is without benefit of being able to interact with the instructor. In effect it is the worst of a lecture format.

2. This is actually related to the above. If you are going to use a classroom lecture or a conference presentation for your course, make sure that it is miked properly. If it is difficult to hear, then that compounds the difficulty students will have in understanding and engaging in the materials. Most video editing programs allow you to modify the sound in a video. Modify the content to be easier on the ears.

3. If you must narrate a PowerPoint or series of PowerPoints as content, there has to be some creativity in the presentation. Two most common problems that I have seen so far have been using the same bank of a dozen background slides for numerous presentations and allowing computer screen items to pop up on the screen while narrating. The former gives the impression that little thought went into the presentation. By this I mean that even in  a series of short videos designed to create a 60 minute presentation you need some variety. Using the same dozen slides over and over again doe not build student engagement. In my case it breaks concentration while I wonder where I have seen the image before. I suspect that I am not alone here. The other issue is having something like an Adobe acrobat update notification pop up in the middle of the recording. It is distracting and does not lend itself to a professional looking content piece. Also it takes little time to correct, just pull your video into an editor and replace the popup with the slide for the duration.

4. Do use closed captioning. In a MOOC, many of your students will be taking your class outside of the normal work day. Your students will thank you if they can watch your videos with the sound turned down while sitting with a baby or significant other sleeping next to them.

5. Do use a transcript. I'm thinking of how much easier it would be to review for a quiz or test (see number 1 above) if there is a transcript to read rather than having to scan through the 40 minute video in order to locate a concept that you would like restated. At the very least, if possible, make the outline for your lecture downloadable from the course.

6. Make your lecture video available as audio in MP3 format. It is not difficult to use a video editor or Audacity to strip the audio portion from your video. Making this available for your online students gives another means for them to access the lectures. Again, many students will be non traditional, and allowing them to listen to your lecture while driving to work or working out at the gym is just another way to help make them successful. It is also a much smaller file size and can easily be downloaded even with a slow Internet connection.

7. Allow your video lectures to be downloaded. Again it gives students more opportunity to view the videos away from the computer. It is easy enough to load a lecture in the proper format onto a multimedia device or cell phone and the view it at leisure.

8. Do use video in your online class, but be sure that your audience can access it. Use common formats and always be aware of your audience. I realize that high speed internet seems ubiquitous to those living in highly populated areas, but if your audience is rural, or low income, there is a chance that they will not be able to view streamed video of high quality.

Building faculty participation in blended learning: Scaffolding the experience to create more interest.

Right now, as I go through BlendKit2014, I am thinking about how one entices faculty into using the LMS and then eventually teaching regularly in it. I actually started thinking about this a few weeks ago when a friend of mine, who is an instructional designer/other duties as assigned type, told me that her university required all faculty to upload their syllabi into the LMS for student access. It initially didn’t kick in as to what that might accomplish. It occurred to me that doing so requires that all faculty have at least the basic training to be able to accomplish that task.

It was sort of an AHA!! moment for me. The first step toward acquainting faculty to the LMS had been accomplished.

This week in the readings for BlendKit2014, I am faced with the same idea. To get faculty to start looking at blended learning as an opportunity, you can simply start with the syllabus. Let’s think about that for a moment. As a teacher, I have always put my course syllabi in a prominent place in the course online. The benefit there is that no student can claim to have not had access to the syllabus. I make it available in Word and PDF formats for download, and also an HTML version to read online. It is actual a bit freeing knowing that you will not get the excuse that the student did not see the assignment or deadline in the syllabus. If they have access to the LMS, they have access to the syllabi.

So making faculty put syllabi on the LMS, even if they use it for nothing else, provides a great perk. It also potentially opens the way for other time saving additions such as posting class materials and other content that insures that students can get to the assignments and lecture materials anywhere, again creating a situation where the responsibility for providing the content is moved from the instructor.

So now we have a very positive slippery slope. As instructors rely more and more on the LMS, they may be more and more tempted to see how it can not only simplify their teaching but also provide different means of teaching. At that point we begin to leave the realm of web-based course supplement into the realm of blended learning.

Thinking about blended learning

So I have started reading up on blended learning as an alternative to both online instruction and completely face to face. I have done a great deal of both of these modes of delivery, but I have never really considered (above and beyond having to incorporate online resources into a F2F class, or having an in-person or synchronous web conference in an online class) quite what the differences might be in implementing something intended to be used equally and of equal importance in both . I guess you could say that it never occurred to me to put the chocolate AND the peanut butter together.

 Right now I am working through what pedagogical advantages there are to connecting these two modes. The first I can take from all of the experience that I have in online teaching and learning.

Blended creates:
1. A sense of place. This is so hard to accomplish in an online course.
2. Makes instructor and other students "Real". Suddenly there are real humans with real mannerisms involved.
3. Group work that doesn't suck. Yes, there are real people working on the assignment, and it is so much more difficult to not participate when you can't simply walk away from the computer.
4. Community can be built in a number of different ways. If you have ever been in a strictly online course, you have probably run into someone who you would like to meet in person. Here you can.
5. Allows the instructor to do what they do best in person. Let's face it, some of us are dynamite lecturers in a way that is hard to duplicate on video. Alternately, some of us are better online and should probably stay there most of the time and flip the class during F2F sessions.
6. the Technologies can be a major or minor consideration depending on how they support class objectives.
7. You have more types of tools to build the course with.
8. Students can use resources for self-study that just aren't practical otherwise.
9. It is easier to change content for the online component in reaction to student interests and  understanding.

BlendKit 2014 starts next week!

Free MOOC for teachers starting next week. It is offered in part through EDUCAUSE and University of Central Florida. From the research that I have been reading lately, blended learning is the fastest growing segment of distance education. After all it allows you to move some of what you do from the classroom to an online mode, saving time for instruction in the classroom among other things. Note that there is also an optional certification that might be useful for proof of professional development/growth. Have a look!

BlendKit 2014 - Becoming a Blended Learning Designer.

So how do we assess the casual MOOC student?

 This is actually a continuation of the post below.

I have continued playing around with the idea of student participation in MOOCs and have come up with some thoughts about the nature of student involvement and motivation. I am wondering, now that I have been in three different MOOCs, if we might be measuring success using the wrong criteria. In general, success is measured by retention and grades. From my limited experience so far it seems to me that many people do not necessarily come into a MOOC with the intention of finishing. Instead they are present for the solutions to specific questions. If they are only there for a short period, but get from the materials and the discussions what they joined the course for, then how do we measure that success? If they complete then that is reflected in the conventional sense, but to not complete does not necessarily mean that the individual’s objectives were not met. Perhaps the measurements are too linear? Or perhaps they depend too much on conventional academic “trip wires”. So how do we build in a means of measuring the success of a course that does not necessarily have to be completed in order to meet the learner’s needs?

I think the most obvious would be a survey of some type, but that too is very conventional and since there is no requirement, because of the nature of the free MOOC, for a student to respond to a survey, then there is really no guaranteed means of returning useful data for evaluation of course success. Another thought that was rattling around in my skull re this problem was some sort of embedded assessment. Let’s say something similar to what is in this course, where there is discussion in support of a set of concepts that culminates in a product of some sort. One could put some kind of assessment at the end or perhaps embedded in the materials itself to gauge understanding and satisfaction, Likert scale maybe? But that really doesn’t do it either. The student does not have to respond to that either.

Then it occurred to me that one of the best methods might be in the discussions themselves.  That is the permanent persistent record. After all it is just one large database of information. If one could export all of the database fields related to the discussions, sort these by the user, date, and initial post vs. response, and then run some sort of search for key terms within it, then that could indicate from within the discussions that there was a question/solution within that discussion. That might be one way to handle evaluating MOOCs for success. You would not be able to do that with any of the major private MLS providers because the database schema is proprietary, however it could be accomplished if the course were in Moodle, Claroline, or Sakia, (and probably others) which use an open source database as a backend.

The key could be in designing a series of searches for key words or phrases that  demonstrate a question and an answer or series of answers. I don't know just how fuzzy these searches would have to be, but probably the more potential "action words" the better. If the search were run against both post and response then a pattern might emerge that indicates that learning has taken place. If the response that most closely answers the question of the original poster is the last post that the OP makes, then there is a probability that the answer has been found. Likewise, if we could search the posts that the OP accesses, we might be able to find, based upon the last few that were viewed, that the OP has found what they came into the MOOC for.

 I am thinking that the casual MOOC student has a great deal to draw upon from the community that is established by those who are in the course for the long term. These people, as I posited in an earlier post, become the community of practice/learning community, and so act as a pre-fab resource for the casual participant. They become a kind of corporate memory that the casual user can use to answer those specific questions they are participating for, complete with background information provided by the preexisting discussion posts. Who the casual user responds to and in  what context might also be helpful in developing a sense of what defines success for them. That again might be something that could be mined from the database of discussions. 

Thought on MOOCs, retention, value, and community (of practice?)

A friend of mine posed a question that I have been reflecting on. I am in the process of participating in no less than four MOOCs, and have noticed some patterns.I've also been reading about poor retention rates for MOOCs and also the nature of community in MOOCs. This has led me to the following questions and ideas about the dynamics of communities and the nature of learners in these environments.

Several people enrolling in MOOCS that I am participating in seem to check in for a short period of time and then leave, not completing the entire course. That seems natural enough to me given that most MOOCS, at least the ones that I have been involved in so far, have no tuition, and no credit hours associated with them. That means no investment, and so no requirement to complete. A few in the introduction discussions state flat out that they are in the course for the parts that interest them specifically..

So when I was asked what a MOOC that was designed for the short term learner would look like, I suspect it would look much the same as the MOOCs that I am in now, where the goal, at least ideally is to have students complete the entire course. My reasoning for this is that ultimately, unless we are talking about modules for e-learning for some sort of incremental professional development, the goal of a course is to build ideas and concepts across a series of lessons to a consummate whole of some sort. Individual modules don't do that. There are always students in any fairly well designed course who will stay for the duration in order to get the whole picture, obtain the certificate, or simply learn all that they can about the subject matter. So maybe the MOOC has a dual purpose where some view it as a complete course while others view it as a shorter term educational opportunity.

The benefit to those that stay with the course only long enough to gain information on some subset of the content is that they do not have to sit through an entire course in order to get it. It is essentially just-in-time education to them. But what is fascinating me is the role of those students who complete the whole course.

If we assume that one of the building blocks of a successful online course is the establishment of a learning community, then those who stay with the course the entire length and participate in the discussions in an engaged and deep sense develop into that learning community. They also become a sort of corporate knowledge for the course. If that is the case, then those who are there only for a short period have access not only to the content that is specific to their needs and goals, but also to a community that has developed around these materials. So for the length of time that students participate in the MOOC they are also gaining the benefit of a community that reflects more depth and variety of understanding than could be had if they were only accessing the content itself.

So in the end, the long term MOOC participants add value for those only attending for specific content and ideas. The learning community becomes a source of instant expertise and support for those who are only there a short period of time.

I'm finding this a very interesting concept. So far I have seen a few people enter the MOOCs with what appears to be a single question that they would like to have answered. Maybe that is part of why MOOCs succeed. There is depth enough for those seeking it and at the same time a community that can address the needs of those who are there to simply mine the course for specific answers.

It could be that current means of measuring retention in a MOOC is not reflective of its success. If we only measure those who complete the entire course we may be missing some different kinds of learning that the MOOC is successful providing. Also, perhaps creating a MOOC that gives some sort of incremental credit based upon time in the course might make a better measure of the relative success of the course. Interesting to ponder.

Moocs

OK, I admit it. I have been somewhat against the idea of the MOOC for serious academic work for some time. Part of my trepidation has stemmed from the lack of a structured community that can be created using a traditional course shell and a great deal of instructor/student interaction. I have always felt that it is the facilitator that makes the course, acting as that "guide to the side" who helps assist students in their journey to understanding without overtly forcing content down their collective throats (now there is a picture).  Students should have the ability to build relationships to better create community and so a higher level of synthesis ala classical constructivism. This model has been my mantra for many years.

Since I have time on my hands, I have decided to dive into MOOCs and see what they have to offer from the inside. I have done this in the past with online courses when I have wanted to see the methods (for better or worse) that others use in facilitating and designing courses. I have tried one MOOC, shortly after Stephen Downes started advocating them a few years back. It was a less than stellar experience. At the time I was not impressed, as it resembled to me the online equivalent to a mosh pit. Little organization and even less self restraint in discussion.

I am currently participating in two upcoming MOOCs. The first is through Swinburne University on a type of rapid online course development known as Carpe Diem. The second will be yet another distance education certificate, this time through Coursera. I'm already detecting some real differences between what I saw in the past and the current MOOC. The Swinburne team has had the foresight to start a facebook page on the course, which I think is acting to help acclimatize students to the course prior to actually beginning the Blackboard based exercises. This is one of those AHA! moments for me, in that using a familiar widely used social platform can be a gateway to helping people transition to a system that they may not otherwise have used in the past.

More to come...