synthesis of Dept of Ed study

The Dept of Ed report is a meta- analysis

This type of study s is defined as a statistical technique for amalgamating, summarizing, and reviewing previous quantitative research. A meta-analysis provides a systematic overview of quantitative research which has examined a particular question. This type of study combines all the research on one topic into one large study with many participants. The danger is that in amalgamating a large set of different studies the construct definitions can become imprecise and the results difficult to interpret meaningfully.

“A meta-analysis is a two-stage process. The first stage is the extraction of data from each individual study and the calculation of a result for each individual study (the “point estimate” or “summary statistic”) with an estimate of the chance variation we would expect with studies like that (the “confidence interval”).

The second stage involves deciding whether it is appropriate to calculate a pooled average result across studies and, if so, calculating and presenting such a result. Part of this process is to give greater weight to the results from studies that give us more information, because these are likely to be closer to the truth.” (Singapore Med Journal, 2005, Vol 46(6) pp 270, Evidence Based Medicine and Healthcare, S Green)


The Department of Education study included the following:

1. A search of journals that returned 1132 studies
2. Resulted in a return of 176 online learning research studies that used an experimental quasi experimental design that objectively measured student learning outcomes.
3. Of these 99 had a face-to-face contrast component 9 of these involved K-12.
4. Of the 99-- 46 proved to have sufficient data for the study.


The 46 studies were:
1. Published in or after 2004.
2. Included 7 contrasts from five studies that included K-12 learners
3. The rest were evenly split between community college and graduates students or adults receiving professional training.
4. Demonstrated weakness in the studies were small sample sizes, failure to report retention rates, sample size is positive for undergraduate and older learners but not K-12.

The study sought to answer the following:


1. How does the effectiveness of online learning compare with that of face-to-face instruction?
2. Does supplementing face-to-face instruction with online instruction enhance learning?
3. What practices are associated with more effective online learning?
4. What conditions influence the effectiveness of online learning?


What is hidden in the study?

1. The study does not present that online is superior as a medium of delivery

2. The study does represent that the time spent on task, curriculum and pedagogy of online learning maybe critical variables.

3. The study does support the redesign of instruction to incorporate additional learning opportunities online.

4. The study found there was no significant effect from using different media in presenting instruction.


The study identified the following 10 practices, which are intervention implementation :


1. Pedagogy/learning experience
2. Computer mediated communication with instructor
3. Treatment duration
4. Media features
5. Time on task
6. One-way video or audio
7. Computer based instruction elements
8. Opportunity for face to face with the instructor
9. Opportunity for practice
10. Feedback provided

What conditions, which are status variables that are fairly impervious to external factors, that influence the effectiveness of online learning?

1. The year of the study
2. Learning type:
a. K-12 students
b. Undergraduate
c. Graduate student /other
3. Subject matter:
a. Medical/health care
b. Other



Further observations:


Blended Compared with Pure Online Learning

The studies reviewed did not provide a basis for determining the benefits of online versus blended learning modalities or conditions.

Media Elements


The fact that the majority of studies found no significant difference across media types is consistent with the theoretical position that the medium is simply a carrier of content and is unlikely to affect learning per se (Clark 1983, 1994).

....that the way in which a medium is used is more important than merely having access to it ...


“In summary, many researchers have hypothesized that the addition of images, graphics, audio, video or some combination would enhance student learning and positively affect achievement. However, the majority of studies to date have found that these media features do not affect learning outcomes significantly.”

Learning Experience Type

..the power of collaborative learning ...


…offered by the author is that students working in groups may scaffold each other’s work, hence reducing the benefit derived from externally provided questions…

Computer-Based Instruction

The study found the platform used to deliver quizzes and course content did have some impact on student success ...


The results of three studies exploring the effects of including different types of online simulations were modestly positive. Two of the studies indicated a positive effect from including an online simulation; however, one study found no significant difference ...this does have some implications for simulated learning environments in virtual worlds.



...online learning literature has also explored the effects of using computer-based instruction elements to individualize instruction so that the online learning module or platform responds dynamically to the participant’s questions, needs or performance ....


Supports for Learner Reflection ....


Overall, the available research evidence suggests that promoting self-reflection, self-regulation and self-monitoring leads to more positive online learning outcomes. Features such as prompts for reflection, self-explanation and self-monitoring strategies have shown promise for improving online learning outcomes


Moderating Online Groups ..

Organizations providing or promoting online learning generally recommend the use of instructors or other adults as online moderators, but research support for the effects of this practice on student learning is mixed.


Scripts for Online Interaction ....

..mechanisms such as scaffolds or scripts for student group interaction online have been found to influence the way students engage with each other and with the online material, but have not been found to improve learning…

US Dept of ED report on Online Ed ---overview

The report can be found here...I merely summarized key points from the report:

http://www.ed.gov/about/offices/list/opepd/ppss/reports.html


The report sought to answer the folllowing;


1. How does the effectiveness of online learning compare with that of face-to-face instruction?
2. Does supplementing face-to-face instruction with online instruction enhance learning?
3. What practices are associated with more effective online learning?
4. What conditions influence the effectiveness of online learning?




Students who took all or part of their class online performed better, on average, than those taking the same course through traditional face-to-face instruction.

Instruction combining online and face-to-face elements had a larger advantage relative to purely face-to-face instruction than did purely online instruction.


Studies in which learners in the online condition spent more time on task than students in the face-to-face condition found a greater benefit for online learning.



Blended and purely online learning conditions implemented within a single study generally result in similar student learning outcomes. When a study contrasts blended and purely online conditions, student learning is usually comparable across the two conditions.

• Elements such as video or online quizzes do not appear to influence the amount that students learn in online classes. The research does not support the use of some frequently recommended online learning practices. Inclusion of more media in an online application does not appear to enhance learning. The practice of providing online quizzes does not seem to be more effective than other tactics such as assigning homework.

• Online learning can be enhanced by giving learners control of their interactions with media and prompting learner reflection. Studies indicate that manipulations that trigger learner activity or learner reflection and self-monitoring of understanding are effective when students pursue online learning as individuals.

• Providing guidance for learning for groups of students appears less successful than does using such mechanisms with individual learners. When groups of students are learning together online, support mechanisms such as guiding questions generally influence the way students interact, but not the amount they learn

Main points from part 4:



NARRATIVE SYNTHESIS OF STUDIES COMPARING VARIANTS OF ONLINE LEARNING
Blended Compared With Pure Online Learning

The studies reviewed did not proivde a basis for determining the benefits of online versus blended learning modalities or conditions.

Media Elements


The fact that the majority of studies found no significant difference across media types is consistent with the theoretical position that the medium is simply a carrier of content and is unlikely to affect learning per se (Clark 1983, 1994).

....
that the way in which a medium is used is more important than merely having access to it ...


In summary, many researchers have hypothesized that the addition of images, graphics, audio, video or some combination would enhance student learning and positively affect achievement. However, the majority of studies to date have found that these media features do not affect learning outcomes significantly

Learning Experience Type

..the power of collaborative learning ...


offered by the author is that students working in groups may scaffold each other’s work, hence reducing the benefit derived from externally provided questions.

Computer-Based Instruction ...The study found the the platform used to deliver quizes and course content did have some impact on student success ...


The results of three studies exploring the effects of including different types of online simulations were modestly positive. Two of the studies indicated a positive effect from including an online simulation; however, one study found no significant difference ...this does have some implications for simulated learning environments in virtual worlds.



...online learning literature has also explored the effects of using computer-based instruction elements to individualize instruction so that the online learning module or platform responds dynamically to the participant’s questions, needs or performance ....


Supports for Learner Reflection ....


Overall, the available research evidence suggests that promoting self-reflection, self-regulation and self-monitoring leads to more positive online learning outcomes. Features such as prompts for reflection, self-explanation and self-monitoring strategies have shown promise for improving online learning outcomes


Moderating Online Groups ..
Organizations providing or promoting online learning generally recommend the use of instructors or other adults as online moderators, but research support for the effects of this practice on student learning is mixed.


Scripts for Online Interaction ....
mechanisms such as scaffolds or scripts for student group interaction online have been found to influence the way students engage with each other and with the online material, but have not been found to improve learning.




Delivery Platform ......Studies are too few to determine..

New to virtual worlds

One may wonder what keeps you coming back to virtual worlds or Second Life especially if that first entre' into a VW is less than enjoyable..well speaking from experience there has to be some connection with something that you find meaningful or passionate about. For me it is the link between virtual and learning.

My first experiences in Second Life were awkward and well..newby. I can recall bobbling around at an in world music event and falling into the water surrounding the event..well at least 20 times. I would then pick myself up and fly or get back on dry land only to do it again.

I really did have that newby look down to an art...duck walk, pink polka dot dress and a tendency to go where no newby should go. After a couple of months of sporadic visits to second life I gave up and did not go back in world until my job required that I have some sort of knowledge of second life and virtual worlds.

I then came back, with a mission. Maybe it was because it was winter, I was bored with trying to figure out how to watch TV or the fact that I read Second Life for Dummies but I was now able to fully immerse myself in the life and times of Second Life. I was hooked

I bought and sold land, uploaded pictures, set up media players, joined groups, chatted went to events and even did a little teaching.

I feel more than comfortable in world now. Everyday I learn more about how people learn, how they live in second life, and the changes they have made and that have been made to them as a result of their second life experience.