Research/wiki/Developing a rubric for assessing the quality of an OER

Based on advantages and issues described.. the following rubric has been created.


 * Wiki Resource: (Title/Url)


 * Type of Learning object: part of a lesson, single lesson, topic, paper
 * A learning object in this context is what wiki page or pages satisfy the learning objectives.
 * For example: if a single lesson is about resizing images, this may be a single wiki page,


 * Objectives of Learning object:

Technology issues
Hosting solution, Behind an institutional firewall, Hosted by author (vmvwiki), Hosted by private wiki available to public (WikiSpaces) Hosted by public wiki (WikiEd, WikiVersity)

Content is connected, All content in wiki, .., content exists on multiple servers

Content is CC, Has (C), available as CC,BY or CC

Spam and vandalism are managed, Has many pages,, is actively monitored

Content and context issues
Including external content

One of the benefits of OERs is that they can be re-used or re-mixed into your own content. One way that is proving very successful is the iFrame insertion. While this has many advantages, including handling the issue of copyright, there have been instances where the linked page has been removed, or the page has become protected (this happens in some of the news feeds where the content is made available free for a short time, then archived and accessed via a payment system).

Further, some inserted pages include large page headers and significant advertising, so all you see is a banner ad rather than the content.

Another issue is that control over updates and deletions is lost. Several times content that has been embedded has been removed (e.g. SlideShare, and YouTube), and in some cases the way they are accessed was changed.

Content out of context

As users have a wide range of abilities, an issue that surfaced in earlier iterations of the virtualME knowledge framework (before the wiki) was the problem of content out of context. For example, where should content be added if there is no wiki page? In the vMVwiki there has not been a noticeable problem, however, this could also mean students have not added an entry rather than placing the content in the wrong context. To some extent this is addressed through the individual user pages. A user survey could be conducted to determine the extent of this.

Having incomplete content

As a wiki allows for continuous editing, there is a temptation to create a filler or starter page. The effect of this can be seen on the wikiversity page on "Intermediate JavaScript" ( http://en.wikiversity.org/wiki/Intermediate_JavaScript ). This can be problematic when students are expecting content but find the page incomplete. However, this could also be seen as an opportunity for students and users to add this content.

Content continually changing

From a learner perspective this can create frustration issues, especially for those students who prefer to print the content. This was noticed when the virtualMVwiki server was unavailable for a few days and students reverted to the backup server copy. Content that had recently been added was not available causing postings to the class discussion forum to ask for the updated content.

Content of varying quality

As adding content requires the use of wiki markup, and can be edited by anyone with a userid, the skill levels of the authors (users) can determine not only the reliability of the content but the way it is presented. Templates can be used to assist in this but authors need to be skilled in their use.

Sourcing content and digital rights

One big issue is constructing the content and managing digital rights. Fortunately there are repositories of Open Content such as Wikimedia commons ( http://commons.wikimedia.org/ ) that can provide a wealth of images. As the content is placed in the public domain rather than restricted to an educators presentation slides (like PowerPoint) much more care needs to be taken. For the teaching vMVwiki much more attention to citing has been done, than probably would have been done if presented in a closed format. This is also observable in Wikipedia where the citings/references play an important part.

OER is normally distributed using a Creative Commons licence. There are many to choose from, however in order to satisfy the OER the two licences are CC-BY-SA and CC-SA.

Learning wikicode

One of the strengths (and weaknesses) of Mediawiki is the necessity to use wiki-code. The advantage is that it is text based so easily edited in any web browser, however the cryptic syntax does take a little getting used to.

This can be enhanced through the use of templates, for example to standardise headers and footers, and constructs such as boxes.

Academic staff issues
Development time

Developing content in a wiki takes a significant amount of time. Unlike a self contained presentation tool, where templates are used to provide professional looking presentations, where text fonts are easily changed, multimedia effects added, and media inserted directly, a wiki requires building the content in wiki-markup, uploading images separately and changing formatting manually.

Loss of income

Nancy George posted the following comment to the OER University (OERu) discussion forum "Universities are also pressuring their academics to bring income into the institution to help them survive; therefore, I find it improbable that they are going to give their blessing to their faculty to develop learning materials that could earn the institution money and then give relinquish the right to that money." There has been significant discussion on this in the OERu discussion forum during May 2011.

Losing control of personal content

This is a more complex issue as it involves a philosophical concept which is very personal. A course on "Open content licensing 4 educators" run by Wikieducator (OCL4Ed, 2011) considers it in one way as "Is teaching a vocation or profession?". The discussion can be found on the Wikieducator page "Educators care: Why open matters?" (http://wikieducator.org/Educators_care/Introduction )

A sticking point for many authors is the Non-commercial attribution, where another person is able to take the work and modify it in a commercial setting. For a detailed discussion refer to Erik Möller's (2007) discussion "The Case for Free Use: Reasons Not to Use a Creative Commons -NC License" ( http://freedomdefined.org/Licenses/NC )

Student issues
Reluctance to edit

While the concept of adding and editing content by users of the site is the cornerstone of a wiki, actually getting users to actively change the content as opposed to passively viewing the content remains a challenge.

One of the students sent me the following comments "I have some thoughts about student contributions to the wiki – barriers, perceived or otherwise. As a student, your wiki is referred to as Michaels Wiki, straight away setting a defining line in regards to ownership. I also think its a lot harder as a young person to realise that as a pupil you also may have something to contribute, we come from years of schooling where its teacher/learner with set roles. So to make that move from student receiver to student contributor is a big step. Getting the students to contribute as part of an assessment – excellent idea :) . "

The final comment, that is, encouraging student participation by including it as an assessable item is often discussed in the context of participation in discussion forums. Some studies further suggest that the level of engagement in discussion forums can be moderated by factors such as teacher’s involvement and possibility of receiving course credits ( (Garrison et al., 1999, 2001, 2003], (Guzdial & Turns, 2000) and (Taradi & Taradi, 2004) cited in Chenga, Paréa, Collimoreb, & Joordensa (2011) ). This has been demonstrated in the virtualMV wiki where classes when comparing contributions from students with and without assessment requirements. As a side effect, once students gain confidence in adding to the wiki they are more likely to add content.

Knowledge of the use of Web based technologies is required

As the content is delivered on a web platform students must be competent at using web technologies to access the content. This includes have Internet access of a suitable quality to display the content. The vMVwiki has primarily been used for Information Technology students, and students in other learning domains could have difficulty in navigating the web technologies required.

Ability to read and process screen based content

Not all users are comfortable with reading content on-screen, and indeed in some instances students have produced complete hard copies of the content. From a personal perspective the author has found that dual monitors make the task of reading on-screen content significantly easier, and has observed a trend among colleagues to having dual-monitors.

Benefits of using a wiki
Given that a wiki provides many challenges, what are some of the benefits?

Technology benefits
Content is available

As a wiki is cloud based, as long as you have access to the internet the content/teaching resource is available. When used to deliver content in a computer laboratory, students can have the content on their screens while following the projected presentation. This is particularly useful where students find it hard to see over/around other students or have difficulty reading the projected image.

Content is browser based

There are many advantages here. One of the design features of MediaWiki is that it performs well in most browsers. As it is browser based issues such as media compatibility are addressed by the browser, rather than what is installed on a local PC (as is the case for a CD/DVD based multimedia presentation). The text editor is fairly basic which has the advantage that changes to content can be done easily inside the browser. Open Office and Microsoft provide add-ons so that you can work offline, however it is preferable with the wiki to work on the live system.

Content can be viewed on many devices

As the number of device types increases and includes Personal Computers, Pad based computers (like Apple iPad), and mobile devices (such as smart phones), the ability to present content on all of these devices becomes more important. As Mediawiki is browser based effectively any device running a browser and linked to the Internet is capable of accessing the content. MediaWiki also allows for a variety of stylesheets and has a Mobile style sheet extension. This can be seen when accessing Wikipedia on a smartphone.

Content and context benefits
Content is always up to date

As the content is stored in one place and can be updated directly as it is cloud based, students are always presented with the latest version of the content. During lectures the content can be updated in real time (There are instances where corrections are made while content is being presented).

Another benefit is that the content can be used in many contexts. For example, resizing graphics is important when discussing web page performance, multimedia construction, and inclusion in Office documents.

Content may be re-used

As the content is in the public domain and for an OER usually comes with a Creative Commons licence (BY-SA) the content may be reused by other organisations and even translated into other languages. In Taiwan, the Opensource Opencourse Prototype System (OOPS) is translating OER courseware into Chinese, and is also an Massachusetts Institute of Technology - Open Coursesware (MIT OCW) official translation affiliate. The Open Learning Network (OLnet) is undertaking research in the use and reuse of open learning content.

Student issues
There are many benefits that students can gain through the use of a wiki. Guth (2007) identified several pedagogical benefits;

Collective authoring: critical reading and responsible writing

As the wiki entry is open to the peers and potentially the world this encourages a high quality of work.

Collective ownership of work

Encourages the concept of sharing personal knowledge. However In Lund and Smordal’s (2006 cited by Guth (2007) ) wiki experience, “learners did not immediately embrace any notion of collective ownership or epistemology but continued a practice where the institutionally cultivated individual ownership persisted.”

Writing as a process and knowledge sharing over time

Guth (2007) identified that this has two advantages: first, it facilitates writing as a process rather than a product and, second, it promotes the continued use of a wiki in different contexts.

Benefits of publishing online

Guth (2007) also identified that wikis give students the opportunity to focus on issues such as referencing online sources and considering copyright issues of the multi-media content (e.g. images).

Students are familiar with the environment

As the content is delivered in an environment similar to wikipedia most students have used this interface.

Students can take ownership

Effective learning engages the students. As they are able to add and modify the content students can contribute in a real way to the material being displayed.

Students have a built in personal note system

It is easy to add personal notes to the wiki via the personal user page which is accessible from every page.

Prepares students for Web 2.0 technologies

For students that wish to, having exposure to the wiki way, means they will develop skills that can be used on other public wikis, and can participate in major international projects such as WikiEducator, Wikiversity and Wikipedia. Promotion tool

Content in a Public wiki is openly available wiki so this provides a marketing platform for the institution. "The educational experience at an institution is more than the consumption of educational materials, so students who are inclined to enroll in university are not disinclined to enroll because they can access materials freely." (Mary Lou Forward, 2011, OERu discussion forum)

Do students actually use the wiki as a wiki?
A student commented as follows "One of the attractions of using a wiki is that anyone with the necessary permissions can modify content on any page. Until we understand how to change content this is often a barrier to actually changing the content". As such the small assessment items have been built into the papers, where students are asked to change or add something to the wiki - even if it is done away from the main wiki on their user page.

A related issue is will students continue to contribute and develop a wiki community. In one case, a student regularly sent an email notifying spam. Eventually they were given administrator access. This student took things a stage further and started categorising pages (which was a good thing) and even created a page "Help wanted" link on the Navigation bar which connected to a page containing links to pages started but needing work.

Another recent development was a group of students creating a page to manage a Web Application Development project (WAD) (http://www.virtualmv.com/wiki/index.php?title=TeamKea_WAD&redirect=no). The page was begun, but replaced with the Open Source Project Management tool, Redmine ( http://www.redmine.org/ ), that managed the processes and documents.