Step III: Criteria for Comparison of the Annotation Methods

After an introduction of the different Annotation Methods, it is now time to collect criteria for the comparison of the Methods. Which Annotation Program will be the best one to annotate medical documents? The criteria are based on my meetings with Dr. Pieter Van Gorp, the FHIES article (Van Gorp et al. 2012), and the literature review by (Novak et al. 2012).

The criteria are classified in three categories:

1. Annotation options 
2. Output 
3. Compatibility 

Annotations options:

This category covers the options needed for the annotation process. Two main criteria are discovered: 
a. Collaborative annotating: medical experts have to be able to collaborate in the annotating process. The medical documents have to be annotated by multiple experts to ensure correct annotations. Furthermore, discussions about or explanations of annotations need to be possible. Therefore, group annotations and notes are selected as needed annotation options. 

b. Functional annotations: this indicates the option to give functional meaning to certain annotations. For example, if a symptom is annotated, this annotation should be given the function ‘symptom’. This will make the annotation process structured and it will make the creation of models possible after the annotation. Since the creation of models is a known step after the annotation, functional annotations is selected as a needed annotation option.

Output:

It is wanted to be able to create models after the annotation process; therefore, the output of the annotation process is of interest. The export of the annotated parts is wanted. The content and type of output file are of interest. The output needs to be computer-interpretable to automatically create models after the annotation. Furthermore, the content must include the annotations and their function; and a link with the original document must be maintained. 

Compatibility:

To create ease of use, the annotation program/method, must be compatible with:

c. Multiple operating systems 
d. Multiple devices, like PCs or tablets 
e. Different types of documents: To be able to annotate all different types of medical documents and to create a generally applicable program. 

Furthermore, synchronization needs to be in place to prevent double and/or conflicting annotations. 

Figure 1: Spiderweb of Criteria

In Fig. 1: Spiderweb of Criteria, the different categories and criteria are shown. My next step will be to discuss these criteria once more with Dr. Van Gorp. After approval, I will compare the different annotation programs/methods with these criteria.

Fig. 1: Spiderweb of Criteria

Sources: 

• Van Gorp, P. et al., 2012. MDE Support for Process-Oriented Health Information Systems: from Theory to Practice.
• Novak, E., Razzouk, R. & Johnson, T.E., 2012. The Educational Use of Social Annotation Tools in Higher Education: A Literature Review. Internet and Higher Education, 15(1), pp.39–49.

Step II E: Selection of Web Annotation Tools

Based on the findings of Novak et al. (2012) and a quick search on Google (Web Annotation Tool), the following web annotation tools are investigated: 

1. HyLighter
2. VPen 
3. SpreadCrumbs 
4. EDUCOSM 
5. Diigo 
6. Zotero.

HyLighter

HyLighter is a browser-based collaboration platform. Novak et al. (2012) describe it as an online SA system that allows users to highlight and annotate in an electronic text and share it with others. One has to buy a license for HyLighter to be able to use it.

VPen

Virtual Pen (VPen) is a Web-based online annotation system developed by (Hwang et al. 2007). This tool was built to investigate the influence of annotation and the sharing of annotations on learning performance.

SpreadCrumbs

Novak et al. (2012) state that SpreadCrumbs has been developed to explore how students annotate electronic resources. Kawase et al. (2009) designed SpreadCrumbs as “an annotation system that provides an interface for adding post-it notes and crumbs (i.e., personal reminders) to any point within a Web page”. Furthermore, bookmarks and the sharing of annotations are possible.

EDUCOSM

Nokelainen et al. (2003) have developed EDUCOSM. This tool focuses on the possibilities of collaboration and the open-ended nature of the Web. Sharing and annotation of arbitrary Web-pages are possible. Furthermore, the EDUCOSM system consists of a set of tools including search, filters, and others for asynchronous collaborative knowledge construction (Novak et al. 2012).

Diigo

Diigo is a Web 2.0 SA tool. On its website developers describe Diigo as “a cloud-based personal information management system”. It runs on different operating systems and web browsers. Annotations, bookmarks, and highlighting are possible and saved in an online personal library. Collaboration and sharing are optional. Furthermore, this annotation tool is freely available.

Zotero

“Zotero is a free, easy-to-use tool to help you collect, organize, cite, and share your research sources”, according to its developers. Zotero is also a Web 2.0 SA tool. Zotero can be used to organize and tag documents, however, it is not possible to annotate or highlight documents.

Sources

• Hwang, W.Y., Wang, C.Y. & Sharples, M., 2007. A study of multimedia annotation of Web-based materials. Computers \& Education, 48(4), pp.680–699.
• Kawase, R., Herder, E. & Nejdl, W., 2009. A comparison of paper-based and online annotations in the workplace. Learning in the Synergy of Multiple Disciplines, pp.240–253.
• Nokelainen, P. et al., 2003. Evaluating the role of a shared document-based annotation tool in learner-centered collaborative learning. Advanced Learning Technologies, 2003. Proceedings. The 3rd IEEE International Conference on, pp.200–203.
• Novak, E., Razzouk, R. & Johnson, T.E., 2012. The Educational Use of Social Annotation Tools in Higher Education: A Literature Review. Internet and Higher Education, 15(1), pp.39–49. 
• HyLighter Website
• Diigo Website
• Zotero website

Step II D: Web Annotation Tools

The second category of tools are Web Annotation Tools. In this blog I will present general information about Web Annotation Tools. 

Web Annotation Tools (WATs) allow annotations on HTML documents, i.e. on a web resource, like a web page. Rau et al. (2004) describe the major functionalities of WATs as “highlighting texts, inserting and editing annotations, organizing and presenting annotations hierarchically, and sharing annotations”. The annotations occur in a separate annotation layer, so the original source is not modified.

WATs can be used for individual or shared annotating. According to Glover et al. (2007), this sharing of notes (with people who have the same annotation system) is one of the two major advantages of inserting annotations into web resources. The second major advantage is the possibility to access the annotations from any web enabled computer.

If WATs are used for sharing than they are a type of Social Annotation (SA) tool. Social Annotation Technology is “an online social bookmarking tool that allows annotations on an electronic resource and supports easy online information sharing” (Novak et al. 2012). SA technology also provides a social platform for interactions and discussions.

Collaborative annotating is one of the key conditions for the annotating tool that is needed in this project, since medical experts must be able to annotate the medical documents in collaboration. Therefore, the larger category, Social Annotation Tools, is closer investigated. Novak et al. (2012) have found that the number of social annotation technologies grows due to “the need for using such tools in various settings and the benefits this technology offers”. One of these benefits is the created sense of community among the users in a given system and the resulting involvement in the community (Bateman et al. 2006). Novak et al. (2012) conclude that educationally used SA tools are found beneficial for collaborative learning. These benefits are positive for this project, however, they also state that it is best to use small teams (2-3 people) for collaborative SA activities. This is a recommendation to keep in mind when implementing the annotating tool.

Sources:

• Bateman, S., Farran, R., Brusilovsky, P. & McCalla, G., 2006, November 8-10. OATS. The open annotation and tagging system, Paper presented at the Third Annual Inter- national Scientific Conference of the Learning Object Repository Research Network, Montreal. 
• Glover, I., Xu, Z. & Hardaker, G., 2007. Online annotation-Research and practices. Computers & Education, 49(4), pp.1308–1320.
• Novak, E., Razzouk, R. & Johnson, T.E., 2012. The Educational Use of Social Annotation Tools in Higher Education: A Literature Review. Internet and Higher Education, 15(1), pp.39–49.
• Patrick Rau, P.L., Chen, S.H. & Chin, Y.T., 2004. Developing web annotation tools for learners and instructors. Interacting with Computers, 16(2), pp.163–181.