Step III: Criteria for Comparison of the Annotation Methods

After an introduction of the different Annotation Methods, it is now time to collect criteria for the comparison of the Methods. Which Annotation Program will be the best one to annotate medical documents? The criteria are based on my meetings with Dr. Pieter Van Gorp, the FHIES article (Van Gorp et al. 2012), and the literature review by (Novak et al. 2012).

The criteria are classified in three categories:

1. Annotation options 
2. Output 
3. Compatibility 

Annotations options:

This category covers the options needed for the annotation process. Two main criteria are discovered: 
a. Collaborative annotating: medical experts have to be able to collaborate in the annotating process. The medical documents have to be annotated by multiple experts to ensure correct annotations. Furthermore, discussions about or explanations of annotations need to be possible. Therefore, group annotations and notes are selected as needed annotation options. 

b. Functional annotations: this indicates the option to give functional meaning to certain annotations. For example, if a symptom is annotated, this annotation should be given the function ‘symptom’. This will make the annotation process structured and it will make the creation of models possible after the annotation. Since the creation of models is a known step after the annotation, functional annotations is selected as a needed annotation option.

Output:

It is wanted to be able to create models after the annotation process; therefore, the output of the annotation process is of interest. The export of the annotated parts is wanted. The content and type of output file are of interest. The output needs to be computer-interpretable to automatically create models after the annotation. Furthermore, the content must include the annotations and their function; and a link with the original document must be maintained. 

Compatibility:

To create ease of use, the annotation program/method, must be compatible with:

c. Multiple operating systems 
d. Multiple devices, like PCs or tablets 
e. Different types of documents: To be able to annotate all different types of medical documents and to create a generally applicable program. 

Furthermore, synchronization needs to be in place to prevent double and/or conflicting annotations. 

Figure 1: Spiderweb of Criteria

In Fig. 1: Spiderweb of Criteria, the different categories and criteria are shown. My next step will be to discuss these criteria once more with Dr. Van Gorp. After approval, I will compare the different annotation programs/methods with these criteria.

Fig. 1: Spiderweb of Criteria

Sources: 

• Van Gorp, P. et al., 2012. MDE Support for Process-Oriented Health Information Systems: from Theory to Practice.
• Novak, E., Razzouk, R. & Johnson, T.E., 2012. The Educational Use of Social Annotation Tools in Higher Education: A Literature Review. Internet and Higher Education, 15(1), pp.39–49.