Difference between revisions of "Data Jamboree 1/Annotation Experiment"

From phenoscape
(Results and Conclusions)
(Background and Participant Preparation)
Line 1: Line 1:
 
====Background and Participant Preparation====
 
====Background and Participant Preparation====
  
An annotation experiment was conducted on day 2 of the Phenoscape Data Jamboree in order to assess curation consistency among the four trained participants. Participant training consisted of a hands-on group annotation exercise on day 1, and individual work on each participant's own publications with assistance from project personnel on days 1 and 2. Participants were also given an Annotation Guide with examples of character types commonly encountered in the fish systematic literature.  Participants were given 2 hours to annotate 10 characters (plus one extra credit) taken from three publications (Hilton 2003, Sanger and McCune 2002, Zanata and Vari 2005).
+
An annotation experiment was conducted on day 2 of the Phenoscape Data Jamboree in order to assess curation consistency among the four trained participants. Training consisted of a hands-on group annotation exercise on day 1, and individual work on each participant's own publications with assistance from project personnel on days 1 and 2. An Annotation Guide with examples of character types commonly encountered in the fish systematic literature was also given to participants.  Participants were given 2 hours to annotate 10 characters (plus one extra credit) taken from three publications (Hilton 2003, Sanger and McCune 2002, Zanata and Vari 2005).
  
 
====Results and Conclusions====
 
====Results and Conclusions====

Revision as of 18:47, 24 April 2008

Background and Participant Preparation

An annotation experiment was conducted on day 2 of the Phenoscape Data Jamboree in order to assess curation consistency among the four trained participants. Training consisted of a hands-on group annotation exercise on day 1, and individual work on each participant's own publications with assistance from project personnel on days 1 and 2. An Annotation Guide with examples of character types commonly encountered in the fish systematic literature was also given to participants. Participants were given 2 hours to annotate 10 characters (plus one extra credit) taken from three publications (Hilton 2003, Sanger and McCune 2002, Zanata and Vari 2005).

Results and Conclusions

Completeness of annotations

All participants recorded the character number, textual description, and selected the appropriate voucher specimen for each annotation. Only two of the four participants recorded evidence codes for each annotation.

Variability of EQ statements among participants

A summary of annotation consistency among participants is presented in the table below (incomplete annotations due to software issues are excluded). Three of the four participants attempted annotations for all 11 characters, while one participant finished only 7 characters.

Participants annotated only one character identically. Variation in the other annotations was due to several reasons. Most commonly, participants differed in how they post-composed entity terms, particular in how entites were related in the post-composition (for example, use of part_of/has_part).


Summary Statistics
Character # # Participants with Annotation* % Consistency with Key Specific Issues
1 4 100
2 3 0 post-composition of Q term for relative length
3 3 0 incorrect recording of count values
4 4 0 TAO term definition confusion (bone vs. cartilage)
5 3 33 E post-composition; choice of appropriate Q
6 4 0 E post-composition
7 4 50 E post-composition
8 3 33 choice of appropriate Q term
9 3 0 E post-composition; choice of appropriate Q term
10 2 50 choice of appropriate Q term
EC 2 25 anatomy term post-composition; choice of appropriate quality term
  • *incomplete annotations due to software issues were excluded

References

  • Hilton EJ. 2003. Comparative osteology and phylogenetic systematics of fossil and living bony-tongue fishes (Actinopterygii, Teleostei, Osteoglossomorpha). Zoological Journal of the Linnean Society 137: 1-100.
  • Sanger TJ, McCune AR. 2002. Comparative osteology of the Danio (Cyprinidae: Ostariophysi) axial skeleton with comments on Danio relationships based on molecules and morphology. Zoological Journal of the Linnean Society 135:529-546.
  • Zanata AM, Vari RP. 2005. The family Alestidae (Ostariophysi, Characiformes): a phylogenetic analysis of a trans-Atlantic clade. Zoological Journal of the Linnean Society 145: 1-144.