Overall Percent Agreement

1. Often times one category is more clear than others. Thus there is likely to be considerable agreement between the two observers about subjects in this category. Therefore the percent agreement may be high only because of the large number of subjects clearly in this category. A correction for this problem is to calculate the percent agreement without the number of times the raters agree in this particular category.
2. Another problem is that the procedure does not take into account that the raters are expected to agree solely by chance. A correction for this problem is to use a Kappa statistic to measure the agreement between raters.