Getting to know your corpus

Hits: 6179
Research areas: Year: 2012
Type of Publication: In Proceedings Keywords: corpora, corpus similarity, keywords, keyword lists, Sketch Engine
Editor: Sojka, P., Horak, A., Kopecek, I., Pala, K. Volume: 7499
Book title: Proceedings of the 15th International Conference on Text, Speech and Dialogue (TSD2012)
Series: Lecture Notes in Artificial Intelligence Pages: 3-15
Address: Brno, Czech Republic
Organization: 15th International Conference on Text, Speech and Dialogue (TSD2012) Month: September 3-7
Corpora are not easy to get a handle on. The usual way of getting to grips with text is to read it, but corpora are mostly too big to read (and not designed to be read). We show, with examples, how keyword lists (of one corpus vs: another) are a direct, practical and fascinating way to explore the characteristics of corpora, and of text types. Our method is to classify the top one hundred keywords of corpus1 vs: corpus2, and corpus2 vs: corpus1. This promptly reveals a range of contrasts between all the pairs of corpora we apply it to. We also present improved maths for keywords, and briefly discuss quantitative comparisons between corpora. All the methods discussed (and almost all of the corpora) are available in the Sketch Engine, a leading corpus query tool.
JRESEARCH_FULLTEXT: Kilgarriff_TSD2012.pdf