Ms. Sari Sulkunen's Academic dissertation 17.11.2007

Sulkunen, S. 2007. Text authenticity in international reading literacy assessment. Focusing on PISA 2000. Jyväskylä: University of Jyväskylä, 2007, 277 p. (Jyväskylä Studies in Humanities ISSN 1459-4331; 76) ISBN 978-951-39-2976-3 (PDF), 978-951-39-2951-0 (bind.)

 

Abstract

The purpose of this study was to investigate how authentic in terms of text typicality and interest the texts used in PISA 2000 reading literacy assessment were to Finnish 15-year-old students. The study consisted of two phases. In the pilot phase, an 8-member panel reviewed the texts from the point of view of text typicality. The results showed that the most essential textual features in evaluating text authenticity were the topic, the genre, the layout and the language used. Based on these results a questionnaire for the main study was designed. Questionnaire was presented for 240 15-year-olds who evaluated and rated the typicality and interest of PISA texts’ topic, genre, layout and language on a scale from 1 to 4. The results showed that the PISA texts were overall only moderately authentic in terms of typicality and interest to 15-year-old Finnish students. Furthermore, texts tended to be more typical than interesting. There were clear gender differences on which texts were found authentic but also texts that were equally typical and interesting for both genders. Overall, girls rated the texts more positively than boys. Based on the results it was concluded that the aim of choosing authentic texts for reading literacy test is important from the point of view of test validity. PISA texts, however, succeed only partially in achieving this aim as they form rather a pale reflection of texts authentic to 15-year-olds. In future, special concern should be in selecting texts more interesting to the target-group, particularly to boys.

Keywords: authentic texts, reading tests, test validity and reliability, literacy, discourse/text genres