Determining the Level of a Language Test with English Profile: A Forensic Linguistics Case Study

  • Núria Gavaldà Universidad Internacional de La Rioja/Laboratorio SQ-Lingüistas Forenses
  • Sheila Queralt Laboratorio SQ-Lingüistas Forenses

Abstract

This article deals with a forensic linguistics case study of the determination of the level of a B1 English multiple-choice test that was challenged in court by numerous candidates on the grounds that it was not of the appropriate level. A control corpus comprising 240 analogous multiple-choice questions from B1 exams aligned with the Common European Framework of Reference for Languages (CEFR) was compiled in order to establish a threshold for the percentage of questions of a level higher than that being tested which can be expected in such exams. The analysis was carried out following a combination of qualitative and quantitative methods, with the help of the tool English Profile, which provides Reference Level Descriptions (RLDs) for the English language within the CEFR. The results of the analysis of the control corpus established a baseline of 5 to 7% of questions that include key items classified as higher than B1, while the percentage was 68% in the case of the disputedexam. Thus, the present study proposes a further application of the tool English Profile within the field of forensic linguistics and puts forward the concept of Level Appropriateness Threshold (LAT), analogous to other thresholds established in forensic linguistics, which can serve as a baseline for determining the appropriateness of B1 English multiple-choice exams and a model for other levels and skill areas.

Author Biographies

Núria Gavaldà , Universidad Internacional de La Rioja/Laboratorio SQ-Lingüistas Forenses
Núria Gavaldà holds a PhD in Linguistic Communication and Multilingual Mediation (specialisation in Forensic Linguistics) from Universitat Pompeu Fabra. She is currently a lecturer in English phonetics and linguistics at Universidad Internacional de la Rioja, and she carries out research on L2 speech, forensic phonetics and forensic linguistics. She has published on phonetics and forensic linguistics and has collaborated in several forensic linguistic cases.
Sheila Queralt, Laboratorio SQ-Lingüistas Forenses
Sheila Queralt holds a PhD in Translation and Language Sciences from Universitat Pompeu Fabra. She is currently the director of the Laboratorio SQ-Lingüistas Forenses. She is author of Atrapados por la lengua and the Decalogue for Requesting a Linguistics Expert Report (2020), and coauthor of the books Soy lingüista, lingüista forense (2019) and Fundamentos de la lingüística forense (2019). She collaborates with different Spanish police forces as an expert witness and she is mentor of the National Cyber League organised by the Guardia Civil.

References

Archer, Dawn et al., eds. 2003. Proceedings of the Corpus Linguistics 2003 Conference. UCREL Technical Paper, 16. Lancaster: Lancaster University.

Arsland, Abdullah and Ali Erarslan. 2019. “Lexical Analysis of a Textbook Based on the EVP.” International Journal of Languages’ Education and Teaching 7 (1): 1-12.

Banerjee, Jayanti and Dina Tsagari, eds. 2016. Contemporary Second Language Assessment. London: Bloomsbury.

Barker, Fiona. 2010. “How Can Corpora Be Used in Language Testing?” In O’Keeffe and McCarthy 2010, 633-45.

—. 2014. “Using Corpora to Design Assessment.” In Kunnan 2014, 1013-28.

Callies, Marcus and Sandra Götz, eds. 2015. Learner Corpora in Language Testing and Assessment. Amsterdam and Philadelphia: John Benjamins.

Cambridge Assessment English. 2020. “Validity and Validation.” [Accessed online on October 30, 2019].

Cambridge University Press. 2015. English Profile – What the CEFR Means for English. [Accessed online April 2, 2019].

Carlsen, Cecilie. 2012. “Proficiency Level: A Fuzzy Variable in Computer Learner Corpora.” Applied Linguistics 33 (2): 161-83.

Coulthard, Malcolm. 2004. “Author Identification, Idiolect and Linguistic Uniqueness.” Applied Linguistics 25 (4): 431-47.

Coulthard, Malcolm and Alison Johnson, eds. 2010. The Routledge Handbook of Forensic Linguistics. London and New York: Routledge.

Council of Europe. 2001a. The Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge UP. [Accessed online on October 30, 2019].

—. 2001b. Structured Overview of all CEFR Scales. [Accessed online on October 30, 2019].

—. Reference Level Descriptions (RLDs) (Language by Language). [Accessed online on October 30, 2019].

—. 2020. Reference Level Descriptions (RLDs) Developed so Far. [Accessed online on October 30, 2020].

Cushing, Sara T. 2017. “Corpus Linguistics in Language Testing Research.” Language Testing 34 (4): 441-49.

Escola Oficial d’Idiomes Barcelona Drassanes. “Mission, Vision and Values.” [Accessed online on October 30, 2019].

Garayzábal Heinze, Elena, Sheila Queralt Estévez and Mercedes Reigosa Riveiros. 2019. Fundamentos de la lingüística forense. Madrid: Síntesis.

Gibbons, John and M. Teresa Turell, eds. 2008. Dimensions of Forensic Linguistics. Amsterdam and Philadelphia: John Benjamins.

Grant, Tim. 2007. “Quantifying Evidence in Forensic Authorship Analysis.” The International Journal of Speech, Language and the Law 14 (1): 1-25.

Harrison, Julia and Fiona Barker, eds. 2015. English Profile Studies 5. English Profile in Practice. Cambridge: CUP.

Knight, Ben. 2015. “Applications of English Profile.” In Harrison and Barker 2015, 93-105.

Kunnan, Antony John, ed. 2014. The Companion to Language Assessment. Vol 2, Approaches and Development. Hoboken, NJ: Wiley-Blackwell.

Lavery, Urban A. 1921. “The Language of the Law”. American Bar Association Journal, 7 (6), 277-83.

Leńko-Szymańska, Agnieszka. 2015. “The English Vocabulary Profile as a Benchmark for Assigning Levels to Learner Corpus Data.” In Callies and Götz 2015, 115-40.

Ley Orgánica 8/2013. Boletín Oficial del Estado, December 10.

Marquina, Montse and Sheila Queralt. 2014. “Similarity Threshold to Detect Plagiarism in Spanish.” RAEL: Revista Electrónica de Lingüística Aplicada 13 (1): 79-95.

Montolío, Estrella. 2012. “La modernización del discurso jurídico español impulsada por el Ministerio de Justicia. Presentación y principales aportaciones del Informe sobre el lenguaje escrito.” Revista de llengua i dret 57: 95-121.

Nicholls, Diane. 2003. “The Cambridge Learner Corpus: Error Coding and Analysis for Lexicography and ELT.” In Archer et al. 2003, 572–82.

O’Keeffe, Anne and Michael McCarthy, eds. 2010. The Routledge Handbook of Corpus Linguistics. London and New York: Routledge.

Park, Kwanghyun. 2014. “Corpora and Language Assessment: The State of the Art.” Language Assessment Quarterly 11 (1): 27-44.

Poblete, Claudia, Lisbeth Arenas, Alejandro Córdova, Emmy González and Daniela Tapia. 2018. Estrategias en comprensión del discurso escrito en contextos jurídicos. Valparaíso: Ediciones Universitarias de Valparaíso.

Stygall, Gail. 2010. “Legal Writing: Complexity”. In Coulthard and Johnson 2010, 51-64.

Tiersma, Peter. 1999. Legal Language. Chicago, IL: U of Chicago P.

Turell, M. Teresa. 2007. “Plagio y traducción literaria.” Vasos Comunicantes 37 (1): 43-54.

—. 2008. “Plagiarism”. In Gibbons and Turell 2008, 265-99.

—. 2010. “The Use of Textual, Grammatical and Sociolinguistic Evidence in Forensic Text Comparison.” The International Journal of Speech, Language and the Law 17 (2): 211-50.

University of Cambridge Local Examinations Syndicate (UCLES) and Cambridge University Press (CUP). 2011. English Profile: Introducing the CEFR for English Version 1.1. [Accessed online on April 2, 2019].

University of Cambridge ESOL Examinations. 2011. Using the CEFR: Principles of Good Practice. [Accessed online on April 2, 2019].

Weigle, Sara Cushing and Sarah Goodwin. 2016. “Applications of Corpus Linguistics in Language Assessment.” In Banerjee and Tsagari 2016, 209-23.

Published
2020-12-23
Section
Articles