Combining Multimodal Techniques to Approach the Study of Academic Lectures: A Methodological Reflection

  • Edgar Bernad-Mechó Universitat Jaume I


This article offers a methodological reflection on the use of multimodal techniques for the study of academic lectures. Three distinct multimodal approaches have been put forward to explore the use of language holistically, namely, multimodal social semiotics (MSS), multimodal discourse analysis (MDA) and multimodal interaction analysis (MIA). These approaches differ in their main focus—the social context, the system of semiotic resources available to the speakers and the social actors, respectively—and the tools they provide to conduct multimodal analyses. To exemplify how analyses may be conducted within each of the paradigms in the context of academic lectures in English, I examine an excerpt extracted from an African-American history lecture from Yale University by a native English speaker in which he organizes his discourse in between content sections. Through the use of short multimodal transcriptions, I discuss how MSS can be used for reflections on the social contexts of academic lectures, MDA describes the use of semiotic resources employed by the lecturers, and MIA can be used to look into how lecturers structure their speech into sequences of actions. Ultimately, I suggest a combination of multimodal methodologies to obtain a broader account of the intricacies of discourse in academic settings.

Author Biography

Edgar Bernad-Mechó, Universitat Jaume I
Edgar Bernad-Mechó holds a PhD in Applied Linguistics. He is a member of the “Group for Research on Academic and Professional Discourse (GRAPE)” at Universitat Jaume I. His research interests focus on the study of academic English in lectures and multimodality studies and methodologies. He is currently working on the organization of lectures from the perspective of metadiscourse and the ensemble of semiotic resources by lecturers.


ADAMI, Elisabetta. 2017. “Multimodality.” In García, Flores and Spotti 2017, 451-72.

ARCHER, Arlene and Esther Odilia Breuer, eds. 2016. Multimodality in Higher Education. Leiden: Brill.

BERNAD-MECHÓ, Edgar. 2017. “Metadiscourse and Topic Introductions in an Academic Lecture: A Multimodal Insight.” Multimodal Communication 6 (1): 39-60.

—. 2018. “A Multimodal Approach to Metadiscourse as an Organizational Tool in Lectures.” PhD diss., University Jaume I.

BERNAD-MECHÓ, Edgar and Inmaculada Fortanet-Gómez. 2019. “Organizational Metadiscourse across Lecturing Styles: Engagement beyond Language.” In Sancho-Guinda 2019, 321-40.

BIBER, Douglas, Susan Conrad and Viviana Cortés. 2004. “ʻIf You Look at...ʼ: Lexical Bundles in University Teaching and Textbooks.” Applied Linguistics 25 (3): 371-405.

BONSIGNORI, Veronica and Belinda Crawford Camiciottoli, eds. 2016. Multimodality across Communicative Settings, Discourse Domains and Genres. Newcastle upon Tyne: Cambridge Scholars.

BUCK, Gary. 2001. Assessing Listening. Cambridge: Cambridge UP.

CALZOLARI, Nicoletta et al., eds. 2006. Proceedings of LRforthEC 2006, Fifth International Conference on Language Resources and Evaluation. Genoa: European Language Rescources Association.

CRAWFORD CAMICIOTTOLI, Belinda. 2015. “Elaborating Explanations during OpenCourseWare Humanities Lectures: The Interplay of Verbal and Nonverbal Strategies.” In Crawford Camiciottoli and Fortanet-Gómez 2015, 144-70.

—. 2016. “A Multimodal Analysis of Interaction in Academic Lectures: A Case Study.” In Bonsignori and Crawford Camiciottoli 2016, 65-92.

CRAWFORD CAMICIOTTOLI, Belinda and Inmaculada Fortanet-Gómez, eds. 2015. Multimodal Analysis in Academic Settings: From Research to Teaching. London and New York: Routledge.

DEPPERMANN, Arnulf, ed. 2013. “Conversation Analytic Studies of Multimodal Interaction.” Special issue, Journal of Pragmatics 46 (1).

DUDLEY-EVANS, Tony. 1994. “Variations in the Discourse Patterns Favoured by Different Disciplines and Their Pedagogical Implications.” In Flowerdew 1994, 146-58.

FLOWERDEW, John, ed. 1994. Academic Listening: Research Perspectives. Cambridge: Cambridge UP.

FLOWERDEW, John and Lindsay Miller. 1992. “Students’ Perceptions, Problems and Strategies in L2 Lectures.” RELC Journal 23 (12): 60-80.

FORCEVILLE, Charles J. and Eduardo Urios-Aparisi, eds. 2009. Multimodal Metaphor. Berlin: De Gruyter.

FORTANET-GÓMEZ, Inmaculada. 2004. “The Use of ‘We’ in University Lectures: Reference and Function.” Journal of English for Specific Purposes 23: 45-66.

FORTANET-GÓMEZ, Inmaculada and María Noelia Ruiz-Madrid. 2014. “Multimodality for Comprehensive Communication in the Classroom: Questions in Guest Lectures.” Ibérica 28: 203-24.

FOX, Janna and Natasha Artemeva. 2013. “The Cinematic Art of Teaching University Mathematics: Chalk Talk as Embodied Practice.” Multimodal Communication 1 (1): 83-103.

GARCÍA, Ofelia, Nelson Flores and Massimiliano Spotti, eds. 2017. The Oxford Handbook of Language and Society. Oxford: Oxford UP.

GEE, Paul and Michael Handford, eds. 2012. The Routledge Handbook of Discourse Analysis. London and New York: Routledge.

GUNEL, Murat, Brian Hand and Sevket Gunduz. 2006. “Comparing Student Understanding of Quantum Physics When Embedding Multimodal Representations into Two Different Writing Formats: Presentation Format versus Summary Report Format.” Science Education 90 (6): 1092-112.

HALLIDAY, M. A. K. 1978. Language as a Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold.

—. 1985. An Introduction to Functional Grammar. London: Edward Arnold.

HYLAND, Ken and Brian Paltridge, eds. 2011. Companion to Discourse. London: Continuum.

IEDEMA, Rick. 2003. “Multimodality, Resemiotization: Extending the Analysis of Discourse as Multi-Semiotic Practice.” Visual Communication 2 (1): 29-57.

JEWITT, Carey. 2014a. “Different Approaches to Multimodality.” In Jewitt 2014b, 28-39.

—, ed. 2014b. The Routledge Handbook of Multimodal Analysis. London and New York: Routledge.

JUST, Marcel A. and Patricia A Carpenter. 1976. “Eye Fixations and Cognitive Processes.” Cognitive Psychology 8 (4): 441-80.

KRESS, Gunther. 2010. Multimodality: A Social Semiotic Approach to Contemporary Communication. London and New York: Routledge.

—. 2012. “Multimodal Discourse Analysis.” In Gee and Handford 2012, 35-50.

KRESS, Gunther and Theo van Leeuwen. 1996. Reading Images: The Grammar of Visual Design. London and New York: Routledge.

—. 2001. Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.

MANUSOV, Valerie L. and Miles L. Patterson. 2006. The SAGE Handbook of Nonverbal Communication. London: Sage.

MCNEILL, David. 1992. Hand and Mind: What Gestures Reveal about Thought. Chicago, IL: U of Chicago P.

MORELL, Teresa. 2018. “Multimodal Competence and Effective Interactive Lecturing.” System 77: 70-79.

NORRIS, Sigrid. 2004. Analyzing Multimodal Interaction. A Methodological Framework. London and New York: Routledge.

O’HALLORAN, Kay, ed. 2004. Multimodal Discourse Analysis: Systemic Functional Perspectives. London: Continuum.

—. 2011. “Multimodal Discourse Analysis.” In Hyland and Paltridge 2011, 120-37.

O’HALLORAN, Kay et al. 2012. “Interactive Software for Multimodal Analysis.” Visual Communication 11 (3): 363-81.

PAK-HIN KONG, Anthony et al. 2015. “A Coding System with Independent Annotations of Gesture Forms and Functions during Verbal Communication: Development of a Database of Speech and GEsture (DoSaGE).” Journal of Nonverbal Behavior 39 (1): 93-111.

QUEROL-JULIÁN, Mercedes and Inmaculada Fortanet-Gómez. 2012. “Multimodal Evaluation in Academic Discussion Sessions: How Do Presenters Act and React?” English for Specific Purposes 31: 271-83.

ROUNDS, Patricia L. 1987. “Characterizing Successful Classroom Discourse for NNS Teaching Assistant Training.” Tesol Quarterly 21 (4): 643-69.

SANCHO-GUINDA, Carmen, ed. 2019. Engagement in Professional Genres: Deference and Disclosure. Amsterdam and Philadelphia: John Benjamins.

SCOLLON, Ron. 2001. Mediated Discourse: The Nexus of Practice. London: Longman.

TAN, Sabine, Kay O’Halloran and Peter Wignell. 2016. “Multimodal Research: Addressing the Complexity of Multimodal Environments and the Challenges for CALL.” ReCALL 28 (3): 253-73.

THESEN, Lucia. 2016. “The Past in the Present: Modes, Gaze and Changing Communicative Practices in Lectures.” In Archer and Breuer 2016, 31-52.

VAN LEEUWEN, Theo. 2005. Introducing Social Semiotics. London and New York: Routledge.

—. 2008. Discourse and Practice: New Tools for Critical Discourse Analysis. Oxford: Oxford UP.

WITTENBURG, Peter et al. 2006. “ELAN: A Professional Framework for Multimodality Research.” In Calzolari et al. 2006, 1556-59.

YALE UNIVERSITY. 2021. Yale University’s OpenCourseWare. [Accessed February 9, 2021].