Combining Multimodal Techniques to Approach the Study of Academic Lectures: A Methodological Reflection
AbstractThis article offers a methodological reflection on the use of multimodal techniques for the study of academic lectures. Three distinct multimodal approaches have been put forward to explore the use of language holistically, namely, multimodal social semiotics (MSS), multimodal discourse analysis (MDA) and multimodal interaction analysis (MIA). These approaches differ in their main focus—the social context, the system of semiotic resources available to the speakers and the social actors, respectively—and the tools they provide to conduct multimodal analyses. To exemplify how analyses may be conducted within each of the paradigms in the context of academic lectures in English, I examine an excerpt extracted from an African-American history lecture from Yale University by a native English speaker in which he organizes his discourse in between content sections. Through the use of short multimodal transcriptions, I discuss how MSS can be used for reflections on the social contexts of academic lectures, MDA describes the use of semiotic resources employed by the lecturers, and MIA can be used to look into how lecturers structure their speech into sequences of actions. Ultimately, I suggest a combination of multimodal methodologies to obtain a broader account of the intricacies of discourse in academic settings.
ADAMI, Elisabetta. 2017. “Multimodality.” In García, Flores and Spotti 2017, 451-72.
ARCHER, Arlene and Esther Odilia Breuer, eds. 2016. Multimodality in Higher Education. Leiden: Brill.
BERNAD-MECHÓ, Edgar. 2017. “Metadiscourse and Topic Introductions in an Academic Lecture: A Multimodal Insight.” Multimodal Communication 6 (1): 39-60.
—. 2018. “A Multimodal Approach to Metadiscourse as an Organizational Tool in Lectures.” PhD diss., University Jaume I.
BERNAD-MECHÓ, Edgar and Inmaculada Fortanet-Gómez. 2019. “Organizational Metadiscourse across Lecturing Styles: Engagement beyond Language.” In Sancho-Guinda 2019, 321-40.
BIBER, Douglas, Susan Conrad and Viviana Cortés. 2004. “ʻIf You Look at...ʼ: Lexical Bundles in University Teaching and Textbooks.” Applied Linguistics 25 (3): 371-405.
BONSIGNORI, Veronica and Belinda Crawford Camiciottoli, eds. 2016. Multimodality across Communicative Settings, Discourse Domains and Genres. Newcastle upon Tyne: Cambridge Scholars.
BUCK, Gary. 2001. Assessing Listening. Cambridge: Cambridge UP.
CALZOLARI, Nicoletta et al., eds. 2006. Proceedings of LRforthEC 2006, Fifth International Conference on Language Resources and Evaluation. Genoa: European Language Rescources Association.
CRAWFORD CAMICIOTTOLI, Belinda. 2015. “Elaborating Explanations during OpenCourseWare Humanities Lectures: The Interplay of Verbal and Nonverbal Strategies.” In Crawford Camiciottoli and Fortanet-Gómez 2015, 144-70.
—. 2016. “A Multimodal Analysis of Interaction in Academic Lectures: A Case Study.” In Bonsignori and Crawford Camiciottoli 2016, 65-92.
CRAWFORD CAMICIOTTOLI, Belinda and Inmaculada Fortanet-Gómez, eds. 2015. Multimodal Analysis in Academic Settings: From Research to Teaching. London and New York: Routledge.
DEPPERMANN, Arnulf, ed. 2013. “Conversation Analytic Studies of Multimodal Interaction.” Special issue, Journal of Pragmatics 46 (1).
DUDLEY-EVANS, Tony. 1994. “Variations in the Discourse Patterns Favoured by Different Disciplines and Their Pedagogical Implications.” In Flowerdew 1994, 146-58.
FLOWERDEW, John, ed. 1994. Academic Listening: Research Perspectives. Cambridge: Cambridge UP.
FLOWERDEW, John and Lindsay Miller. 1992. “Students’ Perceptions, Problems and Strategies in L2 Lectures.” RELC Journal 23 (12): 60-80.
FORCEVILLE, Charles J. and Eduardo Urios-Aparisi, eds. 2009. Multimodal Metaphor. Berlin: De Gruyter.
FORTANET-GÓMEZ, Inmaculada. 2004. “The Use of ‘We’ in University Lectures: Reference and Function.” Journal of English for Specific Purposes 23: 45-66.
FORTANET-GÓMEZ, Inmaculada and María Noelia Ruiz-Madrid. 2014. “Multimodality for Comprehensive Communication in the Classroom: Questions in Guest Lectures.” Ibérica 28: 203-24.
FOX, Janna and Natasha Artemeva. 2013. “The Cinematic Art of Teaching University Mathematics: Chalk Talk as Embodied Practice.” Multimodal Communication 1 (1): 83-103.
GARCÍA, Ofelia, Nelson Flores and Massimiliano Spotti, eds. 2017. The Oxford Handbook of Language and Society. Oxford: Oxford UP.
GEE, Paul and Michael Handford, eds. 2012. The Routledge Handbook of Discourse Analysis. London and New York: Routledge.
GUNEL, Murat, Brian Hand and Sevket Gunduz. 2006. “Comparing Student Understanding of Quantum Physics When Embedding Multimodal Representations into Two Different Writing Formats: Presentation Format versus Summary Report Format.” Science Education 90 (6): 1092-112.
HALLIDAY, M. A. K. 1978. Language as a Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold.
—. 1985. An Introduction to Functional Grammar. London: Edward Arnold.
HYLAND, Ken and Brian Paltridge, eds. 2011. Companion to Discourse. London: Continuum.
IEDEMA, Rick. 2003. “Multimodality, Resemiotization: Extending the Analysis of Discourse as Multi-Semiotic Practice.” Visual Communication 2 (1): 29-57.
JEWITT, Carey. 2014a. “Different Approaches to Multimodality.” In Jewitt 2014b, 28-39.
—, ed. 2014b. The Routledge Handbook of Multimodal Analysis. London and New York: Routledge.
JUST, Marcel A. and Patricia A Carpenter. 1976. “Eye Fixations and Cognitive Processes.” Cognitive Psychology 8 (4): 441-80.
KRESS, Gunther. 2010. Multimodality: A Social Semiotic Approach to Contemporary Communication. London and New York: Routledge.
—. 2012. “Multimodal Discourse Analysis.” In Gee and Handford 2012, 35-50.
KRESS, Gunther and Theo van Leeuwen. 1996. Reading Images: The Grammar of Visual Design. London and New York: Routledge.
—. 2001. Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.
MANUSOV, Valerie L. and Miles L. Patterson. 2006. The SAGE Handbook of Nonverbal Communication. London: Sage.
MCNEILL, David. 1992. Hand and Mind: What Gestures Reveal about Thought. Chicago, IL: U of Chicago P.
MORELL, Teresa. 2018. “Multimodal Competence and Effective Interactive Lecturing.” System 77: 70-79.
NORRIS, Sigrid. 2004. Analyzing Multimodal Interaction. A Methodological Framework. London and New York: Routledge.
O’HALLORAN, Kay, ed. 2004. Multimodal Discourse Analysis: Systemic Functional Perspectives. London: Continuum.
—. 2011. “Multimodal Discourse Analysis.” In Hyland and Paltridge 2011, 120-37.
O’HALLORAN, Kay et al. 2012. “Interactive Software for Multimodal Analysis.” Visual Communication 11 (3): 363-81.
PAK-HIN KONG, Anthony et al. 2015. “A Coding System with Independent Annotations of Gesture Forms and Functions during Verbal Communication: Development of a Database of Speech and GEsture (DoSaGE).” Journal of Nonverbal Behavior 39 (1): 93-111.
QUEROL-JULIÁN, Mercedes and Inmaculada Fortanet-Gómez. 2012. “Multimodal Evaluation in Academic Discussion Sessions: How Do Presenters Act and React?” English for Specific Purposes 31: 271-83.
ROUNDS, Patricia L. 1987. “Characterizing Successful Classroom Discourse for NNS Teaching Assistant Training.” Tesol Quarterly 21 (4): 643-69.
SANCHO-GUINDA, Carmen, ed. 2019. Engagement in Professional Genres: Deference and Disclosure. Amsterdam and Philadelphia: John Benjamins.
SCOLLON, Ron. 2001. Mediated Discourse: The Nexus of Practice. London: Longman.
TAN, Sabine, Kay O’Halloran and Peter Wignell. 2016. “Multimodal Research: Addressing the Complexity of Multimodal Environments and the Challenges for CALL.” ReCALL 28 (3): 253-73.
THESEN, Lucia. 2016. “The Past in the Present: Modes, Gaze and Changing Communicative Practices in Lectures.” In Archer and Breuer 2016, 31-52.
VAN LEEUWEN, Theo. 2005. Introducing Social Semiotics. London and New York: Routledge.
—. 2008. Discourse and Practice: New Tools for Critical Discourse Analysis. Oxford: Oxford UP.
WITTENBURG, Peter et al. 2006. “ELAN: A Professional Framework for Multimodality Research.” In Calzolari et al. 2006, 1556-59.
YALE UNIVERSITY. 2021. Yale University’s OpenCourseWare. [Accessed February 9, 2021].
The authors retain copyright of articles. They authorise AEDEAN to publish them in its journal Atlantis and to include them in the indexing and abstracting services, academic databases and repositories the journal participates in.
Under the terms of the Creative Commons Attribution NonCommercial ShareAlike 4.0 International Licence (CC BY-NC-SA 4.0), for non-commercial (i.e., personal or academic) purposes only, users are free to share (i.e., copy and redistribute in any medium or format) and adapt (i.e., remix, transform and build upon) articles published in Atlantis, free of charge and without obtaining prior permission from the publisher or the author(s), as long as they give appropriate credit to the author, the journal (Atlantis) and the publisher (AEDEAN), provide the relevant URL link to the original publication and indicate if changes were made. Such attribution may be done in any reasonable manner, but not in any way that suggests the journal endorses the user or their use of the material published therein. Users who adapt (i.e., remix, transform or build upon the material) must distribute their contributions under the same licence as the original.
Self-archiving is also permitted, so that authors are allowed to deposit the published PDF version of their articles in academic and/or institutional repositories, without fee or embargo. Authors may also post their individual articles on their personal websites, again on condition that the original link to the online edition is provided.
Authors are expected to know and heed basic ground rules that preclude simultaneous submission and/or duplicate publication. Prospective contributors to Atlantis commit themselves to the following when they submit a manuscript:
- That no concurrent consideration of the same, or almost identical, work by any other journal and/or publisher is taking place.
- That the potential contribution has not appeared previously, in any form whatsoever, in another journal, electronic format or as a chapter/section of a book.
Seeking permission for the use of copyright material is the responsibility of the author.