Data and Interpretation: Digital Humanities and the missing link with Multimodal Semiotics

Abstract

Digital Humanities (DH) has established itself worldwide as a rapidly expanding web of activities with many research centres and international associations. The field has both practical and more theoretical, philosophical dimensions. On the one hand, the explosive growth in ‘data’ of all kinds, and from all areas, raises the need to manage and support interaction with that data. On the other hand, the driving theoretical notion of DH is that (usually large) bodies of digitally managed data can be leveraged for research questions of interest to, and in the best case defined by, the humanities. ‘Doing humanities digitally’ is thus often proposed as opening up new methods and possibilities for hermeneutic interpretative inquiry.

In this talk I offer a brief sketch of some current initiatives and directions taken in DH, suggesting something of the breadth of data, models, infrastructures and methods under development and giving pointers for further information. The very breadth of this enterprise is not, however, unproblematic: accounts too often end up limiting themselves to suggestive visualisations of sets of formal features. Questions of which formal features may be significant and why demand closer engagement with more humanistic notions of data interpretation. I conclude the talk, therefore, arguing this to be a major challenge, perhaps the major challenge, now facing the future development of DH. I do this by drawing on a complementary approach to meaning in diverse media proposed more recently within (multimodal) semiotics.


About the Speaker

John Bateman is Professor for Applied Linguistics in the English and Linguistics Departments of Bremen University. He studied linguistics and computer studies at Lancaster University and obtained his PhD at Edinburgh University in Artificial Intelligence within the ‘Epistemics’ postgraduate programme combining AI, linguistics, psychology, and philosophy. After working and researching on computational linguistics, multilingual and multimodal document generation, and formal ontology at the universities of Kyoto, Southern California, Saarbrücken, and Stirling and the Gesellschaft für Mathematik und Datenverarbeitung (GMD) in Darmstadt, he joined Bremen University in 1999.
Over the past 15 years his research has focused increasingly on communication that combines diverse forms of expression, including verbal language, images, moving images, image sequences, sound and movement, in all cases applying and extending models of textuality generalised from linguistics and semiotics. 2017 saw the publication of a broad transdisciplinary textbook on the theory and practice of multimodality research as a field of study in its own right (with co-authors Janina Wildfeuer and Tuomo Hiippala). Current work includes expanding notions of semiotics to combine discourse and embodiment (in the context of Human-Robot-Interaction in the ‘Everyday Activity Science and Engineering’ SFB of the university), broadening notions of textuality and narrative for diverse media (for example in the university’s interdisciplinary and collaborative research platform ‘World of Contradictions’ Lab on multimodal narrative), applying principles of formal ontology to the modelling of semantics for language and other modalities, and pursuing more robust empirical connections between data and theory for diverse media.

Speaker:
Prof. Dr. John Bateman

---

The Data Science Center is funded by: