The Epistemic Reverse Side of Instrumental Images

Project description

Principal Investigator: Sigrid Weigel (ZfL)
Associated Investigator: Stefan Zachow (ZIB)
Associated Member: Margarete Vöhringer (Universität Göttingen)
PhD Candidates: Martin Grewe (ZIB), Lisa Schreiber (ZfL)
Design associates: Lena Galitsch (ZfL), Moritz Wehrmann (ZfL)

Image Knowledge Gestaltung. An Interdisciplinary Laboratory
Complex problems cannot be solved within the boundaries of a single scientific discipline. They require the knowledge and expertise of researchers from various fields of science who are united in a Cluster. The individual disciplines themselves benefit here from a subsequent strengthening and enrichment. The interdisciplinary laboratory Image Knowledge Gestaltung is one such union of social, natural and technological sciences, medicine, and – for the first time in basic research – the disciplines of design and architecture as well. In this interdisciplinary laboratory, more than 25 different disciplines come together to investigate the fundamental Gestaltung processes of the sciences.

The Epistemic Reverse Side of Instrumental Images (base project)

Instrumental images are used as tools by human agents in fundamental research, therapy, and medical practice, and the base project examines them in terms of the knowledge that their ›reverse sides‹ impart. This concealed knowledge includes, for example, infirm concepts and open-ended research questions that were relevant at the time the image was created. Hidden from the eye, this knowledge has thus faded from use in practical applications. The images under examination depict faces and hands. Two subprojects grounded in the history of science and the history of images concentrate (1) on pictures used in the field of hand surgery (a cultural, medical, and pictorial history of the hand) and (2) on the psychological research done on facial expressions (a scientific and media history of FACS). The projects further share an interest in the interrelations between drawings, schemas, and digital images. An empirical subproject (3) seeks to construct a face database to serve as an archive of facial expressions using three-dimensional images from facial morphology. The database will benefit from the accompanying work being done on the relationships between image technology, interpretive patterns, and the epistemic problems surrounding the semiotics of facial expressions (as pertaining to the history of FACS and critical responses to it). The concrete objective is a digital atlas of facial expressions that surpasses a logic of simply coding expressions. The atlas will ultimately act as a collection point for the findings.

The project is part of Humboldt University's Cluster of Excellence Image Knowledge Gestaltung. An Interdisciplinary Laboratory 2013–2018


Instrumental Images in Hand Surgery: From Hand-Drawn to Digital Images

Associate Researcher(s): Nina Samuel

The hand, Aristotle writes, is »the instrument of all instruments « (instrumentum instrumentorum). More recently, this body part has become central to theoretical debates thanks to smartphones, tablets, and other new technology (Ruf). While case studies of the hand exist in the fields of medical, cultural, and art history (Leroi-Gourhan; Gadebusch Bondio), a study of the hand from an image studies perspective has not yet been undertaken. This project takes the hand as its object of study in order to fill this important gap in research.

The project examines the hand as a dialectical object between form and function and addresses questions concerning the hand as a figure in drawings and instrumental imaging in the context of hand surgery and its emergence as a unique discipline. The study will also look at the field’s epistemological prehistory together with its role in the history of images. Other important areas to be touched upon are technological developments in medicine and how they relate to advancements in arms technology as well as the geometrization and disciplining of the body with regard to both Early Modern theoretical debates on art and architecture and current issues concerning changes in media. In addition, the project will analyze the practice of drawing as an epistemic tool and its embeddedness in pictorial practices used in scientific research and other realms of collecting, contrasting, and organizing information. The study also looks at exchanges between different media and drawing, for example, photography (slides, x-rays), print media such as newspaper pictures, and reproductions of artworks. Informed by the theory of drawing (Kemp) and the history of hand-related knowledge (Sherman), the project utilizes the posthumous archive of the hand surgeon J. William Littler as the basis for investigations into the material culture and pictorial practices of hand surgery. This archive contains images of the hand used for research and in patient consultations. Key questions ask about the concept of »style« in surgery, the interrelationship between photography and hand-drawn pictures, as well as how changes in media technology play a role in the use of images in hand surgery.

Coding Feelings. The History of Knowledge of the FACS in Affective Computing

Dissertation project 2013–2018
Associate Researcher(s): Lisa Schreiber

Within the research area Affective Computing digital devices have been developed which automatically capture and calculate emotions based on physical  patterns. Based on large databases the new technologies promise to capture a wide range of bodily expressions and ›true‹ emotional experience. The Facial Action Coding System (FACS) established in 1978 is used as standard tool of capturing emotional expressions. However, the FACS reduces emotional patterns to only six basic emotions and the muscular movements of  the face.

From a media and culture studies perspective, the dissertation project examines the use of empirical and image based methods that capture emotions in psychology laboratories from the mid-20th century. In addition, the dissertation analyzes digital tools that capture emotions in the field of Affective Computing with attention to specific cases in the field of psychology in which digital tools are used as with cyberpsychology and the treatment of autism spectrum disorders. The overarching goal is to investigate the lingering effects of historical knowledge within the realm of current affect studies as they pertain to the development of FACS in the age of Affective Computing.

Building a Database of Facial Expressions in 3D

Dissertation project 2013–2018

Associate Researcher(s): Martin Grewe (ZIB)

Models of human facial morphology and its dynamics provide the foundations for developing computer-based tools to plan and analyze facial surgery. They are also crucial in the neurocognitive study of how facial expressions are perceived and processed. Medical and neuroscientific applications have long used models of facial expression such as the Facial-Action-Coding-System. However, these models pose significant limitations: In addition to promoting a categorical coding system and a terminological and methodological conflation of emotion with induced facial expressions, they are also restricted by their primarily qualitative description of morphological patterns of expression (e.g., the FACS uses photographs of faces as examples). Furthermore they reduce facial dynamics to mere muscular movement. New applications and questions, however, call for a quantitative model of dynamic facial forms that is better able to take into account subtle differences.

The dissertation project therefore focuses on establishing a highly detailed 3D face and expression database (the ”Mimik Archiv”) intended to support the statistical analysis of the quantitative characteristics of facial morphology. The resulting morphological patterns make it possible to generate a combined model of facial expressions, called the »Mimik-Atlas«. It is intended to serve as a tool for medical and neuroscientific applications.

The first phase of the project aims to develop and improve existing stereo-photogrammetric methods for the acquisition of 3D facial morphology. The Camera Facialis project at the Konrad-Zuse-Center for Information Technology (ZIB) created a multi-camera setup that makes it possible to precisely measure the 3D geometry and colors of facial surfaces within a fraction of a second. The level of detail contained in the digital models, both in terms of geometric and photometric data, surpasses the capacities of currently available systems. Over the course of the project, the Camera Facialis will be used to collect a wide database of facial expressions.

Further research will be concerned with the development of new methods for the automatic processing of newly captured facial expression data. The goal is to enable the rapid integration of digital 3D models into the archive as well as to facilitate retrieval and analysis of the stored morphological data.


Sigrid Weigel (ed./eds.)

Das Gesicht
Bilder, Medien, Formate

Wallstein Verlag, Göttingen 2017, 207 pages
ISBN: 978-3-8353-3146-4
Nina Samuel

Die Form des Chaos
Bild und Erkenntnis in der komplexen Dynamik und der fraktalen Geometrie

Wilhelm Fink Verlag, München 2014, 551 pages
ISBN: 978-3-7705-5776-9

Carl Martin Grewe
(with Hans Lamecker, Stefan Zachow): Landmark-based Statistical Shape Analysis, in: Michael Hermanussen (ed.), Auxology – Studying Human Growth and Development. Stuttgart: Schweizerbart Verlag 2013, p. 200–203

(with Hans Lamecker, Stefan Zachow): Digital morphometry: The Potential of Statistical Shape Models, in: Anthropologischer Anzeiger. Journal of Biological and Clinical Anthropology Vol. 68, No. 4 (2011), p. 506


Nina Samuel
Images as tools. On visual epistemic practices in the biological sciences, in: Studies in History and Philosophy of Biological and Biomedical Sciences, Vol. 44, Issue 2 (June 2013), p. 225–236


Lisa Schreiber
Gefühls-Montagen. Fotografisch-psychologische Praktiken der Aufzeichnung von Emotionen bei Paul Ekman, in: Fotogeschichte. Beiträge zur Geschichte und Ästhetik der Fotografie. Die Seele in der Silberschicht? Fotografische Praktiken in der Psychologie der Moderne, ed. by David Keller, Steffen Siegel, Vol. 36, No. 140 (Summer 2016), p. 49-58

(with Carl Martin Grewe): Digitale Bildarchive. Archivierung und Codierung der Gefühle, in: Nikola Doll, Horst Bredekamp, Wolfgang Schäffner (eds.), +ultra. gestaltung schafft wissen, Leipzig: E. A. Seemann 2016, p. 280–285

Advanced Wellbeing: Digitale Techniken der Vermessung von Affekten in der Cyberpsychologie, in: Mediale Kontrolle unter Beobachtung. Kulturwissenschaftliche Perspektiven auf die strittige Gestaltung unserer Kommunikation (Online journal), No. 5.1 (2016): Privatheit und Quantifizierbarkeit, ed. by Martin Degeling, Florian Püschel and Eva Schauerte

(with Carl Martin Grewe, Stefan Zachow): Fast and Accurate Digital Morphometry of Emotional Expressions for Facial Surgery, in: Facial Plastic Surgery, Vol. 31, No. 5 (2015), p. 431–438


Margarete Vöhringer
Die Maske als Medium. Zur Gesichtsdarstellung im frühen Film, in: Tobias R. Klein and Erik Porath (eds.), Kinästhetik und Kommunikation. Ränder und Interferenzen des Ausdrucks. Berlin: Kulturverlag Kadmos 2013, S. 328–340


Sigrid Weigel
Gesichter – Zwischen Spur und Bild, Codierung und Vermessung, in: Sigrid Weigel, Grammatologie der Bilder, chapter 3. Berlin: Suhrkamp 2015, p. 70–137

Phantom Images: Face and Feeling in the Age of Brain Imaging, in: Zeitschrift für Kunst und Kulturwissenschaften. Kritische Berichte, No. 1 (2012), p. 33–53


Lecture at the Museum für Kommunikation
14 Feb 2017 · 6.30 pm

Lisa Schreiber: Abschied von der Weltformel? Wie neue digitale Techniken Individualität und Vielfalt von Gesichtsausdrücken erfassen

Museum für Kommunikation, Leipziger Str. 16, 10117 Berlin

read more
Präsentation im Martin-Gropius Bau
09 Nov 2016 · 4.30 pm

Digitale Erfassung von Gesichtsausdrücken im Affective Computing

Martin-Gropius-Bau, Niederkirchnerstraße 7, 10963 Berlin

read more
Präsentation im Martin-Gropius Bau
02 Nov 2016 · 4.30 pm

Digitale Erfassung von Gesichtsausdrücken im Affective Computing

Martin-Gropius-Bau, Niederkirchnerstraße 7, 10963 Berlin

read more
15 Mar 2016 · 10.30 am

Gesichtsausdrücke. Messen, Analysieren, Deuten

Urania Berlin, An der Urania 17, 10787 Berlin

read more
20 Nov 2015 – 21 Nov 2015

2. Jahrestagung des Interdisziplinären Labors Bild Wissen Gestaltung

Berlin-Brandenburgische Akademie der Wissenschaften Jägerstr. 22/23, 10117 Berlin

read more
17 Feb 2015

Facial Expressions: Technologie – Experiment – Codierung

ZfL, Schützenstr. 18, 10117 Berlin, 3. Et.

read more
15 Nov 2014

1. Jahrestagung des Interdisziplinären Labors Bild Wissen Gestaltung

Berlin-Brandenburgische Akademie der Wissenschaften Jägerstr. 22/23, 10117 Berlin

read more
Lange Nacht der Wissenschaften
10 May 2014 · 5.00 pm

Camera Facialis: Ein 3D-Portrait-Studio

Konrad-Zuse-Zentrum für Informationstechnik (ZIB), Takustr. 7 (Zugang auch von Arnimallee 6 und Altensteinstraße 23), 14195 Berlin, Foyer/Bibliothek

read more
Interdisziplinäre Kontroverse mit Sigrid Weigel und Peter Deuflhard (FU Berlin)
12 Dec 2013 · 6.30 pm

»Code« aus Sicht der Kulturwissenschaft und der Mathematik

Interdisziplinäres Labor Bild Wissen Gestaltung, Sophienstr. 22a, 2.HH, 3.OG, 10178 Berlin

read more
Eröffnungsveranstaltung mit Workshops, Experimenten u. Festakt
01 Jun 2013 · 2.00 pm

Das Interdisziplinäre Labor: Bild Wissen Gestaltung

HU Berlin, Sophienstraße 22a und Haus der Kulturen der Welt, John-Foster-Dulles-Allee 10, Berlin

read more

Media Response

14 Feb 2017
Können Computer Emotionen erkennen?

Radio talk by Irina Grabowski with Lisa Schreiber, in: RBB Inforadio, Sendung: WissensWerte, 14 Feb 2017, 10.25am