How to use video to study body language, verbal and non-verbal communication

If you've been following the recent debate about some of the more outrageous claims about non-verbal behaviour and body languge (e.g. HERE and HERE), you may have been wondering why I think that so much research in this area falls short of the methodological constraints associated with the approach of conversation analysis.

There’s a long version, that has to do with the theoretical debates taking place within sociology during the 1960s–70s that I certainly don’t intend to get into here.

There’s also a much shorter version that has to do with the way conversation analysis offered a viable alternative for any researcher with doubts about sociology’s heavy reliance on surveys and official statistics, psychology’s equally heavy reliance on artificially contrived experiments and linguistics’s use of invented examples, and/or denying that there’s any point at all in looking at how language actually works (e.g. Chomsky).

What conversation analysts did was to move away from the previously dominant hypothetico-deductive model of science towards a much more inductive alternative of the kind that had given birth to ethology in biology/zoology.

This was made possible by what I believe will eventually recognised as having been as important for our understanding of talk and interaction as the invention of the telescope had been for understanding astronomy: by the late 1960s, high quality audio and video recording technology meant that anyone could record real everyday talk, put it under the microscope and study it at a level of detail that had never before been possible. And the first people to do so were the founders of conversation analysis.

When I wrote Our Masters’ Voices (1984), I was aiming at more general readership than the professional academic community of sociologists, psychologists and linguists. But I still thought it necessary to say at least something about the observational methodology that had made the findings possible.

The most important elements of this approach to observation were that (a) the researcher’s claims are severely constrained by what’s there in the empirical data (so you can’t just speculate, say whatever you like or make it up), and (b) any claims that you do make can be checked out by anyone else who has access to the same data (which is about as powerful a form of verification as there is anywhere else in the social and behavioural sciences).

The following refers to speeches because that's what the book was about, But I still believe, as I did then, that the methodology can be used to study pretty well any data on human interaction and communication that’s been recorded on audio or video tape:


Once a speech has been recorded, it can be studied with all the advantages that television viewers of an action replay of a sporting incident have over those who actually saw it happening live. Unlike them, television viewers get a chance to look at it again and again. Finer points that may have been missed the first time around are brought into sharper focus as the action is replayed, slowed down, or frozen for even closer inspection.

While footballers and spectators may know who scored a goal, they often have no more than a vague impression of the events leading up to it. By contrast, viewers of the action-replay can track the sequence as it unfolds and see exactly how the different actions were organised and combined to produce the goal. They are therefore in a far better position to understand how a particular move worked than those who saw it only once.

All this applies equally to the study of any other form of human behaviour that can be preserved on video tape, including the behaviour of politicians. If, for example, the saying of something which results in applause is to the orator what scoring a goal is to a footballer, then action replays can be put to work in a similar way by looking to see how the words, gestures and other bodily movements combined together to produce the desired response.

Another well known feature of the action replay is also crucially important for the way the observations can be read and evaluated. Replays of sporting incidents are almost always accompanied by more commentary from the commentator(s) on the events we are seeing again, the object of the exercise being to supply a more detailed and informed analysis.

But viewers can also see the sequence of events just as well for themselves, and are therefore in a position to draw their own conclusions about what actually happened. This means that they can also judge the adequacy or otherwise of the commentator's description and analysis.

If the commentator’s claims about how the event occurred are out of line with what the viewers saw, the television company's switchboards will be jammed within minutes. And, if a sports commentator persists in making excessively personal, subjective or eccentric observations about what he (and everyone else) is seeing, he’s unlikely to hold down his job for very long.