Visualising and Reality

'Visualisation' (or 'visualization') is, it seems, a difficult and problematic word. It is worth clarifying the meanings that have developed in various research contexts because of the potential for confusion in its application, and also to note a distinction in how this word is applied differently in different research environments. This is important because of the increasing overlapping of disciplines for which this word is a frequent verb and sometimes a noun, but also because the different methods of inquiry of some of these areas is intellectually incommensurate, however visually satisfying or apparently adjacent the supporting technology may be.

For a start, it is difficult to spell. 'Visualization', an Americanism that appears to mean the same thing, is somewhat rampant in its application by British-based colleagues whom I hope will excuse my use of the English spelling. There may, however, be a case for developing a distinction that we might base on the spelling if only clarify our intentions when invoking it as a concept. This is because of the crude duality of its meaning to scientists and artists. For artists, whether they work in digital technologies or not , 'visualisation' suggests the reification of a vision, to visualise an idea or a thought according to their expressive needs. Artists refer to 'visualisation' or 'visualising' as a verb, the act of art-making or image creation. For those using the word in this way, the assumption of a visualisation practice is to bring into being something new and original. Their concerns are based on whether the resulting image, or the successive approximations they are developing, contain the spirit of their idea, and genuinely represent their 'vision'; the motivation for making the art or image in the first place.

In the scientific world the word is used quite differently. The 'visualisation' process is also a product, given that 'visualisation' in the domain of science often seeks to create a comprehensible model of an otherwise complex subject. 'Visualisation' becomes the way in which scientific information can be communicated, sometimes with dramatic consequences or in crucial situations. The purpose of 'visualisation' in this context is less to do with originality and more to do with fidelity to a research concept or a body of evidence. 'Visualisation' technologies are employed at the point where the data is too complex for users to grasp or master quickly without support, or where the conditions are too unstable or remote to allow for the proper time for contemplation. Given the impressive data volumes that can now be extracted from single molecule for example, 'visualisation' is an important tool in presenting research outcomes and making them accessible. The most famous of these examples remains the DNA double helix, extrapolated from Watson & Crick's stick and ball models in their famous 1953 paper. It is interesting to note the differences between Watson & Crick's original physical model and any of the vast number of visualisations made of DNA since, of which a few examples are illustrated.

The original DNA model by Subsequent 2D models freely available online.
Watson and Crick.
Photo: Cold Spring Harbor
Laboratory Archives

A 'visualisation', using the word as a noun, is intended to clarify the research process by accurately rendering something into visual form. It is meant to provide a model of reality, and when it fails in this task it is thought to be thoroughly useless. For the scientist then, the reliability of the visualisation depends on the accuracy of the data that supports it and the fidelity of the translation of that data into a visual form.

These two applications of this word in the research environment lead to some questions about what and how we make 'visualisations', in either definition. The most pressing appears to be something well known to artists about the visual world and where their expertise can be of significant value to the 'visualisation community'. The capacity to be deceived by what we see is a quality exploited in the visual arts for some time. However, the very portability of visual systems to a large number of paradigms creates an illusion of usefulness. The accuracy of the DNA model cannot be assessed by recourse to the beauty of its structure or it's use of colour (normally the satisfying primaries plus green) to validate how things are in reality. In this I am reminded of the excellent model of the Melnikov House, made by ParallelGraphics and available to view at:

http://www.parallelgraphics.com/products/isb/examples/melnikov/

provenance, there is an implicit suggestion that somehow there is a specific state of This demonstration of the 3D graphics that can be developed with Cortona is hugely impressive, and must have seemed an excellent subject for demonstrating Cortona's utility in bringing understanding to an otherwise obscure, yet deserving, architectural monument. Melnikov's famous house in central Moscow is not particularly accessible, has suffered from lack of maintenance and is squeezed between other apartment blocks all around, and on a narrow little street. In the Cortona version, the viewer can manipulate the building at will, or change their own perspective of the site, flying through walls and floors, looking down ceilings and tilting the building in all kinds of ways. The problem with this usage of visualisation technologies is two-fold. First, they are convincing enough in detail and experience to appear to render a site inspection obsolete, and secondly that they suggest a way of experiencing the building that is simply not possible. In the first case, many of the details are actually incorrect or based on supposition. Even if the data source can be given a proper provenance, and thus authority to represent, the representation suggests a finite state to Melnikov's building, and we know from his writing that this simply is not the case, with moveable windows and floors. One of the issues is that unlike the pristine presentation of the model, a building must exist in a physical location that is effectively changing it. The relative instability of this, in relation to a representation of an absolute state in a visualisation, suggests the misrepresentation of the building as an entity. Secondly, the problem with visual representations without the constraints of the physical world is that they can mislead us about what it is like to stand there in front of a building, or walk its stairs or see it from the angles it's architect anticipated when he designed it. Melnikov's visualisation of an architectural concept gives rise to misinterpretations through the technology of visualisation, an especially dangerous situation should the visuals be quite so convincing as they are here, replacing the complicated experience of actually going to Moscow and getting to see it in person.

In all these approaches, it is striking to note the authority of digital technology in determining our view of the world in general and of the visualised one in particular. The complexities of the technologies and their flexibility in application lead us, as viewers hopelessly well trained by television, into believing that what we see is a more accurate representation of the world than the world itself might present. Visualisation is invariably used as suggested above, when the data is too complex or the situation too remote for our first hand assessment of the reality that it purports to represent. When it does so, it creates a persuasive and plausible argument in and of itself that it has been capable of capturing the data that can give us a model of reality. As has been suggested here, this is sometimes not quite what it seems, and that what we really have is the reality of the model and the model-making process in front of us. The visual arts, with its rich experience of understanding the impressions made by visuality, has much to offer the complex practices of 'visualisation', in terms of helping to apply a critical eye to the representations of the world around us as they manifest in the powerful and persuasive technologies of visualisation.

Gregory Sporton
August 2007

Transcirpt of Presentation at VizNet event, Birmingham

19th June, 2007

Gregory Sporton

Ok I guess most of you will already have seen the presentation I gave last time at Loughborough, and I thought today I would just have a look at one or two things. One issue that for me seems to come out fairly regularly, as people know my background is, I work here in the department of art, and I thought it was worth looking at something like this. This is a VRML reconstruction of Konstantin Melnikov, the Dom Melnikov, I don’t know if you know this building but it is in Moscow, and it is not one of my own I will hasten to add,

VRML model of Melnikov House

but it is extremely interesting because I am able to take this very nice building and turn it around and I can do all this fun stuff with it. You can see around the back there these fantastic windows and Melnikov was a constructivist who worked just after the revolution, slightly before, but mostly after the revolution, up until about 1930 when Stalin put a stop to him being an architect but in fact this is a very important historical building as well as a very beautiful aesthetic building, given that it was one of the few private houses, or one of the few homes in private ownership in the former Soviet Union, up until very recently or at least the last 10 years or so.

You can see that one of the things we can do with this sort of VRML stuff, we can turn this all around and do all kinds of fun stuff and see it from all kinds of interesting angles. We can also go inside it, there is a means by which we can traverse the place from indoors, and indeed pass through the walls and do all sorts of fun stuff.

Melnikov had a particular view of how new building ought to take place and that was that it ought to be providing, every part of the environment ought to be providing you with different sorts of stimulus, for what you are up to.

Now the reason to show this may not be immediately clear but I hope it is a little more clear after I do this. This is the actual Melnikov house.

This is Dom Melnikova. You can see - just here is the actual sign on the front of the house - you might have seen on the visualisation there is a little sign there but this is the actual sign for non-Russian speakers it says 'A monument of architectural and historical importance, this is the house of the architect Constantin Stepanovic Melnikov'. There is quite an interesting difference in some senses between that and the kind of indications that you are getting in the visualisation, which is after all the way most people are going to experience this house these days. This is the front facade, interestingly you can see the sycamores growing, the day I visited they had cut down the day before, they had actually cut the grass because it was about 4 ft high, so they cut the grass but they hadn’t got rid of all the sycamores around the property,.

This again, far from the nice smooth surfaces that we were looking at before in the visualisation, you actually see how those windows actually work. They are rendered concrete laid over brick and there are a few photographs of how they were originally done, you can see there the grass that they have just cut, and Russian weeds, I don’t know if you have ever been in Russia during the summer, but the vegetation grows, like you can watch it growing, it has a very short growing season, so it grows virtually infront of your eyes.

This is a little bit of the side of the house, again you get some idea of the colour change, there is a red brick chimney that sticks out at an oblique angle, through the middle of it, and because of course we are actually seeing it in its appropriate colours, it makes a bit more sense.

This is the chap who helped organise my little visit, he is the head of looking after architectural monuments in Moscow so he has a pretty busy time because he doesn’t have much money, and there are a lot of people demanding various things from him. This is actually him with his fiancee, and we are at the back of the house, looking towards the front, as it were.

This gentleman is actually Melnikov’s son. And Melnikov’s son is a fairly irascible sort of chap, he is about 87, and there has been a lot of problems with this property simply because he won’t let you inside. he is very adamant that nobody goes in. One of the interesting features of Melnikov’s original design was that the bed is made out of concrete, and can’t be moved, so upstairs and indeed his own bed, in the original design, Melnikov and his wife sleep in one part of a very large room up the top and Melnikov’s son, there is a specific area for Melnikov’s son which is really just across a kind of threshold, but again that is only according to the plans, nobody has actually been inside this building in any serious way , there have been some photographs that I have seen taken inside this building, but they are comparatively rare. So we have some idea from the plans and things how these things work.

But as you can see he is not a man to be trifled with, I can promise you. I brought the prettiest girl I could find to charm him into letting us into the house, but it just didn’t work.

You can see this is us coming along the side here giving you some idea of the height of this property, but you will note that again we are back at the front of the house, and you will note that I haven’t been able to show you anything like the way the visualisation begins and that is because this is the view that we actually see. Just to go back to the visualisation.

if I take you back to its very front, you can’t possible see the house from this angle. It is just not possible, the street is too narrow, the fence is too high, the sycamores are too thick, there is no chance of seeing it this way. There is also no chance of seeing it this way. because the yard is too small, and the apartment building to the left of it is simply too close, so you won’t see it this way. And you won’t see it this way, because Melnikov’s son won’t let you.

Now I think there are some very interesting lessons to be drawn from that, in terms of what it is we are doing when we are visualising and I mentioned this once before that we have the tendency to develop a model of reality from the reality of a model. I think probably for me that would be the central argument about how we conduct ourselves doing visualisation.

The other example I might have shown today and I did a bit of work on it but for some reason we can’t get a proper internet connection, but what I was going to show was a range of 3D visualisations of the DNA, the most famous visualisation of all was the DNA, the double helix, It is amazing when you go through the numbers of variations of that visualisation, how many variations there is of that, and how many people have been taught, presumably different things, as a result of looking at that visually, and we use these visual cues, in order to give us, a very clear idea of the world, and that is why we get ourselves involved in visualisation,. simply because the data is too thick, for us simply to take the text or the numbers, and turn it into something that we can comprehend, and that is what we are doing when we are doing visualisation , that is why it behoves us, to be important, the manner in which our visualisations are developed and can be used.

And, With & Through; The technologies of performance

And, With & Through; The technologies of performance

Working with digital technology in the context of performance has now diversified significantly to allow us to develop taxonomies about application and production. This presentation looks at categories of emerging practice in the performing arts and seeks to suggest some ways of distinguishing different levels and types of engagement with the digital processing power. Assessing the relative impact of technology on the practices that encounter it is an important way of understanding what we think technology can do for performing practices. This will be augmented by some practical demonstrations from the Visualisation Research Unit from UCE Birmingham presented via AccessGrid.

The 'e' Prefix: e-Science, e-Art & the New Creativity

The 'e' Prefix: e-Science, e-Art & the New Creativity

The division of labour that has kept the sciences and the arts apart for the past two hundred years is collapsing with the development of the 'e' prefix. As the craft practices of both areas proceed in their own directions, the 'e' component of both the sciences and the arts have created communities of interest that suspect they hold the key to the future development of the craft worlds in which they trained. Their mutual interest in exploring the possibilities of new technologies ahead of the crystalisation of e-practices provides an historic moment where the categories and values that separate science and art come under examination. The task of the early-adopters and innovators of the 'e' culture is to imbue the prefix with meaning and application whilst revelling in the opportunity to use it to transform their disciplines. This presentation will explore where the locus of creativity relocates under these conditions and how that may impact on the culture of the arts in particular.

The Networked Multi-disciplinary Performance Environment

For performance artists, the development of stable, networked performance environments has relied heavily on the importation of models of the ‘social’ inter-personal communication type. Given that such communication responds to merely an aspect of the needs of artists, in practice this proves to be an imperfect and incomplete tool. In response to the challenge of producing a more rounded environment, the authors, through their work at the Visualisation Research Unit (VRU) at BIAD, have developed Collaborative Online Digital Arts (CODA) as a platform for drawing together networked resources from different disciplines.

The benefits, drawbacks, needs and future possibilities of existing networked performance will be discussed, as well as the omissions and shortcomings of CODA and other real time applications. However, the principle of creating a virtual performance environment has been abandoned for enhancing the traditional physical performance environment.

CODA was developed specifically to facilitate inter-disciplinary communication in multi-disciplinary performances. Central to CODA is the concept of the node. A node can be a computer with a specialist function, a sensor network, a human-computer interface device or even a person. All nodes broadcast media-in-specific data to a virtual data pool from which every node has access. This allows performers to interact with the environment without the necessity of having in-depth knowledge of how the data is being generated. For example, this allows the output from a video analysis node to be easily mapped onto a sound synthesis parameter on another node.

CODA could be thought of as a single ‘super instrument’ with which all performers can interact simultaneously. The technological infrastructure of the instrument is hidden, but the interface is not. Performers do not need to learn how it works, instead only what interactions it understands and what the results of those interactions may be. This methodology improves performer spontaneity considerably and lends itself particularly well to improvisation incorporating different disciplines where ad hoc exploration and experimentation are important.

This suggests that the network is itself the instrument, and leads to a further interesting phenomena. The data produced can be automatically archived and endlessly reformulated after real time presentation.

The paper will be presented with a short demonstration of CODA.

The New Passive: Opportunities for Spectators in Interactive Performance

Distributed digital media has produced a new space for performance that has much in common with old spaces: opportunities for spectator anonymity and passivity in particular. The shady interactions of the lurker are difficult to identify and often lead to misapprehensions about the success or failure of online performance. The web statistics rarely count how quickly the back button was pressed. This is partly because of an assumption on the part of developers of performance for the web: that they are virtualising a real-world style experience related to either performance or broadcasting or a mixture of both.

The Web, however, presents more difficulties than this as a push-pull medium. The dynamics of the relationships between people who surf in and people who create frameworks are not dichotomised in the same way as in a performance. In Web 2.0 technologies, it is often the visitor who creates or contributes to the content. Extrapolating this idea to the realm of performance, this presentation will discuss how spectating can be redefined and realigned as contributing. Working beyond the computer hardware and into the realms of the flesh and blood, spectators are now accessible as contributors. The potential for using these new 'input devices' to develop performance work suggests a different relationship in creative practice.