Pictures, mockups and animations: On the ecological validity of environmental simulations

Rainer Guski
Ruhr-University Bochum, Dept. of Psychology
Workgroup Cognition and Environment
Rainer.Guski@uni-bochum.de

Contents

Keywords: psychology, experimental procedures, validity, ecology.

Abstract

The paper introduces the notion of ecological validity in laboratory studies on man-environment relations. Ecological validity refers to the extent to which a stimulus/response-combination indicates what it is claimed for a species. This requires species-typical stimuli, special-typical responses, and a species-typical experimental context. The concept does not necessarily require an experimental design with a representative sample of species-typical stimuli and responses, but it requires stimuli containing rich (and controlled) information, responses that fit clearly to the task at hand and do not require special training. Several examples from laboratory studies using pictures, mockups, large screens, or virtual reality facilities are given.

Introduction: The notion of Ecological Validity

Scientific experiments should be completely under control of the experimenter and be replicable in other contexts, and/or other times and places. This prerequisite cannot be completely fulfilled in real environments. Therefore, environmental psychologists try to simulate and manipulate those aspects in the laboratory that are essential for their question at hand. One of the central methodological questions for environmental psychologists is: Under which circumstances do we produce valid results?

The term “validity” generally refers to the extent to which information indicates what it is claimed to give. Psychologists distinguish several forms of validity, e.g., Content validity, Face validity, Factorial validity, Empirical validity, Predictive validity, Concurrent validity, Test validity, and Ecological validity. It is beyond the scope of this paper to explain all forms of validity; I will only handle “ecological validity”: Ecological validity refers to the extent to which a stimulus/response-combination indicates what it is claimed for a species.

Species-typicality means that a species (e.g., a human being) does not need much learning in order to perform the experiment. The species-typicality must be defined along three dimensions:

1.. Stimulus objects presented,

2.. Response measured,

3.. Experimental context.

With three dimensions and all possible combinations, 8 types of experiments are possible, differing in the grade and type of ecological or non-ecological approach [1].

Ecological validity per se does not mean that the results of a laboratory experiment must be generalizable to real world settings, but if we want to transfer lab results to the real world we must show that the results are generalizable, i.e., independent of the specific experimental stimuli, responses and context. It seems necessary to compare the results of systematic variations of experimental stimuli, responses and contexts; one single experiment will never be enough.

What are Species-typical stimuli

Typical visual stimuli used in psychological experiments are abstract and meagre, e.g., lines, rectangles, circles, arrows, points etc. on uniform backgrounds. They provide little information about context (size, distance, affordance). In his history of psychology, Egon Brunswik showed single stimulus dimensions (e.g. visual angle of an object) to provide no valid information about object size. He proposed to use multiple dimensions (e.g., the visual angle of an object plus the visual angle of egocentric distance for the task of judging the size of an object). James Gibson [2] stressed that visual stimuli should be displayed on structured backgrounds in order to provide size and distance information.

Many so-called “visual illusions” rely on the fact that humans are not able to judge perceptual dimensions without a surrounding context (e.g., colour, size, and speed).

One example: Depending on the visual context, the black man in Fig. 1 looks taller or smaller than the red figure. If we look at Fig.1 as it is shown here, the black schematic man looks taller than the red one. Now imagine the figure without perspective lines at the floor, the walls, and the ceiling – in this case, the black figure would look smaller than the red one.

Fig. 1. Depending on the visual context, the black man (right) looks taller or smaller than the red one (left)..

In the latter case, i.e., in the absence of context information the human perceptual system uses “default assumptions” (e.g., figures are located at the same distance). This tendency to assume “default values” may produce valid or invalid results. Sometimes, abstract visual displays provide sufficient information for certain tasks, e.g., a horizontal wedge facilitates movements toward the direction of its tip, or animated point-light displays provide sufficient information about the gender of a walking person.

If we use real-world stimuli instead of abstract displays, we should be aware that real-world stimuli usually are well-known and contain semantic and emotional information. Semantic and emotional information both depend upon the cultural context, but the cultural context changes over time. Emotional and semantic information should be controlled in experiments, using pretests or statistical means (e.g., covariance analysis).

One of the main tasks for subjects in environmental psychology laboratories is to recognize objects (e.g., places, houses etc.). It is well known that the perceptual recognition of objects depends on several factors, e.g., the uniqueness of an object; the size of the display; static vs. dynamic stimuli; the direction and field of view. When we compare the results of recognition studies between lab and field studies, we sometimes see a high covariation, but there is a large variance, and it is generally believed that still photos are at the lower end of the continuum of the covariation between lab and field results.

There is little psychological research using mockups (design models). Pyron [3] used plastic models of house modules and made black and white endoscopic traveling shots through the settings. The camera angle of view was 50 degrees; eye movements were recorded while subjects viewed the videos. At the end of the session, subjects were to recall the location of 4 specific houses.

Fig. 2: One of the stimuli used by Pyron (1971).

In analyzing the eye movements, Pyron found that the foveal coverage increased with increasing degree of syntactical information. The mean recognition errors were about 20%, and they were independent of the spatial arrangements used. He concluded that the human eye needs structural variation in order to maintain exploratory behaviour. Unfortunately, the paper provides no information about reliability and validity.

Let us turn to essential stimulus dimensions of computer-based studies. A number of dimensions have been shown to influence subject's behaviour: Screen size (regular (small) vs. large); screen type (translucent / opaque); motion (still pictures / animations / real movies; sounds (no sounds / .. / sounds fit to image [4, 5, 6, 7]; degree of interaction (no choice of views / .. / subjects can choose paths, velocity etc. [8]) and the degree of immersion (presence) resp. the "elimination of mediation" [9].

Turning to interactive large-screen environments, I like to mention two of our own studies:

1.. Generally, we use a digital data projector, a 2.5 X 2.5 m translucent screen, and subjects sitting 1 m in front of the screen. Some studies ask for subjective security in environmental settings. Garstka [10] performed a lab experiment with interactive 360° views of underground stations and university campus sites, he varied some stimulus variables, e.g. the brightness of the illumination, and perceivable opportunities of escape. Subjects scaled the degree of perceived subjective security. It turned out that the brightness of iIlumination was a stronger determinant of subjective security than opportunities of escape (Fig. 3).

Fig. 3: Panoramic view of one of the stimuli used by Garstka (2004).

2. The second experiment was performed by Blöbaum & Hunecke [11] in the field (on the campus of the Ruhr-University) and asked 122 students to scale subjective security (German “subjektive Sicherheit”, or “subjective Unsicherheit”) in real life settings under different degrees of illumination. It turned out that only 26 % of the variance in subjective security could be explained (Fig. 4), and perceived opportunities of escape was the strongest determinant of subjective security. This stands in sharp contrast to the lab study mentioned before. It is uncertain whether our lab studies produce valid results, or whether our field studies produce valid results.

Fig. 4: Results of the study of Blöbaum & Hunecke (2005).

Turning to immersive virtual environments, I like to mention the study by Conroy [12]: She used a helmet display with a large field of view (105 x 41°), and a 3D mouse for navigation. In one experiment, she compared real observations of pedestrians in the Tate Gallery with observations of virtual 'pedestrians' in a virtual Tate Gallery. It should be mentioned that there were no pictures in the virtual Tate. For certain behaviour variables, r-square of 0.5 (virtual / real) was observed. This may be a trivial result: people do not run into walls!

Fig. 5: Screenshot of one of the stimuli used by Conroy (2001).

Further experiments by Conroy [12] used wayfinding tasks in five virtual worlds. The floor layout and exterior design of houses was systematically varied, and the subjects' traces on their trips through the environments were registered. Although there was a certain covariation between floor layout and exterior of houses, she concluded that subjects usually move on linear paths, following long sight-lines, with pauses in configurationally 'integrated' locations offering strategic visual properties, long lines of sight, and large isovist areas. The validity question cannot be answered by this study.

There has been a number of desktop virtual environments, but I like to mention only one by Rohrmann & Bishop [7].. The authors simulated a suburban environment and varied the quality of illumination (day/sun, day/fog, night), personal shadow (yes/no) and sound (on/off). They asked 147 subjects about perceived simulation quality, comprehension, recollection and appreciation of the simulated environment. Here are the main results:

1. The simulations were perceived as valid and acceptable;

2.. The appraisals differ according to lighting and time-of-day conditions, and

3.. The provision of sound enhances the perceived quality of presentations.

What are Species-typical responses

As said in the beginning, ecological validity refers to the extent to which a stimulus/response-combination indicates what it is claimed for a species. Species-typicality means that a species (e.g., a human being) does not need much learning in order to perform the experiment, i.e., species-typical responses should not require much learning. Automatic (reflex) responses require the least amount of learning, but they are sometimes of little scientific value. The choice of responses mostly involves a compromise between ease of performance and scientific usefulness.

Classical psychological experiments mainly use two response classes:

1.. Overt behaviour, like verbal reports (free, bound to questionnaire items, etc.), subjective scaling or rating (intensity, frequency, evaluation etc.), and body movements (eye movements, navigation, button pressing etc.),

2.. Covert behaviour, like reaction or decision time, and physiological responses (encephalographic, electrodermal etc.).

Before starting an ecological experiment, we should first observe the typical behaviour in real-world settings, then choose forms of behaviour that are both close to typical behaviour, and recordable and analyzable in an objective manner. It should be noted that even free verbal behaviour can be analyzed in an objective manner, e.g., by using content analysis methods.

The context of Stimulus-Response combinations

Past psychological experiments have shown that the experimental context determines responses to a great extent. For instance, loudness judgments of speech depend on:

a) the range of objective sound levels used [13],

b) the frequency of objective stimulus events used [13],

c) the order of stimulus events used, and

d) the manner of speech (whispering, normal, shouting, cf. [14].

It has been shown that subjects develop response strategies in the course of experiments, i.e., they learn to compare different stimulus elements of the experiment, and how to cope with them. This has been considered as a problem, and two problem solutions have been proposed: (1) single shot studies using many subjects only one time, (2) repeated measurements of few subjects in long sessions. Both solutions have their pros and cons.

Another contextual problem relates to differences between verbal reports about behaviour and observed actual behaviour.. When it is asked about “typical behaviour” in past situations, people often report what they want to do (e.g., sleep with windows open, use public transportation often), but the observed actual behaviour may be different (e.g., people sleep with windows closed, mainly use private cars). At present, we don't have any solution for this problem.

References

[1] D.B. Miller, Methodological issues in the ecological study of learning. In: T.D. Johnston & A.T. Pietrewicz (Eds.): Issues in the ecological study of learning. Hillsdale: Erlbaum, pp. 73-95, 1985

[2] J.J. Gibson, The senses considered as perceptual systems. Boston: Houghton Mifflin, 1966.

[3] B. Pyron, Form and space diversity in human habitats. Perceptual responses. Environment and Behavior, 3, 382-411, 1971.

[4] J. Hetherington, T.C. Daniel & T.C. Brown, Is motion more important than it sounds?: The medium of presentation in environment perception research. Journal of Environmental Psychology, 13, 283-291, 1993..

[5] L.F. Becher, J. Vogt, M. Schreiber & K.T. Kalveram, Effekte der visuellen Umwelt auf das Erleben ökologischer und künstlicher Geräusche. Zeitschrift für Lärmbekämpfung, 44, 195-200, 1997.

[6] S. Viollon, C. Lavandier & C. Drake, Changing soundscape appraisal through visual information. Inter-Noise '99, Paper 1165, 1999.

[7] B. Rohrmann & I. Bishop, Subjective responses to computer-simulations of urban environments. Journal of Environmental Psychology, 22, 319-331, 2002.

[8] I.D. Bishop, J. Wherrett & D. Miller, Assessment of path choices on a country walk using a virtual environment. Landscape and Urban Planning, 52, 227-239, 2001.

[9] G. Riva, F. Davide & W.A. Ijsselsteijn, Being There. Concepts, Effects and Measurements of User Presence in Synthetic Environments. Amsterdam: IOS Press, 2003.

[10] A. Garstka, Subjektive Sicherheit an U-Bahnstationen. Ein computergestütztes Experiment mit navigierbaren Panoramabildern.Unveröff. Diplomarbeit, Bochum: Ruhr-Universität, Fakultät für Psychologie, 2004.

[11] A. Blöbaum & M. Hunecke, Perceived Danger in Urban Public Space: The Impacts of Physical Features and Personal Factors. Environment & Behavior, 37, 465-486, 2005.

[12] R. Conroy, Spatial navigation in immersive virtual environments. Dissertation, London (UK): Faculty of the Built Environment, University of London, 2001.

[13] A. Parducci, Context effects: a range-frequency analysis. In: E.C. Carterette & M.P. Friedman (Eds.): Handbook of perception. Vol. 2, pp. 127-141. New York: Academic Press, 1974.

[14] M.B. Gardner, Distance estimations of 0° or apparent 0°-oriented speech signals in anechoic space.. Journal of the Acoustical Society of America 45, 47-53, 1969.