Building environment maps by looking around

An environment map is a collection of images, seamlessly ``stitched'' together, into some unified representation of the quantity of light that has arrived from each angle in space, over the range of angles for which there exist measurement data. Examples of environment maps appear in Fig 8.

  

Figure 8: Environment maps captured from a first-person perspective, through the process of looking around, are called ``lookpaintings''. Lookpaintings are characterized by irregularly-shaped boundaries which capture the gaze pattern of the wearer. (a) Lookpainting made from 4 input images. Individual image boundaries are clearly visible. Note the unified perspective and the lack of distortion (e.g. lines on the ceiling tiles are almost perfectly straight, despite the extreme perspective). (b) Lookpainting made from 226 input images. This composite image illustrates the nature of first-person perspective. Note that both the author's hands are visible in the picture. Because the apparatus is wearable, it captures a new point of view, while at the same time, capturing what is important in a scene. (Here, for example, the lookpainting has included the video surveillance camera on the ceiling because the author has looked there.)

In constructing these environment maps, the computer performs basic calculations which it is good at, while the human operator makes higher level decisions about artistic content, what is of greatest interest or importance, etc..

As described earlier, the human becomes at one with the machine (in this case the camera) through a long-term adaptation process, so that, as one experiences one's life through the apparatus (living in a computer-mediated world), the subject matter of interest is automatically captured by the human operator. Note that in this simple case, there is no Artificial Intelligence, but instead, there is a synergy between human and machine, where the ``intelligence'' arises through having the human operator in the feedback loop of the overall image acquisition process.

Personal imaging, which is facilitated through WearComp, image processing, machine vision, and computer-mediated reality, enables the user to effortlessly capture high-quality images of a new genre characterized by not only enhanced tonal range and spatial resolution, but also by the ability to include and exclude areas of interest (Fig 9).

  

Figure 9: This image depicts a group of people to whom the author is lecturing. Here the author is able to quickly sweep out the important details of this scene (namely all of the participants), while leaving out areas of the room where nobody is seated. Note how the image is close-cropped, leaving out the two empty chairs in the center, while at the same time, extending out to include the feet of those sitting to the left and right of the empty chairs. This natural selection of subject matter happens often without conscious thought or effort, through the process of simply ``looking around'' at everyday scenes or objects, and is characteristic of the symbiotic relationship between human and machine that arises when the two become inextricably intertwined through a constancy of user-interface extending over a time period of many years.