Living in a 2-D world

In addition to adapting to transformations of the perceptual world, the author noticed a loss of the perception of 3-D depth. (A typical video camera lacks depth from stereo, depth from focus, etc..) In this sense, the author developed a ``photographic mindset'' in which an enhanced sense of awareness of light and shade, and of simple renaissance perspective were attained. It was found [4] that this effect persisted, even when the apparatus was removed, and that the effect would revisit in the form of 2-D ``flashbacks'', so that the world was seen in two ways, much like we see the Necker cube illusion in two possible ways. This discovery gave rise to the fingerpointing process (Fig 6)

  

Figure 6: Living in a 2-D world, through long-term adaptation. Fingerpointing from the perspective of life through the screen. Adaptation was in the ``rot90'' coordinate transformation described in the text. Notice how natural interactions with the world (such as pointing at the video surveillance cameras on the ceiling of the department store, or the signage on the wall) take place in 2-D projection on the image plane rather than 3-D space.

where it was found that pointing at objects was as though though a 2-D plane projection. The notion of attaching a light to the finger arose out of various expressive lightpainting efforts, where the world was viewed as 2-D video, while a light source, attached to the finger, was moved around in 3-D space (Fig 7), while recording the process simultaneously on film (a beam splitter being used to combine video and film cameras).

  

Figure 7: Examples of tracing out a locus of points in 3-D space that are mapped onto a 2-D image. Here a small light source, attached to the author's finger, takes the form of a pointing device, which is used to outline objects in 3-D space, but falling upon their 2-D projection. (a) One of the early lightpaintings using this technique. (b) Image which won best color entry, in the National Fuju Film competition, 1986. Here the method is perfected somewhat, where we begin to see how the Personal Imaging system functions more like the artist's sketch pad than like a traditional camera. (C) Steve Mann, 1985.

In Mediated Reality the drawing takes place right on top of the video stream, so that registration is, for all practical purposes, exact to within the pixel resolution of the devices [5], in contrast to the registration problem of Augmented Reality [10]. This characteristic of Mediated Reality (perfect registration) has been suggested as a means of using the finger as a mouse to outline actual objects in the scene[2]. This form of interaction with the real world, through the apparatus, is yet another example of the human-machine symbiosis that is at the core of Personal Imaging.