Steve Mann
(steve@media.mit.edu)
MIT Building E15-389,
20 Ames Street, Cambridge, MA02139
Now with University of Toronto, 10 King's College Road, Toronto,
Ontario, Canada, M5S 3G4, (416) 946-3387
appears in:
Personal Technologies, Vol. 1, No. 1, March 1997.
@ARTICLE{mannpt,
author = "Steve Mann",
title = "Smart Clothing: The Wearable Computer and WearCam",
journal = "Personal Technologies",
publisher = "Springer",
month = "March",
note = "Volume 1, Issue 1",
year = "1997"
}
Equipped with various sensors which measure heart rate (and waveform), respiration, footstep rate (and waveform), etc., the apparatus can function as a personal safety device for reducing crime, as well as a personal health monitor for improving health care by encouraging individuals to take an active role in diagnosis and body-maintenance.
Presently, conductive cloth in various forms is available, due in part to certain people's desire to shield themselves from radiation effects of cellular telephones and the like, as well as for decorative purposes (the aesthetics of metallic fibers).
The "wearable computer" apparatus is imbedded within nontransparent clothing which provides shielding, (radio-frequency privacy, and solitude). Conductive fibers are also been used for the antenna structure, as well as for connectivity.
Smart clothing has been proposed as a form of existential media [Mann97]. Existential media defines new forms of social interaction through enhanced abilities for self expression and self-actualization, as well as through self-determination. The aspects of existentialism pertaining to existential media are best understood through a reading of [Frankl94].
Examples of smart clothing include internet-connected shoes (Figure 1) which allow one to run with a jogging partner located in some distant place, but connected via the network. Viewpoints might also be shared using the "eye-to-eye" glasses (where a portion of each runner's visual field comes from the other runner [Mann94].
[FIGURE 1]
[CAPTION]
One of author's shoes, equipped with sensors, measure
the impact of various parts of the shoe with the ground.
This gives the wearer's computer a sense of the pace, so that,
for example, joggers might pace themselves with a distant
partner connected through wireless communications.
Such technologies enhance one's capabilities of self-expression.
The SONY "Walkman" is another example of "existential media". Its ability to reclaim a portion of this sort of personal space that has largely been lost to "muzak" (a use of technology that has stolen much of our solitude) affords the user a good deal of self-determination. The proposed "wearable computing" apparatus allows this functionality (playing music) to be "disappear" into clothing and be integrated in a more natural manner.
Smart clothing contributes to the wearer's eudaemonic space [Mann97] which defines a boundary around the user, providing both privacy and self-expression.
The existential computer, which exists entirely within the eudaemonic space of the user is owned, operated, and controlled by the wearer, in order to prevent it from becoming a means of controlling the wearer. This principle is best understood through the distinction between "smart clothing" and "smart uniforms" provided in [Mann96a]. (An extreme example of external control is given in [Hoshen95].)
Furthermore, in contrast to either the desktop "on-the-job" computer, or the "smart uniform" [Mann96a], the existential computer is used for all facets of living (not just at a place of work), and there is an increased familiarity with the interface apparatus through its continuous use.
Although the proposed apparatus will replace much of the environmental technology, a goal is also to work in harmony with some remaining degree of environmental technology. Furthermore, not all environmental technology is counter to the principles of existential media. Most notably, is Ishii's so-called {\em ambient media} [Ishii]. Ambient media provide a natural and unobtrusive environmental awareness to the user. Natalie Jeremijenko's beautiful instrument called "Live Wire" [Jeremijenko] (called "dangling string" by many), which she developed as an artist in residence at Xerox PARC [Ishii][Weiser] is one example of tangible media; the string vibrates in response to network traffic and provides clues for peripheral attunement. It violates neither solitude (by virtue of its ambient nature, it remains in the perceptual periphery), nor privacy (for it has no capability to measure or record user-specific information).
Having a patient wear his or her entire medical history would solve much of the medical records privacy problems we face today. With various biosensors, the most current and up-to-date information would be readily available within the very clothing that's taking the measurements. Of course, one would want to have one's medical records replicated (backed up) in the clothing of selected friends and relatives, to prevent data loss in the event of clothing failure.
This approach would eliminate the need for, and the possible abuses that can arise with, a central database of medical records, and would eliminate the need to for a person to venture through bureaucratic procedures to access his or her own medical information (e.g. when an individual wishes to obtain a second opinion from a different doctor). It would also eliminate the problems associated with smart cards, as clothing is almost always worn, while cards may be misplaced and inaccessible in times of emergency care. Epidemilogical research would still be possible with the patient's data --- participating patients could make the data accessible to organizations doing the research, but this would be done through a query to each participating patient's online "smart clothing" each time the data were needed, so that the patient's clothing would be kept "in the loop", that is, access logs would be automatically generated in the "smart clothing", so that patients could trace the history/usage of their data at a later date if desired.
The first "existential computers", designed and built in the late 1970s and early 1980s (Figure 3) were originally developed for applications in the visual arts, in particular, still-life and landscape imaging [Mann96] --- combining multiple exposures of a static scene to a variety of different light sources, using principles of linearity and superposition of light [Ryals].
[FIGURE 3]
[CAPTION] Evolution of author's "existential computer":
Large head-mounted CRT and separate inbound and outbound
communications antennas of the late 70s were awkward.
Author's waist-mounted television of the mid 80s was somewhat
more comfortable but not constantly visible.
Small viewfinders from consumer video cameras of the late 80s
made possible an eyeglass-based system which later evolved toward
author's current embodiment built into ordinary eyeglasses
with the advent of newer solid-state display technology.
The current state of this simple invention exists as a miniaturized multimedia computer built unobtrusively into a pair of ordinary sunglasses (Figure 4).
[FIGURE 4]
[CAPTION] Current state of author's "existential computer"
invention comprises a complete multimedia computer, with
cameras, microphones, and earphones, all built into
an ordinary pair if eyeglasses (except for some of the
electronics items that fit into a small box into a shirt
pocket, which will soon "disappear" altogether).
This rig is currently running the Linux 2.0 operating
system, with XFree86 (variant of X-windows).
with a small box that clips on the author's belt or fits into a shirt pocket. The input device is fashioned into an ordinary belt, using microswitches, so that one can type or issue commands unobtrusively. Other variations of the input device include microswitches sewn into clothing and the like.
The small size of the apparatus (negative size if one considers what it replaces --- cellphone, pager, walkman, camcorder, Personal Data Assistant (PDA), etc.), provides for a single unified point of contact --- all of the personal electronics we normally carry become subsumed into this one unit. Clearly the unit is capable of email, "i-phone" (internet telephony), and other forms of multimedia communication, so that it eliminates (or will eliminate, in time, once other people have such units) the need to carry a cellular phone or pager.
The system also has sound capability, so that it can function as a personal sound system, thus subsuming that form of existential media which provides the wearer musical self-determination. However, new forms of interaction --- interaction that goes beyond the aggregate functionality of all of the consumer electronics we normally carry --- is possible because they are all subsumed into a single unit.
For example, while one is jogging, one's shoes (Figure 1) can provide the system with an awareness of the running pace, so that music can be adjusted appropriately to maintain an even pace under program control. This allows the runner to use music as a "clocking" mechanism. Furthermore, since heart-rate is known to the system, it could establish a training schedule where the music serves a clock, in the feedback loop of a process that maintains constant heart rate, so that the pace is decreased or increased in accordance with the wearer's training goals. Such a system would also screen calls appropriately to delay them into the rest periods of the training schedule, so, if the wearer were doing interval training, and nearing the end of a max-level set, and an incoming call were detected, the apparatus might silence the first two or three "rings" of the "phone", rather than disturb the wearer during an instant when maximum concentration is desired.
Clearly, today's digital cameras fail to meet the hands-free operation and processing/transmission capabilities. Another, perhaps less obvious yet important missing element from today's digital cameras is the electronic viewfinder. Again, it is quite obvious to anyone skilled in the photographic arts, that not only should there be a means of previewing previously captured images, but the importance of having the human in the loop of the creative process should be regarded as absolutely essential. Just as in {\em Deconstructing the automatic copier}, the benefit of realtime display of imagery should be obvious.
However, the WearCam viewfinder goes beyond merely setting the camera correctly. Since the apparatus is worn over an extended period of time, one adapts to it, and it begins to function as a true extension of the body. In this way, the viewfinder transcends being just a compositional tool, toward allowing the camera to "become" the eye of the wearer.
Long term use of WearCam gave rise to a heightened sense of awareness of light and shade; by living in a "mediated reality" environment [Mann94]. WearCam was an electronic/computational realization of the same sort of adaptation processes/experiments that have been previously done optically [Stratton1896][Kohler64][Dolezal82]
Combining this algorithm with the "wearable face recognizer" proposed in [Mann96c], a virtual name tag, that appears together with other information, is possible. The "existential computer" recognizes the face, and then the flowfield of the entire room is used to give an approximate illusion of a ridid planar patch (only exact for if the wearer and named individual remain in the same place while the wearer looks around).
One such personal documentary, "ShootingBack", was a meta-documentary of sorts --- it was a documentary about making a documentary. In "ShootingBack" the author carries a standard camcorder into organizations characterized by totalitarian surveillance (e.g. department stores where photography is prohibited). Typically a conversation with the representative of the organization, where the representative states (in response to a querry as to why surveillance cameras are being used) something to the effect that only criminals would be afraid of cameras, or that the author must be trying to steal something if he were afraid of cameras, or that cameras should be of no concern. Then, pulling the camcorder out of a sachel, the viewer sees the eyecup of the camcorder moving up toward the "eye" (which is actually the eyeglass-based camera), and the resulting response from a representative (Figure 6).
[FIGURE 6]
[CAPTION] "ShootingBack" is a meta-documentary (a documentary about making
a documentary. Here the viewer sees the process of shooting
with a conventional camcorder, as recorded by WearCam. Because,
after time, the wearer forgets that he is wearing the WearCam
apparatus, the process of using the camcorder is captured
in a natural manner giving the viewer the impression that he/she
is living the life of the documentary artist.
Because there is also the output from the camcorder, in "ShootingBack", a split-screen effect, reminiscent of the movies "The Boston Strangler" and "The Thomas Crown Affair" is used. However, keeping to the integrity and reality-metaphor of the documentary, the natural window sizes are used at all times (e.g. nothing is cropped as is often done in other uses of split-screen). (See Figure 7) The effect is also reminiscent of the split-screen spatial division multiplexers used for "4-up" recordings of surveillance cameras.
[FIGURE 7]
[CAPTION] Split-screen format shows both the view from
the viewer's "eye", as well as the output of the
camcorder. Here a representative of an organization
using totalitarian video surveillance responds to the
author's camcorder.
Most of ShootingBack is shot in "rot90" (90 degree rotated) format, as this brings the viewer in closest proximity to the person being shot. The aspect ratio of the human face fits well this 3:4 format (as opposed to the usual 4:3 ratio of TV). The adaptation to "rot90" took place over a number of years of having worn the apparatus in that configuration, and it was found that a dual-adaptation space formed (e.g. where the author could adapt to seeing the world rotated 90 degrees, and than adapt back to seeing normally without the WearCam apparatus). The adaptation was quite pronounced, and confirmed the author's hypothesis that mappings near the identity have a longer-lasting per-unit effect (e.g. when WearCam was used in a non-rotated format, it was harder to switch back and forth).
This creates an interesting form of balance. In the past, there have often been hidden cameras in the architecture (e.g. there is a large market for smoke detectors with built in cameras, clocks with hidden cameras inside, etc., and many of the companies selling these cameras offer government discounts). But with WearCam there is balance, for now a visitor to a new space can attain the same kind of respect that the owner of the space might have. WearCam gives the nomad a form of personal protection once available only to establishments.
In some sense, WearCam is also less of an invasion of privacy than the cameras of the establishment, for at least one still knows that when one is alone, one has privacy from WearCams. There is already in place a natural tendency to behave in a dignified manner in the presence of others, so it is really the cameras of the establishment that pose the gravest threat to privacy. Furthermore, a community of WearCams gives rise to a more distribued form of crime reduction than a network of cameras wired into the police station or the like.
A further improvement to the personal safety device includes the use of biosensors where the quotient of heart rate divided by footstep rate (as measured using the apparatus of Figure 1 provides a visual saliency index. Suppose that someone were to draw a gun and demand cash from the wearer. The likely scenario is that the wearer's heart rate would increase without an increase in footstep rate to explain it (in fact footstep rate would no doubt fall to zero at the request of the assailant). Such an occurrence would be programmed to trigger a "maybe I'm in distress" message to other members of the personal safety network ("SafetyNet"). These members, comrpised of friends and relatives, would then look at the video signal and might conclude merely that the wire has come loose from the shoe, or might find, in fact, that the distress is genuine. Such a system provides an alternative to Orwellian surveillance cities like Liverpool or Baltimore. Instead of relying on some distant and unaccountable force to keep watch over the society, individuals form small social networks that function much like small towns in a global neighbourhood watch.
The SafetyNet might be comprised of some members linked in with their home computers, as well as various members who have WearCams or just ordinary wearable computers (Figure 8)
[FIGURE 8]
[CAPTION] Personal Safety Network ("SafetyNet"). In the future,
groups of individuals connected together wirelessly
might create a virtual small-town global neighbourhood
watch to reduce crime.
With technolgies like automatic face recognition, an individual can quickly verify that the person knocking on their door is really a police officer, or contractor, and not, for example, the Boston Strangler. This of course would require that police organizations and the like put images of their employees online for all to see. Mutual observability might make face capture more acceptable than the current totalitarian forms of face databases (e.g. that only law enforcement officials can have access to). Mutual observability need not even be assured, merely that there be the possiblity that the WearCam apparatus function. Such possibility serves as a reverse of the unseen guard [Foucault77].
[Dolezal82] Hubert Dolezal. Living in a world transformed. Academic press series in cognition and perception. Academic press, Chicago, Illinois, 1982.
[Foucault77] Michel Foucault. Discipline and Punish. 1977. Translated from "Surveiller et punir".
[Frankl94] VIKTOR E. FRANKL. MAN'S SEARCH FOR MEANING. WASHINGTON SQUARE PRESS, 1230 Avenue of the Americas, New York, N.Y. 10020 ISBN: 0-671-83465-7 1994.
[Hoshen95] Joseph Hoshen, Jim Sennott, and Max Winkler. Keeping tabs on criminals. IEEE SPECTRUM, pages 26--32, February 1995.
[Jeremijenko] Natalie Jeremijenko. Bureau of inverse technology, fivaonline. http://www.fivaonline.com or http://cdr.stanford.edu/
[Kohler64] Ivo Kohler. The formation and transformation of the perceptual world, volume 3 of Pshchological issues. International university press, 227 West 13 Street, 1964. monograph 12.
[Mann94] S. Mann "mediated reality". TR 260, M.I.T. Media Lab Perceptual Computing Section, Cambridge, Ma, 1994.
[Mann96] Steve Mann. Wearable computing: A first step toward personal imaging. IEEE Computer, Feb 1997.
[Mann96a] Steve Mann. Smart clothing: The shift to wearable computing. Communications of the ACM, pages 23--24, August 1996.
[Mann96b] S. Mann. "smart clothing". TR 366, MIT Media Lab Perceptual Computing Section, Cambridge, Ma, Feb. 2 1996
[Mann96c] Steve Mann. Wearable, tetherless computer--mediated reality: Wearcam as a wearable face--recognizer, and other applications for the disabled. TR 361, MIT Media Lab Perceptual Computing Section, Cambridge, Ma, Feb. 2 1996. Also appears in AAAI Fall Symposium on Developing Assistive Technology for People with Disabilities, 9-11 November 1996, MIT.
[Ryals] Cynthia Ryals. Lightspace: A new language of imaging. PHOTO Electronic Imaging, 38(2):14--16, 1995. http://www.novalink.com/pei/mann2.html.
[Stratton1896] George M. Stratton. Some preliminary experiements on vision. Psychological Review, 1896.
[Weiser] Mark Weiser and John Seely Brown. Designing calm technology, 1996. http://powergrid.electriciti.com/1.01/calmtech-wp.html.