ISAST LEONARDO, Vol. 36, No. 3, pp. 199,207-210, 2003 (PDF)
INTELLIGENT FIXTURES AND SYSTEMS:
Steve Mann, Mediocrat
(S. Mann is Currently an Assistant Filing Clerk Trainee of EXISTech Corp.)
284 Bloor St. West, Suite 701, Toronto, Ontario, Canada, M5S 3G4
ABSTRACT
EXISTech Corporation's computer networks and control systems, together
with infrared image sensors, facilitate hygienic touchless control of
plumbing fixtures. Two of EXISTech's sensors are described in detail:
an active infrared faucet sensor (320x240 pixels) and a passive infrared
autoflush sensor (160x120 pixels). The internetworked plumbing systems
help facility managers and law enforcement personnel remotely monitor
the operation of fixtures in various bathrooms. A single infrared sensor
array can also be used for controlling several showers, faucets, urinals,
or water closets in large bathroom complexes. Intelligent bathroom
fixtures and systems based on computer vision help enhance the privacy
of users by ensuring that law abiding users need not be disturbed by
police foot patrols into the restroom areas. The intelligent quantimetric
sensing technology eliminates the need for privacy-invading foot patrols.
New computer vision algorithms also automatically detect slip-and-fall
accidents as well as vandalism and contraband disposal, to assist remote
monitoring by law enforcement.
INTRODUCTION
This paper describes new approaches to the design of automatic sensor
operated bathroom fixtures, systems for controlling bathroom fixtures,
and methods of bathroom fixture design, control, and management,
through the use of new computer vision technologies, and image processing.
Networked plumbing systems also help facility managers monitor the
operation of various bathroom complexes from a central remote location.
SMART BATHROOMS WITH VIDEO BASED MOTION DETECTION
By using video based sensors, the intelligent bathroom provides a safe,
hygienic, and user friendly environment that responds automatically to the
needs of the user. Concomitant functions, such as security, drowning
prevention, and other safety features are possible with the intelligent
bathroom. It is assumed that the benefits of automation of fixtures will
cause sensors to be installed in virtually all bathrooms of the future.
It is also expected that the most economical sensors will be single
chip video cameras, which now only cost $10 in mass production, whereas
other sensors such as infrared position sensing devices now used in
electronic plumbing systems cost much more because they are specialized
devices. Similarly radar and sonar systems commonly used for occupancy
detection (for automatic door openers, lighting control, etc.) now cost
more than low cost single chip video cameras. Once these cameras are
installed in most bathrooms, new uses, such as automation of lighting,
automatic bathroom door openers, automatic call for help in case of
slip and fall injury, and automated detection of vandalism will emerge.
Sensor systems used in many of EXISTech Corporation's research prototype
bathroom facilities comprise standard video cameras with infrared
blocking filters removed to make them sensitive in the infrared.
Because of mass production, a single camera costs much less than one
of the specialized sensors commonly used to automatically flush toilets
and urinals. Moreover, further cost savings are introduced by the
fact that a single video camera can monitor a number of fixtures. For
example, one video camera can monitor a whole row of urinals, to flush
only the urinals that have been used. Intelligent computer vision systems
monitor optical traffic flow patterns, so that a weaker flush is used when
others are standing in line to use a urinal, whereas a stronger
flush is provided after the last user of a group of users has finished.
Thus intelligent video based sensor technology will reduce initial
installation costs as well as operating costs, when compared to other
sensor technologies.
Camera based sensors can provide a much more intricate and sophisticated
form of control, because they can detect user behaviour, usage patterns,
optical traffic flow patterns, and other attributes not evident in
traditional bathroom presence or occupancy sensors.
Different kinds of optical elements are being used in EXISTech Corporation's
experimental bathrooms [1]. For example ceiling domes that provide the sensor
with a wide field of view are being used for controlling a large number of
bathroom fixtures with just one sensor. Many of the lowest cost cameras
have wide angle lenses. A camera with a fisheye lens can be
placed on the ceiling of a shower room, so that it can monitor the entire
shower room. Therefore a single sensor can be used to automate many
showers. Such a single sensor is out of the way of vandalism, soap scum
buildup, or other problems that would arise if sensors were distributed
along walls of the shower room with one sensor under each shower head.
Users will enjoy a nice hot shower, without having to adjust the
temperature, or even touch anything at all. Users simply step into the
viewing area, comprised of a plurality of detection zones, and the shower
within their detection zone turns on. Instantly, out comes water at the
perfect temperature! When a user steps away to lather up with soap, the
water turns off. When the user steps toward the spray head for a rinse,
the water turns on again.
Moreover, maintenance and installation are simplified by having one image
sensor controlling various shower spray heads. Additionally, the one image
sensor may provide other features such as automatically warning building
staff if a person has slipped and fallen, or automatically recognizing
faces of users, and providing each user with water tempered to the
preference of each individual user. The video based motion detection
sensor will also act as a deterrent to crime and vandalism in the shower
room. Additionally, multiple spray heads at each station can spray a
user with water in such a way that very little is wasted. A beam pattern
of spray can adapt to the position and orientation of the user's body.
EXISTech's intelligent bathroom control systems can be reprogrammed to
respond to users in slightly different ways, and therefore user behaviour
can be modified slightly. Through slight modifications in user behaviour,
efficiency and restroom throughput can be increased. For example,
the system might detect that, in a row of hand faucets, the furthest one
is used excessively during certain times of day. It might be determined
that a homeless person is using it for hair washing purposes, especially
if it is somewhat hidden from view. The system can detect this pattern
of deviant use, and correct it by adjusting the timing on that particular
fixture so that it will time out sooner than the others. This would
effectively move that user to another faucet. Thus slight changes in
system parameters can be used to effect slight changes in user behaviour.
Data output from an infrared sensor operated faucet is shown in
Figure 1. By representing the numerical values in the array as
shades of grey, we can see the detection zone, where the hands of
a bather are present.
[insert FIGURE 1 http://wearcam.org/fig1.tif and CAPTION somewhere near here]
[1 column width]
The sensor produces its own source of light, so that it operates
irrespective of ambient light (if any, e.g. it will also work in
a completely dark room). The faucet runs GNU Linux and is connected
to the Internet for remote maintenance and monitoring.
A central computing facility at an undisclosed branch of EXISTech's
headquarters monitors a worldwide network of bathroom facilities.
However, each fixture or group of fixtures has its own processor
so that the systems will continue to run during network outage.
Statistical information is logged locally and files are updated
using the UNIX rsynch feature, when the network comes back online.
Figure 2 shows the result of statistical analysis performed by a
miniature computer system [2].
[insert FIGURE 2 http://wearcam.org/fig2.tif and CAPTION somewhere near here]
[1 column width]
SMART TOILETS
In addition to automation of water flow to sinks, showers, bathtubs, etc.,
EXISTech sensors may be used to automatically flush toilets and urinals.
A single sensor can control a row of fixtures. With toilets that are
in stalls, one sensor is needed for each two toilets. However this still
provides a savings, in that if there are an even number, N, of toilets,
then only N/2 sensors are needed.
Typically, a passive infrared sensor is used for automatic flush,
because passive sensors tend to perform better at detecting recently
deposited waste products (owing to heat from the body which causes
waste products to be warmer than their environment). Since toilets
and urinals are supplied with cold-only plumbing (as opposed to
showers and sinks which usually receive both hot and cold water
supply), the passive thermal sensor is the best choice for
automatic flush.
Computer vision systems with sensor arrays save water by
distinguishing the difference between urination and defecation,
as illustrated in Figure 3. When urination is detected a smaller
quantity of water is used, whereas more water is used to clear the
solid waste matter from the bowl after defecation is detected.
[insert FIGURE 3 http://wearcam.org/fig3.tif and CAPTION somewhere near here]
[3 column widths, across page]
Sloppy aim, forgetting to flush, and other acts of negligence can also
automatically be detected and appropriate action taken (notification of
authorities, automatic flushing of toilets, automatic release of
perfume or pepper spray, etc.)
Advantageously, ceiling mounted infrared sensors can also be used to
automatically categorize inter-stall activity, such as distinguishing
changing of clothes from the passing of contraband, as illustrated in
Figure 4.
[insert FIGURE 4 http://wearcam.org/fig4.tif and CAPTION somewhere near here]
[3 column widths, across page]
SMART SHOWERS
Additional features of image-based motion detection can also be applied
to shower rooms to provide user safety and security, by way of watching
the user to make sure that the user is attended to when encountering
danger through tripping and falling, such as when slipping on a soapy
shower room floor.
High quality pictures assist facility managers in monitoring usage
patterns, to help reduce or eliminate deviant behaviour such as excessively
long showering, shaving in the shower room, vagrancy, the washing of clothes
in the shower room. Using the appropriate software, with artificial
intelligence, management can be sure to maximize user satisfaction by
making certain one inconsiderate user does not decrease the
user-satisfaction of other users.
In addition to deterring crime and vandalism, intelligent plumbing
fixtures can also be used to respond to acts of terrorism.
A regular shower facility in a typical locker room setting
can also double as a mass decontamination facility in times of emergency.
In this case, having full color video feeds assists remote decon officers
in determining, for example, if a powder on a patient's body is grey
powder such as anthrax, or some other color of powder.
A unique hexagonal architecture, as shown in Figure 5, speeds
throughput, when used with a six person sensor operated column
shower installed in each of the men's and women's shower rooms.
[insert FIGURE 5 http://wearcam.org/fig5.eps and CAPTION somewhere near here]
[2 or 3 column widths, across]
A six person column shower having six nozzles around a hexagonal/round
column, is used to match this hexagonal architecture.
Optics is comprised of a single sheet of smoked polycarbonate that
was heated and bent around the outside circumference of the round column,
and then inserted inside, after six viewing holes were drilled.
A typical installation of this invention uses optics with approximately
15% transmissivity, so that the degree of light coming back from light
that first passes into the viewing window and back out is 2.25%, which
falls nicely below the 4% level of light reflected from typical such
material. This allows color cameras to be concealed within the
column, out of sight of vandals.
An adhesive sealant makes the inside of the column water tight.
Six video cameras are installed in the column with a 45 degree mirror on
each one. Every second camera is pointing up from underneath, while the
other three point down from above. An embedded computer embodies video
capture devices in the column to locally control six separate solenoid
valve actuators, while also providing remote connectivity.
Appropriate software detects the presence of users, and turns on the
appropriate showerheads where flesh is detected. In this way no water
is wasted.
In a typical installation, one such column is placed in the hexagonal
men's and women's shower rooms of a mass decontamination facility [3].
Since there are six cameras in each shower and six cameras in the
central triage room there are a total of 18 cameras, which can be
displayed on two television sets as a 3 by 3 mosaic of images (a
9-up image on each TV). This allows two TV sets to be used, one for
the men's side and the other for the women's side. A square lattice
of images ensures the same aspect ratio of any one image, so that
the images efficiently use the TV screen real estate at the decon
officer's station.
A typical screen shot of the sensor operated column shower in actual
usage (men's shower room, anthrax mailroom decon exhibit [4],
July 2001) is shown in Figure 6.
[insert FIGURE 6 http://wearcam.org/fig6.tif and CAPTION somewhere near here]
[3 column widths, across page, or perhaps run sideways to get wider]
CONCLUSION:
Infrared sensor operated plumbing fixtures can incorporate recent
advances in computer vision. Passive infrared sensor arrays are
optimal for computer control of toilets and urinals, whereas
active infrared sensor arrays are most suited to the control
of sinks and showers. However, for emergency preparedness,
where telemedicine and remote triage are essential, showers
should be equipped with full-color sensing capability, especially
helpful in the identification of perpetrators who may be among
the victims of a bioterror attack.
References and Notes:
1. See http://existech.com and http://wearcam.org/safebath/index.htm
2. This work is based on a new kind of Knowledge Representation
theory called "SETS", with example in http://www.wearcam.org/faucet.htm
3. See Canadian Patent 02303611, filed April 1, 2000, and link
from a July 2001 exhibit at Gallery TPW, http://wearcam.org/tpw.htm
with curatorial essay in http://www.existech.com/tpw/index.html.
4. See the press release, "Mailroom begins operations June 30th;
public tours July 5, 2001, at 7:00pm" and the call for volunteers
in a mass casualty anthrax decontamination drill,
http://www.existech.com/press_release_deliverify.htm
BIO: S. Mann was born in Canada, and spent much of his childhood days
inventing various kinds of smart clothing and wearable computers. He
went to the United States to study at Massachusetts Institute of
Technology, receiving his PhD degree in 1997, prior to returning
to Canada as an Assistant Filing Clerk Trainee for EXISTech Corp.
He is also a faculty member at University of Toronto.
--------figure captions--------
All photo credits are to S. Mann, and artwork is by S. Mann.
See file mannfigs.txt, reproduced below:
FIGURE 1, http://wearcam.org/fig1.tif
CAPTION: Data output from an infrared sensor operated faucet, wherein an
infrared emitter, and infrared detector are up inside the spout
of a gooseneck faucet. The infrared detector returns a byte array
having 240 rows and 320 columns. By representing the numerical
values in the array as shades of grey, we can see the detection
zone, where the hands of a bather are present. A small part of
the spout (visible at the bottom of the array), as well as most of
the sink, is visible to the infrared sensor. Note also the
specular highlights from the infrared source that is built into
the sensor.
FIGURE 2, http://wearcam.org/fig2.tif
CAPTION: Statistical analysis of faucet sensor array data. This array
represents a Boolean array of bather present/absent. Such an
array may be used to automate sinks, shower facilities, and bath
tubs. Statistics of the bather-absent basin with the faucet
running, subtracted from a known average of known samples of the
array, of the empty basin with water running, are used to automate
the operation of the fixture.
FIGURE 3, http://wearcam.org/fig3.tif
CAPTION: Classification of waste type, and detection of negligence.
Passive infrared heat sensors are used for automatic flush,
where fixtures are connected to cold water only. The computer
vision system can detect waste products based on their thermal
signature. Urination, e.g. as happening in Stall 1, where
the stream of warm urine as well as the urine in the bowl,
is visible can automatically be distinguished from the
defecation taking place in Stall 3 where solid waste
matter is visible in the bowl, by virtue of the
thermal contrast between warm bodily waste and the colder
surrounding water. Note also that in Stall 4, there is a
visible thermal butt print on the toilet seat, and two thermal
footprints where the cement floor is still warm. This heat
signature indicates evidence of a recent usage of the toilet
in Stall 4.
FIGURE 4, http://wearcam.org/fig4.tif
CAPTION: A person in Stall 1 undresses and places clothes up on the
stall divider, temporarily. The computer can recognize that
all of the articles of clothing are eventually retrieved by
the user of Stall 1. However, when articles are passed from
one stall to another, the computer vision algorithms track
the passing of articles, such as in Stalls 3 and 4, where
contraband is passed underneath the stall divider. In this
way an alarm is sent to the authorities, together with live
video of only Stalls 3 and 4, while masking out the
information in Stall 1, to protect the privacy of the person
who is changing his clothes.
FIGURE 5, http://wearcam.org/fig5.eps
CAPTION: Hexagonal architecture increases bathroom throughput and
efficiency for mass casualty bioterror response preparedness,
such as in response to a spilled salt shaker or other
mysterious white or grey powder. Separate high security
Rotogate (TM) turnstiles for men and women lead into
gender-specific undressing areas. There six men, and six
women, remove all clothing, jewellery, wearable computers,
and other personal effects. When all six have complied by
stripping completely, they are allowed to pass into the
shower rooms, through turnstiles that are unlocked by
a guard in the central control room. Another six men
and six women are then herded into the undressing rooms
while the first group are showering. To protect the
privacy of those undergoing decon, the walls of the
central guard tower are made of darkly smoked bulletproof
glass or polycarbonate and there are black cloth baffles
inside the control room. Thus men cannot see through to
the women's side, or vice versa. After showering,
victims are then allowed to enter the examination rooms,
through another remotely unlocked turnstile. Telemedicine
facilitated by high resolution videocameras allows remote
triage staff to perform a medical exam, and permit healthy
persons to leave.
FIGURE 6, COLOR FIGURE, http://wearcam.org/fig6.tif
(if it is not possible to have 1 color fig, use fig6grey.tif)
CAPTION: Screen shot from the male decontamination officer's station.
The screen window is approximately 2048 pixels across, for
being displayed on a high resolution computer screen (the
kind of display typically used for an air traffic control
screen). Here a time-lapse video (live video of examination
combined with slightly delayed shower video, and much delayed
undressing video) displays the same victims appearing at three
different stages of decon at the same time.
S. Mann, 80 Spadina Avenue, Toronto, July 5th, 2001.