Wearable Computing and The Veillance Contract

Section 1

Slide 1 now showing

My grandfather taught me to weld when I was 4 years old. It was an experience that was at once both exhilaratingly fun and frightening. Frightening because I saw the world "through a glass, darkly".

Slide 2 now showing

You see, while wearing a welder's helmet both eyes see the world through one piece of glass that is very dark. Through this dark glass everything except one little "pinprick" of light == the electric arc == appears almost black.

Slide 3 now showing

Throughout my childhood I experimented with television -- a then new technology -- as well as electric circuits, computation, and the like. All as a way of seeing better - something I felt was a basic human need.

This led me to the invention of HDR (High Dynamic Range) imaging, now used in most commercial cameras including the Apple iPhone.

Here is what I see through my glass while welding.

... Seen for the first time in human history.

Section 2

Fast forward a bit; Our family was one of the first few in our city to get a computer.

Slide 4 now showing

Back in the 1970s my childhood experiments in computing involved wearing the computer together with a helmet-mounted television system/station to see the world photographically.

Slide 5 now showing

This was around the same time that the SpeedGlas, the first electronic welding helmet, appeared in 1981.

Speedglass darkens uniformly everywhere and thus doesn't help the wearer see better than traditional fixed-shade glass during the actual welding.

Slide 6 now showing

MannGlass adapts the image, improving visibility during actual welding. This led to my invention and patenting of HDR (High Dynamic Range) imaging == a technology now used in most digital cameras.

I spent much of my childhood as an inventor and amateur scientist, tinkering, creating, and exploring.

Now beyond my simple need to see the welding arc, I began to look at my everyday life through this photographic and computational glass, using a variety of "wearable computing" contraptions I invented, designed, built, and wore.

By the early 1980s I had miniaturized my glass from the 1970s helmets into an eyeglass device suitable for creating an "Augmediated Reality" in my everyday life.

Slide 7

A decade later, I took this invention to MIT and founded the MIT Wearable Computing Project.

Slide 8 (video)

[play 45 seconds of Nicholas Negroponte video == old 1990s televion broadcast]

[http://www.glogger.mobi/v/75560] [12 to 58 seconds = main point: founding of MIT Wearable Computing Project]

Slide 9

Many people from around the world came to see my work, including Mark Spitzer and a handful of others who built upon these ideas and created a commercial embodiment of the Digital Eye Glass (but without the camera). I showed them how they could build a camera into their MyVu product. It wasn't long after that MyVu was acquired by Google.
>

Slide 10

While still at MIT, I invented the augmented reality pendant, a gestural interface that contains both a data projector and camera, which I completed in 1998, and named "Synthetic Synesthesia of the Sixth Sense". 11 years later, another student presented this work at TED, which he called "SixthSense", and it became one of the top 20 most viewed TED talks of all time:

Section 3

Slide 11

Presently, I'm the Chief Scientist at Meta, makers of Spaceglasses.com, and my PhD student Raymond Lo is the CTO (Chief Technology Officer).

Slide 12 (video)

[Play the first 40 seconds of http://glogger.mobi/v/211654] [up to and including the wearable face recongizer]

Here you can see, for example, the "wearable face recognizer" I published 16 years ago (in 1997) in a paper entitled "The Visual Memory Prosthetic".

Section 4

Slide 13

Many inventions come from a research lab, but wearable computing, Digital Eye Glass, and HDR are inventions that came from my everyday life.

In this respect, these technologies are very much "in and of the real world".

Slide 14

I was very much inspired by the tagline of the world's largest technical society, the IEEE. Their tagline is "Advancing Technology for Humanity".
[Figure: IEEE logo]

SEEING AID CLOSE FOCUS

I've always felt this importance of "Technology in SERVICE of HUMANITY" and many of my inventions and ideas are driven by this ethos -- not just ideas, but ideals too.

Slide 15

So when I saw people bent up like pretzels, hunched over their computers, I thought the computer should be the thing that's "bent up" ---- the computer should be twisted and wrapped around the body and head ---- i.e. wearable computing.

And when I saw the GUI and "desktop metaphor" I came up with something else, for which I coined the term "Natural User Interface" back in 2001. Spaceglasses give us a natural interaction with the real world rather than being hunched over a desktop metaphor!

Section 6

Slide 16

For more than 35 years I've been inventing, designing, building, and wearing computers in my day-to-day life, and this has uncovered some strange paradoxes about society in general.

In the 1970s people would often cross the street to avoid me, but by the 1980s I'd finally developed a form of the technology that was much less offputting, and suitable for use in everyday life. However, despite the everyday acceptance, there were still specific members of society that had problems with it.

Slide 17

In 1985, 28 years ago, I was stopped by a security guard while getting onto the subway.

MEMORY STORAGE CAPACITY ... understand why security guards ...

It was mostly shopkeepers, security guards, and other authority figures who objected to my computerized seeing glass.

Ironically, it was those members of society who install and use surveillance cameras.... Those were the people who seemed to object to the glass ---- the very same people who were building a world of watching ---- were afraid of being watched.

At the time, my computer system didn't have the memory or storage capacity to capture and record much, if anything, of what I saw.

But nevertheless, having been physically assaulted by security guards afraid that the seeing glass might be recording them, I began to try and understand why ordinary people accepted it more readily than those in positions of authority who had such an adverse reaction to it.

Slide 18

Back in the 1970s and 1980s I was probably the only person walking around wearing a computer with wireless communications, but these days many people are walking around with a small computer, such as a smartphone or other device that includes a camera, computer, and wireless data link.

Slide 19

And I'm no longer the only one facing these issues. Last year when I was physically assaulted by McDonalds staff, in Paris, France, I did a little research and found that I wasn't the first. A travel agent had previously been assaulted by a McDonalds staff for photographing the menu in this Paris location.

PRIVACY OF MENU

Slide 20

Many people today also use smartphone applications that translate signs from a foreign language, or use their phones to read QR codes.

There is a certain irony in the recent proliferation of "no photography", "No cameras" and "No Cellphone in store, please" signs along with QR codes that give pre-purchase product information.

The QR code is like a sign that says "Cameras required", often seen together with signs that say "cameras forbidden".

Slide 21

Most of the places that forbid cameras are quite full of cameras. For example, at the US Consulate, when picking up my daughter's passport, they had a rule that no electronic devices were allowed in their building. So I was both required to enter to pick up my daughter's passport, and also forbidden from entering because I myself, am an electronic device. By their rules, I was existential contraband == contraband by my mere existence.

I've sometimes pointed out this irony to the shopkeepers and establishment owners: "Cameras are forbidden but you have quite a few of them here", to which they reply, "but those are SURVEILLANCE cameras".

Slide 22

Somehow a "surveillance camera" is not a camera! So I started trying to understand surveillance. Perhaps if I streamed my eye glass video to the police, maybe I'd be allowed to use it as a seeing aid, because then it would be a "surveillance camera" which they say is not really a camera so it would be allowed?

Slide 23

"Surveillance" is a French word which means "watching from above" or "watching over". "Veillance" means "watching" == from the French verb "veiller" which means "to watch". "Sur" means "over" or "from above", as in "sur tax" or "sur charge".

Slide 24

Random House Dictionary defines "surveillance" as "a watch kept over a person, group, etc., especially over a suspect, prisoner, or the like: The suspects were under police surveillance."

Under assault by these security guards I felt like the suspect or prisoner in this definition. Whatever I had invented was the opposite of surveillance, so I made up a new word to describe or define it: "sousveillance", from the French prefix "sous" which means "under", as in "sous-chef".

So now there are two veillances: surveillance and sousveillance. My six-year old daughter made a couple of nice drawings that illustrate these two veillances:

and
[Figure: http://www.glogger.mobi:81/users/mann/image/2013_05_30_16_14_39_91545000.png]

Children see things in an amazingly clear way. For example, she also asked "Why do buildings have more rights than people?" (e.g. why are buildings allowed to wear cameras when people are not?), and "why is merchandize more valuable than people (e.g. why is merchandise allowed to protect itself with cameras when people are not?).

DASHCAM NEVER A PROBLEM IN A DRIVE_IN THEATRE

Obviously its really the shopkeepers and building owners that have these rights.

Slide 25

For example, a New York police officer was caught planting crack cocaine, and then photographing the "evidence", in a shop when he thought no one was looking. But the shop's surveillance cameras caught the officer planting the evidence, from two different camera angles.

When the shopkeeper was arrested for posession of crack cociane, he presented his shop's surveillance video and was then released.

In this case, his surveillance cameras were doing some sousveillance.

Imagine what might have happened if the only recording of this incident was one made by the police cameras they used to photograph the "evidence".

Slide 26 ("CrackPack")

Outside his shop he should still have the same right to capture his own memories for the purpose of proving his own innocence -- in case his backpack became a crackpack -- but authorities and property owners are quick to forbid such personal recordings.

ARCHITECTURE OF ONE

Slide 27 ("McVeillance")

This one-sided nature of surveillance came through when I asked McDonalds to investigate their assault against me, and they refused to consider their surveillance videos. They only wanted to consider their employee testimony. In this sense, the one-sided nature of surveillance concentrates evidence into the hands of one entity, and therefore creates a possibility of fraud through an inherent conflict-of-interest.

Slide 28

Therein lies the hypocrisy of surveillance. Surveillance is the veillance (watching) that says "we're going to watch you but you're not allowed to watch us".

Thus surveillance is the veillance of hypocrisy.

Sousveillance -- the new word that I made up -- is the opposite of surveillance.

What is the opposite of hypocrisy...?

[pause].

Slide 29

The opposite of hypocrisy is integrity.

Sousveillance, therefore, might be regarded as the veillance of integrity.

Section 7

Slide 30

I've looked at human rights approaches to veillance, e.g. having governments step in and make it illegal to forbid the use of a seeing aid, or visual memory aid, on the basis of discrimination. Indeed, business establishments like T&T Supermarket in Toronto are actually in violation of the law because they forbid the use of a seeing aid in their establishment.

I've also considered liabalization: An organization forbidding digital eye glass is mandating a change in the way we see (or don't see) the world, and therefore could be liable. For example, I've got a form for people to sign, accepting liability if I should slip and fall because of a sudden change (from digital eye glass to none) mandated by their organization.

Slide 31

However, these approaches rely on government intervention ---- something some people are in favour of, but others (especially libertarians) are against. Some believe a business owner should be able to say "this is my store and cameras are not allowed here for any reason".

SPECIFICALLY AGAINST SEEING AID BUT OK WITH NECKLACE

So, I have a different answer == one that would even appeal to nearly everyone, even the libertarians.

Slide 32

So I want to now present a concept I call the "Veillance Contract".

We all know what a contract is. Suppose, for example, you and I agree to form a business partnership and we both draft up a 100 page contract. The last page is the signature page, that we both sign. We both keep copies of our contract. If I lose or destroy my copy I am still bound by the contract.

But suppose I am forbidden from having a copy of the contract. Legally, the contract is not valid. Why?

[pause].

You see, if you're forbidden from receiving a copy of the contract, there's an increased incentive for fraud, e.g. the other party could change some of the pages leading up to page 100 = the signature page.

That is why the contract is not valid unless both parties are allowed to receive a copy of it.

Contracts are binding whether written or oral, but until recently we've not had an easy way to document oral contracts.

Now the veillance contract is a contract for the digital and new media age. There is a certain "social contract" or "ad idem" mutual understanding about how we should and do conduct ourselves in the world.

So my proposal is very simple: If party "A" wants to record the activities of party "B" and also wants to forbid party "B" from having or making any recording of the activity, then any recording that "A" makes should be inadmissible as evidence in a court of law.

I call this principle the "Veillance Contract". So the shopkeeper or business owner who asks the government to stop meddling in his or her affairs, gets exactly what they asked for ---- they shouldn't come crying to that same government when they want to use that government's courtrooms and prisons to prosecute shoplifting crimes, for example, where they've forbidden suspects from recording their own evidence for use in their own defense.

In particular, T&T Supermarket should not be able to use their video surveillance recordings as evidence, because they've, in effect, destroyed (or prevented the collection of) a person's own evidence the person might use to exonerate themselves.

Section 8

Slide 33

Surveillance is the centralized veillance of the single authority, whereas sousveillance is the distributed veillance of the many.

Slide 34

Traditionally surveillance housings looked like this:

Slide 35

Zabou's interpretation of sousveillance:

seems dated.

Slide 36

but now they're often housed in mysterious domes so we can't see which way they're "looking"

In many ways, surveillance tends to be secretive.

So imagine if we followed the example of our governments and industry leaders, and their ideals of secrecy, i.e. if we were to each wear a spherical dome around our head so that nobody could see which way you were looking.

Slide 37

But sousveillance draws its strength not from secrecy, but from the opposite of secrecy: openness.

Slide 38

Sousveillance, by its nature, tends to be open and distributed, where videos that document corruption and wrongdoing go "viral" on social networking sites.

Data integrity is often protected by distributed storage, e.g. instead of recording data in only one location -- where it can be lost accidentally or deliberately -- we spread the data around through a "RAID" = redundant array of inexpensive disks -- and even through cloud-based offsite data backups.

The opposite of data integrity is data corruption.

Thus surveillance and secret centralized storage is the veillance of hypocrisy and corruption.

Sousveillance counterbalances surveillance, by bringing to it the possibility of integrity.

Section 9

Let me end with two examples, one of data corruption due to centralized storage, and one of data integrity due to distributed storage and dissemination.

The first example was of a Brazilian electrician, Jean Charles de Menezes, shot dead (8 times) by police on the London subway system. It was a case of mistaken identity (they thought he was a terrorist but it turned out they had the wrong person). What's most salient is that there were four CCTV recordings seized by police who claimed all 4 recordings were blank.

Slide 39

[Cite: Row over 'blank' CCTV tapes at station Rosie Cowan and David Hencke The Guardian, Tuesday 23 August 2005 00.08 BST]

The second example comes much closer to home. I live with my family on Dundas Street West, where we all ride the "505" streetcar to school each day.

Slide 40

On that very same streetcar, Sammy Yatim was alone with a police officer who shot him 8 times (the officer fired 9 shots and one missed). After being shot 3 times, and falling to the ground, there was a long pause. Then Sammy was shot 6 more times, and then, sometime later, Sammy was tasered.

Like most of our streetcars, this streetcar had four surveillance cameras in it. Here's a picture of one of the surveillance cameras on the 505 streetcar, which I took before the shooting:

[Figure: http://www.glogger.mobi:81/users/mann/image/2013_07_04_09_28_46_25717100.jpg]

Jean and Sammy were both shot 8 times, under the surveillance of four cameras. In Jean's case the only recordings were surveillance recordings. In Sammy's case, there was both surveillance and sousveillance.

Slide 41

It was the presence of sousveillane -- and its open nature (in contrast to the secrecy of surveillance), through crowdsourcing and social networking, which brought the situation to the pubic eye. In Jean's case, no charges were brought to the officers. In Sammy's case, the officer was charged with murder.

Slide 42

While opposed by security guards, police, and other members of the establishment, sousveillance may be the answer to a balanced veillance, completing the other side of the Veillance Contract.