I invented a camera whose output could be authenticated. Nikon and Canon stole the idea. What happened next will shock you.

"Deepfakes" have been around for years, like this Coca-Cola commercial mixing living and dead celebrities.  But how can you prove if a video hasn't been manipulated?

(Note: This is an expanded version of the article I wrote for Cameracraft magazine.  I'm sharing it now because the idea needs to be out there.  Plus, I was on tap to give a TEDx talk about this subject in April 2020, but of course it was cancelled due to pandemic concerns.  So I'm putting it out there.  The world needs this invention!)

Once upon a time there was a saying: “The photograph doesn’t lie”.  While mostly true, you could still lie in the old days by attaching false captions or using a forced perspective.  Lying by manipulation came much later -- it was used heavily by the Soviets during the time of Stalin, and then by the advertising industry (which is synonymous with lying, really) with the invention of the Scitex imaging workstation in the 1970’s.  But the ability to really lie via manipulation didn’t reach the masses until Photoshop came along. 

1989.  US Ambassador Vernon Walters presents photographic evidence at the United Nations Security Council, supporting his claim that a Libyan MiG-23 shot down by US fighters had been armed.  “It is untrue!” exclaimed the Libyan ambassador.  “The pictures were fabricated; they were directed in the Hollywood manner!”

People (journalists and academia, mostly) started to sound the alarm in the 1970s about how you can’t rely on the photograph for evidentiary purposes anymore.  And I started to collect examples of famous photo manipulations in history which made a difference (good and bad).  Some historic examples are sprinkled throughout this article.

Back in my NASA days I identified this as a problem that needed solving – in my view, society was relying too heavily on the image whose sanctity was eroding, and I made it my mission to restore it.

Photographic proof that Rambo and Groucho were at the Yalta conference.

My Solution

The solution I came up with imposed no restrictions on image use and required no special or restrictive formats.  Taking a picture with my special digital camera would produce two files: The first is an industry-standard image file like a .jpg or RAW file, which can be used as you would any other image file (you can even manipulate it if you wish).  The second file is a smaller encrypted “hash” file called a digital signature which is kept alongside the first file to enable validation later on. 

If the image hasn’t been manipulated, you’ll be able to prove it in court by feeding the image in question, the encrypted hash file, and the camera’s serial number into freely available verification software.  If even a single bit has been altered, the validation will fail. 

The comic strip Bloom County was one of the first to call this out as a problem in the 1990's.

Wait, there’s more!  My invention also sought to thwart the classic, non-manipulation methods of lying as well, such as attaching false captions or taking a picture of a manipulated image.  To combat these time-honored methods, my camera would add a colorful border around each image which contained bitmapped text describing when and where the camera was: It would note all the usual data covered in EXIF fields (shutter speed, f/stop, ISO, white balance), the time and date, a compass showing what direction the camera was pointed to, the distance the lens was focused to (to catch copying a manipulated image), and even the GPS co-ordinates of where the camera was using a built-in GPS receiver.  (Clearly this was an optimistic prediction, because back in the 1990’s when this invention was proposed, GPS receivers were so big they took up a goodly portion of the B2 bomber’s electronics space.)

Another time-honored way of lying with the image; no manipulation required.  My system would help thwart that technique as well.

How the camera works would take too long to describe in this short blog post; however if you’d like to learn the details, I refer you to read the full technical paper, published in the Nov. 1993 edition of IEEE Proceedings of Consumer Electronics:

http://friedmanarchives.com/~download/blog/TDC/Trustworthy_Digital_Camera_Technical_Paper.pdf

In my proposed system, the environmental variables are authenticated as well as the image itself – in this case information in the colorful border, indicating the time, date, exposure variables, white balance, what direction the camera was pointing, and where you were in the world when the shot was taken.   Here’s an image of youthful me standing next to Captain Kirk, holding another invention – a one-handed keyboard for astronauts.

As a NASA employee, whenever you invent something, NASA owns it whether it was work related or not.  They unleash their army of patent lawyers to file a patent and pursue it until it’s granted.  And they give me a $500 award, which NASA thinks is a great deal because most patents never make any money and the engineer doesn’t have to spend tens of thousands of US Dollars getting the patent on their own.  NASA also has the means to advertise the patent to large U.S.-based companies for licensing, again something individual engineers cannot do.  Fair enough.

The Patent

There was interest.   Polaroid Corporation, who had been searching for a solution to this very problem for over a decade, flew a cadre of executives to California to hear me give a talk on the subject, with the intent of possibly licensing the idea.  But Polaroid collapsed before it could be acted upon.

Polaroid was extremely interested in licensing the technology; they even flew a cadre of high-level managers out to hear me give a presentation.  I threw this prototype magazine ad together, thinking past the sale: "Here’s what you’ll be able to communicate to the world!"

Then something interesting happened.  In 2003, Canon stole my idea.  With the introduction of the EOS-1Ds, Canon also introduced the Data Verification Kit DVK-E1 which works only with this high-end camera.  Three years later, Nikon did the same thing with the introduction of the Nikon Image Authentication Software, which worked only with their high-end D2Xs DSLR, and later the D2HS, D2X, and D200 cameras. 

Both Canon and Nikon came out with their own version for their high-end digital cameras.

Frustrated, I appealed to the NASA patent office, pointing out the blatant theft of intellectual property, but they did nothing.  As it turns out, the patent was only filed for and granted in the U.S. (Canon and Nikon are both Japanese companies), and besides, it turns out NASA is not in the business of defending IP violations – just filing for patents so they can help justify their existence to Congress.

Just as I was starting to learn to let go of the outcome (a valuable lesson for inventors and non-inventors alike), the whole thing collapsed.  A Russian research firm called Elcomsoft identified a flaw in Canon’s implementation, exposing the private encryption key and rendering all of the cameras using this product inert.  Nikon’s scheme was similarly broken six months later.  In both cases, the secret encryption key had been extracted from the camera, a sign that the hardware system was designed by engineers unfamiliar with data security techniques.  Had either of those companies hired me to be their product manager, I would have made sure that one compromised camera would not have invalided the system for all other cameras.

Scientific American also called this out as being a problem.

I was even more disheartened that neither company made any attempt to fix their product or come out with a new, stronger version.  While Canon made a tiny announcement that the system had been compromised and invited customers to contact them, Nikon was completely silent.  The products just died, just like Kodak’s PhotoCD standard.  And the world either hasn’t noticed or doesn’t seem to miss it.  I interpret this as "The product didn't sell very well", and "Global interest in this problem is weak".

Like all young inventors, I thought my solution was going to save the world, especially in situations where truth is valued, like when the photo was being used as courtroom or insurance evidence.  Shortly after the patent was granted I approached a lawyer friend of mine and excitedly told him about my invention.  He shrugged it off.  “If we ever question the authenticity of an image submitted by the other party, we hire our own photographer and take our own pictures from our own perspective”.   This was an important lesson I would learn time and again as a frustrated inventor – if you have to explain to people that there’s a problem before you can present the solution, you’ve already lost the race to a product's success.

Today we have deepfake videos whose manipulations are getting increasingly harder to detect, making a solution like this even more important than it was in the 1990's when I invented it.  But the patent has since expired, and I'm pretty much done trying to change the world.  I'm sharing this idea in hopes that someone will take it and run with it.  

===
Until next time,
Yours Truly, Gary Friedman

*** UPDATE 10/28/23 ***

The idea lives!  A new standard for image authentication which pretty much copies my idea is being drawn up by the Coalition for Content Provenance and Authenticity (C2PA).  Sony, Leica, Canon, Nikon, Adobe, Arm, BBC, Intel, Microsoft, and Truepic, and Twitter have signed on, and the first camera to employ this "Content Credentials" technology was the Leica M11-P which costs a mere $9,125. :-)   

Of course my patent has long expired, and this adds to a long list of ideas I first thought of back in the 1990's that others are getting rich off of now.