A system for traceable photography. Provisional patent application filed Jan 2001, abandoned Jan 2002, placing my rights to the invention in the public domain. I think a professor at Princeton filed a similar patent in Sep 2000.
Video Authentication Steganography
Inventor: Shanti Rao
Background
Photographic steganography (the hiding of information within a picture) is typically used for storing copyright information within a still picture (U.S. Patents 5,687,236 and 5,613,004) or for embedding control signals within a video stream (U.S. Patent 6,026,193).
As it becomes possible to alter images in real-time, the authenticity of video images may be called into question. Thus, there is clearly a need for methods for the authentication of video, allowing for authentication and detection of alterations to a video.
Summary
The invention provides a system and methods for placing a digital signature on video, to allow for authentication and detection of alterations within the digitized video.
encrypting the digest with the encoder's (e.g. a camera) secret key with hardware or software, and transmitting the the digest and image as part of a video. The video can be received. The system also can authenticate the video and can notify a user or viewer of alterations made to the video after encoding.
The digital signature is created within digital video data by an encoder, for example, a video camera. The digital signal also can be created after capture of an image, by a video encoding device. As used herein, the term camera is to be interpreted to include a video encoding device which processes an acquired image. An integrated circuit controls the image sensor of a digital video camera. A suitable image sensor can be, for example, a CCD sensor, a CMOS sensor or a hybrid CMOS-CCD sensor. Each encoder has a unique identification number and public key pair. As it reads pixel data from the image sensor, this circuit uses digital steganography techniques to encode information into a watermark for each frame. The message stored in each frame can include:
Additionally, information which does not vary from frame to frame may be stored in partially in each frame, including:
Frame encoding
An image watermark consists of a low-level distortion pattern spread over the image, encoding a digital message. Watermarks, which are not detected by people, can be read by a computer. They have been used primarily for placing copyright information within an image.
The encoder calculates the digest of the frame, then bundles this with other information to create the frame message. The watermark is applied so that the message is stored over the entire image, ensuring that the decoder can detect a cropped frame.
A decoder reads the watermark message from several frames and extracts the camera's public key. It then calculates a digest of the message, and compares this to the digest and sequence number in the watermark, decrypted with the public key. The decoder can detect:
Digest algorithms
Video authentication steganography is relevant for broadcast over lossy channels, such as compressed digital video or analog broadcast. Thus, the digest algorithm is designed to detect major changes in the picture, and is not the same as a digital signature.
The digest is sensitive to:
The invention also provides a method of steganographically marking a video.
The method includes acquiring a digital video image from an image sensor. The acquisition of a digital video image is well known to one of skill in the art and can be accomplished with a number of techniques and camera systems. The camera can be a digital camera having a digital image sensor. The image sensor can be, for example, a CCD sensor, using a variety of techniques known to one of skill in the art. Such acquisition techniques also can be applied to acquire images using other image sensors. Suitable digital image sensors include CMOS detectors and CMOS-CCD hybrid detectors. Video images also can be acquired using an analog camera with an analog to digital conversion. A variety of preferred embodiments are shown in the table below.
The invention provides a method for placing a digital signature within a digitized video. According to the invention, a digital video image is acquired, for example, using a camera equipped with an image sensor, for example, a CCD. A video image also can be acquired from an analog source through analog to digital conversion. The video images can represent color images or black and white images.
The image is stored, by pixels in a suitable memory device including, for example, static, dynamic, flash, magnetic core, persistent, volatile memory devices. The pixels can be stored using any suitable strategy including, for example, interlaced or noninterlaced formats.
The image is digitally processed using software running on a processor or coded into the integrated circuit to calculate a message digest. The message digest can represent: the amount of information that can be stored in the image with the steganography algorithm without reasonably altering the appearance of the image; the date and/or time that the image was acquired; the camera ID; or information derived from the image itself.
The message digest can represent the date and/or time the image was acquired, and how the clock was set with information obtained from a clock in the camera, from an external clock, synchronized with a GPS or other time reference set by the operator. The message digest also can represent the unique camera id which can, for example, be hard-coded into the camera during manufacture, laser marked on a chip, uniquely etched on the chip, stored in ROM, stored in a Programmable Gate Array, or etched into the PCB.
The message digest also can include the frame number. Frame numbers can be assigned directly by the camera circuit numbers every image it takes. The numbering of frames can be sequential or not sequential. It can, for example, be a pseudorandom sequence based on the camera id. The numbering system also can start indexing at any number, which could be a function of the camera id, date, and/or time. Using such a system, when the camera stops recording, it causes a break in this sequence, allowing detection of discontinuous camera shots. When the camera resumes recording, it resumes the sequence. The frame index, or the number used to generate the frame index, is stored. Image storage can be accomplished using any technique available to one of skill in the art. Images can be stored, for example, in flash or other nonvolatile memory; on a disk; in volatile memory. When an image is stored in volatile memory, it is generally preferred that the image data be moved to nonvolatile memory when the device loses power.
The message digest also can include information derived directly from the image itself. Such image derived data include, for example, using a digital checksum of the brightness and color of the image. Such a digest can be calculated at several image resolutions, so that a later detect loss of detail can be detected, for example, using an algorithm that can allow detection of changes to the image, regardless of overall change to brightness, overall change to contrast or minor degradation from uniform noise. Sources of noise include, for example, minor degradation from noise typically found on TV signals (diagonal black lines, etc....), station watermarks on the corners or borders of the image, horizontal or vertical squeezing, to make room for ads on the TV screen.
The message digest also can be calculated using a periodic checksum function, including , for example, a discrete or continuous sine or cosine transform, or a wavelet transform. The periodic checksum function also can be generated using a pseudorandom or nonrepeating pattern that can be generated, for example, by a cyclic polynomial function (like CRC) or can be retrieved from a stored table. The message digest also can be calculated by measuring the noise and information content of the image from the spectral density of the image.
After calculation of the message digest, the digest is then encrypted with the camera's secret key using, for example a DSP or a specialized circuit. The can be used with the RSA public key, or any other public key algorithm.
The system can then bundle the digest and the digital image together. The bundling process can be implemented by a specialized chip or by a DSP handling the encryption. When the video data is bundled for transmission as a digital video, the message digest can be transmitted along a different channel from the video stream. If the desired data format uses lossy digital compression, like JPEG, the digest may be added between lossy and non-lossy stages, so that the message digest can be reconstructed without data loss. The digest also can be overlayed onto the image with a steganography algorithm. When the video file is to be transmitted as an analog video, the digest is preferably overlayed oonto the image with a steganography algorithm.
A digital video image thus encoded can be transmitted as part of a video stream for digital video in a compressed or uncompressed form. When the video stream is to be sent in digital form, the data can be compressed by sending only the differences between successive frames, or where each frame is compressed independently. The video stream can be transmitted Along parallel wires or in serial on a wire, through fiber optic cable, wirelessly. The digital vidoe signal also can be modulated and demodulated, using digital compression, error correction, and detection.
A digital video image also can be transmitted as analog video with pixel values converted to voltage & current or using a digital-to-analog converter (R/2R ladder, digital pot, multiplexer, etc.). An analog video signal can be transmitted on wires (e.g., differential or single-ended, coaxial cable. analog fiber optic, paired cables , R/G/B/Sync, Y/L/C/Sync). The signal also can be broadcast in a variety of formats. The signal can be encoded, modulated, transmitted, demodulated, decoded, can be demodulated and re-modulated at a relay or can be digitized, compressed, and transmitted in packets. Such a video signal can be converted back into an analog or digital TV signal for display.
According to the invention, the encoded video data can be transmitted to a device that records and/or replays the video or transmits it to viewers. The encoded video image can be squeezed or stretched, so that it appears in a little box on a screen, or with a visible mark to indicate ownership (e.g., a station or network logo) or an invisible mark to indicate ownership.
The encoded video can be displayed to a device including a television, computer projector, through an analog or digital video recorder. Authentication of an encoded video of the invention can be accomplished with such a device by reading the message digest. When the video is transmitted as an analog signal, the video image is first digitized with an A/D converter, a DSP or specialized circuit or with an algorithm in software.
reading the steganographic messages on the video images
optionally removing these messages from the video images, for example, through digital subtraction or through image degradation, to discourage copying
The video data can be validated, for each frame, by: determining if the message digest was validly constructed with a known digest algorithm, for example, by re-calculating the message digest, using the same algorithm as in the encoding step. The video data also can be validated by inspecting the digital signature on the digest, and determining if it was signed by the circuit that generated the digest. Alterations to the video data can thus be detected by comparing the message digest to the video image to detect: image frames presented out of sequence; changes to the images that can be detected with the digest algorithm; which portions of the frame, if any, were altered. Such a comparison also can detect the amount of image quality loss, which could be used to disguise an alteration from the "information content" field of the message digest, from calculating the amount of information that can be stored in the image, using steganographic techniques.
The system can then notify the user or viewer whether or not the video was altered after it was captured with the camera or after encoding by the encoding system. Such notification can include notification if only portions of the image have been altered, which portions, of how much information has been lost in transmission (wich may indicate alteration) and can include to whom the camera belonged by looking up the camera id in a database on a computer. The system also can extract the date and time at which the pictures were taken, and the reliability of the date/time. The system also can detect if any frames were rearranged. The system also can detect if any frames from different sources were put together.
The system can, for example, notify the user by overlaying text on the video image being displayed, which can turned on or off by control panel or computer interface, or by transmitting information to a secondary image display device (computer, TV, projector) or by showing a copy of the image, with authentic or non-authentic areas highlighted, displayed, or removed or by re-ordering frames that were transmitted out of order. The system also can store authentication data in a computer file with the digitized image or with tracking numbers for synchronizing with the video stream. The system also can store authentication data on an analog storage device, for example, magnetic tape, with the authentication data stored digitally alongside the video stream.
Particular embodiments of the invention are summarized in the following table:
Sequence of events:
general purpose microcontroller, DSP or ASIC