Video Orbits v1.0, or, how to ``paint with a video camera''

This chapter is a brief tutorial on how to use Video Orbits version 1.0, while further information can be obtained from http://wearcam.org/orbits

Video Orbits version 1.0 partially implements the methodology of Chapter~4 of PhD thesis ``Personal Imaging''. (The work of Chapters 2, 3, and 5 will be incorporated into Video Orbits version 2.0.) More on Personal Imaging and related research is at http://wearcam.org/research.html

Video Orbits 1.0 allows you to ``stitch together'' multiple pictures of the same scene or object, taken from a fixed center of projection or of a flat surface. These assumptions may also be violated to some extent with successful results see some of the examples in http://wearcam.org/orbits/gallery.html

Video Orbits 1.0 can reassemble overlapping pictures of the same scene into one large "image composite", given the assumption that they lie in the same orbit of the projective group of coordinate transformations (e.g. static scene, constant lighting, and zero parallax, which happens if either the camera is at a fixed point in space but still free to pan, tilt, rotate about its optical axis, or zoom, or if the subject matter is flat). Given enough overlap in the input images, and a reasonable approximation to images being in the same orbit, the software can function without user intervention.

One can regard the process as embodying a ``painting'' metaphor, where the camera ``paints'' out images onto an empty ``image canvas''. If the camera ``paints'' back on itself (e.g. path loops around in a self-intersecting way so that it covers an area already ``painted''), some cumulative error may become visible, because release 1.0 does not re-adjust to distribute the error over various possible frame pairs (this will be fixed in release 1.1).

The four main programs you need to use to assemble such image sets are estpchirp2m, pintegrate, pchirp2nocrop, and cement (Computer Enhanced Multiple Exposure Numerical Technique).

The programs use the ``isatty'' feature of the C programming language to provide documentation which is accessed by running them with no command line arguments (e.g. from a TTY) to get a help screen. The sections for each program give usage hints where appropriate. Future versions will support the ``pipe'' construct (e.g. some programs may be used without command line arguments but will still do the right thing in this case rather than just printing a help message).

The first program you need to run is estpchirp2m, which estimates coordinate transformation parameters between pairs of images. These ``chirp'' parameters are sets of eight real-valued quantities which indicate a projective (i.e., affine plus chirp) coordinate transformation on an image.

The images are generally numbered sequentially, for example, v000.ppm, v001.ppm, ... v116.ppm (e.g. for an image sequence with 117 pictures in it).

After you run estpchirp2m on all successive pairs of input images in the sequence, the result will be a set of sets of eight numbers, in ASCII text, one set of numbers per row of text (the numbers separated by white space). The number of lines in the output ASCII text file will be one less than the total number of frames in the input image sequence. For example, if you have a 117-frame sequence (e.g. image files numbered v000.ppm to v116.ppm), there will be 116 lines of ASCII text in the output file from estpchirp2m.

The first row of the text file (e.g. the first set of numbers) indicates the coordinate transformation between frame 0 and frame 1; the second row, the coordinate transformation between frame 1 and frame 2, \ldots A typical filename for these parameters is ``parameters\_pairwise.txt''

These pairwise {\em relative} parameter estimates are then to be converted into ``integrated'' {\em absolute} coordinate transformation parameters (e.g. coordinate transformations with respect to some selected `reference frame'). This conversion is done by a program called pintegrate.

This program takes as input the filename of the file containing parameters from the ASCII text file produced by estpchirp2m (e.g. ``parameters\_pairwise.txt'' and a `reference frame' (specified by the user), and calculates the coordinate transformation parameters from each frame in the image sequence to this specified `reference frame'.

The output of pintegrate is another ASCII text file which lists the set of chirp parameters (again, 8 numbers per chirp parameter, each set of 8 numbers in ASCII, on a new row of text), this time one parameter per frame, designed to be used in order. That is, the first row of the output text file (first set of 8 real numbers) provides the coordinate transformation from frame 0 to the reference frame, the second from frame 1 to the reference frame\ldots

The program called pchirp2nocrop takes the ppm or pgm image for each input frame, together with the chirp parameter for this frame % from pintegrate, and `dechirps' it (applies the coordinate transformation to bring it into the same coordinates as the reference frame). Generally the parameters passed to pchirp2nocrop are those which come from pintegrate (e.g. {\em absolute} parameters, not relative parameters). The output of pchirp2nocrop is another ppm or pgm file.

The program called cement (CEMENT is an acronym for Computer Enhanced Multiple Exposure Numerical Technique.) assembles the dechirped images (which have been processed by pchirp2nocrop) and assembles them onto one large image `canvas'.

Additional notes

The estimation program, estpchirp2m, can take optional initial parameter ``guess'' inputs. This is useful when incoming images do not have a lot of overlap (i.e., taken with a still camera rather than a video camera).

Initial estimation parameters can be generated using the Abelian pre-processors (still to be ported to C), or idr.c (interactive dechirp/rechirp) may be used to experiment with manual specification of the starting ``guess''.

The history of Video Orbits

The collection of tools comprising the Video Orbits software was originally developed over a period of several years by Steve Mann while at McMaster University, University of Toronto, HP Labs, and M.I.T.. Others at M.I.T., including Shawn Becker, Nassir Navab, Chris Ggraczyk, Jeffrey Levine, and Kenneth Russell have recently contributed extensively to this project (see acknowledgements section of PhD thesis ``Personal Imaging'').

Related Research

The Wyckoff Principle (incorporated into VideoOrbits 2.0)

References:

IEEE Trans. Image Proc. Sept. 1997
U.S. Pat. 5706416