Fun note: A children's guide to comparametric equations (simple experiments one can do at home to understand comparametric equations) is included in this directory, in the file, photocell.pdf (requires a cheap photocell, ohm meter, and lamp). It is not necessary to read this paper, to do the lab, but it may help by providing a fun and creative interpretation of comparametric equations.
You may submit this assignment as a gzipped tar file which contains a .tex file and the associated .eps files. As long as the document compiles, this will most likely make the T.A. marking this assignment most happy. Otherwise, a postscript file which may be read with gv is acceptable, or a .pdf file which is viewable using xpdf is also acceptable. Send these files to corey@eyetap.org.
For this lab, the following two images [1][2] will be used:
The first image has an exposure time of 1/16th of a second, the second has an exposure time of 1/8th of a second. If you click on the images, you will get the full-sized versions. Since the full size versions contain more data, it is preferable to use them. (Better statistics result from the use of more data.)
Once you have the full sized images, you may want to decompress the jpeg images to get uncompressed ppms using a program such as djpeg. For now, you may also want to convert the images to greyscale, or possibly only use the green channel. To understand why comparametric research is important in image processing, consider the first question on this assignment:
Be creative about this. Describe the intent of your method in your report, as well, hand in the octave code to carry out the method. Include the resulting image as well as a difference image and the sum of squares error. To do this effectively, what who need to devise is some function which will take pixels from the first image and make them approximate the corresponding pixels in the second image. The function may be as simple as a scalar multiplication. However, you are free to make the function any location independent function you want to bring the pixels into a better approximation of the second. The location independence restriction requires that the function act only on pixel values, irrespective of where they are located. This is just to prevent doing something like merely selecting the second image as your result. Therefore your algorithm must work on other image pairs that you have not yet seen. The TA will test your algorithm on another pair of images (taken with the same camera, one at 1/16th of a second, the other at 1/8 of a second) that you have not yet seen. The function should work on any two images taken from the same camera, as long as the exposure difference is the same as the two in this assignment.
Don't spend too much time on this question, it's meant to simply outline the nature of the problem. The later questions will take more work, time, and thought.
Consider this really simple example, using a twelve pixel camera (four pixels across and three down). We have the following 3x4 images (shown in pixel values). The second one is from a 12 pixel camera having a doubled exposure time to the same subject matter:
image 1:
| 1 | 3 | 2 | 3 |
| 3 | 2 | 1 | 2 |
| 0 | 0 | 2 | 0 |
image 2:
| 2 | 3 | 3 | 3 |
| 2 | 2 | 2 | 2 |
| 0 | 1 | 3 | 0 |
To understand what a comparagram is, consider the following example:
A comparagram is formed by first creating an N by N matrix where N is the number of possible greyscale values in the image (usually this is 256 however, in this toy example, it will only be 4). First, the matrix is filled with 0's. Now consider the first pixel in the first image, in location (1,1). In the first image the value of this pixel is 1. In the second image with the doubled exposure, the value of the pixel in the same location is 2. Therefore, in the comparagram's (1,2) entry, we add 1. We continue in this way, accumulating entries, for all the pixels in the images, giving the following comparagram:
| 2 | 1 | 0 | 0 |
| 0 | 0 | 2 | 0 |
| 0 | 0 | 2 | 2 |
| 0 | 0 | 1 | 2 |
As the comparagram is now with the noise around the underlying function, processing the comparagram becomes quite difficult. For this reason, we need to recover the underlying function. Some reasonable method to recover this function from the comparagram is needed. To make the later computations easier, each row of the comparagram may be slenderized into single points. This transforms the comparagram into a "comparametric function", which is also sometimes called a "comparagraph". If the above comparagram was slenderized, we might get the following comparagraph:
| 1 | 0 | 0 | 0 |
| 0 | 0 | 1 | 0 |
| 0 | 0 | 1 | 0 |
| 0 | 0 | 0 | 1 |
Also, the matrix form which has been used up to now can be a confusing manner in which to view the comparagram. Using octave, rotate the matrix such that the top left corner of the matrix is in the bottom left corner. If you view the comparagram in this manner, it's easier to see the comparagram as a function, where the origin is in the bottom left corner (just like what you are probably used to). Now, look at individual columns of the comparagram. Before our rotation, these were matrix rows, now they are columns. We want to reduce each column to a single value. Note that even though the values represented in the comparagram are integers, our final resulting value need not be an integer. For example, if you were to use a method of first moments on each column, the number you would get would not be an integer, but this is fine.
Print out your rotated comparagram (once again compressing it to pixel values). Also, print out your comparagraph (slenderized comparagram). Ideally, you want to show the comparagraph on top of the comparagram. This will show how well your algorithm slenderized the data. Also submit your slenderization code and a small explanation of the intent of the algorithm (a few sentences).
Lets now consider a simple model for a digital camera. Incoming light passes through one or a series of lenses and strikes a sensor array. We won't worry about the function of the lenses. For now all we care about is the sensor array.
Consider first one sensor in the sensor array of a camera, and assume that it represents 1 pixel. If no light falls on the sensor, we expect that the resulting pixel value will be zero. We also assume that if much light is collected by the sensor, we will get a value of 255. Unfortunately, the sensors used in cameras do not respond exactly like our eyes (i.e. are not photometric) or respond uniformly to all wavelengths of light (i.e. are not radiometric). This means the data collected by the sensors needs a new unit of measure which shall be referred to as a photoquantity. It is known that photoquantities (q) are linear. After the data is collected by the sensor, a camera response function is applied to the photoquantities to compress the dynamic range. The value is then quantized to form a pixel value.
It is logical to think that the camera response function is monotonically increasing. For example, if the amount of light striking a particular sensor is increased, we do not expect the resulting pixel value to lower. For this reason, the camera response function (f) must have an inverse. We could apply this function f inverse to the pixel values of the image to get photoquantities. Such an image which has had f inverse applied to it is called a portable lightspace map. A portable lightspace map (PLM) is usually represented as IEEE double-precision values, and as mentioned, these values are linear in the photographic quantity, q.
Consider what a comparagram is. From the viewpoint of the sensors, some amount of light has struck a particular sensor. The camera response function is applied to get the imagespace picture (the jpeg image). If we call the camera response function, f, and the vector of photoquantities from the sensor q, then the first uncompressed image (our ppm image) is f(q). The second picture is of the same subject matter, but the exposure time is twice as long. Thus, the photoquantimetric values from the first image have been doubled or we have multiplied the original vector of photoquantities by 2 to form 2q, then the camera response function has been applied. The second image is therefore f(2q). This means that the comparagraph (slimmed comparagram) which was computed previously is actually a "meta-function" of f(q) against f(2q). Such a functional relationship between a function f(q) and a function f(kq), for some scalar constant, k, where the relationship does not depend on q, is called a comparametric equation. For example, the straight line of slope sqrt(2) given by g(f) = sqrt(2) f (where g(f)=f(2q)) is a comparametric equation with a solution given by f = sqrt(q).
(What this question means is that a plot of f(2q) as a function of f(q), is a straight line that does not necessarily pass through the origin, so it's a slight generalization of the above example.)
(Straight lines are often used in comparametric theory, e.g. Google search "comparametric function" returns paper http://iul.eng.fiu.edu/candocia/Publications/Barros_ICASSP2002.pdf which uses various piecewise straight lines to approximate comparagrams, but the above question in this lab is a lot easier because you're just using one straight line for the entire comparagram, so don't get too distracted by the complicated nature of Barros_ICASSP2002.pdf).
This sort of function composed of functions for which the original function f is unknown is termed a first order function equation, or in this context a comparametric equation. Finding closed form solutions for f can be very difficult. Also, just as in solving differential equations of integrals, usually a family of functions is found. Interestingly, the same proof using measure spaces, measure theory, connected sets, etc. which shows the existence of solutions to differential equations may be applied to show the existence of solutions to comparametric equations. However, this is not needed for our problem. For the purpose of bringing pixels to photoquantimetric values, all we need is a numerical solution.
Right now, the comparagraph which you have derived is only defined at discrete points. For deriving a solution for f, it is helpful to define f on all values in the range [0,255]. You may use a spline using the the 255 data points you have in the comparagraph. Warning, the next question is quite difficult and is therefore a bonus question. You will only need to use splines for interpolation if you are going to attempt the next question. If you are indeed going to try the next section, you will want to become familiar with the octave-forge functions spline and ppval.
The lookup table should look something like:
| |pixel values| | photoquantimetric value| |
| 0 | 0 |
| 1 | q_1 |
| 2 | q_2 |
| 3 | q_3 |
| 254 | q_254 |
| 255 | q_255 |
If the unrolling of the comparagraph is too difficult, you may simply use this pre-calculated lookup table, available here