Lab 5 (10 marks)

HDR (High Dynamic Range) imaging

Step 1 (1 mark)

Acquire a plurality of pictures of the same subject matter that differ only in exposure. This might be for example 3, 4, or maybe 5 images, depending on the subject matter's dynamic range. The exposure ratio ("k") must be the same across all successive pairs. A good choice of "k" is k=2, such that each exposure is twice the previous in the sequence. Thus we might have:
  1. v1=f(k1 q);
  2. v2=f(k2 q);
  3. v3=f(k3 q);
  4. v4=f(k4 q);
  5. v5=f(k5 q);
You can designate one of these as the "reference" image, i.e. the image to which all the others refer, and in this sense let's for example, pick the middle image as the reference image.

Thus, without loss of generality, k3=1, and we have:

  1. v1=f(q/k^2);
  2. v2=f(q/k);
  3. v3=f(q);
  4. v4=f(k q);
  5. v5=f(k^2 q),
and with k=2:
  1. v1=f(q/4);
  2. v2=f(q/2);
  3. v3=f(q);
  4. v4=f(2 q);
  5. v5=f(4 q),

Step 2 (1 mark)

Compute the comparagrams, pairwise. Thus you'll have one less than the number of images. Note the data compation. For example, if we had five images each 42 megapixels, we begin with 210 megapixels, which gets compacted down to four comparagrams each 256 by 256 (65536 pixels), which totals about 16 kilopixels: more than 13,000 times more compact. Display the comparagrams on a nice tonescale that makes the tonal range clear and visible: 1 mark.

Step 3: "Comparasum" (1 mark)

Sum up the four comparagrams. Explain why. For this you can look at the four comparagrams and compare them to the "comparasum". Note which areas of the input comparagrams and the comparasum are populated. Show this (compared to the input comparagrams) on a nice tonescale: 1 mark.

Step 4 (2 marks)

I derived three equations in Monday's lecture. Fit the 3rd equation to your comparasum. Determine the best fit "a" and "c" parameter for your particular camera. Plot the comparagraph and compare it with the comparagram to make sure the fit is good.

Specifically, for 2 marks, as follows: determine the "a" and "c" values for 1 mark, and plot the comparagraph corresponding to these "a" and "c" values directly on top of the comparagram for 1 mark.

Step 5:

Use the second equation with the "a" and "c" values you found, in order to convert your pictures into "lictures" (lightspace pictures, i.e. from functions of "f" into functions of "q"). These will be "float" or "double" arrays.

Lighten the dark lictures and darken the light lictures, appropriately. Specifically, you'll end up with a plurality of lictures that each estimate the true quantity of light, q, falling on the image sensor:

  1. q~1=4 f^1(v1);
  2. q~2=2 f^1(v2);
  3. q~3=f^1(v3);
  4. q~4=0.5 f^1(v4);
  5. q~5=0.25 f^1(v5).

Step 6 (4 marks)

Combine these estimates together. Let's do this in a way that teaches the principle, even if it is less efficient in terms of memory space in computation.

Begin by constructing "certainty" images from each of the input images. The certainty image is the input image evaluated at the pixelspace Certainty function, C[0..255] which is for each pixel value, indicative of the certainty (confidence) in that pixel value. Clearly pixel values of 0 or 255 have zero certainty, whereas more moderate pixel values have greater certainty. C is the slope of the response function in pixel space. See chapter 4 of the book, or this paper,, Equation 13.

For an intuitive understanding of this, see Figures 7 and 9 from the above paper. For each of the five images in Figure 7, there is a corresponding certainty image in Figure 9. Intuitively speaking, the certainty images show us how moderate ("midsy") the input images are. Moderate areas of the input image give rise to high certainty values where the certainty image is white or almost white. Extreme areas of the input image (very black and very light) give rise to dark areas of the certainty image.

For 2 marks, show the certainty images.

Compute a weighted sum to get the best estimate of q. Weight by the certainty images:

q=sum(Ci qi)/sum(Ci).

For the remaining 2 marks, do something useful with the result, q.

You can feed it into tensor flow, or face recognition, or anything else, directly, where you can show a better result with the q, than with any of the individual input images.

Alternatively, you can generate a picture, v = f(p(q)), where p is some processing. This will show the true power of quantimetric image processing on an licture (lightspace picture) comprised of multiple exposures. A simple example of p is unsharp masking, deblurring, or image sharpening, to produce a picture that is sharper and clearer than anything else you can do with any of the input images.

Post your results to the following Instructable: