"MathWorks tools enabled us to push the limits on the next piece of the technology that we were developing."
Hanumant Singh, Woods Hole Oceanographic Institution
Underwater optical imaging enables scientists to study everything from biological ecosystems to ancient shipwrecks, but the underwater environment limits the scale and quality of the images obtained. Lack of ambient light makes it impossible to image wide areas or large objects, and images are often distorted when the camera perspective interacts with irregularities on the sea floor. Photomosaicking, which involves collecting overlapping images and then stretching and warping them until they fit together, helps address these problems, but existing photomosaicking methods were developed for land-based projects and are difficult to adapt to underwater conditions.
Researchers in the Deep Submergence Laboratory at Woods Hole Oceanographic Institution (WHOI) developed photomosaicking software specifically for underwater imaging. WHOI used several MathWorks products throughout the project.
"Our work would have been impossible without MATLAB and the Image Processing Toolbox," says Hanumant Singh, associate scientist in the Applied Ocean Physics and Engineering Department at WHOI. "Trying to do the same thing in C would have taken a lot more time and effort, and would have been much more complex."
Underwater photomosaicking involves surveying extended areas of the ocean floor, collecting overlapping images, identifying their common features, and then merging the images to form larger mosaics.
“Taking pictures underwater is like driving in fog,” says Singh. “There are far more variables involved. For example, since electromagnetic radiation doesn’t travel very far under water, you must use your own light sources. And since there are no straight lines or walls underwater, trying to identify matching features in several images is difficult. In addition, the object that you are photographing has shadows that change size as you move away from it.”
Singh and his team wanted to develop photomosaicking methods to address the unique challenges of underwater imaging.
Traveling a few meters above the ocean floor with a submersible vehicle, the researchers took several photographs of an underwater site. MathWorks tools enabled them to solve many of the imaging problems that they encountered. Lighting underwater was irregular, which made it critical to capture digital images with a dynamic range exceeding 8 bits per pixel. “We could do this because MATLAB can manipulate 16-bit images,” says Singh.
The next step was to preprocess the images to remove lighting artifacts, such as shadows, by means of histogram equalization. They wrote MATLAB routines that work well for underwater imagery. The spatial image transformation (image warping) capabilities of the Image Processing Toolbox enabled them to preprocess the images using “in-situ” camera calibrations to correct for radial lens distortion. As a final preprocessing step, Singh’s team constructed multiresolution pyramids from each corrected image, using the Image Processing Toolbox for Laplacian filtering.
They then extracted features to match between overlapping images. As Singh explains, “High-quality feature extraction requires convolution with large image kernels. We accomplished this very efficiently using the two-dimensional fast Fourier transforms in MATLAB.” Then, using the Optimization Toolbox, they fitted successively more advanced sets of parameters to achieve pair-wise image registrations.
Moving toward global registration, they represented the image-to-image topology of the mosaic with a network matrix having a row and column for every image collected. These were sparse matrices containing zeroes, except where a row-column pair corresponded to a pair of overlapping images. “The ability to represent and manipulate sparse matrices in MATLAB made this approach practical and efficient,” says Singh.
The Optimization Toolbox enabled the researchers to perform topology refinement via a nonlinear Levenberg-Marquardt algorithm. “MathWorks tools have amazing flexibility,” says Singh. “For example, when we were stretching and warping our images, we could pose this as an optimization problem, pick up a function from the Optimization Toolbox, and quickly run it through.”
The WHOI researchers have applied their MATLAB routines to solve a host of underwater imaging problems and to image shipwrecks, plane crashes, and underwater archeological sites.
To survey extended areas of the ocean floor
Use MathWorks tools to process multiple, overlapping images and combine them into a single photomosaic