20 Color
-
Upload
jennifer-evans -
Category
Documents
-
view
243 -
download
0
Transcript of 20 Color
-
7/28/2019 20 Color
1/6
Image processing operations in color space using finite-state machines
Frederick M. Waltz
2095 Delaware Avenue, Mendota Heights, MN 55118-4801 USA
ABSTRACT
The SKIPSM (SSeparated- KKernel IImage PProcessing using finite-SState MMachines) has recently been extended with excellent
results to various grey-scale operations (morphology, Gaussian Blur, ranked filters, etc.) as well as 3-dimensional binary
operations. Another interesting direction, initiated in this paper, is the extension to color spaces.
After a discussion of some possible goals of such image processing, this paper presents implementation techniques and
examples. Because of the extra costs involved with the printing of colored images in journals, the example images are
presented here in grey-scale only. Color images will be provided via e-mail on request.
Keywords: image color spaces, binary morphology, grey-scale morphology, separability, finite-state machines
1. INTRODUCTION
The motivation for this paper is the opposite of the usual one: Instead of looking for techniques to solve a specified problem,
we here investigate possible problems to which a certain technique will apply. To paraphrase a common expression, we have
at our disposal an excellent and powerful hammer, and we are looking for things to nail with it. The excellent hammer
is the class of image processing implementation techniques called SKIPSM (SSeparated-KKernel IImage PProcessing using
finite-SState MMachines)1-20, which provides significant speed increases (in comparison to conventional implementations) in a
wide range of neighborhood image processing operations. For binary morphology with very large structuring elements and
such operations as the Grassfire transform, the speed increases are truly phenomenal, with speedup ratios of 100 or more
being common. For other operations, the speedup ratios are typically not as large, but are still worthwhile.
Recently, the technique has been extended to a number of grey-scale operations15, 18, 19, 20 and even to 3-dimensional and n-
dimensional morphology13, with excellent results. This paper explores a few of the possibilities of applying SKIPSM to
color images.
As a starting point, let us discuss Adobe Photoshop, a widely-used multi-platform software program for manipulating
images, and especially color images. A fairly wide range of standard image processing operations is built into this program,including Gaussian and other blurring operations, edge enhancement, nonsharp masking, something they call a median filter,
etc. Their standard user documentation does not give theirdefinitions of their operations, but it appears (on the basis of
operations investigated so far) that all operations applied to complete color images operate in RGB (Red-Blue-Green) space,
with the same grey-scale operation being performed on each color plane. The user can also separate the color planes into
three separate images, operate on each plane separately, and then recombine the results to form a new color image. Outputs
can be saved in CMYK (Cyan-Magenta-Yellow-Black) format and some other formats, but apparently not in HSI (Hue-
Saturation-Intensity) format, which is the one that generally makes the most sense for industrial inspection. In no case
examined to date is there any interaction between the RGB color planes, as there would be if the RGB information was first
converted to HSI, these signals were processed separately, and then the results were converted back to RGB.
An aside: For experimental purposes (although obviously not for on-line applications), it has proven to be very useful to be
able to move images back and forth between Photoshop and a spreadsheet, to allow virtually any kind of image processing
operation to be performed without the need to get inside Photoshop or some other image processing engine, or to spend
time writing and debugging C code which will the be discarded in a few minutes after your experiment is completed.
Techniques for moving images between Photoshop and Excel can be provided on request.
2. SOME POSSIBLE OPERATIONS
As indicated above, conventional color-space operations, on all planes together or on individual planes, include edge
enhancement, neighborhood averaging, etc. Figure 1 shows another possibility: Grey-scale erosion or dilation on one of the
color planes (RGB or HSI), an operation for which SKIPSM is particularly appropriate15, 19, especially if the desired
Image processing operations in color space using finite-state machines SPIE Paper 3521-33 Frederick M. Waltz SK20-1
SPIE Conf. on Machine Vision Systems for Inspection and Metrology VII Originally published Boston, Nov. 1998
Copyright July 1998 by F. M. Waltz. Duplication by permission only.
-
7/28/2019 20 Color
2/6
structuring element is not a simple rectangle or circle. The wavy line in
this figure is an attempt to represent the three-dimensional surface of the
color variable.
One possible place where operating on one of the RGB planes might
provide useful results is that in which there are image spikes or noise of
predominantly one color, say red. Eroding only the red plane wouldreduce this noise without affecting the other color planes, producing a
better signal. Figure 2 presents a synthetic and highly simplified 32x32
example of this type, in which 3x3 spikes in the red plane are completely
removed by eroding with a 5x5 circular structuring element (shown
below), with only minor changes a 2-pixel enlargement of the central
dark square in the main structure in this plane. (Because color plates are
not practical in this Proceedings volume, grey-scale versions of the
before and after images are shown here. These contain only the
intensity information (the I plane of the HSI representation). The smallest discernible squares in these images
represent one pixel. Color versions of these and the other images from this article will be provided on request via
e-mail. Contact the author at .
Similarly, dropouts or dark spots can be reducedby grey-scale dilation on one or more of the
color planes. Figure 3 shows grey-scale
representations of a color image before and
after the application of a diamond-shaped 7x7
dilation operator (shown at right) to the green plane only.
There has, of course, been some distortion of the signal
aspect of the image, in that the central light area of the green
plane of the image has been enlarged by three pixels all
around. Except in very unusual situations, a filtering
operation distorts the signal while removing the noise. The
point being made here is that if additional kinds of imageprocessing operations are available, there is a better chance
that one can be found to reduce the noise effectively
without distorting the signal too much.
These are clearly highly artificial examples. Should larger
erosion or dilation operators be desired, SKIPSM can provide
efficient implementations, including automatically-generated
compileable main-loop computer code19.
Image processing operations in color space using finite-state machines SPIE Paper 3521-33 Frederick M. Waltz SK20-2
SPIE Conf. on Machine Vision Systems for Inspection and Metrology VII Originally published Boston, Nov. 1998
Copyright July 1998 by F. M. Waltz. Duplication by permission only.
xy
huesaturation
or intensity
image plane
erosion
Figure 1. Grey-scale erosion on individual
color-space variables.
Figure 2. A simple 32x32 synthetic image before and after
erosion of the red color plane with a 5x5 circle.
Figure 3. A simple 32x32 synthetic image before and after
dilation of the green color plane with a 7x7 diamond.
Red Before
Green
Grey Before
Red After
Blue
Grey After
-
7/28/2019 20 Color
3/6
3. USING DIFFERENT OPERATIONS SIMULTANEOUSLY ON THE RGB COLOR SIGNALS
Figure 4 shows another possibility: Applying the same operation (erosion, in this example) to all the color signals or planes,
but using different-sized structuring elements. This would be appropriate for RGB images if the noise to be filtered out had
different size and shapes in the color planes note likely, but certainly possible, depending on the source of the noise. For
example, varying amounts of light in the red band might be affecting the
acquired image adversely. Varying light intensity and color has been a realproblem in outdoor automated vehicle recognition and control systems on
high-speed highways, because the ambient light is much more red near
dawn and sunset. (Human eyes readily adjust to this, so that humans are
generally unaware of slow ambient light changes, but electronic cameras
usually dont handle this problem very well.)
Rather than continue with more morphology examples at this point, a
linear processing technique will now be considered. A standard method of
finding particular features in an image, such as holes or corners, is to use
linear filters designed to trap the features in question. These are called
n-tuples in the PIP image processing environment16, but are not
available in typical image processing designed to produce images for
human viewing. The n-tuple operation involves computing the correlation
of pixels in the image with the selected (usually non-contiguous) pixels of
the n-tuple. SKIPSM can implement this operation rather well.
In the example shown in Figure 5, three
independent
n-tuple
detectors are
applied to
the RGB planes. In the red plane, the detector looks for bright
spots surrounded by a darker area (pupil of eye). In the green
plane, the detector looks for a light area below a darker area
(lower eyelid). In the blue plane, the detector looks for a lightarea above a darker area (upper
eyelid). The combined grey-
scale intensity image (Figure
5e), because it merges the three
fairly good individual detectors,
results in not very good
detection of any of the three
entities, showing one of the
advantages of using separate
color planes for multiple
detection tasks. Note that
because the center spot also
meets the other two definitions,
it provides a strong response on
all three planes. Larger n-tuples could be used to eliminate these ambiguities.
Another interesting possibility for which SKIPSM is well adapted: grey-scale openings and
closings. Figure 6 shows a case in which grey-scale erosion of one or more of the color
variable is followed by grey-scale dilation of the result. Figure 7a) shows the application of
this sequence using a 5x5 circular structuring element to all three color planes (RGB) of the
Image processing operations in color space using finite-state machines SPIE Paper 3521-33 Frederick M. Waltz SK20-3
SPIE Conf. on Machine Vision Systems for Inspection and Metrology VII Originally published Boston, Nov. 1998
Copyright July 1998 by F. M. Waltz. Duplication by permission only.
x
y
image plane
intensity(or B)
erosion
x
y
hue(or R)
erosion
x
y
saturation(or G)
erosion
Figure 4. Using different operations on
the individual color-space variables.
Figure 5. a) 32x32 image of an eye. b) pupil detector - red.
c) lower eyelid detector - green. d) upper eyelid detector -
blue. e) combined detectors - intensity only.
upperdetector
lowerdetector
bluegreenspot
detector
red
positive weight
negative weight
center pixel
Key:
a)
b) c) d)
e)
-
7/28/2019 20 Color
4/6
eye image of Figure 5a). The resulting RGB results and the
grey-scale intensity-only information are shown in Figure 7
a real black eye. Figure 8 shows the RGB images and the
resulting grey-scale intensity-only image resulting from
subtracting these color images from the original color image.
Note the way that the pupil has been highlighted.
4. PROCESSING IN HSI SPACE
All the actual examples (as opposed to the discussion) in this
paper have involved processing in RGB space. This was done
primarily because inputs and outputs in RGB format (but
apparently not HSI) are available in Photoshop, the tool
used for preparing and publishing this paper. (Photoshop is
primarily designed for the publishing business.) Thus, even
though the actual image processing shown here was done in
Excel, which can handle any kind of computation that one
could desire to do, there was insufficient time to make the
complicated nonlinear conversions to and from HSI space.
Therefore, such examples are not provided here.
The fact remains, however, that for many image processing
applications, operations in HSI space can provide significant
advantages. Of course, operation in Intensity space is familiar
to all, because this is nothing other than the conventional
grey-scale processing used in most machine vision system.
For example, a line or feature on an object of one
predominant color but varying intensity and purity will appear
as a highly variable signal in two or perhaps all three of the
RGB planes, as well as the S (saturation) and I (intensity)
planes in HSI space. But in H (hue) space, it will show us as a
contiguous set of pixels with a small color range, making
detection and location of the feature relatively easy.
Because the goal of this paper is to present possible uses of
the kind of processing SKIPSM can provide, no more will be
said here on this subject. The author expects to explore
further the implications of SKIPSM for automated inspection
using color images in HSI as well as RGB space.
Image processing operations in color space using finite-state machines SPIE Paper 3521-33 Frederick M. Waltz SK20-4
SPIE Conf. on Machine Vision Systems for Inspection and Metrology VII Originally published Boston, Nov. 1998
Copyright July 1998 by F. M. Waltz. Duplication by permission only.
ximage plane
intensityetc.
erosion
intensityetc.
xy
dilation
originalcontour
erodedcontour
erodedcontour
dilatedcontour
originalcontour
y
firstoperation
secondoperation
Figure 6. Grey-scale erosion followed by grey-scale dilation
of an individual color variable.
Red
Blue
Green
Intensity
Red
Blue
Green
Intensity
Figure 7. Result of applying a 3x3 erosion diamond followed
by 3x3 dilation diamond to Figure 5a).
Figure 8. Result of subtracting Figure 7 from original image
shown in Figure 5a) to highlight things removed.
-
7/28/2019 20 Color
5/6
5 . SUMMARY AND CONCLUSIONS
This paper has presented ideas for using the high-speed and/or large-neighborhood capabilities of the SKIPSM paradigm to
facilitate automated inspection involving colored images. The emphasis has been on presenting possibilities for processing,
not on the advantages of one or another color inspection approach. This is a large field in its own right. It is hoped that
experts in this field will find reasons here to investigate SKIPSM implementations for operations that might otherwise be too
time consuming to be practical for on-line inspection.6. BIBLIOGRAPHY
1. F. M. Waltz, SKIPSM: Separated-Kernel Image Processing using finite-State Machines, Proc. SPIE Conf. on
Machine Vision Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 36,Boston, Nov. 1994
2. F. M. Waltz and H. H. Garnaoui, Application of SKIPSM to binary morphology, Proc. SPIE Conf. on Machine
Vision Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 37,Boston, Nov. 1994
3. F. M. Waltz and H. H. Garnaoui, Fast computation of the Grassfire Transform using SKIPSM, Proc. SPIE Conf. on
Machine Vision Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 38,Boston, Nov. 1994
4. F. M. Waltz, Application of SKIPSM to binary template matching, Proc. SPIE Conf. on Machine Vision
Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 39,Boston, Nov. 1994
5. F. M. Waltz, Application of SKIPSM to grey-level morphology, Proc. SPIE Conf. on Machine Vision Applications,
Architectures, and Systems Integration III, Vol. 2347, Paper No. 40, Boston, Nov. 1994
6. F. M. Waltz, Application of SKIPSM to the pipelining of certain global image processing operations, Proc. SPIE
Conf. on Machine Vision Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 41,Boston,
Nov. 1994
7. A. A. Hujanen & F. M. Waltz, Pipelined implementation of binary skeletonization using finite-state machines, Proc.
SPIE Conf. on Machine Vision Applications in Industrial Inspection, Vol. 2423, Paper No. 2,San Jose, Feb. 1995
8. A. A. Hujanen & F. M. Waltz, Extending the SKIPSM binary skeletonization implementation, Proc. SPIE Conf. on
Machine Vision Applications, Architectures, and Systems Integration IV, Vol. 2597, Paper No. 12,Philadelphia, Oct.
1995
9. F. M. Waltz, Application of SKIPSM to binary correlation, Proc. SPIE Conf. on Machine Vision Applications,
Architectures, and Systems Integration IV, Vol. 2597, Paper No. 11,Philadelphia, Oct. 1995
10. F. M. Waltz, SKIPSM implementations: morphology and much, much more, Proc. SPIE Conf. on Machine Vision
Applications, Architectures, and Systems Integration IV, Vol. 2597, Paper No. 14,Philadelphia, Oct. 1995
11. F. M. Waltz, Automated generation of finite-state machine lookup tables for binary morphology, Proc. SPIE Conf.
on Machine Vision Applications, Architectures, and Systems Integration V, Boston,Nov. 1996
12. F. M. Waltz, Binary openings and closings in one pass using finite-state machines, Proc. SPIE Conf. on Machine
Vision Applications, Architectures, and Systems Integration V, Boston, Nov. 1996
13. F. M. Waltz, Implementation of SKIPSM for 3-D binary morphology,Proc. SPIE Conf. on Machine Vision
Applications, Architectures, and Systems Integration VI, Vol. 3205, Paper No. 13,Pittsburgh, Oct. 1997
14. F. M. Waltz, Binary dilation using SKIPSM: Some interesting variations, Proc. SPIE Conf. on Machine Vision
Applications, Architectures, and Systems Integration VI, Vol. 3205, Paper No. 15,Pittsburgh, Oct. 1997
15. J. W. V. Miller and F. M. Waltz, Software implementation of 2-D grey-level dilation using SKIPSM, F. M. Waltz
and J. W. V. Miller, An efficient algorithm for Gaussian blur using finite-state machines, Proc. SPIE Conf. on
Machine Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 18,Pittsburgh, Oct. 1997
Image processing operations in color space using finite-state machines SPIE Paper 3521-33 Frederick M. Waltz SK20-5
SPIE Conf. on Machine Vision Systems for Inspection and Metrology VII Originally published Boston, Nov. 1998
Copyright July 1998 by F. M. Waltz. Duplication by permission only.
-
7/28/2019 20 Color
6/6
16. R. Hack, F. M. Waltz, & B. G. Batchelor, Software implementation of the SKIPSM paradigm under PIP, Proc. SPIE
Conf. on Machine Vision Applications, Architectures, and Systems Integration VI, Vol. 3205, Paper No. 19,Pittsburgh,
Oct. 1997
17. F. M. Waltz, The application of SKIPSM to various 3x3 image processing operations, Proc. SPIE Conf. on Machine
Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 30,Boston, Nov. 1998
18. F. M. Waltz, R. Hack, and B. G. Batchelor, Fast, efficient algorithms for 3x3 ranked filters using finite-statemachines, Proc. SPIE Conf. on Machine Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 31,
Boston, Nov. 1998
19. F. M. Waltz, Automated generation of efficient code for grey-scale image processing, Proc. SPIE Conf. on Machine
Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 32,Boston, Nov. 1998
20. F. M. Waltz and J. W. V. Miller, An efficient algorithm for Gaussian blur using finite-state machines, Proc. SPIE
Conf. on Machine Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 37, Boston, Nov. 1998
Image processing operations in color space using finite-state machines SPIE Paper 3521-33 Frederick M. Waltz SK20-6
SPIE Conf. on Machine Vision Systems for Inspection and Metrology VII Originally published Boston, Nov. 1998
Copyright July 1998 by F. M. Waltz. Duplication by permission only.