Color matching

---

• See also
User Guide:
Advanced color matching settings and concepts Color spaces and converting between them Distance between colors Relative color calibration
Reference:
McolAlloc McolAllocResult McolControl McolDefine McolDistance McolDraw McolFree McolGetResult McolInquire McolMask McolMatch McolPreprocess McolSetMethod McolTransform MimConvert
Availability
Not available in MIL-Lite


Available in MIL
Previous
Next

Color matching is the process of finding a match between the color of a target area and one or more predefined color-samples. Color-samples are composed of color elements. These color elements can be added to or removed from the color-sample to control the color-sample's color properties during the matching process. When you first define a color-sample using McolDefine(), you are also automatically adding the first color element to the sample.

The match is performed, in part, by calculating the color distance between the target area and the color-sample. This information can be retrieved after calling McolMatch(). However, if you are only interested in color distances, you can use McolDistance(). For more information, including the differences in distances between McolMatch() and McolDistance(), see the Distance between colors section earlier in this chapter.

Various conditions, such as different cameras and illuminants, can cause color from identical images to appear dissimilar. If this occurs, you can call McolTransform() to perform a relative color calibration before matching colors. Relative color calibration ensures all colors are consistent according to a specified reference color. For more information, see the Relative color calibration section earlier in this chapter.

Steps to performing color matching

The following steps provide a basic methodology for using the MIL Color Analysis module to match colors:

Basics of color matching

Color matching can be used to perform one of two basic tasks: color identification or supervised color segmentation. In either case, the match is based on the defined color-samples, whose color elements can come from a source image or from explicit color values. For more information, see the Defining and adding color-samples and color elements subsection of this section.

Color identification

Color identification refers to matching the general color of each target area with the best color-sample within a group of predefined color-samples. When performing color identification, you might find it useful to locate the target areas with the MIL Geometric Model Finder module before performing the match. To produce image results for a target area, use McolMatch() or McolDraw() with M_DRAW_AREA_MATCH_USING_LABEL or M_DRAW_AREA_MATCH_USING_COLOR.

Supervised color segmentation

Supervised color segmentation refers to matching the color value of each pixel in the target image (or target area) with the best predefined (supervising) color-sample. For supervised color segmentation, you must use McolSetMethod() with M_MIN_DIST_VOTE, which matches colors on a pixel-by-pixel basis, and produce image results using M_DRAW_PIXEL_MATCH_USING_LABEL or M_DRAW_PIXEL_MATCH_USING_COLOR.

By matching the color value of each pixel with the best predefined color-sample, you can use the Color Analysis module to separate objects by their color information. This type of matching is useful to, for example, retrieve the relative presence of each color in an image (this is also referred to as the spatial coverage for each of the colors).

Defining and adding color-samples and color elements

As previously discussed, color-samples are composed of color elements. When you first define a color-sample, it is automatically considered to have a single color element. Therefore, when you add a color-sample to the color matching context, you are by default adding the first color element to the color-sample. You can define a color-sample (and consequently the first color-sample element) from either a region of an image (M_IMAGE), or from three explicit color component values (M_TRIPLET), using McolDefine().

For image type color-samples, MIL performs an estimation of the color based on color statistics, such as the mean. MIL considers this the color of the color-sample and uses it whenever required (for example, McolTransform()). The size of the color-sample does not typically affect speed or effectiveness.

To add color elements to color-samples that already exist in the context, use M_ADD_COLOR_TO_SAMPLE. The color estimation that MIL performs for the color-sample uses the new color element(s) that you specify for the color-sample, as well as the existing element(s) of the color-sample. MIL now considers the resulting estimate the color of the color-sample.

You can also use McolControl() to control settings related to color-samples.

When you add, modify, or delete a color-sample, you must preprocess the color matching context (McolPreprocess()) before any subsequent call to McolMatch().

For more information about color-samples, see the Color-samples and color elements subsection of the Relative color calibration section earlier in this chapter. Defining color-samples for color matching is nearly identical to defining color-samples for relative color calibration.

Area identifier image

The area identifier image is a grayscale image, specified with McolMatch(), that defines the independent target areas where color matching will occur in the target image. You must use an area identifier image to match multiple target areas. Each unique, non-zero label in the area identifier image defines an independent target area. Typically, the area identifier image is the same size as the target image.

In the following example, the 4 target areas in the area identifier image are defined according to the objects found with the MIL Model Finder module. The color of each corresponding target area, in the actual target image, is then matched (identified) with the best color-sample.

To produce image results on an area basis (as illustrated above), use McolMatch() or McolDraw() with M_DRAW_AREA_MATCH_USING_LABEL or M_DRAW_AREA_MATCH_USING_COLOR. For more information, see the Image results subsection of this section.

Acceptance

The acceptance levels determine the minimum scores required for a successful match between a target area and a color-sample. You can set an acceptance level for the color-sample's match score, and for the target area's relevance score, using McolControl() with M_ACCEPTANCE and M_ACCEPTANCE_RELEVANCE.

Color-sample acceptance (for the color-sample's match score)

M_ACCEPTANCE is applied to the match score (McolGetResult() with M_SCORE), which indicates the similarity between the color of the color-sample and the color of the target area. The higher the acceptance, the closer the colors must be for them to match. For example, if you are using an M_STAT_MIN_DIST operation mode (McolSetMethod()), and you set M_ACCEPTANCE to 100, the colors will only match if they are absolutely the same. For an acceptance of 100 while using an M_MIN_DIST_VOTE operation mode, all pixels in the target area that are not outliers must have voted for the same color-sample to have a successful match.

For more information on the actual match score calculated, see the Match score and relevance score subsection of this section.

Relevance acceptance (for the target area's relevance score)

M_ACCEPTANCE_RELEVANCE is applied to the target area's relevance score (McolGetResult() with M_SCORE_RELEVANCE), which indicates the significance (relevance) of the match score (M_SCORE). In statistics, this is similar to the confidence level. A high relevance score indicates that the best-matched color-sample was a vastly superior match compared to the other color-samples, while a low relevance score indicates that another color-sample came very close to being the best-matched color-sample.

If you are using the M_STAT_MIN_DIST operation mode (McolSetMethod()), and you set M_ACCEPTANCE_RELEVANCE to 100, the match will only be successful if the best-matched color-sample was the only color-sample considered (this can occur if, for example, all the other color-samples fell outside of the distance tolerance (M_DISTANCE_TOLERANCE)). For a relevance acceptance of 100 while using an M_MIN_DIST_VOTE operation mode, all pixels in the target area must have voted for the same color-sample.

For more information on the actual relevance score calculated, see the Match score and relevance score subsection of this section.

Distance tolerance

The distance tolerance refers to the maximum color distance, between the color-sample and the target area, allowed for a successful match. To specify the distance tolerance, use McolControl() with M_DISTANCE_TOLERANCE set to M_INFINITE, a specific value, or M_AUTO (the default). The greater the distance tolerance, the greater the distance (difference) between matching colors can be. For example, if your target area is green, you can set the distance tolerance to 0 to only match with the exact same green. However, by increasing the tolerance, you can match with colors that are progressively different than the original.

When setting the tolerance, you must consider the specified distance type, the color space, and the color space encoding. For example, a distance tolerance of 1.0 when using an M_MAHALANOBIS distance type is not the same as when using an M_MANHATTAN distance type.

Tolerance mode

When setting M_DISTANCE_TOLERANCE to a specific value or M_AUTO, MIL determines the distance tolerance according to the tolerance mode, which you can set with M_DISTANCE_TOLERANCE_MODE.

If you set the tolerance mode to M_ABSOLUTE (the default), MIL applies the M_DISTANCE_TOLERANCE value as is. For M_AUTO, the M_DISTANCE_TOLERANCE value is infinite.

If you set the tolerance mode to M_RELATIVE, MIL internally calculates the distance between each pair of color-samples and finds the smallest distance value; half this value is then multiplied by the M_DISTANCE_TOLERANCE value. For example, if you have three different color-samples (sample 1, sample 2, and sample 3), the distances between each pair would be the distance between sample 1 and sample 2 (distance 12), sample 1 and sample 3 (distance 13), and sample 2 and sample 3 (distance 23). The tolerance value for sample 1 would then be the smaller value of distance 12 and distance 13, multiplied by M_DISTANCE_TOLERANCE. The tolerances for each of the other samples are calculated the same way. For M_AUTO, the M_DISTANCE_TOLERANCE value is 1.

If you set the tolerance mode to M_SAMPLE_STDDEV, MIL specifies a tolerance strategy based on the color-sample's standard deviation.

For color-samples defined from images (McolDefine() with M_IMAGE), the color-sample's standard deviation is computed as the distance between each pixel in the color-sample and the average color of the color-sample. MIL calculates this distance according to the current distance type set with McolSetMethod(). The resulting distance, which in this case is considered to be the color-sample's standard deviation, is multiplied by M_DISTANCE_TOLERANCE and is used as the acceptable distance tolerance. If the computed color-sample's standard deviation is less than 1.0, MIL uses 1.0 as the standard deviation.

For color-samples composed of triplets (McolDefine() with M_TRIPLET), MIL uses 1.0 as the standard deviation. In this case, M_DISTANCE_TOLERANCE remains unchanged and is used as the acceptable distance tolerance.

For M_AUTO, the M_DISTANCE_TOLERANCE value for M_SAMPLE_STDDEV is 3.

Operation mode and distance type

Before calling McolMatch(), you can use McolSetMethod() to set the match's operation mode to one of the following:

Color matching operation modes only take color information into account; target areas are therefore only distinguishable if their color composition differs. Further details for each operation are described later in this section.

You can also use McolSetMethod() to set the type of distance with which to perform the match. For RGB and CIELAB color spaces, a Euclidean color distance is calculated by default. For HSL color spaces, Manhattan is the default. For more information, see the Color distance types subsection of the Distance between colors section earlier in this chapter.

M_STAT_MIN_DIST operation mode

When using the M_STAT_MIN_DIST operation mode, color statistics are calculated (typically the mean/average) for each target area and for each color-sample defined in the context, and the color distance between each target area and each color-sample is determined.

The resulting distances determine the score of the color-samples. The closer the colors, the higher the score. If a distance is not within a color-sample's distance tolerance, the color-sample's score is 0%. The color-sample with the highest score above the acceptance level is the target area's best-matched color-sample.

In the following example, the target area's average color is calculated, and then matched with color-sample 1, which is the closest (best-match) color.

For M_STAT_MIN_DIST, you should accurately define the target area, and it should generally consist of the required color, as indicated in the example above.

M_MIN_DIST_VOTE operation mode

When using the M_MIN_DIST_VOTE operation mode, color statistics are calculated (typically the mean/average) for each color-sample defined in the context, and the color distance between each pixel in each target area and each color-sample is determined. Each target pixel then votes for the color-sample with the closest color, and which is also within the distance tolerance.

The number of votes that a color-sample accumulates determines its score. The greater the number of votes, the higher the score. The color-sample with the highest score above the acceptance level is the target area's best-matched color-sample.

In the following example, each target pixel votes for the color-sample that has the closest color. In general, grapefruit pixels will vote for color-sample 1, while background pixels will vote for sample 2. Since there are more grapefruit pixels, color-sample 1 is the best-match.

Since M_MIN_DIST_VOTE operates on a pixel-by-pixel basis, it provides more detailed results (for each pixel) than M_STAT_MIN_DIST and is generally more robust (but slower). M_MIN_DIST_VOTE is also less sensitive towards the accuracy of the target areas; as indicated in the example above, the target need not consist of only the grapefruit.

M_HISTOGRAM_MATCHING and M_HISTOGRAM_VOTE operation modes

MIL offers two histogram operation modes: M_HISTOGRAM_MATCHING and M_HISTOGRAM_VOTE.

When using the M_HISTOGRAM_MATCHING operation mode, color histograms are calculated for each target area and for each sample defined in the context. A match score is then calculated between each target area histogram and the sample histograms. The sample with the highest match score above the acceptance is the target area's best-matched sample. When using M_HISTOGRAM_MATCHING, you must set M_DISTANCE_TOLERANCE_MODE to M_ABSOLUTE.

In M_HISTOGRAM_MATCHING mode, pixel values are grouped together by subdividing their color space into bins of equal size. You must specify the number of bins for each color band using McolControl() with M_NB_BINS_BAND_.... The match score is based on the comparison of histogram frequencies for each bin, rather than each individual pixel value.

Note that the histograms shown above are approximations intended for explanatory purposes.

M_HISTOGRAM_MATCHING is generally the most robust operation mode when dealing with color-samples that contain a mixture of colors. For more detailed information on histogram matching, see the Histogram matching concepts subsection of the Advanced color matching settings and concepts section later in this chapter.

The M_HISTOGRAM_VOTE operation mode uses a pixel voting process to find a target area's best-matched color-sample. When using this operation mode, MIL computes color histograms for each sample defined in the context. Then, each pixel in the target area votes for the color-sample that has a non-empty histogram bin containing the pixel's color. The number of votes that a color-sample accumulates determines its score. The color-sample with the highest score above the acceptance (McolControl() with M_ACCEPTANCE) is the target area's best-matched color-sample. In the case of a tie, either during the vote or after all votes have been cast, the color-sample with the highest label value is chosen as the best-match.

When using an M_HISTOGRAM_VOTE operation mode, MIL does not perform typical distance calculations. When passing M_HISTOGRAM_VOTE to McolSetMethod(), you must also set the distance type to M_NONE. Also, MIL ignores distance settings such as McolControl() with M_DISTANCE_TOLERANCE_MODE and M_DISTANCE_TOLERANCE.

Like M_HISTOGRAM_MATCHING, you should generally use M_HISTOGRAM_VOTE when color-samples are a mixture of colors. Although typically slower than M_HISTOGRAM_MATCHING, M_HISTOGRAM_VOTE is a good option when dealing with outliers or images containing unneeded pixels/colors.

Basic results

You can use McolGetResult() with the identifier of a color matching result buffer to retrieve context, target area, and color-sample results, such as the index of the target area's best-matched color-sample (M_BEST_MATCH_INDEX). To retrieve such results, you must have first used McolMatch() to write all results to a result buffer, by setting its ControlFlag parameter to M_DEFAULT. If you used McolMatch() to write the results to an image buffer (M_DRAW_...) instead, you will not have a result buffer to specify in McolGetResult(). For more information on calculating image results, see the Image results subsection of this section.

Depending on the type of result to retrieve, you must set the appropriate label or index value for the target area (AreaLabel) and color-sample (ColorSampleIndexOrLabel) parameters, and use the proper data type to hold the results. The following table lists various types of results and the settings you should specify to retrieve them.

Type of result	AreaLabel parameter	ColorSampleIndexOrLabel parameter	Data type for result
A color-sample result for a specific color-sample. For example, you want to retrieve a color-sample's match score (M_SCORE). Note that to get any color-sample result, you must specify the target area(s).	A specific target area.	A specific color-sample.	Data type: double.
A color-sample result for a specific color-sample in all target areas. For example, you want to retrieve the color distance of a color-sample in each target area (M_COLOR_DISTANCE).	All target areas.	A specific color-sample.	Data type: array of type double. Array size: M_NUMBER_OF_AREAS.
A color-sample result for all color-samples in a specific target area. For example, you want to retrieve the color distance of all color-samples in a target area (M_COLOR_DISTANCE).	A specific target area.	All color-samples.	Data type: array of type double. Array size: M_NUMBER_OF_SAMPLES.
A color-sample result for all color-samples in all target areas. For example, you want to retrieve whether each color-sample in each target area fulfills the color matching conditions (M_SAMPLE_MATCH_STATUS).	All target areas.	All color-samples.	Data type: array of type double. Array size: M_NUMBER_OF_AREAS x M_NUMBER_OF_SAMPLES.
A general target area result for one target area. For example, you want to retrieve the index of a target area's best-matched color-sample (M_BEST_MATCH_INDEX).	A specific target area.	M_GENERAL.	Data type: double.
A general target area result for all target areas. For example, you want to retrieve the index of each target area's best-matched color-sample (M_BEST_MATCH_INDEX).	All target areas.	M_GENERAL.	Data type: array of type double. Array size: M_NUMBER_OF_AREAS.
A general result for a color matching context. For example, you want to retrieve the number of target areas or color-samples (M_NUMBER_OF_AREAS or M_NUMBER_OF_SAMPLES).	M_GENERAL.	M_GENERAL.	Data type: double.

Best-matched color-sample

You can either return the index or the label of each target area's best-matched color-sample, using M_BEST_MATCH_INDEX or M_BEST_MATCH_LABEL. If no color-sample has matched, -1 is returned for M_BEST_MATCH_INDEX; for M_BEST_MATCH_LABEL, the value set using McolControl() with M_OUTLIER_LABEL is returned.

Match status

You can either return the match status of a color-sample, using M_SAMPLE_MATCH_STATUS, or the match status of a target area, using M_STATUS.

The status of a color-sample (M_SAMPLE_MATCH_STATUS) returns M_MATCH if that color-sample fulfills the match conditions, with respect to the acceptance (M_ACCEPTANCE) and the distance tolerance (M_DISTANCE_TOLERANCE); otherwise, M_NO_MATCH is returned. The status of a target area (M_STATUS) returns M_SUCCESS if at least one color-sample fulfills the match conditions, and if the target area's relevance score passes the relevance acceptance level (M_ACCEPTANCE_RELEVANCE); otherwise, M_FAILURE is returned.

The color-sample with the highest score (M_SCORE) is referred to as the best-matched color-sample. To determine if a specific color-sample is the best-matched color-sample, compare the color-sample's index or label with M_BEST_MATCH_INDEX or M_BEST_MATCH_LABEL.

Match score and relevance score

The color-sample's match score is based on the operation mode specified, using McolSetMethod():

Similarly, the target area's relevance score is also based on the operation mode specified, using McolSetMethod():

A low relevance score indicates that you should be cautious about the match results, even if the match score is high. For example, a high match score and a low relevance score could mean that the target area matched very well with a color-sample, but came very close to matching with a different color-sample; this implies that a slightly different target image, or even a difference in lighting, could change your results. You should therefore set appropriate acceptance levels for each of these scores. For more information, see the Acceptance subsection of this section.

The following example shows two cases illustrating the difference between match and relevance scores in M_MIN_DIST_VOTE mode:

Outlier coverage and color-sample coverage

The outlier coverage (M_OUTLIER_COVERAGE) quantifies, as a percentage, the proportion of pixels in the target area that did not vote for any color-sample, while the sample coverage (M_SAMPLE_COVERAGE) quantifies the proportion of pixels that did vote for a specific color-sample.

Since coverage results are based on pixel votes, they are only available when using an M_MIN_DIST_VOTE operation mode.

Color distance

The color distance (M_COLOR_DISTANCE) returns the distance (difference) between the color of the target area and the specified color-sample, when using an M_STAT_MIN_DIST operation mode.

You can also return the maximum color distance (M_MAX_DISTANCE), which is the greatest color distance between the target area and all its matching color-samples.

Image results

To produce image results, you will typically use McolMatch() with the ControlFlag parameter set to M_DEFAULT, and pass the identifier of a result buffer in which to write the results of the color matching operation. You will then use this result buffer with McolDraw() to draw any image result (M_DRAW_...). You can also use this result buffer to retrieve any other type of result, using McolGetResult().

In some cases you can avoid calling McolDraw() and produce an image result directly using McolMatch() by setting the ControlFlag parameter to M_DRAW_..., and passing the identifier of an image buffer in which to write the results of the color matching operation. Since you do not have to call McolDraw(), producing an image this way is typically done to save time when you require just one specific image result. For example, when performing color segmentation, you might only be interested in the label of the color-sample for which each pixel voted; therefore, using McolMatch() with M_DRAW_PIXEL_MATCH_USING_LABEL to get that one resulting image is sufficient. Note that when using McolMatch() in this way, you are not producing a result buffer and therefore will not be able to retrieve results using McolGetResult().

If you use McolDraw() to draw the images, you must first enable the required control types to save the resulting images in the result buffer, using McolControl() with M_SAVE_AREA_IMAGE and M_GENERATE_.... For example, to draw M_DRAW_AREA_MATCH_USING_COLOR, you must first enable M_SAVE_AREA_IMAGE and M_GENERATE_SAMPLE_COLOR_LUT.

Unless otherwise specified, all M_DRAW_... images are available with either McolDraw() or McolMatch(). Images drawn using individual pixel match results (M_DRAW_PIXEL_MATCH_...) can only be specified when using an M_MIN_DIST_VOTE distance type.

Note that when image results include color elements or entire color-samples, they are drawn using either their triplet values (M_TRIPLET) or their estimated values (for example, the average) taken from a source image (M_IMAGE), depending on how you defined them (McolDefine()). Color elements and color-samples are not drawn using the color of the target area.

Target areas

For each target area, you can draw:

The distance image (M_DRAW_DISTANCE) and the label images (M_..._USING_LABEL) draw numerical values and are therefore grayscale images. The other images (M_..._USING_COLOR) draw colors. For information on the size and depth of the required image buffer, use McolGetResult() or McolInquire(), as required. For example, to return the depth per band (in bits) required for the image buffer in which to draw M_DRAW_AREA_MATCH_USING_COLOR, use McolGetResult() with M_SAMPLE_COLOR_SIZE_BIT.

The following image illustrates an example case where color matching is performed and the five target area results are drawn:

You can also further process the distance image using other MIL modules. For example, you can use the Blob Analysis module to identify blobs using the grayscale distance image and compute their features, such as area, perimeter, and min/max diameter, or use the Edge Finder module to detect crests and contours (note that Edge Finder can also be used directly with color images).

Color-samples and color elements

For each color-sample, you can draw:

These images are only available with McolDraw(); they cannot be produced directly with McolMatch().

Background and outliers

Background pixels are pixels that are outside the target areas and are not used in the matching operation, while outlier pixels are pixels inside a target area, but do not match with any color-sample.

When drawing image results, the destination buffer's bit depth must account for background and outlier pixels, if they are drawn. When drawing the distance image, (M_DRAW_DISTANCE), outliers are also drawn and can be difficult to identify if the outlier color is similar to the resulting distance. For example, if the outlier color is 0.0, you wouldn't be able to distinguish it if the resulting distance is also 0.0. In this case, you can use M_DRAW_..._USING_LABEL, which draws the outlier label.

To set the outlier and background pixel value, use McolControl() with M_BACKGROUND_DRAW_COLOR and M_OUTLIER_DRAW_COLOR. To set the outlier label, use McolControl() with M_OUTLIER_LABEL. Note that unlike the outlier label, the background label is not specified explicitly; instead, it is taken from the background pixel color.

Inverting colors

You can add the combination value M_INVERTED_COLORS to certain M_DRAW_... settings in McolDraw() and McolMatch() to invert the color of the best-matched color-sample in the resulting image. For inversion, MIL uses the difference between each band value of the best-matched color-sample and the maximum possible value in the image. For example, with an 8-bit image, the color-sample band values are modified as follows: [255 - ColorSampleBand0, 255 - ColorSampleBand1, 255 - ColorSampleBand2]. In this case, if you use M_DRAW_AREA_MATCH_USING_COLOR and the best-matched color-sample value is [240, 10, 255], that color becomes [15, 245, 0] when you invert it (M_DRAW_AREA_MATCH_USING_COLOR + M_INVERTED_COLORS).

In general, you can use M_INVERTED_COLORS to help increase the contrast between the pixels of the best-matched color-sample and the pixels of the underlying destination image. This can prove especially useful when both M_BACKGROUND_DRAW_COLOR and M_OUTLIER_DRAW_COLOR are set to M_TRANSPARENT.