| Customize Help

Camera calibration propagation


Camera calibration contexts store a lot of information. When you associate a camera calibration context with an image or a digitizer, some of this information is copied and some of it is just referenced. In addition, when an operation is done on a calibrated image, its camera calibration information is typically propagated to the destination image or result buffer.

The three types of information stored in a camera calibration context are:

  • The non-linear mapping between the world and pixel coordinate systems.

  • Properties of the calibration setup, such as the type of grid used and its size.

  • Camera calibration information, which represents the most functional part of the calibration, and consists primarily of a copy of all coordinate systems.

When a digitizer is associated with a camera calibration context (McalAssociate()), the digitizer only retains a link to the camera calibration context. When you grab images with the digitizer, the grabbed images are automatically associated with the camera calibration context. A digitizer with an associated camera calibration context is known as a calibrated digitizer. To disassociate a camera calibration context from an image or digitizer, use McalAssociate() with M_NULL.

When an image is associated with a camera calibration context, it receives a copy of the calibration's coordinate systems and a reference to the camera calibration context for all other settings, including the intrinsic attributes. This data is referred to as the camera calibration information. An image with an associated camera calibration context is known as a calibrated image. If you change the relative coordinate system or camera position of the camera calibration context after association, the change will not affect the image, because the image has its own camera calibration information. If you change any other setting of the camera calibration context after association, the change will affect the calibrated image because those settings are retrieved through the reference to the camera calibration context.

A result buffer used by an analysis operation can also contain camera calibration information. If the MIL module supports returning results in real-world units, the camera calibration information stored with the source images will be copied to the result buffer. For more information on getting results with respect to real-world units, see the Working with real-world units section later in this chapter.

When a child buffer is allocated in a calibrated image, the child buffer receives a copy of the camera calibration information stored in the parent, taking into account the different origin in the pixel coordinate system. Consequently, the absolute and relative coordinate systems of the child buffer are initially the same as the parent image, when the child is allocated. Since the child buffer has its own copy of the coordinate systems, you can displace its relative coordinate system to any new location. However, there exists a rigid link between the parent and child's relative coordinate systems, so if you move the parent's relative coordinate system, the child buffer's relative coordinate system will be displaced to maintain the same position and orientation that was previously established with respect to the parents relative coordinate system. For more information, see the Multiple relative coordinate systems subsection of the Calibration and one or many fixtures section of Chapter 27: Fixturing in MIL.

You can also use McalAssociate() to copy the camera calibration information from an image or result buffer to another image or result buffer. MbufCopy() will also copy all the camera calibration information associated with a buffer to the specified destination buffer.

Processing calibrated images

When a camera calibration context is associated with an image, it has certain implications on processing operations applied to that image. Depending on the type of operation performed on the calibrated image, the destination image also receives a copy of the camera calibration information for the following circumstances:

  • When performing point-to-point or neighborhood processing operations, the destination image receives a copy of the camera calibration information from the source image.

    If the operation uses more than one source image, the camera calibration information gets copied to the destination image only if all source images have the same camera calibration information; otherwise, the camera calibration information of the destination image is left unchanged.

  • Geometric functions (MimFlip(), MimPolarTransform(), MimResize(), MimRotate(), MimTranslate(), and MimWarp()) always result in an uncalibrated image, even if the source image is calibrated.

    You can use McalWarp() to warp an existing calibration context and associate it with a warped uncalibrated image. For more information, see the Propagating camera calibration information after performing a geometric operation section later in this chapter.

  • The function MbufClear() always results in an uncalibrated image. You can use MgraClear() to clear a buffer while preserving the camera calibration context associated with the empty image.

  • Functions for which the source data is always considered uncalibrated always produce uncalibrated images. These functions include, for example, MbufImport...() and MbufLoad().

  • A result buffer will receive a copy of the camera calibration information from the source images used to generate the result if and only if all source images have the same camera calibration information and the module supports returning results in world units; otherwise, the result buffer is left uncalibrated.

Saving and reloading a calibrated image

When saving and reloading a calibrated image, you must account for its associated camera calibration context. You can either save it with the image file, or save it in a separate file. By default, the image's camera calibration information is not saved and, therefore, is not reloaded.

To inquire whether an image is calibrated, use McalInquire() with M_ASSOCIATED_CALIBRATION.

Saving the calibration with the image file

To save the image's associated calibrated object with the image file, use MbufExport() with M_MIL_TIFF + M_WITH_CALIBRATION. To restore the associated camera calibration context, use McalRestore().

Typically, when restoring the camera calibration context, you will also restore the associated image, using MbufImport(), MbufLoad(), or MbufRestore(), and then associate the restored image buffer to the restored camera calibration, using McalAssociate(). In this case, the calibrated image will always be in the same state as it was when it was previously saved. The following is a general example of how to save/restore the calibration (and the image) using M_WITH_CALIBRATION.

/* Save the image along with its associated calibration in a file. */
MbufExport(MIL_TEXT("Image.mim"), M_MIL_TIFF+M_WITH_CALIBRATION, ImageId);

/* Free the image. */
MbufFree(ImageId);

/* ... */

/* The following code can be in another application. */

/* Restore the image from the file. */
MbufRestore(MIL_TEXT("Image.mim"), SystemId, &ImageId);

/* Restore the calibration object from the file. */
McalRestore(MIL_TEXT("Image.mim"), SystemId, M_DEFAULT, &CalibrationContextId);

/* Associate the restored calibration context to the restored image. */
McalAssociate(CalibrationContextId, ImageId, M_DEFAULT);

When using M_WITH_CALIBRATION, the restored camera calibration context contains information relevant only to the restored image (of the same file); it is not recommended that you associate it with another image. For example, if you restore the camera calibration context (McalRestore()) from an image file that had been saved using MbufExport() with M_WITH_CALIBRATION, the restored camera calibration context and the camera calibration context that was originally associated with the saved image, can have different settings. This can occur because the calibration settings that are specific to the saved image (such as the position of the relative coordinate system) are also present in the restored camera calibration context.

When restoring an image with a constant pixel size (including depth maps), use MbufImport() with M_MIL_TIFF + M_WITH_CALIBRATION. This imports the image along with its uniform camera calibration information. In this case, the camera calibration is associated with a default uniform camera calibration.

Saving the associated camera calibration context in a separate file

In certain cases, you might not want to save the associated camera calibration context in the same file as the image (MbufExport() with M_WITH_CALIBRATION). For example:

  • You do not want to save the image in a MIL file format.

    M_WITH_CALIBRATION must be used in combination with M_MIL_TIFF. You cannot, for example, use an M_JPEG... format.

  • You want to associate the same camera calibration context to multiple images, but you only want to save it once (for example, to conserve disk space).

To save a calibrated image and its camera calibration context in a separate file, perform the following:

  1. Obtain the identifier of the image's camera calibration context, using McalInquire() with M_ASSOCIATED_CALIBRATION.

  2. Save the camera calibration context, using McalSave() or McalStream().

  3. Save the image buffer, using MbufSave() or MbufExport().

To reload the image and associate it with its camera calibration context, perform the following:

  1. Restore the image buffer, using MbufRestore().

  2. Restore the camera calibration context, using McalRestore().

  3. Associate the restored image with the restored camera calibration context, using McalAssociate().

The following is an example of how to save two calibrated images and their common camera calibration context in separate files, and then reload the images and associate them with the restored camera calibration context:

/* Save 2 images in JPEG files. Both images:
   - Are already associated to the same calibration contexts.
   - Are not child images.
   - Have not been corrected using McalTransformImage.
   - Have relative coordinate systems that have not been modified. */
McalInquire(Image1Id, M_ASSOCIATED_CALIBRATION+M_TYPE_MIL_ID, &CalibrationContextId);
McalSave(MIL_TEXT("Calibration.mca"), CalibrationContextId, M_DEFAULT);

MbufExport(MIL_TEXT("Image1.jpg"), M_JPEG_LOSSY, Image1Id);
MbufExport(MIL_TEXT("Image2.jpg"), M_JPEG_LOSSY, Image2Id);

/* Free the images and the calibration context. */
MbufFree(Image1Id);
MbufFree(Image2Id);
McalFree(CalibrationContextId);

/* ... */

/* The following code can be in another application. */

/* Restore the images from the files. */
MbufRestore(MIL_TEXT("Image1.jpg"), SystemId, &Image1Id);
MbufRestore(MIL_TEXT("Image2.jpg"), SystemId, &Image2Id);

/* Restore the calibration context from the file. */
McalRestore(MIL_TEXT("Calibration.mca"), SystemId, M_DEFAULT, &CalibrationContextId);

/* Associate the restored calibration context to the restored images. */
McalAssociate(CalibrationContextId, Image1Id, M_DEFAULT);
McalAssociate(CalibrationContextId, Image2Id, M_DEFAULT);

Note that if one or more of the following conditions are true, this save/reload procedure can result in a calibrated image that might not be in the same state as the previously saved image:

  • The previously saved image was corrected, using McalTransformImage().

  • The previously saved image was calibrated, using McalUniform().

  • The previously saved image was a child image.

  • The relative coordinate system of the previously saved image was moved.

When you associate a camera calibration context with an image, its child images are also associated with the camera calibration context; their offset from the parent image is automatically taken into account. If you save a child image on its own into a file and restore it, it becomes an ordinary stand alone image; if you reassociate it with the original camera calibration context, the camera calibration context cannot automatically know that the image's pixel coordinate system should be offset from the original mapping.

If the camera calibration context is restored from a separate file, you can restore the calibration of the image (that used to be a child image) by associating it with the restored camera calibration context, and then specifying its original offset from its original parent that was calibrated, using McalControl() with M_CALIBRATION_CHILD_OFFSET_X and M_CALIBRATION_CHILD_OFFSET_Y. The following is a general example of how to perform this procedure:

/* Associate a calibration to an image and allocate a child of this image. */
McalAssociate(CalibrationContextId, ImageId, M_DEFAULT);
MbufChild2d(ImageId, ChildOffsetX, ChildOffsetY, ChildSizeX, ChildSizeY, &ChildId);

/* Save the child images and the calibration context in separate files. */
MbufExport(MIL_TEXT("Child.mim"), M_MIL_TIFF, ChildId);
McalSave(MIL_TEXT("Calibration.mca"), CalibrationContextId, M_DEFAULT);

/* Free the image and the calibration context. */
MbufFree(ChildId);
McalFree(CalibrationContextId);

/* ... */

/* The following code can be in another application. */

/* Restore the image from the file. */
MbufRestore(MIL_TEXT("Child.mim"), SystemId, &ChildId);

/* Restore the calibration context from the file. */
McalRestore(MIL_TEXT("Calibration.mca"), SystemId, M_DEFAULT, &CalibrationContextId);

/* Associate the restored calibration context to the restored image. */
McalAssociate(CalibrationContextId, ChildId, M_DEFAULT);

/* Specify the original offset of the restored image from its parent
   (at the time it was originally calibrated). */
McalControl(ChildId, M_CALIBRATION_CHILD_OFFSET_X, ChildOffsetX);
McalControl(ChildId, M_CALIBRATION_CHILD_OFFSET_Y, ChildOffsetY);

If you create a camera calibration context for a child image, associate it to the child image, and then save the image separately from its parent, you will not need to specify its offset from its parent. Similarly, if you save the camera calibration context and the child image in the same file, you will not need to specify the child offset from its parent.