| Customize Help

Preparing a container for display or processing

Your 3D sensor might transmit 3D information in a format that is not natively displayable or processable by MIL. For example, MIL can only process containers that have a range component with a particular structure, but your 3D sensor might be a stereoscopic camera that transmits a disparity component. You can use MbufConvert3d() to convert grabbed 3D data to a 3D-processable and/or 3D-displayable point cloud container. This function also applies some modifications to the coordinates transmitted by your 3D sensor, such as scaling or offsetting all coordinates by a specified value to convert the 3D data to natively calibrated coordinates.

Typically, if your 3D sensor is compliant (transmits data suitable for grabbing into a container), you do not need to modify grabbed data before making it 3D-processable or 3D-displayable using MbufConvert3d().

If your 3D sensor transmits several sets of 3D components, you might need to create a child container that contains only a single set of 3D components before converting the data (for more information, see the Child containers section later in this chapter). If your 3D sensor does not transmit required 3D settings for the range or disparity component, you must set those before converting the data.

3D settings for the range or disparity component

When you pass a 3D container as a source to MbufConvert3d(), 3D settings in the range or disparity component are applied to the 3D data. For example, the X-coordinates of all points in the destination point cloud container are offset by the value specified by the M_3D_OFFSET_X setting. Additionally, some settings (such as M_3D_REPRESENTATION) must be set to convert the data. The 3D settings are listed in For specifying settings useful with components that store 3D data.

If the M_3D_REPRESENTATION setting of your range or disparity component indicates a uniform distribution of X and/or Y values (for example, M_CALIBRATED_Z_UNIFORM_XY), you should specify M_3D_SCALE_X and/or M_3D_SCALE_Y to produce natively calibrated coordinates.

Scale settings are not exclusively used with uniform 3D representations; coordinates are always scaled using these settings during conversion, regardless of how those coordinates are stored.

If you are grabbing 3D data from a laser profiler or linescan device that is not perpendicular to the conveyor belt, you can compensate for the angle using the M_3D_SHEAR_X and M_3D_SHEAR_Z settings. For example, if your conveyor belt slopes downward relative to your 3D sensor, you can specify an M_3D_SHEAR_Z to offset the Z values in each slice from those in the previous slice by a set amount.

For a disparity component, the disparity-specific 3D settings must be set (using MbufControlContainer() with M_3D_DISPARITY_...). Refer to your camera manual for the correct values.

Note that MbufConvert3d() does not perform unit conversions; the source value for M_3D_DISTANCE_UNIT is always propagated to the destination without changes to the underlying data.

Compensating for missing or ambiguous information

In some cases, you might grab 3D data that is missing required information. For example, you might grab a range component with M_3D_REPRESENTATION set to M_CALIBRATED_Z. This 3D representation means that the underlying data has no specified X or Y-coordinates. By default, you cannot convert this container using MbufConvert3d() because the container is missing information. You can set the ControlFlag parameter to M_COMPENSATE flag to specify that you want MIL to convert the data by making assumptions. In the example above, the range component will be treated as though it has the 3D representation M_CALIBRATED_Z_UNIFORM_XY.

You can use M_COMPENSATE to compensate for the following ambiguities in the source container:

  • The M_3D_REPRESENTATION of the range or disparity component indicates that the coordinates for at least one axis are unspecified.

  • There is no confidence information. Confidence information can be stored in a confidence component, or in the Z-axis values of the range or disparity component when M_3D_INVALID_DATA_FLAG is set to M_TRUE.

  • There is more than one confidence, intensity/reflectance, mesh, or normals component.

    Note that you cannot compensate for having a container with more than one range or disparity component. You can free the extraneous components using MbufFreeComponent(), or allocate a child container to filter out the extraneous components using MbufChildContainer().

  • At least one component is incorrectly formatted. To learn the correct formatting for each component, see the Layout of data in a component subsection of the Working with non-compliant cameras section of Chapter 36: Grabbing from 3D sensors.

Requirements for a 3D-processable container

For a container to be 3D-processable, it must have the attribute M_PROC in addition to one (at most) of each of the following components:

Component type



Data type

Other Settings




Floating point

M_3D_REPRESENTATION set to M_CALIBRATED_XYZ or M_CALIBRATED_XYZ_UNORGANIZED. All 3D settings (from the table For specifying settings useful with components that store 3D data) set to their default, except for M_3D_DISTANCE_UNIT which can have any setting.




8-bit unsigned

The same M_SIZE_X and M_SIZE_Y as the range component.



1 or 3

8-bit unsigned or 16-bit unsigned

The same M_SIZE_X and M_SIZE_Y as the range component.




Floating point

The same M_SIZE_X and M_SIZE_Y as the range component.




32-bit unsigned

If the range component has M_3D_REPRESENTATION set to M_CALIBRATED_XYZ, M_SIZE_Y must be 6.

If the range component has M_3D_REPRESENTATION set to M_CALIBRATED_XYZ_UNORGANIZED, M_SIZE_Y must be 3 or 6.

Any components with a component type not included in the table above will be propagated, but not used, by 3D processing functions and MbufConvert3d(). The container must not have a disparity component.

You can determine whether a container is 3D-processable using MbufInquireContainer() with M_3D_PROCESSABLE.

Requirements for a 3D-displayable container

Any container with the M_DISP attribute that can be passed as a source to MbufConvert3d() (with M_COMPENSATE) is 3D-displayable. A conversion will occur each time the container is modified, unless you convert the container to a format that is natively 3D-displayable. Automatic conversion is primarily useful for displaying 3D data as it is being grabbed.

The requirements for a container to be natively 3D-displayable are the same as the requirements for a container to be 3D-processable, except that the M_DISP attribute is required.

You can determine whether a container is 3D-displayable using MbufInquireContainer() with M_3D_DISPLAYABLE.