| Customize Help

Augmentation


Augmentation allows you to create a plausible variation of an image. This is done by applying a specified ordered set of image processing operations with randomized settings within a specified range. For example, to plausibly mimic the different kinds of images a camera might capture of an object passing under it, MimAugment() can randomly rotate, translate, and blur an image. This lets you generate the images that a camera can grab without the camera actually grabbing them.

Augmentation can prove useful if you want to add a variety of modified images to increase the entries in your dataset when performing image classification with the MIL classification module. You can also use augmentation to add a variety of modified images with which to perform test procedures.

Steps to augment an image

The following steps provide a basic methodology for augmenting an image:

  1. Allocate an image processing context for augmentation, using MimAlloc() with M_AUGMENTATION_CONTEXT.

  2. Allocate an image processing buffer to hold augmentation results, using MimAllocResult() with M_AUGMENTATION_RESULT.

  3. Enable the image processing operations that the augmentation can perform, using MimControl() with M_AUG_..._OP. By default, all operations are disabled.

  4. Optionally, modify the range of values for the image processing operations that the augmentation can perform, using MimControl(). MIL randomly chooses the values to use for the operations from within the specified range.

  5. Optionally, set the priority (order) with which the augmentation can perform the image processing operations, using MimControl() with M_PRIORITY.

  6. Optionally, set the probability with which the augmentation can perform the image processing operations, using MimControl() with M_PROBABILITY.

  7. Perform the augmentation, using MimAugment().

    This function performs the augmentation operation on a source image and places the results in either an image buffer or an augmentation result buffer, depending on the destination parameter setting. The resulting image buffer holds the modified version of the source image. The result buffer holds information about the modifications.

  8. If you called MimAugment() with an augmentation result buffer, retrieve the required results, using MimGetResult().

    Important results include the image processing operations performed, and the values used for them, for each call to MimAugment(). To save a report of such results, call MimStream() with the M_SAVE_REPORT operation.

  9. Optionally, draw the resulting image of the augmentation, using MimDraw() with M_DRAW_AUG_IMAGE.

  10. If necessary, save your augmentation context, using MimSave() or MimStream().

  11. Free all your allocated objects, using MimFree().

The augmentation operation is usually run multiple times, since most augmentation related applications require multiple modified images. Typically, you should save the augmented images (MbufSave()). For more information about how to use the augmentation operation, see the following example:

mimaugment.cpp

Note, you can interactively configure and execute augmentation operations with MIL CoPilot.

Priority

By default, the image processing operations (M_AUG_..._OP) that MimAugment() can perform are prioritized (ordered) as follows:

  1. Affine operations, such as M_AUG_ROTATION_OP.

  2. Structure operations, such as M_AUG_DILATION_OP.

  3. Geometric operations, such as M_AUG_FLIP_OP.

  4. Intensity operations, such as M_AUG_INTENSITY_MULTIPLY_OP.

  5. Linear filter operations, such as M_AUG_SMOOTH_DERICHE_OP.

  6. Noise operations, such as M_AUG_NOISE_GAUSSIAN_ADDITIVE_OP.

To modify this order, call MimControl() with M_PRIORITY. Changing the order of the operations can alter their final effect; for example, smoothing an image before adding noise differs from adding noise before smoothing.

Probability

Each time you call MimAugment(), MIL randomly selects which enabled image processing operations to perform. By default, each operation has an equal chance of being randomly selected. To modify these chances, call MimControl() with M_PROBABILITY.

Note, setting the probability to 100.0 ensures that MIL performs the operation, while setting 0.0 ensures that MIL will not perform the operation (this is the same as disabling the operation).

Randomness and seeds

To establish the randomness required by the augmentation operation, MIL uses an internal random number generator (RNG).

All the random values that MimAugment() uses trace back to a seed; that is, the first initialization value of the internal random number generator. By default, MIL randomly generates a new seed each time you call MimAugment().

To specify an explicit initialization value for the internal random number generator, call MimControl() with M_AUG_SEED_MODE set to M_RNG_INIT_VALUE, and then use M_AUG_RNG_INIT_VALUE to specify the actual value. Using the same seed multiple times lets you rerun (repeat) the same augmentation process with the same randomized settings. This seed is saved with the augmentation context.

You can also specify the seed directly when calling MimAugment(), by using the SeedValue parameter. To do so, you must set M_AUG_SEED_MODE to M_USER_DEFINED_SEED. This seed is not saved with the context.

Note that, you can get the seed that MimAugment() uses (regardless of how the seed was established) with the M_AUG_SEED_USED result, and then use that seed to repeat that exact augmentation operation.