Convert Pipeline

The Convert pipeline converts the raw Bayer image data into a BitmapImage, after moving through the pipeline.

As the first step in the pipeline, the ImageSDK decodes the image, similar to the Decode pipeline (see Decode Pipeline).

Sensor Profiles

When converting the raw Bayer data into RGB, ImageSDK applies what we call Sensor Profiles. A sensor profile is a generic image sensor characteristic. Each IIQ file defines what camera it originated from, and what sensor was in that camera. This information is now used, to correct for nonlinearity in the sensor output data. This means the resulting RGB image data will better reflect the sensors true dynamic range.

ImageSDK comes with a collection of “Sensor Profile”-files, that each define the sensor characteristics for sensors used in Phase One’s cameras. Since these are distributed as stand-alone files, ImageSDK must be aware of where to look for them on the file system. By default, it will look for the files in a folder called SensorProfiles in the current working directory, but you can specify the location manually.

Set the directory where ImageSDK looks for Sensor Profiles, using this global SDK method:

P1::ImageSdk::SetSensorProfilesLocation("path/to/SensorProfiles");

Sdk.SetSensorProfilesLocation(SENSOR_PATH);

Caution

For the SetSensorProfilesLocation method to have any effect, you must call it before you open any RawImage objects. The opening will fail with an error “Unsupported sensor” in case the SDK doesn’t find the appropriate sensor profile.

Ordering

You cannot change the order of the steps in the convert pipeline. The order of the methods in the code itself is unimportant, as the Convert pipeline always will follow its own sequence of execution. This is due to performance optimization of the pipeline.

In the following example, we place the “scale”-step before the “crop”-step in the code:

        config.setCanvasClip(CanvasClipWarpLoose);
		config.SetOutputWidth(1024);
		config.SetCrop(1000, 2000, 1024, 768);

config.SetOutputWidth(1024);
config.SetCrop(x: 1000, y: 2000, width: 1024, height: 768);

Despite the order of the process-steps in the code, the pipeline will still execute in a pre-determined order. I.e., the following example will yield the same result as the previous example:

config.SetCrop(1000, 2000, 1024, 768);
config.SetOutputHeight(500);

config.SetCrop(x: 1000, y: 2000, width: 1024, height: 768);
config.SetOutputWidth(1024);

The output images from the two sample code snippets, will be identical.

The Configuration Object

To use the Convert pipeline, you first create an empty ConvertConfig configuration object:

P1::ImageSdk::ConvertConfig config;

ConvertConfig config = new ConvertConfig();

After creating the object, you can apply a list of adjustments to the image.

Image adjustments

To produce a desired output image, the recorded raw data can be tuned. A list of different adjustments help to render the output in a desired way. See the ConvertConfig documentation for the full set of adjustments.

The pipeline specify working range for those adjustments. If the user tries to set an adjustment outside of its boundaries, the setting will be clamped to the maximum/minumum value.

Coordinate systems

The image goes through a couple of geometric transformation in the conversion pipeline, and to be able to refer to certain coordinates on the image, it is important to define these transformations and related coordinate systems.

When the RawImage is loaded, the pixels are loaded into the Raw coordinate system.

Following that all transformations are applied, including geometric correction and orientation. After this the image is the Canvas coordinate system.

Because of some of the transformations, the image edges may become crooked, the transformed data will not be upright rectangle shaped (see black frame below). Still the image must be an upright rectangle. There is two way of handling this. Either by using the external or internal bounding box as image data. This is illustrated by the red and the blue frames. See CanvasClip for available options.

Finally the image is scaled to the requested output size. Then the image is in Output coordinate system.

The origin in all coordinate system is at the top left corner.

Note: some of these transformation may be identity transform, depending on the ConvertConfig and the RawImage.

Cropping

You can request the pipeline to output a smaller portion of the image defined by a rectangle. This rectangle is defined by its top left corner and width and height in the canvas coordinate system.

Here is an example of how we apply “crop”:

config.SetCrop(1000, 2000, 1024, 768);

config.SetCrop(X: 1000, Y: 2000, width: 1024, height: 768);

The positional offset and dimensions of a crop

The crop will be placed on the source image as defined by the X and Y coordinates: the offset-point is the top-left corner of the cropped image (a positive Y-value is downwards). The width and height then defines the size of the cropped area (see Basic Image Info on extracting the width and height of the image).

Note

An attempt to crop the image outside the range of its own width and height (invalid values) throws an exception.

As the ImageSDK is optimized for performance, the crop size can potentially add a few pixels to the width and height. The width and height of the cropped output image may thus differ a few pixels from the given parameters.

Warning

This means you will very likely not get the exact crop offset or dimension you requested. Most likely it will be a few pixels off.

Scaling

You can apply scaling of an image three different ways. SetOutputScale

Output Scale multiplies the size of the image by the input value
Output Image Width to a given number of pixels
Output Image Height to a given number of pixels

Those settings are mutually exclusive, so settings any of them will eliminate the previous setting.

If none of them are set the output image size will be 100% of the canvas size (see Cropping for details).

config.SetOutputScale(0.5);

config.SetOutputScale(0.5);

Applying conversion

The final step is to process an image through the Convert pipeline. You do this with the ApplyTo method, as follows:

P1::ImageSdk::BitmapImage bitmap = config.ApplyTo(image);

BitmapImage bitmap = config.ApplyTo(image);

When you apply the Convert pipeline to an image, the image conversion (“Demosaic”) is also applied, converting the image from raw Bayer data into a BitmapImage. BitmapImage is an in-memory representation of the RGB bitmap data (the “viewable” image).

From here, you can choose to employ methods of your choice (based on the Host system) to e.g. save the image into the desired image container format (Tiff, Png, Jpg etc.).

It is important to note that the data contained in BitmapImage is not an image file, like a BMP file. Despite the naming overlap. If you write the content of BitmapImage to disk (say, with fwrite), it will not be in any image file format. It will just be a plain data file with RGB pixel data.