.. _camera-liveview-label: LiveView -------- The LiveView feature enables seeing the camera's sensor output in almost real-time. This output is scaled down and converted into RGB (:term:`Debayered`) automatically by the camera. LiveView is received as a series of still images, like frames in a movie. The camera will supply as many frames as the network or USB connection allows. This means the *frame rate* will be variable. As image frames arrive on the CameraSDK side, they are delivered to you as *notifications* of the type ``CameraNewLiveViewImage``. This means you need to listen for that notification *and* activate the subscription to begin receiving LiveView image frames. To begin receiving image frames, you must enable the LiveView subscription, as follows: .. code-tabs:: .. include-tab:: ../../labels/cppEnableLiveView :language: cpp :title: C++ .. include-tab:: ../../labels/csEnableLiveView :language: csharp :title: C# This initiates LiveView in the camera, and it will begin to stream frames to the CameraSDK. Tap into this stream by creating (or re-using an existing) ``Listener`` object: .. code-tabs:: .. include-tab:: ../../labels/cppWaitForLiveView :language: cpp :title: C++ .. include-tab:: ../../labels/csWaitForLiveView :language: csharp :title: C# The ``ILiveView`` object contains e.g. the width and height of the LiveView frames, as well as information about the format of the LiveView frames. .. TODO: See Appendix K for further information on how to handle the frames, e.g. to apply a crop. The LiveView frames are stored in memory "*borrowed*" from the CameraSDK. Therefore, you must always dispose the LiveView image object, as soon as possible. .. warning:: Failing to dispose will cause the internal LiveView receive buffer to run out of memory, and become unable to pass new LiveView frames. When LiveView is no longer desired, stop the LiveView feed from the camera. This reduces both the power consumption and heat generation of the camera significantly. Disabling the LiveView feed can be done as follows: .. code-tabs:: .. include-tab:: ../../labels/cppDisableLiveView :language: cpp :title: C++ .. include-tab:: ../../labels/csDisableLiveView :language: csharp :title: C# Special Notification Behavior ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Given enough transfer bandwidth the camera is able to send 60 FPS, which will all arrive in the notification queue. To ease stress on the queue, LiveView notifications have a special behavior: *All incoming* ``CameraNewLiveViewImage`` *will overwrite any existing LiveView image notification, already present in the queue - in place!* This means, that at any time, there can only *one* single LiveView notification in a *Listener*'s queue. This notification will be overwritten whenever another (newer) notification arrives. This ensures you will always get the latest image frame, and that your queue will not get flooded with image notifications. You can still have two separate ``Listener`` objects, both listening for LiveView image notifications. Each of them will have *one* notification instance in their queue. Both these instances will be overwritten whenever a new notification arrives. Still, similar to other notifications, there is only a single image payload instance (LiveView image buffer object), shared between notifications across all *listener* queues. Again, there can only be *one single* LiveView notification present on a *Listener* queue. When new LiveView images arrive, they *will overwrite* the existing notification - in place! Keep this in mind, when working with LiveView. Configuration ^^^^^^^^^^^^^ The *subscription* class for LiveView includes a set of configuration parameters, that allows you to tailor how LiveView frames are processed in the camera. Most prominently, the configuration allows you to scale and crop the LiveView output. Both these operations happens *in the camera*, no processing is done by the CameraSDK. .. note:: Changing any of these configuration parameters, will *flush* the processing pipeline in the camera. This means there will likely be a brief drop in frame rate and you might even receive frames with visible artifacts. Formatting """""""""" By default all cameras deliver LiveView image in 24-bit RGB (8-8-8) pixels, row first order. This means the image frame is uncompressed, the buffer is the size of the sum of the 3-byte RGB pixels. Newer camera firmwares introduce support for Jpeg encoded image frames, effectively realizing something similar to *Motion JPEG*. In this mode, the image buffer will contain a valid Jpeg file, that could be written to disk and read by any image viewer. Using Jpeg instead of uncompressed 24-bit RGB is much more CPU intensive for the camera, so the maximum frame rate for Jpeg is 12 FPS. The benefit is a much lower bandwidth cost! This is ideal when using LiveView over an (outdoor) Wifi network. Cameras also have an *auto mode* which automatically switches between 25-bit RGB and Jpeg, selecting the one that gives the highest frame rate. .. caution:: Basic LiveView is available on all cameras. However, the capability to receive image frames as *Jpeg images* is introduced in CameraSDK 3.0. This *Jpeg* feature also requires your camera to run a newer firmware: * **iXM**: Firmware *4.23* or later * **IQ4**: Firmware *6.03* or later You can set your desired *format* when activating the LiveView subscription, using a custom ``Subscribe(...)`` method: .. code-tabs:: .. include-tab:: ../../labels/cppEnableLiveViewCustom :language: cpp :title: C++ ..include-tab:: ../../labels/csEnableLiveViewCustom :language: csharp :title: C# Or, you can use the designated ``SetFormat`` method on the subscription object: .. code-tabs:: .. include-tab:: ../../labels/cppLiveViewSetFormat :language: cpp :title: C++ ..include-tab:: ../../labels/csEnableLiveViewCustom :language: csharp :title: C# Not setting the format will keep the camera default setting, which is ``RGB24``. .. caution:: When using the ``Auto`` format, be aware that the camera will change the format, from one incoming frame ro the next. Always inspect the format of every frame and avoid making any assumptions on the format. If the frame rate drops, the camera might try to use Jpeg. On the other hand, if Jpeg frame rate sit consistently on 12 FPS, the camera might occasionally try switching to uncompressed RGB, to see if that can achieve higher than 12 FPS. Crop """" You can ask the camera to only send a portion of the total image to you, using the *crop* feature. Using *crop* you can effectively implement *zoom & pan* in LiveView image streams. The *crop* is defined by an *offset* and a *width and height*, as illustrated in the figure below: .. image:: ../7_imagesdk_features/crop-coords.png :width: 60% :alt: The positional offset and dimensions of a crop :align: center The offset coordinates (X,Y) is in relation to an origo in the upper left corner. Changing the dimension (width and height), will *not* change the final output size of the LiveView frame. It will change the amount of "down scaling" applied to the image. However, you *cannot* crop to a dimension (width and height), below the final output size of the LiveView frame. Set a crop on the LiveView image stream: .. code-tabs:: .. include-tab:: ../../labels/cppSetCrop :language: cpp :title: C++ ..include-tab:: ../../labels/csEnableLiveViewCustom :language: csharp :title: C# This applies a ``[ (x: 100, y: 100), (w: 2000, h: 2000) ]`` crop to the LiveView stream, and immediately flushes the internal processing pipeline. Note that you may still receive LiveView frames using the previous crop, since they may already be *in transit* to your :term:`Host Computer`. Your requested crop is *not* guaranteed to be obeyed precisely by the camera, since the cropping process is done by hardware. So the camera will find the nearest crop, supported by the hardware and use that. This means you will most likely get a crop that *slightly off* what you requested. For this reason, the LiveView image buffer class, ``ILiveView`` defines two crops: * ``SourceCrop()`` the crop you actually got. * ``RequestedCrop()`` the crop that you originally requested. Therefore, take care to inspect the incoming LiveView frames for what crop is applied to them, using the ``SourceCrop()`` method on the LiveView image frame object. .. tip:: Use the method ``ResetCrop()`` on the ``ILiveViewSubscription`` object to return to *no crop*, meaning you will see the entire sensor frame in the LiveView stream. Image Output Size """"""""""""""""" The camera will scale down every LiveView frame to a fixed output size, cropped or not. You can change this size using the *Preferred Max Dimension* option. This option defines the size (in pixels) of the longest edge of the image. Whether this edge is the *width* or *height* will depend on the currently applied *crop*, or if no crop, on the camera sensor's aspect ratio. .. For example, for a final output scaling (*Preferred Max Dimension*) of 1024 and a crop with dimension ``(w: 4096, h: 2304)`` the resulting LiveView image frame will have *source crop* of .. SupportedFormats ImageFormatsSupported() const = 0; void Subscribe(ImageFormat format) = 0; ImageFormat Format() const = 0; ValueRect Crop() const = 0; uint32_t PreferredMaxDimension() const = 0; ValueArea FullImageSize() const = 0; float Lightness() const = 0; void SetFormat(ImageFormat format) = 0; void SetCrop(ValueRect const& cropRect) = 0; void ResetCrop() = 0; void SetPreferredMaxDimension(uint32_t const maxDimensionValue) = 0; void SetWhiteBalancePoint(ValuePoint const& whitePoint) = 0; void SetLightness(float const lightness) = 0;