Using Cameras

Ultra Low Latency Camera Streaming

NatDevice provides an extensive API for using hardware cameras, encapsulated in the CameraDevice class.

Finding a Camera

We begin by finding a camera with a device query.

void Start () {
    // Create a device query to find camera devices
    var filter = MediaDeviceCriteria.CameraDevice;
    var query = new MediaDeviceQuery(filter);
    // Get the camera device
    var device = query.current as CameraDevice;
}

Make sure to add a using NatML.Devices statement at the top of your script.

Starting the Preview

Camera devices work by streaming the camera preview to a user-provided delegate function:

void Start () {
    ...
    // Start the preview
    device.StartRunning(OnCameraImage);
}

The delegate function accepts a CameraImage which contains a view to the pixel buffer provided by the native camera system, along with EXIF metadata about the image:

void OnCameraImage (CameraImage image) {
    // Do stuff
    ...
}

The provided delegate is not guaranteed to be invoked on the Unity main thread, so refrain from using Unity APIs within this method.

Streaming to Texture

The most common use case when using a camera device is to display the camera preview. For this, we will need to stream incoming camera images into a Texture. To do so, NatDevice provides the TextureOutput:

async void Start () {
    ...
    // Create a texture output
    var textureOutput = new TextureOutput();
    // Start the preview
    device.StartRunning(textureOutput);
    // Wait until the texture has been created
    await textureOutput.textureCreated;
    // Access the texture
    Texture2D previewTexture = textureOutput.texture;
}

Make sure to add a using NatML.Devices.Outputs statement at the top of your script.

Capturing Photos

Camera devices allow for capturing high-resolution photos:

// Capture a photo
device.CapturePhoto(OnCameraPhoto);

The provided delegate is invoked with a photo CameraImage:

void OnCameraPhoto (CameraImage cameraImage) {
    // Do stuff
    ...
}

Stopping the Preview

The camera preview can be stopped when it is no longer needed.

// Stop the camera
device.StopRunning();