Recording Textures
Just One More Step
Unity provides two ways to readback pixel data from a
Texture
into system memory. They differ based on whether the client is willing to accept a deferred completion of the readback.A synchronous readback will request pixel data from the GPU and block until the request has been completed.
// Say we have some `RenderTexture`
RenderTexture renderTexture = ...;
var width = renderTexture.width;
var height = renderTexture.height;
// We can perform a synchronous readback using a `Texture2D`
var readbackTexture = new Texture2D(width, height);
RenderTexture.active = renderTexture;
readbackTexture.ReadPixels(new Rect(0, 0, width, height), 0, 0);
RenderTexture.active = null;
Once the above code executes, the pixel data will be accessible through the
Texture2D
.// Commit the pixel buffer
recorder.CommitFrame(readbackTexture.GetPixels32());
Due to the concurrent nature of GPU's, such a readback will typically result in a pipeline stall. This means that both the CPU and GPU cannot do any other work until the request is complete. Pipeline stalls are typically very expensive, and will cause a very noticeable frame rate hit in most cases.
Though expensive, synchronous readbacks exhibit very predictable memory consumption patterns.
// Issue a readback request
AsyncGPUReadback.Request(renderTexture, 0, request => {
// Once complete, access the data container
var nativeArray = request.GetData<byte>()
// And commit the pixel buffer
recorder.CommitFrame(nativeArray.ToArray());
});
This readback will request the data from the GPU but will not wait for the GPU to complete the request. When the transfer is complete, Unity will invoke the provided callback. We can then commit the pixel buffer to the recorder within this callback.
The advantage with this approach is that it provides much better performance over synchronous readbacks. There are a few disadvantages with this approach:
- It adds latency on the order of a few frames
- It has less predictable memory consumption patterns than synchronous readbacks.
NatCorder provides primitives that implement both synchronous and asynchronous readbacks, so that you don't have to. The
TextureInput
class handles synchronous readbacks:// Create a texture input
var recorder = ...;
var clock = ...;
var textureInput = new TextureInput(recorder);
// Commit video frames from a texture
var renderTexture = RenderTexture.GetTemporary(...);
textureInput.CommitFrame(renderTexture, clock.timestamp);
// Create a texture input
var textureInput = new AsyncTextureInput(recorder);
// Commit video frames from a texture
var renderTexture = RenderTexture.GetTemporary(...);
textureInput.CommitFrame(renderTexture, clock.timestamp);
Last modified 10mo ago