Embedded real-time refocusing

Following the derivation of the plenoptic ray model and its efficient refocusing algorithm in spatial domain, this page demonstrates real-time refocusing for FPGA implementation. This may be of interest for the field of motion picture arts, which Lytro Inc. seeks to enter with its announcement to release a professional cinematographic light field camera (reference). A rough schematic of the tested implementation setup is illustrated below with a refocusing processor block in dashed lines being sub-divided into horizontal and vertical processing units with potential parallelization.

Block diagram for FPGA processing

The refocusing computation units for horizontal and vertical direction can be represented by FIR filters with switch controls. The FIR filter design depends on the refocusing parameter a. Switch states are stored in a register matrix given by s(a, w, p) where w and p are column and row indices respectively with each row being loaded after a pixel clock cycle. To prevent clipping, a divider is introduced in the shape of h0 which is all the same for each register.

Semi-systolic FIR filter

Corresponding timing diagram with internal register states can be seen below. Note that colours represent spatio-angular ray samples of the model introduced in the derivation and refocusing sections.

Timing diagram of semi-systolic FIR filter

For a more detailed explanation, please consider to read the research articles provided hereafter.

Related Publications

Real-Time Refocusing using an FPGA-based Standard Plenoptic Camera

C. Hahne, A. Lumsdaine, A. Aggoun and Vladan Velisavljevic, "Real-Time Refocusing using an FPGA-based Standard Plenoptic Camera," IEEE Transactions on Industrial Electronics (TIE), (2018).

[PDF] [BibTex] [VHDL code]

The standard plenoptic camera: applications of a geometrical light field model   

C. Hahne, "The standard plenoptic camera: applications of a geometrical light field model," PhD thesis, Univ. of Bedfordshire, (January, 2016).

[PDF] [BibTex]

If you have any questions, feel free to contact me via   info [ät] christopherhahne.de