Overview

Pyimof actually only supports gray scale images in its implemented methods.

Implemented methods

Optical flow algorithms can mainly be classified into

  • global methods based on pixel-wise matching costs with regularization constraints, ie based on the Horn & Schunck paradigm.

  • local methods based on local window matching costs, ie based on the Lucas-Kanade paradigm.

Pyimof provides one implementation for each of these class of methods:

  • TV-L1: A popular algorithm introduced by Zack et al. 1, improved in 2 and detailed in 3 (See pyimof.solvers.tvl1()).

  • Iterative Lucas-kanade: A fast and robust algorithm developped by Le Besnerais and Champagnat 4 and improved in 5 (See pyimof.solvers.ilk()).

These two algorithms have been selected for theire relative speed. Efficient GPU implementations for both of them have been developped but it is not planned to port Pyimof on this platform.

Datasets

Multiple datasets for the evaluation of optical flow algorithms have been developped (for example Middlebury, MPI-Sintel and Flying Chairs). The two-frame grayscale Middlebury training dataset 6 is accessible via the pyimof.data module functions to ease testing.

IO

Estimated vector fields can be saved and loaded in the .flo file format using the pyimof.io module.

Visualization

Visualizing optical flow is made easy using the pyimof.display module

  • the pyimof.display.plot() function applies a colormap (preferably circular 😉) to the optical flow according to its direction and magnitude. An optional color-wheel showing the color code can also be displayed to ease resulting image understanding.

  • the pyimof.display.quiver() function draws a quiver plot with multiple option for coloring the arrows and displaying a background image.

Moreover, Pyimof gives access to a Matplotlib colormap inspired by the popular color code used by the Middlebury evaluation site for displaying algorithms’ results.

References

1

Zach, C., Pock, T., & Bischof, H. (2007, September). A duality based approach for realtime TV-L 1 optical flow. In Joint pattern recognition symposium (pp. 214-223). Springer, Berlin, Heidelberg.

2

Wedel, A., Pock, T., Zach, C., Bischof, H., & Cremers, D. (2009). An improved algorithm for TV-L 1 optical flow. In Statistical and geometrical approaches to visual motion analysis (pp. 23-45). Springer, Berlin, Heidelberg.

3

Pérez, J. S., Meinhardt-Llopis, E., & Facciolo, G. (2013). TV-L1 optical flow estimation. Image Processing On Line, 2013, 137-150.

4

Le Besnerais, G., & Champagnat, F. (2005, September). Dense optical flow by iterative local window registration. In IEEE International Conference on Image Processing 2005 (Vol. 1, pp. I-137). IEEE.

5

Plyer, A., Le Besnerais, G., & Champagnat, F. (2016). Massively parallel Lucas Kanade optical flow for real-time video processing applications. Journal of Real-Time Image Processing, 11(4), 713-730.

6

Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011). A database and evaluation methodology for optical flow. International Journal of Computer Vision, 92(1), 1-31.