Monthly Archives: December 2015

Head Tracking 3D with OpenCV

Hello, in this post I want to present a small project developed with OpenCV for a simple head tracking with a normal webcam.
By head tracking I mean that the software tracks the user head movement in the image taken by a camera connected to the computer (could be a webcam or other sources). The tracking is actually a very fast and continuous search for faces. With the acquired position of the tracked head, a 3D render is called and is used to draw on the monitor a ‘fake’ window to give the tracked user the illusion of a proper projection.

The tracking and filtering of the data is done with OpenCV, the rendering with OpenGL/FreeGLUT/GLEW.

The software workflow is as follow:

  1. Initialization, where the Kalman filters used for smoothing the head tracking position, loading the 3D model rendered and the rest of the environment
  2. From this moment on, the 3D loop is activated, and does not communicate with the tracking side at all, except for the “SetCam” function, which specify where the user head is, and update the viewing frustum accordingly.
  3. The tracking loop instead opens the webcam stream, and starts tracking the first face found.
    • When a face is found, a second classification occours in the described area to look for two eyes, and to compute a synthetic distance of the user from the camera.
    • A kalman filter is then applied both to the x-y values, and also to the z value found before
    • Finally, values are communicated to the render for updating the projection matrix.

Here a video of the result:

Download link to the binaries, to try this on your computer (Windows only sorry):

Download binaries

And to download the full visual studio project (VS2012) with all included libraries:

Download sources

In the code I will provide, a lot of simplification and optimization can be done, but for my purpose this was good. For any question please feel free to drop me an email.