Real-time Webcam Head Tracking with Speech Balloon

May 16, 2009 by Devin Reimer

I have been doing some experiments with head and eye tracking within flash for the last few months. Just recently I discovered a open source port of a head tracking library called Marilena. This is yet another amazing library from the Spark Project. This library takes care of most of the hard work when it comes to head tracking, but it is does not natively support video.  So once I got these library to work with my application it ran at about 4 fps. The good part was the head tracking algorithm was much better than anything I had created. Some tweaks later I got the application to around 18 fps. Which is better but still not acceptable. So after some searching I found an article by a guy named Mario Klingemann from over at Quasimondo. He had added some performance enhancements to the Marilena library. (Check out his article here). This got my application up to around 22 fps, which was much closer to where I wanted it. Finally I wrote a little function the cuts down the search area depending on if head has been found on the last time around, this at last got the frame rate up to 27 fps, which is completely usable.

This application allows you to type text into a text area which then gets updated into the speech balloon which is positioned directly above your head in the webcam video.

To try out the demo (webcam required) click here.

To get the source click here

2 Responses to "Real-time Webcam Head Tracking with Speech Balloon"

  1. Cory Bell says:

    Hi Devin,

    I really love the things that you are doing. They are so innovative and cool. I was @ the code convention a few months back @ RRCC downtown campus. I would really love to get togeter for lunch some time and chat about web dev and 3d apps. I am a RRC cap/ist student and i have been working for our province doing web pages in french and english for the past year and a half(just accross from fisheries and the smart park @ U of M). The gov policies dont let me get as creative as i would like due to standards and all that, but i see myself delving deeper into these web dev aspeck of computers. Feel free to contace me at either of my email i left on the above reply that is hidden. If not i will try to get intouch with you over the summer. I did meet With Scott last year, a sales manager who Randal Peters connected me with for an understanding of dmt, just to get a better feel of the industry. Oh and wow on this head tracking stuff, neatooo.

  2. nano says:

    it’s fantastic that flash can now do what it took a large computer & c a few years ago 😉

    (opencv + gentoo =

    love the five3d integration

    keep it up !

Leave a response