When I was young I was fascinated by the playstation 2 EyeToy. I used to wonder how it worked and the movement detection intrigued me. During my years at university I took a course on computer vision thinking that it might fill me in on the doubts I had on the subject. The course was great on a theoretical point of view and I thought there was no better way to test out the theories than by making a webcam game. Unfortunately at the time I did not have time but I kept the idea close to heart.
So a few years later I finally had the time and motivation to make this project reality. The idea was simple there are enemies attacking you from all the screen edges and you need to move your hand around to defend yourself. Of course, being part of Sleeping Penguinz, I had to make the enemies angry penguins that threw snowballs at you.
Now that I had an idea all that was left was implementing the webcam input. In unity we can get the webcam input texture like so:
Take into account that this code snippet is an extract of my code and might be missing function definitions to be functional. I am sure you can fill in the wholes. If you struggle for a very long time to do so, feel free to contact me and I will see if I can help you out.
By reading the code above you might have figured out the technique I use to detect motion. It is actually the most basic idea and surprisingly it works quite well. The idea is taking the current input frame from the webcam and subtracting the previous from from it. Comparing each pixel of the result to a arbitrary threshold (depends on the camera and lighting of the room, should be set by the player) we can find every spot on the input image that “moved”/changed from the previous input image.
Now that we know how to detect movement on the webcam input we need to map that movement on the screen. To do so I used the Unity UI canvas and scaled down the resolution of the screen to match the webcam pixels one-to-one.
From there I knew the area and position that a snowball took on screen. To detect if the player hits a snowball I compute the number of pixels that “moved”/changed inside that snowball. If the number of pixels that changed is greater than a third (arbitrary number, it could be anything, but from play testing it felt right) of the area of the snowball then it counts as a hit.
As you see the idea is simple and it works fairly well. It took me two or three days to get the webcam movement detection to work as I wanted. Then I started adding more features, penguins and projectiles like bombs, fish and bananas, that have different patterns from snowballs.
All the beautiful art was made by Moulfry. You can find the game here
In the end I passed roughly a month working on Penguin Move and I really enjoyed making this game.