3D, OpenCV, and Me - Homework#2

3D, openCV, and me - 3Dsav#2 from Kevin Bleich on Vimeo.

This assignment will go down in infamy. I am pleased with my results, but there is still so so so so much more to explore with all this data.

So far, I have been able to calculate and draw the bounding box, and centroids. I next experimented with some blob detection and contour finding. Once I was able to get something cogent together I thought an easy way to track small blobs in 3D would be to change the threshold amount on the contour search along with my movement BACK and FORTH in the z-axis. Here I am only calculating the brightest bits so, for example, if I were tracking fingers on both hands as soon as one hand is farther back than the other, the dual track is lost. Must work on this.

For some quick interaction I decided I would use the blobs to control the pan around the 3D point cloud, which is what you are seeing here. Apologies for the jumpy graphics, I think I was getting some noise in the Kinect because it was being a lot more smooth than that.

In my research of openCV and motion tracking, I realized I knew zilch about coding for gesture recognition. I think I am ready for that challenge. I think I'll be fine though since I really didn't know how to do what I did in the video a week ago.

At the top of my list for questions tomorrow is to ask about great resources for learning how to track and utilize gestures, as well as learn more about how I might be able to track multiple discrete objects in 3D.

I have a few ideas for visualization as well, but I am getting way ahead of myself. 3D baby steps.

Kevin

Week 2: Homework

This week's homework consists of four separate components.

1. Find the bounding box center and centroid for 3d data. Visualize their relationship.
2. Pick a computer vision technique, and extend it to 3d. For example, apply a blur or morphological filter to the 3d data and think about how it is affecting the data. For advanced students, consider writing an optical flow algorithm, extending contour detection/connected components to 3d, or using the depth image to inform a face tracking algorithm.
3. Track a single gesture in 3d. This may use the results of steps 1 and 2. For example, using the result of step 1 to recognize a "superman" gesture.
4. Post a summary of the information from last week's project on the class blog. You may host it elsewhere, but you need to at least provide a link and one picture here. The description should provide enough information that with some ingenuity could recreate it, but it doesn't have to be written as a tutorial.

A basic project for working with 3d data is available on the class github. For instructions on how to use the code, see the readme on that page.

Week 1: Homework

Due next week, you have three assignments.

Make a 3d Scan
In groups of 4, make a 3d scan. The more DIY and low-tech, the better. It just can't be visualized with a Rutt-Ettra height map. Depending on your technical expertise you may:

1. Invent a 3d scanner. It could be super-low-resolution, or single-image instead of realtime.
2. Reimplement a technique that already exists, like using a shadow, laser, or binary subdivision.
3. Get code running for a technique that someone else created, like my structured light code or OpenCV stereo matching.
4. Use a tool that already exists, extract and display the data in your own way. Like using Photosynth, Kinect, or David Laserscanner.

We'll share the results of your work in the next class. Because this assignment is so open ended, you can use whatever tools/environment you like. Future assignments will be written in Open Frameworks only so we can share source code.

Post a Link
Post a cool project related to 3d scanning to this blog with a sentence or two describing it.

Brainstorm for your final project
Start thinking about where you want to take all this! How does all this relate to your work? What techniques, ideas, or aesthetics are you most interested in? You don't need to turn anything in for this, but feel free to email us with questions and ideas or talk to us in person.