Week 1: Intro to 3d

intro to input techniques
radiohead "house of cards" video
data and source code for visualizing it, in processing
see the "making of" for more info
things people made with the data: pin art, lego visualizations...
it got engadget play but yesterday so did a video made with kinect
we have the tools, but what should we do with them?

time of flight infrared
wikipedia has a good overview
example video is 320x240, poor fps. compare to kinect at 640x480 and 30 fps. this video comes from the baumer tzg01

lidar
flying over netherlands in a plane and rendering the massive point cloud
newer (last few years) lidar can run at high speeds, e.g., SICK scanners or the velodyne scanners
velodyne was used for radiohead video (outdoor scenes and party scene)
doing a lidar scan looks something like this
can you make your own lidar scanner? maybe starting with a laser rangefinder?
lidar is similar to sonar + radar (light detection and ranging, sound navigation and ranging, radio detection and ranging). lidar uses pulses of light, radar uses pulses of radio-frequency light, sonar uses pulses of sounds. maybe you can make a diy sonar with an ultrasonic rangefinder and rotating sound-reflective plate?

structured light
major component of the radiohead video. 'no cameras or lights' is a lie. one camera, one projector, used for all the close up shots.
some of my work: started out with gray code subdivision, moved on to three phase scanning. started getting realtime results from the scanner. the process of scanning looks like this. a year ago it started running around 60 fps with some crazy hacks. i shoot and visualize 3d data for the broken social scene video. recently, i collaborated with three other artists the janus machine
the highest resolution structured light comes from debevec, regularly used in hollywood. see 'the digital emily project'
maybe the lowest resolution structured light comes from this iphone app which is actually more similar to 'shape from shading' assuming it calculates surface normals and propagates them.
you can detect edges with multiple flashes (there is a quick explanation in this generally mindblowing lecture from ramesh raskar).
if you don't put any light in the scene, you might still be able to use the known lighting somehow (shape from shading).

kinect
really, a kind of 'structured light', but using an infrared pattern
the video with 2 million views that explained to everyone what it means to have a 3d camera
primesense has developed some great skeletonization software that works with depth images.
super oldschool skeletonization: muybridge stop motion, markers, long exposure
normal approach to skeletonization: motion capture no one ever knows how to get them working
newschool is kinect. passive interaction (no markers needed) can be used for controlling robots!

coordinate measurement machines
lego 3d scanner using touch
this is how pixar did (does) it for a long time, just more expensive machines

optical triangulation
david laserscanner is not open source, but basic version is free. fairly long and involved process
lasers go well with turntables. here are some projects that use turntables with varying levels of diy.
if you need a line, you can use a wine glass stem as a lens.
or you can use the shadow of a stick instead. checkout out the byo3d course notes.
you don't have to automate the movement though, you can be the movement.
for a sort of 'orthographic triangulation', use milk or ink instead of a laser.
carefully watch friedrich's video from the eyebeam mixer.
what is implied by the process of making a 3d scan?
is it theatrical (pool of ink) or is it passive (kinect, airport 3d scanners)
do you have to pose, and what does that pose mean? (see 'hands up' essay)

stereo vision
point grey bumblebee2 camera, a couple thousand dollars.
used by golan levin for double-taker/snout (check out the debug screen)
used by joel gethin lewis for "to the music" music video for colder, and for "echo" dance perforamnce at uva.
lots of other stereo vision software out there, not just point grey. amazing collection of different algorithms and their accuracy at middlebury. opencv has great stereo matching routines, but can be slow if not tuned well.
state of the art for face scanning might be mesoscopic augmentation with stereo matching.
the main advantage of stereo over kinect is it can do daylight. the main disadvantage is that sparse scenes are hard to reconstruct, and the data is much noisier.

structure from motion
a bit like like stereo, but the camera moves over time and the scene stays still.
some are even online (realtime) like ptam.
there is a list of a bunch of sfm software at the end of the wikipedia page.
voodoo is free, boujou is $10k.

multiview stereo
photosynth can be used for large scenes (like all of barcelona) or small scenes (like a single face).
the photosynth data can be intercepted from the server and used in other software, like processing. there are probably better ways to do the export now than manually with wireshark.
photosynth is based on a tool called bundler.
the results of bundler, a sparse point cloud can be fed into cmvs to get a dense point cloud. cmvs used to be called pmvs2. it's pretty high resolution.

mri + ct scan
most hardcore use of 3d scanning: print yourself a new heart, save your life.
mri is expensive (requires massive, expensive magnets) and ct is dangerous (x-rays). not really diy. but they do give you voxels: a full 3d space that sees 'inside' things, not just the shape of their surface from one perspective
like an mri, sharks see em fields. maybe something like this (kinect + em sensor)?

other techniques
cameras have all kinds of properties that vary with respect to depth. one is defocus, and you can use this to estimate depth.

intro to output techniques

3d on 2d screens
if you're displaying 3d on a 3d screen you might get the wiggle. the wiggle can creep into the camera movements of any 3d video, because our brain doesn't really realize it's 3d otherwise.
sometimes it can be useful to have a 3d controller for a camera in an interactive context.
or maybe other cues like defocus (depth of field) can be used to show depth? i've done a little work on this inspired by discussions with open ended group.

laser cutting
olafur eliasson's housebook
jared tarbell's work, making slices and making height maps

3d printing and volumetric laser etching
sophie barret-kahn has explored a bunch of techniques including crystal etching.
services like shapeways and ponoko offer 3d printing for relatively cheap on small scales.

cnc milling
foam can be milled to create reliefs, and the reliefs can be used to make plaster molds
and high end 5-axis cnc machines can do pretty much anything you can imagine