 |
|
|
Highlight: We have a cool 3 minute video (20Mb, WMV format) showcasing some of our demos.
Research
Image Pre-Conditioning for Non-Uniform Out-of-Focus Projector
Blur
Photometric Calibration of Multiple Projectors and
Cameras
Geometric Calibration of Projected Displays
Dynamic Shadow Elimination and Occluder Light Suppression
Demonstrations & Applications
Large-Scale Displays Based on Multiple Casually-Placed
Projectors and Cameras
Projector Light Fantastic: Dynamic Relighting of Indoor
Environments
Virtualized Whiteboard
Persistence of Light (Tracking Spotlight)
E-MELDER: Bridging the Digital-Physical Divide
Hand Pointing Interface
SmartSpace is the primary home to SCE’s research into
the innovative use of projector-camera systems (PROCAMS). The long-term goal of
the PROCAMS project is to develop key technologies that will enable the creation
of ubiquitous displays in intelligent environments, creating a present-day
version of the `holo-deck’ where the walls, tabletops, floors, and other objects
can be transformed into interactive displays.
Such ubiquitous displays will
revolutionize the way interactive media may be delivered to users, and will
enable augmented reality applications to be used without cumbersome heads-up
displays. Furthermore, ubiquitous displays can dramatically enhance our
pervasive access to computational and information resources, breaking free of
confining small handheld devices and creating a substantial impact on work and
home lifestyles. It is impossible to achieve this vision, at least not
practically, by mounting flat-panel displays on all visible surfaces. Instead,
the approach adopted by researchers is to create ubiquitous displays through a
moderate number of casually placed projectors and cameras that span large and
overlapping fields of view. This approach would allow for the creation of
interactive displays on challenging surfaces such as curved pillars, moving
objects, people, and even cloth.
As part of this research initiative, we have a research project entitled Perceptual Fidelity in Projector-Based Ubiquitous Interactive Displays, funded as part of a SCE - Georgia Tech collaboration program. The PI's are myself and Michael Brown, and the collaborators are Wolfgang Müller-Wittig from NTU, as well as Jim Rehg and Irfan Essa from Georgia Tech. NTU PhD students
(Joint work with Michael Brown and Peng Song)
 |
We present a technique to reduce image blur caused by out-of-focus regions in projected imagery. Unlike traditional restoration algorithms that operate on a blurred image to recover the original, the nature of our problem requires that the correction be applied to the original image before blurring. To accomplish this, a camera is used to estimate a series of spatially varying point-spread-functions (PSF) across the projector’s image. These discrete PSFs are then used to guide a pre-processing algorithm based on Wiener filtering to condition the image before projection. Results show that using this technique can help ameliorate the visual effects from out-of-focus projector blur.
Michael S. Brown, Peng Song and Tat-Jen Cham. Image pre-conditioning for out-of-focus projector blur. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New York, NY, volume 2, pages 1956-1963, 2006. [PDF]
(Joint work with Peng Song)
 |
 |
A theory for photometric calibration of cameras and multiple projectors with overlapping displays is presented. The theory is predominantly based on the analysis of isointensity curves in projector input state space – curves which define different projector input intensity combinations that result in the same camera-observed pixel intensities. Three methods, which have different speed and accuracy tradeoffs, are proposed for recovering the projector-to-screen and screento- camera intensity transfer functions. The methods do not require a specific parametric model for the shapes of these functions, nor impose any smoothness constraints. Additional methods are described for calibrating projector offsets and binary light sources, and also extending the perpixel analysis to other pixels in the display. The methods do not require the use of expensive equipment, and may be carried out with a low dynamic range camera with minimal controls.
Peng Song and Tat-Jen Cham. A theory for photometric self-calibration of multiple overlapping projectors and cameras. In Proc. IEEE International Workshop on Projector-Camera Systems (PROCAMS), San Diego, CA, 2005. Joint Best Paper Award. [PDF]
(Joint work with Rahul Sukthankar, Gita Sukthankar, Jim Rehg, Han Chen, Grant Wallace and Matt Flagg)
(Joint work with Jim Rehg, Rahul Sukthankar and Gita Sukthankar; subsequently with Peng Song, Jay Summet and Matt Flagg)
Two related problems of front projection displays which occur when users obscure a projector are: (i) undesirable shadows cast on the display by the users, and (ii) projected light falling on and distracting the users. This paper provides a computational framework for solving these two problems based on multiple overlapping projectors and cameras. The overlapping projectors are automatically aligned to display the same dekeystoned image. The system detects when and where shadows are cast by occluders and is able to determine the pixels which are occluded in different projectors. Through a feedback control loop, the contributions of unoccluded pixels from other projectors are boosted in the shadowed regions, thereby eliminating the shadows. In addition, pixels which are being occluded are blanked, thereby preventing the projected light from falling on a user when they occlude the display. This can be accomplished even when the occluders are not visible to the camera. The paper presents results from a number of experiments demonstrating that the system converges rapidly with low steady-state errors.
Please see a 3 minute video (20Mb, WMV format) showcasing some of our demos.
(Based on an FYP project by Li Koon TANG)
This project involved designing and implementing a self-calibrating system for deploying large displays created through the combined output of multiple casually-placed projectors. It uses real-time geometric calibration to provide a display made up of casually tiled projectors on arbitrary display surfaces, and its set-up takes only a few seconds. Requiring only a computer, low-resolution camera and twin output video card in addition to portable projectors, this is a practical and relatively inexpensive way to build flexible tiled displays.
 |
(Based on an FYP project by Hui Min LOI)
Based on only a single fixed projector, the projector-camera system created in this project is able to cast apparent shading and fake shadows on objects that lead to a user visually perceiving the presence of virtual light sources located anywhere in the environment. Such a system would allow for different interior lighting design to be instantaneously created without the need for wiring up different physical lamps, which would be a costly and tedious process. For example, the system would allow the simulation of top-down halogen lamps located near the wall, before rapidly changing to back-cast fluorescent lighting, before yet again switching to upward plant spot-lighting, without physically needing any of the different lighting systems. With the additional use of 3D position sensors (we deploy a MIT/Crossbow Cricket system), users are able to physically specify the locations of different lighting sources in real-time, including locations in mid-air. The system will create a real in situ experience of the lighting effects resulting from the virtual light sources.
 |
(Based on an FYP project by Miana Aprilia HUSADA)
This project involves augmenting an ordinary white board with
a projector-camera system to create advanced capabilities. The user works on the
whiteboard as in the usual way, drawing or writing with ordinary whiteboard
marker. Whenever the user wants, she can easily grab an electronic copy of the
specified region which contains things that have been drawn or written with the
marker pen. This copy can be loaded and projected back onto the board with user
specified size and location. Features in virtualized whiteboard enable the user
to duplicate, move, enlarge or reduce, and save the physical ink for later
retrieval. The physical ink on the whiteboard may then be erased using a normal
whiteboard eraser.
Meanwhile,
the user may insert image from file or add to the existing electronic copy on
the whiteboard with the marker pen as before. At any time, these new markings
can be saved into digital form to merge with the existing electronic version.
Moreover, features such as slide presentation and drawing are also implemented
in the system to give more flexibility to the user in explaining their ideas.
Most of the features mentioned above can be controlled using a laser pointer.
The main theme of the project is exploring the idea of a shared space for both
physical and digital entities.
(Based on an FYP project by Siang Cheng TAN)
(Based on an FYP project by Huizhuan CAI)
(Based on an FYP project by Maggy Anastasia SURYANTO)
This project presents a projector-camera system-based approach to replace
pointing devices by enabling users to use their arms for interacting with large
immersive projected displays. Users are allowed to walk around freely in a room
and select information projected on a wall screen by using their own hands as
pointing devices. Once captured and tracked in real-time, hand pointing actions
are remapped onto the screen, thus reproducing in an advanced interaction
scenario the behavior of traditional 2-D pointers.
This project also used simple
hand size matching for detection of hand-opening and closing gestures, as an
analogy to mouse click. This approach is inexpensive in term of computational
cost and proves to be accurate as well.
CeMNet SmartSpace - where imagination evolves into reality