Computational Displays

Introduction

Displays have seen many improvements over the years. With advancements in pixel resolution, color gamut, vertical refresh rates and power consumption, displays today have become more personal and accessible devices for sharing and feeding information. Advancements in computer human interaction have provided novel interactions with current displays. Touch panels are now common and provide better interaction with the cyberworld shown on a device. Displays today have many shortcomings still. These include rectangular shape, low color gamut, low dynamic range, lack of focus and context in a scene, lack of 3D viewing, etc. Efforts are being made to create better and more natural displays than 2D flat rectangular screens we see today. Much research has gone into designing better displays including 3D displays, focus and context displays, HDR displays, etc. Such displays work at the device level and use physical (mechanical, metallurgical, chemical, etc) means to create better displays. Such approaches prove expensive and are hard to scale to acceptable standards of color, refresh rate, etc. of current displays. An alternate to this is to use computation to enhance displays. We propose Computational Displays, which employ computation to economically alleviate some of the shortcomings of availabe displays. Specifically we give solutions to the walk-around 3D viewing, focus+context and color resolution problems The systems work on top of existing display methodologies and are independent of any specific display technology. This makes our systems scalable to any display method and enhances the same using computation. In conclusion, we argue that such computation/algorithms should be built into the displays themselves to enhance the visual experience.

View Dependent, Multi-Planar, Walkaround 3D Displays

MultiPlanar

Displays have remained flat for most part since their inception. In this work we focus on non-planar displays made out of planar facets. We focus on displaying 3D scenes in perspective correct manner to such displays. We propose an accurate rendering mechanism using GPU shaders that produces correct color and depth map on each facet, ensuring consistency across facet boundaries. We compare our results with various previous methods such as CAVE and Projective Texture Mapping and conclude that our method is artifact free and superior in terms of visual quality - a requirement for visualization applications. We also provide a scalable GPU culling algorithm to scale our rendering scheme to any display shape, consisting of over a thousand facets. The pipeline proposed uses commodity GPUs to implement the system and can handle any type of scene, even mesh deformations as shown in the video.

Distributed Massive Model Rendering

mpp

Graphics models are getting increasingly bulkier with detailed geometry, textures, normal maps, etc. There is a lot of interest to model and navigate through detailed models of large monuments. Many monuments of interest have both rich detail and large spatial extent. Rendering them for navigation on a single workstation is practically impossible, even given the power of today's CPUs and GPUs. Many models may not fit in the GPU memory, the CPU memory, or even the secondary storage of the CPU. Distributed rendering using a cluster of workstations is the only way to navigate through such models. We present a design of a Distributed rendering system intended for massive models. Our design has a server that holds the skeleton of the whole model, namely, its scenegraph with actual geometry replaced by bounding boxes at all levels. The server divides the screen space among a number of clients and sends them a list of objects they need to render using a frustum culling step. The clients use 2 GPUs with one devoted to visibility culling and the other to rendering. Frustum culling at the server, visibility culling on one GPU, and rendering on the second GPU form the stages of our distributed rendering pipeline. We describe the design and implementation of our system and demonstrated the results of rendering relatively large models using different clusters of clients in this work. The demonstration video shows the interactive rendering of huge scenes of Coal Powerplant (approx. ~96M triangles) and Fatehpur Sikri (~172M triangles) models on a 4-client setup.

Increasing Intensity Resolution on Single Displays

SpatioTemporal

Displays have seen many improvements in spatial resolutions and vertical refresh, to provide a smoother visual experience. Color intensity resolution, however, has not changed much. Most displays are still limited to 8-bits per channel. Simultaneously, much work has gone into capturing high dynamic range images. Mapping these directly to current displays loses information that may be critical to many applications. We present a way to enhance intensity resolution of a given display by mixing intensities over spatial or temporal domains. Our system sacrifices high vertical refresh and spatial resolution in order to gain intensity resolution. We present three ways to mix intensities: spatially, temporally and spatio-temporally. The systems produce in-between-intensities not present on the base display, which are clearly distinguishable by the naked eye. We evaluate our systems using both a camera and human subjects, evaluating if our system scales the intensity resolution and also ensuring the newly generated intensities follow the display model.

View Dependent Parametric/Implicit Displays

Parametric

This work extends the view dependent displays to non-planar parametric shapes. In this the display surface is not defined using planar facets, but by a set of parametric or implicit equations. To render perspective correct 3D on to such a display requires non-linear rasterization. No graphics pipeline at the moment supports this. We approximate it using the tessellation hardware provided by the new Shader Model 5.0 / DirectX 11 GPUs. We break large triangles on the fly to a user defined error threshold using the tessellation hardware. The resulting vertices are then moved using per-vertex raycasting to correct their positions in the observer's eye. This results in correct perspective view in the eye of a user whose head is tracked. The method is an approximate rendering scheme to any parametric display, however does not interpolate pixels as opposed to Projective Texture Mapping. The rendering speed is orders faster than approximating the surface using planes because of single pass rendering and the use of tessellation hardware.

Garuda: A Scalable, Tiled Display Wall using Commodity PCs

Garuda

Garuda is a client-server based display wall system employing distributed rendering to render massive 3D environments at interactive framerates to a tiled display. The system is designed and built using commodity hardware. Features such as client caching and use of server-push philosophy in conjunction with UDP multicast help the system scale to very large tiled configurations. The system uses a novel culling algorithm to find what part of the scene is in which of the tiles. This is a scalable algorithm both in terms of scene complexity and number of tiles. The system is built as a library intercept mechanism over the Open Scene Graph (OSG) API. This ensures that any OSG application can be ported to Garuda without the need of modifying, recompiling or relinking the code. The OSG executable directly runs on the tiled display using this. Rendering capabilities increase with number of tiles as distributed rendering is employed. No machine in Garuda, server or clients, renders the entire environment. Culling is performed at the server and part of the scene graph are sent to clients over standard Ethernet. Clients cache the geometry and evict it based on LRU, this exploits the temporal coherence in the scene. Rendering happens at client end with the server used to to synchronize the buffer swap.

Related Publications

Pawan Harish, Parikshit Sakurikar, P J Narayanan - Increasing Instensity Resolution on a Single Display using Spatio-Temporal Mixing Proceedings of the 8th Indian Conference on Vision, Graphics and Image Processing, 16-19 Dec. 2012, Bombay, India. [PDF]
Revanth N R, P J Narayanan - Distributed Massive Model Rendering Proceedings of the 8th Indian Conference on Vision, Graphics and Image Processing, 16-19 Dec. 2012, Bombay, India. [PDF]
Pawan Harish and P. J. Narayanan - Designing Perspectively-Correct Multiplanar Display IEEE Transactions on Visualization and Computer Graphics, Vol.99, 2012. [PDF]
Nirnimesh, Pawan Harish and P. J. Narayanan - Culling an Object Hierarchy to a Frustum Hierarchy, 5th Indian Conference on Computer Vision, Graphics and Image Processing, Madurai, India, LNCL 4338 pp.252-263,2006. [PDF]

Pawan Harish, Parikshit Sakurikar, P. J. Narayanan - Spatio-Temporal Mixing to Increase Intensity Resolution on a Single Display (Poster) In CVPR Workshop on Computational Cameras and Displays 2012. [PDF]
Pawan Harish, P. J. Narayanan - View Dependent Rendering to Simple Parametric Display Surfaces (Short Paper)" In JVRC of EuroVR-EGVE, 2011. [PDF]
Pawan Harish, P. J. Narayanan - A View Dependent, Polyhedral 3D Display, in ACM SIGGRAPH Virtual Reality Continuum and its Applications in Industry 2009. [PDF]
Nirnimesh, Pawan Harish, P. J. Narayanan - Garuda: A Scalable Tiled Display Wall using Commodity PCs, In IEEE TVCG, 2007. [PDF]