EPSRC logo

Details of Grant 

EPSRC Reference: EP/T01346X/1
Title: Graphics Pipelines for Next Generation Mixed Reality Systems
Principal Investigator: Steed, Professor A
Other Investigators:
Ritschel, Professor T
Researcher Co-Investigators:
Project Partners:
PhaseSpace Inc. Varjo Technologies Oy
Department: Computer Science
Organisation: UCL
Scheme: Standard Research
Starts: 01 May 2020 Ends: 31 August 2024 Value (£): 835,711
EPSRC Research Topic Classifications:
Computer Graphics & Visual. Networks & Distributed Systems
EPSRC Industrial Sector Classifications:
Information Technologies
Related Grants:
Panel History:
Panel DatePanel NameOutcome
06 Nov 2019 EPSRC ICT Prioritisation Panel November 2019 Announced
Summary on Grant Application Form

In the past twenty years display technology has moved on considerably

with improvements such as higher resolution, faster frame rates and

high-dynamic range colour. In the same period graphics processing

units (GPUs or graphics cards) have become significantly faster with

broader functionality. However, we argue that current implementations

of the traditional graphics pipeline, which is based on the

rasterisation and z-buffering, are unsuited to emerging displays.

In particular, the types of near-eye displays used for augmented

reality and virtual reality provide new challenges. Since their

contexts of use are very different, it is not even clear what

properties are ideal. Should the displays be multi-focal or

vari-focal? How fast do display update rates need to be to support

registered augmented reality systems? How do we exploit the properties

of the human visual system to render more efficiently?

The traditional computer graphics pipeline puts emphasis on generating

full images at the display size and rate, where that display might be

1920x1080 at 60Hz or higher. For a near-eye display, it is clear that

such a pipeline is nowhere near suitable: already one consumer HMD is

demanding 2480x1080 at 90Hz, roughly double the bandwidth that a common

desktop display requires. At these rates, there will still be problems

with visual acuity and latency (e.g. there is an inherent 11ms display

lag). Future HMDs are touted with resolutions of 8K pixels across, but

this will require a very significant increase in graphics compute and

thus power consumption. Not only is this expensive, it could limit the

form factor of the device and thus user-acceptance of the technology.

There is very exciting work going on in display technologies at the

moment. For example, Varjo (a project partner) are building a HMD that

has a moving display insert that tracks the eye; Facebook have built a

demonstration display that uses a focal surface to generate multiple

focal depths in the same image.

In comparison, the graphics pipeline is taken mostly as a given.

Recently, the proposers, along with a small group of colleagues in the

field, have started to the challenge the status quo. We don't propose

to ditch the highly optimised compute units in graphics cards, but

rather to study frameworks within they can be exploited more readily.

We believe that by reformulating the graphics pipeline and paying

attention to the very specific needs of near-eye displays, we can

radically reduce the power required from GPUs, and thus make near-eye

display more usable.

We will focus on three connected challenges that we have labelled the

latency, redundancy and bandwidth challenges. First, we will target

extremely low latency displays. We will develop systems that achieve

>1000 fps visual output, with latency under 1ms and study how these

impact visual response. Second, we will explore stronger decoupling of

frame-based rendering from display. We note that in near-eye displays

most pixels are wasted, and thus we target novel spatial and temporal

algorithms that reduce redundancy. Third, to exploit redundancy more

generally, we need to use it to reduce bandwidth between graphics card

and display. Taking inspiration from the concept of surface light

fields, our concept of ambient fields will render to buffers that are

expected to be valid for re-rendering to the user for 10-100ms.

In summary, we believe that the current graphics pipeline and its

associated implementation in GPUs is unsuited to drive near-eye

displays. We want near-eye displays to be low-cost, power efficient

and highly acceptable to users. To achieve this, we propose new

algorithms that can use GPU capabilities more effectively. We target

reducing redundant compute to enable lower latency and lower bandwidth

requirements through the graphics pipeline.
Key Findings
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Potential use in non-academic contexts
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Description This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Date Materialised
Sectors submitted by the Researcher
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Project URL:  
Further Information:  
Organisation Website: