Details of Grant 

EPSRC Reference:	EP/X029336/1
Title:	Cerebellum-inspired parallel deep learning
Principal Investigator:	Costa, Dr R
Other Investigators:
Researcher Co-Investigators:
Project Partners:	Graphcore
Department:	Physiology Anatomy and Genetics
Organisation:	University of Oxford
Scheme:	New Investigator Award
Starts:	01 March 2024	Ends:	28 February 2027	Value (£):	412,790
EPSRC Research Topic Classifications:	Artificial Intelligence
EPSRC Industrial Sector Classifications:	Information Technologies
Related Grants:
Panel History:	Panel Date Panel Name Outcome 03 Jul 2023 EPSRC ICT Prioritisation Panel July 2023 Announced
Summary on Grant Application Form
Deep learning, a type of machine learning, has recently undergone dramatic developments. It is already having an impact across society, from scientific discovery to climate change prediction. However, current deep learning models require billions of parameters which can take several weeks to train, costing millions of pounds with large carbon footprints. It is therefore becoming increasingly important to develop efficient training methods for deep neural networks. One of the key bottlenecks that underlie the inefficiency of deep neural networks is the need to perform a large number of computational steps sequentially. Here, inspired by recent findings on how biological neural networks learn, we propose to develop an efficient training algorithm for deep neural networks. We have recently proposed that a specialised brain region, the cerebellum, enables parallel learning in the brain. The cerebellum is defined by two features that are well placed to facilitate parallel learning: sparsity and modularity. First, the cerebellum contains highly sparse connectivity with only four input connections per neuron, which should result in faster learning. Second, the cerebellum is a highly modular system, which is well placed to enable parallel learning. Inspired by these cerebellar features we will develop a sparse-modular system for training deep learning networks capable of efficient parallel training. In collaboration with industry, the benefits of this approach will be demonstrated using a new type of parallel processor designed to accelerate machine learning. Overall, our work will lead to a novel approach to parallel deep learning, leading to a substantial reduction in training times and costs.
Key Findings
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Potential use in non-academic contexts
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Impacts
Description This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk Summary Date Materialised
Sectors submitted by the Researcher
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Project URL:
Further Information:
Organisation Website:	http://www.ox.ac.uk