| EPSRC Reference: |
GR/M48123/01 |
| Title: |
ANALYSIS OF NATURAL GRADIENT LEARNING FOR STATISTICAL MODELS |
| Principal Investigator: |
Rattray, Professor M |
| Other Investigators: |
|
| Researcher Co-Investigators: |
|
| Project Partners: |
|
| Department: |
Computer Science |
| Organisation: |
Victoria University of Manchester, The |
| Scheme: |
Standard Research (Pre-FEC) |
| Starts: |
01 May 1999 |
Ends: |
31 October 2002 |
Value (£): |
55,227
|
| EPSRC Research Topic Classifications: |
|
| EPSRC Industrial Sector Classifications: |
| No relevance to Underpinning Sectors |
|
|
| Related Grants: |
|
| Panel History: |
|
|
Summary on Grant Application Form |
|
In recent years there has been an explosion in the use of generalised statistical models; currently popular examples include multilayer perceptrons (MLPs) and independent component analysis (ICA). These methods have been applied to many problems of great practical importance and their popularity can in part be attributed to the development of practical optimization algorithms. On-line optimization is a particularly popular and efficient scheme wherby the model parameters are iteratively adapted in response to a stream of training examples. Natural gradient descent (NGD) was recently proposed as a more principled alternative to standard on-line methods. This algorithm respects the Riemannian geometry inherent in the space of statistical models and ensures optimal asymtotic regime is not currently in place. We will use methods from non-equilibrium statistical mechanics to solve the dynamics of NGD for two popular statistical models (MLPs and ICA) in order to provide better theoretical understanding of this algorithm. We will determine general conditions for optimal and robust performance, evaluate NGD compared to other algorithms and develop/evaluate principled variants and approximating algorithms.
|
|
Key Findings |
|
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
|
|
Potential use in non-academic contexts |
|
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
|
|
Impacts |
|
| Description |
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk |
| Summary |
|
| Date Materialised |
|
|
|
|
Sectors submitted by the Researcher |
|
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
|
| Project URL: |
|
| Further Information: |
|
| Organisation Website: |
|