As our software systems grow in size and complexity, increasingly diverse users
have different wants and needs from their languages: the right language for a
statistician (e.g. R) is different from that of someone who formally verifies
safety properties (e.g. OCaml), which is different again from someone creating
user-facing apps (e.g. Javascript). However, different languages inhabit
different silos and interactions between them are crude and slow. Language
composition has long been touted as the solution to this problem, allowing
languages to be used together in a fine-grained way, but has traditionally
struggled to match this promise. In the Lecture Fellowship, my team and I showed
that large, messy, real-world languages can be composed together, even allowing
different languages to be intermingled within a single line of code. We were
able to make the performance of such multi-lingual programs close to their
mono-language constituents, showing that language composition's promise is real.
However, in the course of this research, an unexpected problem became apparent:
Virtual Machines (VMs), the systems used to make many languages run fast (and
which are crucial to the good performance of language composition), do not
perform as expected. In the largest VM experiment to date, we showed
that VMs perform incorrectly in around 60% of cases. Attempts to fix existing
VMs have largely failed, because the problems are so deeply embedded that they
cannot be teased out, even after careful examination. This is a significant
problem for language composition, for which VMs are a foundational pillar.
This Fellowship Extension thus aims to show that VMs can have good, predictable
performance and that they are a suitable foundational pillar for language
composition. However, we cannot expect to create a traditional VM, which often
consume tens, hundreds, or thousands of person years of effort. Instead, my team
and I will create a new meta-tracing VM system, since history shows that these
can be created in a small number of person years. Fortunately for us,
meta-tracing has also been shown as the fastest way to run multi-lingual
programs, so it is a natural fit. We will rigorously benchmark the new
meta-tracing system we create from the beginning of, and throughout, its
development. This will enable us to observe performance regressions soon after
they occur, allowing us to fix them quickly.
We will also take the opportunity to address one of meta-tracing's biggest
weaknesses: its slow warmup, that is the time between a program starting, and
JIT compilation completing. Tracing currently involves a software interpreter
interpreting a software interpreter, with a 100-200x overhead when a loop is
traced. We will use the Processor Trace (PT) feature found in recent x86 chips
to move the software part of meta-tracing into hardware, giving a roughly 100x
speed-up to this critical phase of the system. That will also allow us to be
more aggressive in optimising other parts of the tracer that currently cause
poor warm-up.
At the end of this Fellowship Extension, alongside traditional research papers,
we will produce an open-source release of our new meta-tracing system. This will
allow others to build on our work, be that for language composition, or simply
to make individual languages run fast.
|