[CakeML-dev] relaxing regression test requirement for pull requests

Ramana Kumar Ramana.Kumar at cl.cam.ac.uk
Mon Mar 6 06:40:55 UTC 2017

Hi all,

Currently our situation is as follows:
  - a full regression test takes at least a day, maybe two
  - most of this time is spent in compiler/bootstrap

There have been plans and discussions to improve the testing
infrastructure and to improve our proofs so that they run faster, but
all that requires work.

In the meantime, I think we share the following goals:
  - the tip of master branch stays usable (and a good place to branch off)
  - people are encouraged and rewarded for making timely contributions

I worry that the requirement to submit a pull request that needs to
pass a regression test may currently be discouraging commits at all.
Especially minor changes.

I think we can achieve the above goals by relaxing it a little. So
here is my new proposal:

To commit/merge to master you need to have some evidence that what you
are committing won't break anything _except_ possibly
compiler/bootstrap. If you do break compiler/bootstrap (or anything
else), you need to fix it asap, but this can be fixed on master after
the commit.

The strongest evidence is a full regression test on a merge between
your branch/fork and current master. However, local partial builds
also count. Untested arguments for why things shouldn't break aren't
very trustworthy in practice, but they count for epsilon.

Pull requests are required for major changes. Direct commits to master
are allowed for minor changes, but any breakage must be promptly

Thoughts? Would this help things or make them worse?


More information about the Developers mailing list