🚀 go-pugleaf

"Anne & Lynn Wheeler" <ly...@garlic.com> wrote in message
news:ubsm2wbep.fsf@earthlink.net...
> nm...@cus.cam.ac.uk (Nick Maclaren) writes:
> >
> > In actual fact, there are bugs that can creep by the 3-level test
> > for both compilers and VMs, but they are mind-bogglingly obscure
> > by comparison with the ones that adding the third level picks up.
> >
>
> for the resource manager ... I developed an automated benchmarking
> system that would generate kernels, auto-reboot, run benchmarks, build
> new kernels, run more benchmarks ... and it would run & run & run. The
> basic set was 2000 benchmarks that took three months elapsed time to
> run (before initial product release).
>
> The benchmarking attempting to choose workloads along the edge of the
> "normal" observed operational envelopes, random points withing the
> envelope and some number of operations that were way outside any
> normal envelope ... say saturating a couple fixed-head paging devices
> (hundreds of pages/sec), with huge queues and average service time of
> 1second.
>
> These outside the envelope benchmarks tended to precipitate zombie
> user conditions and numerous timing failures. In order to eliminate
> all such cases (from the normal system), I had to totally rewrite the
> system serialization primitives in order to eliminate all possible
> occurances of zombies & kernel failures because of various asyncronous
> timing problems.



This seems to be a good answer to the question someone raised a few weeks
ago about how do you test an interactive, time-sharing operating system.  It
also squares with my experience (so it must be right!) that stressing the
memory allocation and swapping/paging systems brings out all sorts of
othrwise hard to find timing related bugs.


--
    -  Stephen Fuld