Stable Release UpdatesMay 18, 2011
One of the things I do on a regular basis is co-ordinating and executing the testing for the ‘Stable Release Updates’ which Canonical issues every few weeks. These contain important fixes to the kernel which we want to get out to users as soon as possible, but with these fixes come the possibility of regressions. We take it upon ourselves to run a small suite of tests, tightly focused on essential hardware such as graphics, memory, networking etc. across every system which we have certified for that release. This amounts to nearly two hundred systems when combining the systems we have for all the releases which need to be tested (at the moment it’s two, but this could be up to three). So the effort looks quite large, but luckily we have established a good process for it where I take care of servers which can be tested totally remotely and my colleague in Montreal takes care of laptops (which need a small amount of manual intervention). I also test systems which are located in the US with help from another colleague of mine who looks after the lab there. Between us we do a pretty efficient job of it.
What comes out of this process are reports (such as http://people.canonical.com/~hwcert/sru-testing/wk14_2011/lucid-proposed.html) which tell us how each system did with the testing. Failures are analysed and we check if they constitute a regression in the platform. Up until now we’ve been quite lucky (well for a tester we call it unlucky ;) ) to not find any major issues. This probably means we need to beef up the test suite! After we’ve tested all the systems we report on the updates tracking bug to indicate our happiness with the state of the update. All being well in other places (QA do regression testing too, and the community gets a chance to test it as well) the update gets published and every Ubuntu user sees updates in Update Manager!