Thoughts on hosting a good UDS session

The inaugural online UDS (or vUDS as it’s becoming known) is underway. This brings with it a number of new challenges in terms of running a good session. Having sat in on a few sessions yesterday and been the session lead for several sessions at physical UDS’s going back nearly two years now, I thought I’d jot down a few tips on how to run a good session.

Prepare

Regardless of whether the session is physical or virtual, it’s always important to prepare. The purpose of a UDS session is to get feedback on some proposed plan of work (even if it is extremely nebulous at the time of the session). Past experience shows that sessions are always more productive where most of the plan is already fleshed out prior to the session and the session basically functions as a review/comments meeting. This depends on your particular case though, since the thing you are planning may not be possible to flesh out in a lot of detail without feedback. I personally find this is rarely the case though.

Be punctual

UDS runs on a tight schedule, at least in the physical version, although I don’t see any good reason why this should change for vUDS. Punctuality is therefore important not as good manners but from a practical point of view. You need to time to compose yourself, find notes and make sure everything is set up. For a physical UDS this would have been to check microphones are working and projectors are projecting. For a vUDS, in my brief experience, this means making sure everyone who needs to be is invited into the hangout and that the etherpad is up and the video feed is working on the session page.

Delegate

As the session lead it is your responsibility to run a good session, however it will be impossible for you to perform all the tasks required to achieve this on your own. Someone needs to be making notes on the Etherpad and someone needs to be monitoring IRC. You should also be looking out for questions yourself but since you may be concentrating on conveying information and answering other questions, you do need help with this.

Avoid going off track

Time is limited in a UDS session and you may have a lot of points to get through. Be wary of getting distracted from the point of the session and discussing things that may not be entirely relevant. Don’t be afraid to cut people short – if the question is important to them then you can follow up offline later.

Manage threads of communication

This one is quite vUDS specific, but especially now that audiovisual participation is limited, it is important that all of the conversation take place in one spot. Particularly for people who are catching up with the video streams later on. Don’t allow a parallel conversation to develop on IRC if possible. If someone asks a question in IRC, repeat it to the video audience and answer it in the hangout, not on IRC. If someone is talking a lot in IRC and even answering questions, do invite them into the hangout so that what they’re saying can be recorded. It may not be possible to avoid this entirely, but as session lead you need to do your best to mitigate it.

Follow up

Not so much a tip for running a good session, but for getting the best from a good session. Remember to read the notes from the session and rewatch the video so that you can use the feedback to adapt your plan and find places to follow up.

That’s all there is to say, I really hope this first virtual UDS goes very well and that sessions are productive for everyone involved.

Advertisement

Sorting lists of objects in Jinja

Part of my duties as a Hardware Certification Engineer is to develop and maintain our test and bug reports. Previously these have been Python scripts containing lots of inline HTML (yuck :/), which I’m ashamed to admit it because I didn’t know any better when I initially wrote them. I’m currently in the middle of refactoring (hopefully not ref**ktoring) the scripts to take advantage of a new API that’s available for querying data from the Certification website and have decided to use Jinja (precisely Jinja2) as the template engine.

So I have a list of systems which may be in different locations and I want to sort them by location. Initially I did this in the Python code which gathered the data, using a simple .sort(key = lambda obj: obj.datacentre). Then the template code used the sorted list and all was well. Later on when reading the Jinja template documentation I discovered a way to sort a list in the template itself. I considered this to be a cleaner, more transparent way to modify the list (somebody looking at the template asking ‘why does it come out in this order’ has the answer right in front of them). So I gave it a try by changing

for hw in {%reported_hardware %}

to

{% for hw in reported_hardware|sort(attribute='datacentre') %}

and was unfortunately confronted with a traceback from the template engine:

Traceback (most recent call last):
...
AttributeError: Hardware instance has no attribute '__getitem__'

This looks like it’s saying that the ‘Hardware’ object (which I’ll introduce in a second) isn’t the type that the template engine was expecting.

Initially I defined the Hardware object like this:

class Hardware:

def __init__(self..., datacentre):
...
self.datacentre = datacentre

My first attempt to fix this involved making datacentre a property:

@property

def datacentre(self):

return self._datacentre


But this didn’t seem to do the trick. Eventually I read in the Python documentation about the property built in working only on new-style classes and this set off the lightbulb

class Hardware(object):

This seemed to be the magic ingredient, and the sort in the template engine started working. I also realised that making datacentre a property wasn’t really necessary, so reverted that change.

Update: I found out that this whole problem is avoided by using Python3.x because all classes there are new-style classes. So the advice above only applies if you’re stuck with Python2.x

Network Manager applet getting you down?

I find that sometimes the Network Manager applet in Ubuntu can be a little temperamental (apologies to the maintainer, cyphermox, if he’s reading this – but such is the nature of software). Sometimes it won’t show available routers and if that’s the case then I’ve established a little workaround that I’m telling you about mainly because it involves a script I wrote that lives in a somewhat obscure place in Ubuntu.

Step one in the workaround is needed if you don’t know which networks are available in advance. If you’re sitting in your home then you’ll probably not need this step since most people know their router SSID. If you don’t then you can scan using:

nmcli dev wifi list

This is really reliable and always works if your WiFi hardware is working.

The second step is to use the SSID to create the connection using the script I wrote:

sudo /usr/share/checkbox/scripts/create_connection $SSID --security=wpa --key=$WPA_KEY

If the router doesn’t use any security (which nmcli dev wifi list will tell you) then you don’t need –security or –key. If the router doesn’t use WPA2 (maybe it uses WEP), then you’re out of luck – and deservedly so. Change the routers security settings if you can!

Filing bugs in Launchpad from the terminal (rapidement!)

Obviously being an engineer working on Ubuntu, part of my day-to-day life is working with bugs in Launchpad. Filing them, triaging them and fixing them (etc, etc). For a long time I have felt like the tools provided by Ubuntu to assist with bug filing don’t meet my needs as an engineer working on an upstream project. The well known ubuntu-bug is too heavyweight, for the following reasons:

1. It spends a lot of time gathering logs and other info which in many cases is not useful (for example when filing very simple bugs.
2. It wants you to go through the browser to fill in many details, mainly unnecessary.
3. It complains about ‘non-existant’ or ‘unofficial’ packages, which is often just obstructive.

I’m not complaining that ubuntu-bug is bad or useless by any means, it does many useful things and is ideal for the use case of a user filing bugs against packages. I don’t need the extra stuff and just want to file simple bugs against whatever project I may desire.

Wanting for such a script led me to ask on #ubuntu-devel (FreeNode IRC) if such a tool existed. The consensus from two very senior Ubuntu developers who I would trust to know was that nothing was there but it would be simple to write. I knew that, which made it all the more surprising it wasn’t yet. So a few hours later appears:

lp-file-bug
usage: A script for filing new bugs in Launchpad. [-h] [--project PROJECT]
                                                  [--package PACKAGE]
                                                  title

Most excitingly, it was suggested by Bryce Harrington, maintainer of the ‘python-launchpadlib-toolkit’ package that the script should be added to the package so I proposed a merge request which he accepted. After fixing a couple of bugs, I have got around to getting a build of python-launchpadlib-toolkit into my PPA. You may install the latest version of it for Precise using:

sudo add-apt-repository ppa:brendan-donegan/ppa; apt-get update; apt-get install python-launchpadlib-toolkit

Having done that it is very easy to use:

lp-file-bug ‘The title of the bug’ –project=myproject

An editor will be opened (either ‘nano’ the default editor in Ubuntu, or whatever you have set in $EDITOR) and you write the description there, save and close it and then you’ll be given the link to the new bug, which you may use to view it.

New Years Resolutions

I’m only just back from my holidays and it’s the time of year to be making some resolutions. We all know that these things usually go nowhere, so I don’t usually bother – but this year I’m in a resolution sort of mood. Maybe it’s because there’s another baby on the way that I’m feeling responsible right now!

My first resolution is to learn my family by marriages language, that being Urdu. Particularly because my mother-in-law doesn’t speak English (a mother in law who can’t talk to you – sounds like some peoples dream!) and learning a language will broaden my horizons, since I’ve never really learnt a (human) language properly. I think this one will be difficult, but I’ll be happy as long as I make some improvement by the year end.

The second one is in the technical field. There’s a very interesting technology for message passing called Rabbit and though not directly related to my job, it’s something that brings me back to my university days when I was very interested in all sorts of distributed systems (I did projects on distributed file systems and message passing interfaces). Of course no-one has any significant amount of control over their career direction at the early stages, so I never got a chance to indulge myself in these areas further, having been thrust into the realm of QA as a graduate. So I will take some time to get up-to-speed on using this technology and hopefully get a chance to implement something in it (time willing)

Back to the soft skills again, a great passion of mine is cooking. I didn’t start doing any cooking for myself until I finished University (most of my food preperation before then could have been better described as ‘assembly’ rather than cooking) and I’d say by now I’ve reached the level of ‘competent home cook’. I want to take that to the next level. Not yet exactly sure what that will mean 🙂 This ones going to be easy I think, because basically it’s just getting better at something I consider my hobby.

That’s it for concrete goals for now. I might pick a language or two to learn as well (my languages are becoming stale actually as I’m becoming increasingly focused on Python and Bash exclusively) just to keep myself fresh and on the soft (heh) side again there’s a general goal about fitness, but I won’t bore anyone with that.

If I can rememember to then I’ll post again next year on where I got – wish me luck!

Picking Cherries

Today I needed to use Bazaar to merge the changes from just one file in the project that I’m working on to an older release. Trying to search Google for how to do this, the information wasn’t very clear. I eventually figured it out so I thought I’d share the technique here. Sometimes it’s called ‘cherrypicking’ (thus the title).

First branch the branch you want merge the changes into:

bzr branch lp:checkbox-certification/0.9

then branch the branch which contains the changes:

bzr branch lp:~brendan-donegan/checkbox-certification/wireless_scanning_sru

change into the root of the branch you want to merge into and specify the bzr merge command giving the path to the file you want to ‘cherrypick’ the changes from:

cd 0.9; bzr merge ../wireless_scanning_sru/jobs/sru_suite.txt

This will merge all of the changes from the file wireless_scanning_sru/jobs/sru_suite.txt to 0.9/jobs/sru_suite.txt

Preparing for UDS P

With the release of Oneiric Ocelot just around the corner and the archives firmly in freeze mode, my main focus has turned to preparing topics for UDS P which is taking place in Orlando at the end of this month. As you might know by reading my blog one of my main roles within the team is co-ordinating testing of SRU kernels by Hardware Certification. The next cycle of development is going to take on a strong flavor of SRU testing. Personally I’ll be hosting two sessions at UDS,

The first one is titled ‘Improving automated certification testing of Kernel SRUs‘ and is based around increasing the overall scope and coverage of the test suite used when testing the SRU kernels. Recently I took the time to document the test suite we use during SRU testing, and if you read through it you can see that it’s really quite basic and hasn’t been especially good at picking up regressions so far. I’m quite excited at the prospect of doing this and my definition of success here will be a test suite that starts detecting real problems early. Linked at the bottom of the blueprint are some notes that were brainstormed together on the #ubuntu-kernel channel on FreeNode last week which will form a foundation for the discussion. If you’re interested in ensuring that kernel updates don’t break your system and will be at UDS P then feel free to subscribe to the blueprint (of course you’re free to send your feedback via this blog as well).

The second topic is titled ‘Image creation and publishing for kernel SRU testing‘ and has a less broadly interesting premise but will be important for us nonetheless. At the moment we use quite a complicated lab infrastructure to install all the necessary pieces for SRU testing over the network and it prevents us from easily allowing external parties to perform the same testing themselves. If we can easily automated the creation of images which include everything required for testing then we can get rid of this barrier. If the subject matter interests you then, again, either subscribe or leave feedback here.

From manual to automatic

Every well seasoned tester knows the advantages and disadvantages that manual/semi-manual and automatic tests have when compared to each other. A manual test is easy to create, just a few simple words and you have your test. Automatic tests allow you to (almost) fire and forget about them with your only concern being the PASS/FAIL at the end. A semi-manual test is a funny hybrid of the two, usually only used in a situation where a fully automated test is almost physically impossible (e.g. verifying screenshots and tests involving peripherals). Manual tests are not good in situations where the same test must be run many times across a large number of configurations. This is exactly what we have in hardware certification, where we must run tests across ~100 systems on a very regular basis. To this end we’ve been taking the opportunity this development cycle to update some of our older tests to be more automated.

One of the tests that I updated was one which would cycle through available resolutions on the system (using the xrandr tool) and request the tester to verify that they all looked okay with no graphical corruption. This is the sort of test that is fine when someone is running the tests on a one-off basis, it’s not so good when one tester needs to supervise 50+ systems during a certification run. One of the main problems is that it causes too much context switching, with the tester constantly needing to keep an eye on all the systems to see if they’ve reached this test yet. Obviously, it being a graphical test, it’s difficult to do fully automated verification so a compromise needed to be reached. The solution I came up with was to integrate screen capture into the test and then upload these screens in a tgz file as an attachment with the test submission. Everything going well, the tester can sit down at their own computer and go through the screens and confirm they’re okay. In fact the person verifying the screens doesn’t even need to be in the lab! The task can be distributed amongst any number of people, anywhere in the world.

Another test that looked like a prime candidate for automation was one for testing the functioning of the wireless card before and after suspending the system. Previously the test case was:

– Disconnect the wireless interface.
– Reconnect and ensure you’re online.
– Suspend the system.
– Repeat the first two steps

This was all specified to be done manually. I am currently updating this test to use nmcli to make sure a connection can be made, then disconnect and reconnect just as would happen if the tester did the steps manually using nm-applet. The one thing I haven’t got down pat yet is connecting to a wireless network where a connection didn’t exist before. This step may be optional as it could be expected that the tester will do this manually at some point during the setup of the tests and we can trust a connection to be available already. This will mean this test has gone from manual to fully automated and hopefully should shave potentially some significant number of minutes off the whole test run!

Saving time on our existing tests will allow us to introduce new tests where appropriate, so we’re able to provide even more thorough certification testing.

Ubuntu Bug Control Membership

I attended a company rally in Dublin last week so didn’t get around to blogging, but having received some good news just yesterday I though I’d make a posting about it.

After the Ubuntu Developer Summit in Budapest one of the objectives I set for myself this year was to become more active in the Ubuntu community. Being a QA person, one of the most obvious routes was through the well structured bug handling initiatives that the community has in place (thus my posting about the Bug Squad and bug days). To this end I’ve devoted what spare time I have to these initiatives. Apart from Bug Days I also take part in the 5-a-day program which encourages participants to update five bugs a day (by commenting, changing status or updating titles – anything that could be construed as a valuable change to a bug). This is going pretty well and I’m on a 3-week streak of fulfilling my 5-a-day quota (according to the 5-a-day report).

As a result of all this activity I’d gathered enough experience (and evidence of that experience) to file an application for the ubuntu-bug-control team in Launchpad, who have permission to change bug Importance and set the status to ‘Triaged’. This involved testifying to having read some documentation and then providing some example bugs to demonstrate that I understood the bug triage process. It took a week and a half to get enough positive responses (two) to have my application accepted, but as of yesterday I am officially a member of ubuntu-bug-control!

I’ll do my best to use these powers effectively and diligently and hopefully make a big difference to the effectiveness of bug triage in Ubuntu.

My favourite aliases…

Something I recently (embarrassingly) discovered is that bash supports the concept of aliases, which are like shorthand for commonly used commands. Ubuntu comes with a few as default already in your .bashrc, e.g. ‘ll’ for ‘ls -alF’ (long listing). You’re free of course to add your own in .bashrc, so here I present some of the ones I use:

alias chx='chmod +x'
alias rvim='sudo vim' (if you use VIM that is ;) )
alias sagi='sudo apt-get install -y'
alias sagr='sudo apt-get remove'
alias sagu='sudo apt-get update'
alias saar='sudo add-apt-repository'

I find that especially the apt ones save a lot of typing. Hope you find them useful!

(oh yeah, just put the lines in your ~/.bashrc and run ‘source ~/.bashrc’)