We Got Served

Whilst people are waiting to hear whether they’ve got into the Windows Home Server Beta, (from what I gather, invites will be sent out in the next week or so) I thought I’d write this morning about writing bugs. Beta Testing is great fun - you get to play with new software before anyone else, you have the challenge of breaking it, and then all the joys of trying to fix it again - figuring out what went wrong, and what you need to do to get back on the right rails again. So, you’ll enjoy yourself, learn lots, have the opportunity to discuss things with both MS and your fellow testers, but best of all, have a direct impact on the quality (and maybe the featureset) of the final release.

There is however, just one output you need to focus on - bugs! That’s what we’re interested in here. I was fortunate enough to travel to the MS Campus in Redmond last October for a two-day Vista Technical Beta workshop, and met some of the team running the (massive) Vista beta. Two things struck me from a conversation with a guy called Sam White, who’s worked on most of the Windows betas - firstly, the lack of participation from beta testers. Only 5% or so of people who get into a beta ever send in a bug. Secondly, many of the bugs that are sent in, aren’t well written which makes it very difficult for the team to try to reproduce the bug at their end, confirm it, and send it off to the developers for fixing.

So, I thought I’d put together this little guide to writing AMAZING BUGS - this isn’t specific to Windows Home Server, it applies to most beta programmes. I should, I guess, explain my qualifications for doing this - well, I’ve been beta testing various pieces of Microsoft software since 2004 - mainly Windows Media Center, but also Windows Live releases and most recently, Windows Vista. I also topped this list of bug submissions for Vista, out of about 25,000 people, so I know my way around a bug form!

1. Only Apply If You Have the Time

There’s a few different kinds of beta programme. Managed Betas, and Public Betas - managed betas are predominately by invitation only, and public betas (often called things like Community Preview Programmes). Sometimes, a beta programme will include both elements - let’s take the Windows Vista programme as an example. There was an “invite-only” managed beta programme which lasted from Beta 1 (July 2005) all the way through to release on November 2006. But the team then also ran an open community preview programme from the Beta 2 stage where the general public could download Vista, try it out, and send in bugs if they felt so inclined.

There’s a reason for this difference - if you have time to really test a product, answer surveys, send in great bugs and generally Get Involved, then lots of beta teams would love to invite you to their beta programme. (There’s no coincience that doing a great job on one beta generally gets you invited into others). If you’re interested in checking out a product, having a play with it, seeing if it’s right for you etc, then the prublic preview programmes are for you. Increasingly, previews are being released at beta 2 and release candidate stages, so you still get to see the bits early, but there’s no requirement for bugs.

Beta Testing can eat up spare time, so if you have it and are serious about donating it, apply! If not, wait for the CPP.

2. Be Patient For Your First Opportunity

Every beta needs particular types of users. I’m not sure if the selection process is an art or a science (probably a science :-)) but put yourself in the beta team’s shoes - you have 20,000 people who want in, who do you select? Well, I guess I’d look for a representative sample of testers that would be similar to the final end user, in terms of where they live, what equipment they use, what language they speak, their technical capability, that kind of thing. That’s why you have to fill in all that stuff when applying for betas. If you don’t fit what they’re looking for this time around, no problem, it’s not personal - keep plugging away, and you’ll get in to a beta programme before long.

I was lucky with my first beta - I spotted an article on Neowin (yes, I do check it out occasionally) that MS were looking for Media Center testers. At the time, Media Center hasn’t been released in the UK, and they were only looking for about 50 people. I filled in the survey, told them loads about my TV (!!) and in I got.

Nowadays, Microsoft Connect exists just to communicate with potential beta testers. Make sure you’re registered there, keep your eyes and ears open, and keep applying.

That’s it for Part 1… Part 2 will cover Finding Bugs.