Blogg

I had the pleasure to invite Ingrid Domingues for dinner this week. Ingrid is one of the authors of the book Effect Managing IT that I have read at least three times so far. There has been a lot of discussion in, at least the context driven testing community, on how to increase the value of testing.

An IT-system is built to solve a problem - if the problem is not solved the product does not work!

So we need to focus on the users/customers to ensure that they get good usage of what we build. Effect manging IT puts the focus on what effects we want to achieve and what roles of users that can achieve them. This serves as a basis for requirement and testing where we try to make sure that the most important effects are realised. We then have a pretty clear idea of what the consequences are if we remove a certain function or if we find a problem in a part of our system. Ingrid told me she is a bit tired of waiting for the business to understand the concepts off effect managing and start working with them. Instead they choose to work towards unclear goals with the result of not optimizing their earnings. Well, I have said the same thing about testing for fifteen years, how come the understanding is so slow to penetrate our industry. ISTQB certification is part of the answer. It is hard to change to the better when the old ruts are sold as the one and only solution. Most testers that I respect believe it is useless or plain wrong but a lot of managers and some novice testers think it is important. That often changes when they take a testing class like RST or my own test design class where they are shown alternatives and are encouraged to think for themselves instead of beeing dictated to. I will leave that discusson for now.

Testers should be business analysts, many have said. But how come the important part of business analysis is just thrown at the testers? Shouldn´t the whole project be customer focused? As it is now, not even the customers themselves seem to be business driven! Looking back at most of the projects I have worked on the last 15 years, the greater part of them are function driven or maybe "project driven".  What I mean with that? Well, how come all project management and system development methods focus on getting a bunch of functions completed within in a certain time limit and within budget? There is little engagement from the customer side, lately it seems to have become even less rather than more. The roles system analyst, business analyst or whatever they may be called never seem to have anyone assigned. And what about the roles interaction designer and usability architect. They are surely legends because I have heard about them but never seen them in any project so far.

The truth about IT-projects is that the only ones making money on them are whoever is developing them. Whoever pays the bill hopefully earns some money AFTER the project is put in production. So their goal, and the project's, should be to maximize the earnings after the project has ended. If that would be the focus we would not only look at effects but at all the important quality factors that are alomost always conveniently forgotten by everyone engaged in builing a system. What are the support costs? How can we update, upscale, change platform? Are the users happy, if so the usage of the system will increase hence we make more money.

It is time to stop running IT-projects and start to run business projects! All other industry does it or they go bancrupt.

We are currently evaluating tools to aid us in creating, administrating and executing tests. Development is done in Visual Studio 2008. I would really like to get some input from any testers and developers using this part. Is it good, great or do you hate it and why?

We most likely will evaluate the product this spring but it would be nice to get a head start by reading other peoples reflections beforehand.

I love test design and I think it is an essential part of good testing. I enjoy analysing complex data and functions and often create graphical models or tables of the design. But when it comes to actual test cases I start to hesitate.

What is the reason for me to create test cases? Well the answer must be that I or someone else can use them later as support for executing tests. Executing tests is often interpreted as following some detailed instructions and comparing predicted results with actual results. So what is the reality of test cases, let´s hear it! In most cases I do NOT need any test cases at all in order to do some really good testing! All of the tests I run at the moment are done without any test cases at all. I am doing a menu tour of the whole application, taking one part at a time and at the same time making notes. I am using the user guide as inspiration and am also updating that document with what I find. Every page has my comments, questions and bugs. I am setting up the basic functionality of the system and at the same time doing some software attacks. The most common are the input constraint attack, trying to make all error messages appear, OOPS - making mistakes and trying to correct them. All of these techniques help me find lots of problems, some already known but forgotten. Read How to Break Software by James Whittaker or get some info from James Bach on how to perform this kind of general attacks.

Next step for me will be to test the whle system flow. I have created a simple general process from creating a survey, asnwering, analysing and then presenting results. That is basically what the system does, helps users to collect data and get statistics from it. My plan is to create a number of test charters - high level test cases with test ideas - and collect data in excel from where I can analyse the results.

We are at the moment evaluating an administrative system for test cases, logs and bug report but so far I have only been using the bug reporting part. I find most test cases administrative systems to be really crappy and not supporting any reasonably good way of working. The reason they are built the way they are is that that was the easiest way to build them! They are all built like a file explorer with folders and subfolders and unnatural stepwise descriptions of testcases. Boring and ineffeicient.

SAY NO, when you think it is a bad idea to work a certain way or with certain systems. We testers need to lead the way. Who else is going to create a better way of working for us? And you WILL meet resistance, changing hurts. But befioe you give up, think about if you want to spend the rest of your career working like you are today. The choice should not be that hard when you look at the long term picture.

Let´s get better!