Gå till innehåll

I just finshed reading Rocket Surgery Made Easy. As the title so clearly says it is a do-it-yourself guide to finding and fixing usability problems.

 

The author´s name is Steve Krug and here´s what he looks like according to the book. skrug_cartoonSo next time you happen to be at a usability conference make sure you say hello...

My current personal goal is to become really good at usability testing so I try to read good books and browse the web for videos, talks and other useful stuff like templates for creating prototypes. This post will focus on my interpretation of what Steve says.

 

 

Have You Ever Met a Good Midget Comedian?

This book is just like a really good midget comedian... very short and very funny. It is full of headlines like this one together with those nerdy quotes that we all love to hate and cherish so much. The title may as well have been Usability testing made entertaining. For a long time I tried to make usability a part of the IT-projects I worked in with limited success until I decided to keep it very simple and do it myself. If there is something I have learned from 20 years in business it is that large, complicate and cumbersome stuff never ever gets done! At least not for real or for very long.

In my opinion, the essence of the book is:

      "Let´s make it so easy and still so rewarding that it´s very hard to motivate not doing it!"

Even better, the book in itself is made accessible so there is hope that even the most reluctant readers can get through it. To be really clear, this is a book on how to get you STARTED, not the comprehensive works that can be found elsewhere. The problem with most thicker books is that usually at least half of the words are totally unnecessary making the reading experience not very pleasant and then there is so much information that it is really hard to find out what is most important. So cudos for effectively distilling usability into something more devourable.

So by now you have realised that I really liked this book! I like it so much that I will most likely use it as course litterature for a one day usablity class I am planning to create. I would love to translate it to Swedish to make it even more accessible fo my fellow countrymen.

 

Rocket Surgery Cover Page
Rocket Surgery Cover Page

 

The Core Process of Usability Testing

Let us start with the goal of usability testing made easy:

Our goal is to find and fix the most serious problems in order to make products easier to use.

That´s really all there is to it. We don´t try to find all of the problems and we do not bother about statistical significance in our measurements. The point is that bad usablity shows itself readily and even if we found momore problems, there would be no time to fix them. So if we can find what is hurting the most and fix this right away, we stand a great chance to end up with a great product. This type of usability testing has one large advantage over more advanced versions - this one get´s done(hopefully)!

If you look at the tis on planning and execution it is pretty clear that this is usability the agile way - and agile is good for us, right?

When Do We Start

"Start earlier than you think makes sense" (S Krug)

Right along the lines with Lean UX we start testing early and continue throughout the project. We have a hypothesis and we test it to see how well it works. How? Create a prototype and do a usability test. It is soooo easy to create a prototype in powerpoint or Omnigraffle that it is criminal not doing it! It saves so much time if we test before the developers have started coding. Of course we need to do some background work beforehand mapping the user experience or creating an impact map so we understand what the real problem is. This is not discussed in detail in this book but an earlier blog post. is a good start for learning more.

 

One Hour Well Spent

A test session takes only one hour, an extra half hour extension is possible but in my experience that may be too long both for booking people and for keeping focus.. Make sure that everything is well prepared. Since we have only one hour to spend, there is no time for last minute fixes like not having your test data or equipment ready. The easiest possible configuration is to have a device with your application loaded, and one test manager telling the participant what to do and then observing and taking notes. The recommendation is to have at least one observer - in the same room if you can keep them from talking during the test. I find it a bit hard to take great notes at the same time as I am leading the test.

Now here is an important point that Steve makes. Having more observers gives us many advantages:

  • people make different observations
  • it is a great learning experience for everyone involved
  • it helps getting team consensus on what to fix
  • the understanding for why usability testing is needed increases

These additional observers should be kept in another room since too many people in the same room make he subject feeling watched - which is a correct observation - and that will disturb the testing. So hook up a big screen using GoToMeeting or any other tool you have - Skype or Project Place can work although I have found Skype a bit choppy at times. Use an external microphone and extra speakers to record and replay the talk aloud and instructions. No need to record the face of the tester - audio is enough. Recors the screen if you think there is actually a chance that you will watch the re-run. It can of course be good ammunition for pointing out a problem later.

Now introduce the tester to what they will do and make sure they understand we are testing to see if OUR DESIGN HAS PROBLEMS not if they are skilled! Don´t forget this and don´t tell them more than three times in a session since this will lead them to think the opposite.

Set up of simple think aloud session
Usability testing using think aloud

Now it is great to have another test lead in the observation room to distribute the tasks in written format together wih a place where the observers can write what they see and sum it up in their top three problems.

Post Test Session Processing

Meet with all of the observers. Collect all top three observations. I like to do this on post-its. Do a collective sorting and prioritising. And voila, you have a list of most important things to fix and some other stuff that you will not fix right now. It may seem a bit provocative to some - but if you have not observed - you don´t get to vote. Cause we are not voting on what you think is bad - we make the prioritisation on what you actually observed! This will hopefully make more people wanting to observe. Time well spent I think!

So that´s it. No big report that noone will read, no huge list of to do that we have no chance getting done. Pretty agile, huh!

 

Final Words

Last project I worked in failed miserably at the usabilty testing part. Lack of understanding and knowledge was the biggest culprit. If was to take the advice from Steve I would have run tests from early on although the web interface looked crappy for a very long time. I would have created a quick prototype on my own to start with just to have something that enabled me to get feedback. I would also have used my colleagues and friends as testers since many usability problems have to do with general things like navigation and information that also non-business specialist will run into. I would have recorded sessions and shown them to the members of the project.

I was so pleased after reading this book that I bought and read the updated version of Don't make me think. This is a good read as well but more about that later...

 

 

Scrum does not work. I agree that thins are not working but are we really doing Scrum?

- That Scrum Stuff Does not Work!

… my highly regarded developer colleague exclaimed a while ago. My reptile response was that Scrum Does work – if you do it right. Then I thought some more and realised from my 20 years of working in IT that “People working together” is what really matters”. Regardless of what method was the latest fad and eagerly promoted by people on the front line – sometimes myself on the barricades – when I got in a project with the right people, things got done. I will not deny that a lot of the agile thoughts resonate well with my own experience. I feel an urgent need to share some stuff with you, here it is!

People working together

If we get people really working together problems are more easily solved. Whenever I have had the chance to work in a team that continuously discuss and solve problems together, we have performed well. Needless to say you need skills in order to perform, but skills combined with an arrogant know-it-all-myself attitude seldom (never) leads to success. Actually the opposite is often true. Pebbles in the machinery can be handled but if you get a big rock in the wrong place it can drag the whole team to underperform. I have seen it in different roles from developers or testers claiming to have THE only right answer, requirements analysts that are much reluctant to change THEIR wonderful document or even discuss what you find unclear and project management the old style that kills argumentation with a distinct – Now, this is the way we are going to do it!

My favourite quote is “Tobbe, if you do not do what I tell you to do - you are not a team player!” I mistakenly had though my role as test lead was to question things and to do my best and if I found something that looked really wrong I should act on that. But that turned out to be valid only as long as I did not question the project manager. As you may understand, authoritative leaders, especially those that try to tell me in detail how I should work claiming seniority since they are much older, more experienced, have a degree or certification in something and also once upon a time long ago did some sloppy work - then known as testing – have a really hard time working with me. I do not claim to know everything and I am not trying to make you look bad but I DO want the team to deliver and to do my best. If I know something and I would like to discuss this with you and if you then put me down I come back as a rubber ball on steroids.

As tester and analyst my job description includes questioning what is right. In a healthy project the team wants to perform well. Questions, if they are posed in a respectful way, are readily discussed and agreed on. If you see signs of people more wanting to protect than to discuss their creations or their leadership you are in for some serious trouble.

The same goes for vehemently arguing over the exact process rather than the best way to solve a problem. It is probably a good idea to first adopt then adapt Scrum as a process instead of trying to make a lot of changes without trying the original thing first. A lot of bright and experienced people put a lot of effort in creating Scrum so what makes you think you know better? Read about Johanna Rothman ranting about that.

Embracing change

Not too many years ago a fellow tester exclaimed “If we can just get complete and correct requirements from the beginning, our testing work would be so much easier” Then we had this long argument where I claimed that that is a totally unrealistic expectation since it is built on the following premises:

  1. All client stakeholders have managed to internally understand and  agree from start exactly what they want
  2. The customer or a requirement analyst are able to formulate this complete and correct view in a way that leaves no room for interpretation
  3. It is technically possible to fulfill the requirements
  4. During the project the customer never wants to make any changes based on new needs, cost estimation of what we are supposed to build or any other inpput

Well since many of us have realised that all of these premises are unrealistic and are regularly violated in each and every project we work in we prefer to leave the requirements open for discussion.

Certainly there have been projects where the requirements were more known from the start of the project than others. We did build things before agile in a more waterfall like manner with a one document requirement spec and did succeed pretty well in some cases. A recent project I was in tried to do it as a mix of waterfall and agile – all of the requirements were written down as a start but since we were going to work in Scrum the readable document was split up in user stories. Then we all conveniently forgot/ignored the very important point regarding a user story – that the team has to have a conversation with the customer about what it really means and agree on acceptance criteria. Instead the requirement analyst wrote the detailed requirements in acceptance criteria format from the start. That was a great idea since he then could move to another project and only in rare cases be bothered with any clarification work. Needless to say, we all hated the requirements and the developers did not even bother to read them! It is a far too common misunderstanding that we are using a method just because we are using a template connected to that method. A good place to start is this article regarding the different approaches to requirements. http://www.scrumalliance.org/community/articles/2010/april/new-to-user-stories.

Working Closely with Your Customer

Another important guideline for system development is that we have close collaboration with the customer throughout the whole project. Scrum recommends small collocated teams because we need to discuss in order to understand each other. I do not think that this is something that came with agile rather that has been common knowledge for a long time. Unfortunately common knowledge seems to be rare at times.

In my experience actually sitting together with the customer or user is often impossible due to their constant lack of available time. That does not mean that we cannot still work together on a weekly basis. The most successful projects I have been on have all had very engaged customers and requirements analysts. We have met the customer every week to clarify problems, if the customer has not been able to visit us – we have sent somebody to meet in a place at their convenience. The most successful meetings have been where the user/customer meets the whole team. That is usually at the demo and the sprint planning meeting.

Task flow, prototypes and Teamwork

A problem we have had with demos is that they sometimes become like a scripted test were a few selected flows are shown and all open questions and remaining bugs are avoided. To remedy this we have successfully done two things.

  • Acceptance test environment where customer is encouraged to play around on their own
  • Run usability tests after each new delivery

These two fairly simple things we have found give a lot more feedback than the demo and are good value for the time spent.

To any product owners or users that decide they do not have time to participate in the process I say that is a really bad idea and will cost you a LOT more in the long run. You complain that IT drives the project and engineers that do not understand the true user do the design – well start communicating with us and help changing that!

Make Sure you Focus on the User Experience

I have heard the argument that when releasing often usability is a non-issue and I must disagree strongly. There also seem to be a misunderstanding that Anyone can do UX and I disagree again. Just because something does not require in depth technical knowledge doesn’t automatically make it easy. I have heard the same argument about testers, requirement analysts and project managers – if it does not require an advanced tool anyone can do it Well there is a big difference between doing something and performing something really well. Yes I do have a camera and some photos turn out really well but most of them are not appreciated outside the immediate family. Finding out what a user really needs is a laborious task that requires active observation, good interviewing technique as well as an analytical mind and the ability to be a link between users, business and IT.

There is a very real need for UX and development to agree on a suitable level of what should be ready before coding starts. The agile idea is to avoid BDUF, Big Design Up Front and the UX idea is to first finding out what users real problems are before starting to solve them. This includes some early prototyping which is done in the discovery phase of Scrum. UX then need to be a part of the team building the solution since everything will not be set from the first day of coding. A UX designer can be very helpful in answering questions regarding proposed solutions and someone will have to lead the usability testing effort.

GoodDesign

In a recent project the choice was to completely ignore any usability questions and to instead focus on functionality. I had to argue for making fields large enough to show all the data, you have to think I am joking, right? The whole idea of separating functionality – like selecting the right field in the database – from usability - actually being able to see the data and understand what it means – must be flawed!

Most developers I have worked with are neither skilled nor very interested in usability or UX questions. If you are a developer and this does not apply to you, great! For the rest, at least recognise that the team needs to have the competence of building great wireframes, interaction design and usability testing. Things that have made our applications look and feel professional and usable are:

  1. Making sure that the business analysis is done properly. Experience mapping used in UX works great. This effort in the discovery phase is the back bone for doing the right thing.
  2. Professional help with wireframes from someone that can connect the UX with the solution, not only looking good
  3. A specialist building at least the first version of the CSS
  4. Running usability tests on a regular basis

I recommend reading the principles behind agile - not only the manifesto. It starts with:

Agile principles
First agile principles

User Experience and a good collaborative team is a great start if we want to honor the principle above.

So does Scrum Work or Not?

Just because you have a Scrum board does not mean you are doing Scrum! My advice would be that you first try doing it the proper way - then you can adapt it. To my developer colleague I simply stated " You cannot say Scrum does not work because we are not doing Scrum"  

It is a simple yet powerful framework that I like to have as a tool in my own toolbox. Combine it with other good practices like Experience Mapping, Continuous Integration and Exploratory Testing.

 

1

Mapping the user experience - How to find out what the real problem is before we start solving it.

Working in a project where the users are ignored since “We have to focus on functionality first!” leaves me exhausted. The constant flow of cartoons and jokes about developers not really caring about the real needs of users seem to go on. And just to clarify, with developers I include project managers, testers and anyone in the process of producing software. We laugh at the jokes and go back to repeat them in our own way, often thinking that usability is some other person’s responsibility.

TypDevProcess

To be perfectly honest our craft still seem to be very immature in the art of satisfying the users for what we build. So how can we change this? I for one have made a vow to myself that I will do all I can to help building systems that the users really like to use. I truly believe that change comes from people putting their foot down saying, “No more of this!”.

Moment of Truth

Try this though experiment with me. A lot of people get really sick from using badly designed computer systems. There is a lot of statistics on this. Swedish readers can have a read at Jonas Söderströms blog - the book is an eye opener! English speaking readers can read the slides.  So if we accept the fact that the use of IT systems are making people sick, think about who built the systems. That would be us – the developers, testers, requirement analysts, scrum masters etc. So indirectly we are hurting people by building systems that are hard to use. Now take a moment to reflect on your personal code of ethics. Can you honestly say that you are ok with continuing building bad solutions knowing what the effects are?

Mapping the User Experience

The UX community have some really great ideas on finding out what the real problem is, before we start solving it? Sounds like a fairly reasonable idea to me.

Collecting Data

The Lean movement have a core practice – the cool English-Japaneese term is Go to Gemba.The only way of really understanding what people are doing is observing them doing it, preferably in their natural setting. The same idea goes for mapping a user experience. Observe users as much as you can, ask them questions about what they are trying to do and what problems they are facing. Try not to impose too much since that will disturb the users and may change their behaviour. I find it good to go out and observe in pairs since we observe different things. We never do any video recordings since that disturb users and we never seem to have time to watch the recordings anyway. It can be exhausting to be an active observer. Try to focus on what you see and hear and don’t try to find solutions at this point. Make sure to write down quotes that really make a point like : “ I wish I did not have to enter that data once more”.

It may not always be possible to observe all the users you want but hopefully you can at least meet them for interviews. To be honest, in the projects I have participating in we have mostly worked with interviews and have made only a few visits to observe users working. Done the right way interviews can be very revealing. We still avoid video recording and instead try to be active listeners and ask clarifying questions. Good questions you may ask are open-ended like:

-“Tell me more about what you just did.”

-“What information would be good to have when performing this task?”

You can spend a lot of time collecting data and there are as usual no rules for how much is enough. I have mostly built systems intended for specific professional users which means that the number of user groups is fairly limited. It is a good idea to interview at least three people in each user group in order to get valid personas. For larger systems you may want to interview many more than this but we need to start somewhere.

Analysing Data

Now is the time to start analysing all the data from your observations. A great way of doing this is to arrange a workshop where all the people that interviewed or observed participate. Condense findings by writing the main points on yellow post-its and putting them up on a wall. It is I good idea to put some butcher paper or some taped flip chart papers on the wall to be able to move the information when needed. Group the post-its together for the specific actions users try to do.

Now create a chronological flow of tasks on blue notes where one task may consist of several actions on green notes. I does not have to be perfect as flows in reality seldom are. You now end up with something looking like this:

 

ExMap
Sorted observations of user experiences

Note that what is on the wall now is the group’s collective observations of real user’s actions grouped together. We still have not solved any problems but we ought to have a pretty good picture of what the real problem is. We should all literally be on the same page.

I attended a great one day tutorial by Chris Risdon where our sorted tasks looked like this.

20140402_143724
Tasks for booking a hotel

 

Creating the Experience Map

There are several models for creating a more formal map.

Here is a task map based on the free downloadable templates at cxpartners. The templates accompany the book Communicating the User Experience which is a great start if you want to dig deeper into this matter after reading this post. The flow describes how to order a specific medical drug and is loosely based on a recent project.

TaskMapLic
Task Map for Drug Application

 

Some more examples can be found in Chris Risdon's slides. Read more about customer journeys and mood graphs.

Our hand drawn graph at the course looked like this

20140402_143740

Show  

Since I strongly believe feedback is a key factor to any successful project this is a good time to show the graph to important stakeholders to make sure we all agree. If we find that we still have outstanding questions we may need to conduct some more in depth interviews.

Conclusion

This only a very brief introduction to the starting point of UX. The technicques are not the hard part - the crux is to become a great observer and interviewer. The ideas are not new - business requirements and processes have been a part of many development models. The problem is that thay have to often been ignored. UX is changing all that by really putting focus where it belongs - solving real problems for real users.

If you have read earlier blog posts you will recognise the four step flow of visual problem solving.

Visual Problem Solving
Visual Problem Solving

 

 

 

 

 

 

 

 

 

 

 

 

 

The experience map is a great way to start a project! There is plenty of information if you want to dig deeper in Mapping the user experience

The latest stuff I read and can recommend is

Communicating the User Experience: Caddick, Cable

Smashing UX Design: Allen, Chadley

I wrote a paper some time ago on the powers of visual problem solving. This has now become an e-book thanks to the cooperation of EuroSTAR conferences. The book can be downloaded from the EuroSTAR website. I have also made it available on my own site. As usual no registration or fee is asked for.

Happy reading!

Book review of Outliers by Malcolm Gladwell. Sketchnote attached.

Just finshed reading another great book. This time the famous piece Outliers by Malcolm Gladwell.

An outlier is defined as

1. something that is situated away from or is classed differently from a main or related body

2. a statistical observation that is markedly different in value from the others of the sample

Why do some people achieve so much more than other? Malcolm has done extensive research on the subject, there are ample references in the book.

It is often said that behaind every great man there is a great woman, if we generalize it more it is true that no man/woman make it all on their own. Support from family, friends, business partners or others are nescessary in order to have the energy to keep going.

Another important factor is timing. Beeing born in january means you have an advantage when junior sports teams are picked, that advantage means you are a bit better than the people born later the same year. Given that extra advantage you get picked and thus have oppurtunity to increase that differnce since you get more practice time and better coaching. It is a self-fulfilling prophesy!

You cerainly have an advantage if yu have talent, but you only go so far using only that. Talent helps you getting selected but then there are others that are equally talented. If you have a really low IQ yuo do have a disadvantage, however if you have enough brains to score above 115 it stops making that much of a difference. You just have to be good enough!

High IQ will not help you if you totally lack the social skills to communicate with the rest of the world. There are lots of tales told about really smart (high IQ) people that do not make it!  Growing up in a supportive environment, given the chance to develop and beeing encouraged to trying things will give you a significant advantage.

Outliers Sketchnote
Outliers Sketchnote

 

But I am a self-made man. Sorry but that seems to be only one part of the equation. If you have some luck beeing born at the right time in the right family and several things coincide to give you an opportunity and then you DO take hat opportunity and work hard - THEN you are destined to make it big. Because without hard work, I mean really hard, you are destined to become just average.

Hard work is however a crucial part of becoming an outlier. Far from all hard working people succeed but it would be pretty safe to say that those that DO succeed all have worked really hard. If you put 10 000 hours into what you do - you will eventually become really good at what you do!

Actually this last part is where I lack some additional research. What about the quality of those 10 000 hours? The importance of getting instructions and feedback? Well, maybe that is the content of future book.

In all, this is a joy to read, could hardly put the book down until it was finished. it seem to be thoroughly researched and I DO like it when somebody questions what people think is common knowledge when there turns out to be a different explanation. I am more inspired than ever tu burn down my 10 000 hours in visual problem solving! Waiting for David and Goliat to arrive in my mailbox shortly.