There isn’t of course a system – qual is too creative for that – too much of an art form. We seek understanding, so there are many different paths, Grasshopper. But now I can reveal some of the thought process behind choosing methods – as well as giving you a quick look at the variety available. There is a lot of talk nowadays about the ‘NewMR’ – the reality is somewhere inbetween. Yes, there are a lot of exciting, even transformative methods,but my belief is the focus group still dominates.  And of course there is no reason why you can have only one method…. if you use two or three they can cancel out each other’s drawbacks and give you a more grounded result.

I’m trying to avoid it but the video gets technical in places ( semiotics, gamification) – if you want more information search on this site for slightly longer explanations.

I was asked to be a judge for one of the prizes in the VisionsLive Bulletin Board Christmas competition. The task was to find new and exciting ways of using the board, and the judging  was hard. In the end I had to go for a winner and two ‘honourable mentions’ as there were a number of entries that made the most of the benefits of bulletin boards.

Winner: Where do babies come from – how to explain it to children

This topic illustrates many of the issues that researchers face; working with delicate or sensitive subjects, needing to go beyond the immediate response, understanding how views are shaped by respondents’ own experience, in their own social circles – and giving people permission to admit to shortcomings or mistakes in a non-judgemental way.

Unpack the subject and you will find much to explore in terms of the age of the child, what they already know and how they found it out, differences in parenting styles and so on.

It is also a winner because it is a perfect subject for using a board; in fact it would work better on a board than in a face to face environment. Firstly, a board can offer anonymity and mitigate embarrassment. In the same way as some people deal with the subject by giving their child a book to read, when the participants write their responses, they can be more considered, careful with the way they frame their language, and not be afraid they will blurt out something rude or stupid. It is a subject in which the questioning must evolve and trust must be built up over time. There is a lot of exploring to do and a board gives the ability to go with the flow and not rush participants. 

The board also allows participants to check  things out with their children during the course of the research – what have they been told at school, for example, and this is also going to take time. A good strategy is to find out what the parents think first, and then ask them to check with their children and see how that modifies opinions.

All of this can be done more cost-effectively using a board. A face to face method for this would require reconvened groups, probably separated by age and SEC of parents, whereas one board could handle the full range of sample needed and run for the time necessary to complete understanding of all the issues.

Honourable mention: What makes a truly great music festival? (With participants from around the world).

This subject would be really brought alive with uploads of images, videos and music that would convey participants opinions as much as their words would.

Using different languages with boards is not an issue and there could be a really broad international spread without any travel costs involved. The analysis would be really fascinating too because it would be possible to find all the commonalities but also the cultural differences; you could ask the Taiwanese to comment on the Americans ideas and vice versa. So cool to have your analysis done by your participants!

Honourable mention: Men of the ASP (Association of Surfing Professionals) current world tour – a yearlong bulletin board

Another use of a bulletin board that cannot be cost effectively replicated offline.

This would in effect be netnography without the ethical issues – getting insights into a community, the relationships, the attitudes, the ups and downs over time. It would take some initial skill and effort to get regular participation, and would generate a mountain of information, but you would have both personal thoughts and feelings, and group dynamics as they all react to each other’s posts. And it is long enough to really get to know the participants.

So I’m doing a series of 10 minute videos for use on social media and mobiles. Once you have a bit of knowledge, and, importantly the right mindset, you have a huge number of options for doing fabulous research projects. 

The videos break down the process into steps to give an orientation for those who are less familiar with qualitative research.

Next steps  are How to choose a Research Method, followed rapidly by Groups versus Depths – including online interviewing, and with short illustrations of the methods included. I love new methods – with John Griffiths we just won a prize for a workshop that used Gamification and Crowdsourcing – but I have to admit that groups and depths are still the mainstay of qual.

• How technically could that be achieved?
• How many people might come to the workshop?
• And how would we manage them and what would we crowdsource ?

Although the company that provides technical a.v. support to the conference were very capable – they had even in the past organised a live link to the space station, it turned out that the humble Skype worked perfectly for our needs. (There must have been major sighs of relief in the MRS Conference budget department).

On the day, we were connected live with Jaroslaw Cir in Prague, Hanne Kristainsen, founder of Creative Creatures in New York, Kate Tribe (Tribe Research) in Australia and Han Zantigh, MD of Brainjuicer in China.  They were our crowdsourcing mentors and were in the four corners of the room, available for consultation.

As John and I had recently formed Electric Learning, it seemed logical that we should teach crowdsourcing and how it can be used in research – in the interactive, immersive style that Electric Learning is becoming known for.   I had just read Reality is Broken by Jane McGonigal. Despite the title,  its a very positive book; it finds the value in gaming and discusses how to apply game theory to learning. Not gamification note – game theory, which is about flow, engagement and motivation, and is social as well as individual.

So our workshop delegates were allocated into teams, given a mission, given trade-able resource cards, (see right for some examples) various ways of winning points (including asking good questions of the mentors) and a time limit, to come up with ideas for using crowdsourcing as a research method.  And in true crowdsourcing style, the ‘crowd’ voted on the best idea at the end of the workshop.

Many thanks to those who took part, nominated us and voted for us. We won the Award for Best Workshop.  The judges said

This award is judged on the interactivity, practicality, originality and management of workshops held during the conference. In their highly interactive and original workshop John and Joanna asked competitive teams to consider a group generated issue where crowd sourcing might be relevant. Among the resources available were four leading experts on crowdsourcing techniques in Sydney, New York, Prague and Shanghai who could be individually consulted on Skype and who judged the presentations. The result was a high level of creative energy and involvement from every participant.

  “The session epitomised what a good workshop should be: highly interactive, very original, well structured and managed by the leaders, and with real learnings for the participants”

More about crowdsourcing…

More about gamification and game theory….

I have often resorted to giving clients a one page explanation of group dynamics, or a set of listening tasks, to help them see what happens in groups, rather than just watch.  I would like to have Simons’ guidelines framed in every viewing room in the country.

Over to Simon Riley, from Shore Qual

Hallowe’en night and a haunted viewing room

Who would be a qualitative researcher, eh? This post is not at all influenced by my having something of a difference of opinion last night with some clients over a group discussion …but ever the Polyanna, I’m turning this into something positive, by lecturing everyone on how to avoid the pitfalls of watching discussion groups.

So it was Hallowe’en night, I’d missed my son’s party to leave for the discussion groups, but left him my spooky compilation CD in the true manner of a 40-something Dad (featuring way too much of The Fall, little bits of Bauhaus, The Specials and yes The Monster Mash). All started – and almost until the end, continued – very well. I’d negotiated my way through the white water ride of a potentially tricky group, doing some creative NPD with some 18-24 year old lads. I’d got through a tight guide nicely; and more importantly, the lads had really engaged with the discussion (not always the case in 18-24 male groups) and told us as much about themselves as we dared hope for. I was just preparing myself to accept the plaudits modestly from the viewing room.

For a moderator, the machinations in the viewing room can be scarier than a cemetery in Transylvania

But when you step out of the hubbub of the discussion room and venture back there, you never quite know what you’re going to get. Especially with new clients. As soon as I opened the door I could sense some kind of spell in the room: deathly quiet; furrowed brows; no eye contact. It was a qual research Hallowe’en nightmare. All that was missing was the bell tolling, a howling wind and a wolf silhouetted by the moon. What had gone wrong back there?

It wouldn’t be appropriate to do a post-mortem here. But assuming I’m not completely deluded about how to moderate a discussion group after 13 years (having been, inter alia, head of marketing qual at Ipsos MORI) how could this happen?

Of course, there is no ‘perfect’ in qual and you always look to improve, but that aside, I’ve long been fascinated with the disconnect between a group that a qual researcher would recognise as a success and one that clients recognise as a success. I also supervise groups, sitting with clients, so have experience on both sides of the mirror here. Watching and commenting on groups is not as easy as it may look, especially if you’re not so comfortable with qual.

So I’ve drawn up a few basic rules to help discussion group viewers get their heads around what they are seeing and hearing – aimed at those who do not hang out in viewing facilities as much as I do (lucky you!):

• Don’t worry if it doesn’t make immediate sense – wait for the analysis. To follow a cooking analogy, you are watching the chef gathering some of the ingredients. You are not watching him/her prepare the dish and you are not going to be able to taste it yet.
• For the same reason, value your first insights but treat them with caution too. An initial insight may be valid, it may not. Again, you can only tell by analysing properly in the round.
• Don’t take what group participants say literally. What they tell us and show us is evidence we use to work them out, it is not gospel.
• What people say spontaneously, unprompted, tends to tell us much more about them than what they say when put on the spot (with some exceptions). Moderators try to minimise their interventions – so the discussions are as natural as they can be, given that we’re in a viewing facility with a bunch of strangers here.
• The moderator hears, sees and picks up a whole lot more, by being in the room and looking people in the eye. Ask the moderator what he/she felt about the vibe, body language etc. But the moderator does not have the freedom viewers have to switch away from the group for a few minutes to take a thought and ponder it. So those viewing have more of a chance to develop thoughts (though they risk missing something important when they do). Conversation between moderator and viewers after the group should be based on this understanding of these relative strengths and weaknesses.
• Follow-up probing is important in groups but it has to be used selectively. It can bore the participants not being probed; it is hungry on time. But worse, it can make people over-rationalise and self-justify, rather than be themselves (not just the person being probed but the others who think, “It’s me next”). Qual research at its best comes at things indirectly; asking the moderator to do more direct probing is not always the way to get more understanding.
• Remember, this is the bit of the research process where we build around participants, not the other way around.
• Different audiences need different moderating styles. A naturally chatty group has one set of challenges – e.g. keeping them on-topic and cutting people short without disempowering them – a quiet group has another. Typically the moderator will seemto be “working harder” with the latter but he/she is really working equally hard with both.
• Good recruitment is important, but you can’t and shouldn’t seek to over-engineer the minutiae of each person you invite. They are people and if they don’t fit neatly into a box, that’s normal. Of course if you just have totally the wrong people, that’s different and is a big issue.
• Qualitative research can be very counter-intuitive. It is not just a research technique – it is a whole approach to gathering and making sense of information that will jar with the way many approach the world and especially their working life. It’s more of a leap for those without an arts or humanities background, because critical analysis skills are the most relevant ones for interpreting the meaning of discourses. It is instinctive to seek to measure, to seek order and to seek direct answers; it is also drummed into most of us in school. But qual explores, highlights and makes connections; it embraces mess and lets it breathe and live so we can study it. Order is only really imposed at the final stage of sense-making. This can feel very uncomfortable for those used to being in control right the way through. It also requires trust in the researcher’s ability to do the sense-making.
• Moderators often “play dumb” in groups as a strategy for getting people to explain themselves. It does not mean they are dumb. Though they can be.

I’m not the first qual researcher to be frustrated by clients having a different take on a group and I won’t be the last. Aren’t we just the misunderstood geniuses? But if one future group viewer takes note of one of these tips, I’ll be at least as happy as Bela Lugosi (and hopefully less dead than him). Of course, sometimes, it is our fault …

Ignore all wording, focus entirely on prices

Technically speaking, its the interpretation part you can save time on. Analysis is really analysis and interpretation; the analysis part consists of reviewing the material, comparing, categorising, coding, kind of like sorting out a large bucket of Lego to see what you have and what you might be able to build with it.  And this you can do as you go along – you don’t have to wait till the end of the project.

The interpretation part consists of asking questions about what you have found. What does it mean to consumers? Why did they say it this way and not that way? What does it mean for the client? What kind of story can you build or structure you can develop?  Experienced researchers also ask some of these types of questions during the research – thereby getting to these higher levels and checking their interpretations faster and more accurately.

If I was American I would call these Power Questions. An example would be ‘why is that important to you?’  Oddly enough it works well when you are asking about something relatively unimportant in the grand scheme of things, such as why some people like their coffee black. What it achieves is to help the respondent think about the higher significance of black coffee – taking you into interpretative territory immediately.