Tuesday, May 17, 2005

A question of questions

Summer sees scores of young people trudging the length and breadth of Bombay, filling out questionnaires. With the exponential rise in B schools and courses like BMM and BMS which favour internships, every Tom , Dick and Hari is doing a summer project. And the project you're most likely to be handed is a 'survey'.

Surveys serve two important functions:
a) They get the trainee out of the boss' hair for most of the duration of the project. This is important because few companies have extra seating space or computers for trainees.

b) The survey, if sincerely done, just might reveal something of use to the client or agency which they can further investigate. The operative word is IF, because survey forms are rarely administered or completed as they should be.

This happens for two reasons:
a) Idiot questionnaires: The survey is 7 pages long and the respondents lose interest by page 2. Asking people to rank and rate 7 attributes on 5 parameters is a pointless exercise but one which the designers of surveys nevertheless insist on.

So the student has no choice but to hurry through the survey, taking down a few answers, guessing/ making up responses to a few others before capturing the most crucial data: name, address and tel no.

Crucial because based on this info, the boss may randomly conduct a back-check - to ensure that the respondents are not a figment of the imagination, and that they were actually questioned. Which is true, but does not reveal the whole story.

b) Lazy/ unethical behaviour: Where there's a will, there's a loophole. And smart (lazy) students know fully well to exploit it. Many students are given a daily 'target' of forms to fill out. Others are paid on a per form basis (this is especially true of undergrads who work directly for market research agenices for pocket money, not experience).

Hence, however decent the questionnaire may be, these students are in a situation where dil maange more.

This is what happened when the DNA people came to the colony I stay in. The 3 page, 7 minute questionnaire was reduced to a 1 1/2 page, 3 minute job. The girl simply skipped over page 2 and was brazen enough to smile and assure me, "Don't worry, I'll fill out the rest myself".

Of course, DNA is using the exercise more to collect a database of names and addresses to subsequently market their newspaper. But what about companies who consider market research to be the 'holy grail'? And there are plenty of them...

Survey strategies
As an MBA student I too went around doing a survey for my summer project. I was working at Lintas and the project was Surf Ultra vs Ariel. I learnt for the first time that there are 17 'wash attributes' - cleans whitest, cleans brightest, and so on and so forth.

I must admit I could not get housewives to rate ALL 17 attributes on 5 parameters (strongly agree.. somewhat agree... etc). That bit I had to extrapolate based on what they said to me. But I did take the trouble to trudge to Nehru Nagar and other lower middle class localities even though there were 400 flats in my colony where I could have easily filled out all the forms.

Eventually I learnt the Great Indian Survey Trick. The single most efficient way to get female respondents is in the second class compartment of the Mumbai local train. The trick is to do it at non-peak time.

First check that the lady is alighting at least 6-7 stations away. Then shoot. 99% of the time the woman is more than happy to share her thoughts with you. Beats going house to house and having doors slammed on your face - and you get a completely random sample.

Good deed for the day
Having 'been there, done that', I have on more than one occassion filled out surveys for forlorn looking trainees. Invariably, however, I find the questionanires are badly designed/ worded and administered with minimum enthusiasm.

Sure, market research is a gruelling and thankless job but treating it as a punishment only makes things worse.

If you're trudging around with a survey in hand this summer, see it as opportunity. To smile at random strangers, to connect with them for a few minutes. And also to deal with rejection, even rudeness, yet not take it personally. To live, to learn, to grow.

I know, I did.

5 comments:

  1. Great post. I put in a link to this one at my weblog here. http://ecophilo.blogspot.com/2005/05/of-surveys.html
    Regreat that I couldnt use the local train idea (for obvious reasons) though!

    ReplyDelete
  2. The Market research/Ad agency do not follow the KISS formulae (Keep it simple - stupid) while designing the questionnaire. It often results in data haze and ends up as irrelevant information gathering excercise.

    ReplyDelete
  3. Your mention of keeping respondent's psyche into mind before designing a survey is crucial. Many times well intentioned but ill designed surveys loose interests of respondent. And of course, if surveyer wants to makeup data, then nothing can be done.

    However this "trick" of using railway comparment, called enriched or bulk sampling, is not likely to give you random sample, unless, of course, your target group happened to be just that, middle class women traving metro in non-peak hours! Though it is better than making data. Getting a "simple random sample" is next most crucial task apart from survey design. However an intelligent designer may put "double check" questions to filter madeup madeup data.

    ReplyDelete
  4. Anonymous11:03 AM

    nice post. I am a market research professsional myself! I havelinked to this post and written more about it on my blog - do check it out sometime.... http://indsight.org/blog/index.php?p=341

    ReplyDelete
  5. Nice post, surveying is a sample of what a sales guy goes through.....

    The other side of this story is doing surveys with large corporates....let me share my experiences on my next post, hilarious to say the least!!

    ReplyDelete

Disqus for Youth Curry - Insight on Indian Youth