Canadian Business Research Intro

Be Careful with Surveys

Click Here for Printer Friendly Version

Many business advisors and business start-up publications suggest new entrepreneurs conduct surveys to gather information about their market and business environment.  Anyone who has attended one my research seminars knows I do not encourage new entrepreneurs to rush out and conduct surveys.  This has nothing to do with any intrinsic hate of surveys or of any underhanded desire to sell surveying services.  Instead it stems from years of experience helping new entrepreneurs research their new business ventures.

Many entrepreneurs use surveys as a means to avoid talking to people.

It is much easier to run through a list of questions or better yet have someone else ask the questions you wrote than have an actual conversation.  The problem here is that you lose out on qualitative information.  Statistics on their own can be very difficult to interpret.  By speaking directly to people in your market or industry you draw on the insights and experiences of individuals involved in your business environment.  Successful entrepreneurs are not shy!

Among marketing professionals there is an on-going debate between qualitative and quantitative research and which is better.  Both are certainly vital to understanding one's marketplace.  However from an entrepreneur's point of view face-to-face qualitative research (talking to people!) has the added benefit of developing a business network.  Contacts and the consequent referrals are vital for small business success.  They are the cheapest and most successful forms of advertising.  Therefore I always strongly advise new entrepreneurs to conduct extensive qualitative research and forego conducting a survey if they are unwilling or unable to do both types of research.  Secondary sources and existing studies can be used to collect hard data numbers.

If an entrepreneur is prepared to both talk to contacts and conduct a survey, then my advice is ensure your survey is sound and that it will provide you with the insight you require.

Beyond the “avoidance issue” is one of quality. Time and time again we see poorly constructed surveys that seriously distort an entrepreneur's understanding of their market or industry. 

The problem can usually be traced to one of three issues:

1. The questions asked were leading or vague
   (e.g. Would you use my service?)
    

2. The nature of the survey sample was skewed or inappropriate
   (e.g. only the in-laws were interviewed)
    

3. The size of the survey sample was too small 
   (e.g. only 10 people were contacted)

Problems with survey design are then compounded by the fact that entrepreneurs tend to consider the results of their own survey as superior to all other data because they have compiled it.  They dismiss contradictory sources as ill informed or irrelevant before they even consider them in context.  They base key assumptions about their business on seriously flawed information. 

If you are going to conduct a survey make sure you take the time to develop a valid set of questions and that you contact enough relevant respondents to make your results viable and useful.

 

There was a “Mini Wheats” commercial on TV a few years ago when the “Mini-Wheat” guys asked people in the mall whether they liked sweet things in the morning and if they liked to eat a nutritious breakfast.  They later congratulated themselves because everyone clearly liked both and so would obviously like “Mini-Wheats”

Regardless of being a silly commercial, it is the perfect example of a poorly constructed survey.  Just because someone likes sweet things in the morning and eating nutritious things for breakfast does not mean they like “Mini-Wheats”.  It does not even mean they eat cold cereal for breakfast.  They might prefer whole-wheat bagels with marmalade.  The survey results are meaningless.

Fortunately Statistics Canada has feature on their web site called: Statistics: Power from Data!  It is primarily published for high school math students but it is useful for anyone undertaking a survey for the first time. It includes definitions, examples and exercises.  It explains how to craft useful questions, the types of data collection available and processing options. 

See: http://www.statcan.ca/english/edu/power/toc/contents.htm

The American Statistical Association also has a booklet entitled What is a Survey which is written primarily for non-specialists.  It covers a variety of topics such as What is a Survey, How to Plan a Survey, How to Collect Survey Data, Judging the Quality of a Survey, What Are Focus Groups, Designing a Questionnaire, What is a Margin of Error.  (See: http://www.whatisasurvey.info/)

Also see the Statistics Canada publication: Survey sampling: a non-mathematical guide (cat# 12-602). This manual is a basic guide on survey sampling for those with little or no formal training in mathematical statistics and it is largely patterned after workshops on survey sampling. It is intended to explain the fundamental concepts and methods of survey sampling by way of examples and illustrations rather than by theory and algebraic expressions.

Another good reference book is The Survey Research Handbook: Guidelines & Strategies for Conducting a Survey by Pamela L. Alreck & Robert G. Settle  (1994) Without technical buzzwords or statistical jargon, this book provides the methods and guidelines for conducting practical, economical surveys from start to finish.

And finally there is How to Conduct Your Own Survey, by Priscilla Salant, and Don A. Dillman (1994).  This book covers the key features of developing a quality survey including: Deciding what information you need, Choosing a Survey Method, When and How to Select a Sample, Writing Good Questions, Questionnaire Design

Make sure your business' success is not undercut by shoddy research.  If you are going to conduct a survey take the time to do it right!

Next: The benefit of business research

©  John White, GDSourcing - Research & Retrieval 2006