Here are the steps you need to take if you are planning to design and administer your own survey:
1. Define as precisely as possible exactly what your information needs are. What questions do you want answered? This component is the most critical in obtaining useful survey results.
2. Define your universe and target audience. Select your sample. Remember that sampling is a very powerful tool and allows you to extrapolate to a large number from a few.
3. Develop your questions. The questions should reflect your information needs and should also be used to test alternatives. Multiple choice questions seem to provide more data than open-ended questions. However, there is a role for both. For example, you need to not only find out if they will buy your product, but what price they are willing to pay for it. Don’t forget to get any demographic data (age, sex, income) that will allow you to target to marketing better.
4. Prepare your data collection instrument. If it is a questionnaire, the wording needs to be very clear and understandable. For an interview schedule, the questions must be laid out in a logical format with prompts to probe. This is especially true if you are not collecting the data yourself. Always ask the demographic questions last; start with a general question. Also, if possible, pre-code your instrument so that the data can be entered into a computer file directly.
5. Collect your data. If you are using a questionnaire, be sure to include a covering letter explaining the purpose of the survey, and specific instructions. Always included a self-addressed stamped envelope. Having a toll-free (800/888) fax number for respondents to fax back their completed questionnaires also works well.
6. Send a reminder postcard. This often increases the response rate. An incentive - discount coupon, an opportunity to win a prize, etc., also often increase response rates.
7. Input the data into a computer file and analyze. This can be done a spreadsheet or a statistical package such as SPSS. Be sure to be thorough. Not only look at the overall results, but take at look at the sub-groups to see if you can identify any unique requirements.
8. Prepare a final report along with recommendations. This can be as short or long as you need it to be.
Developing a really good and effective research instrument is difficult and requires consider skill and experience. If the research is critical to the success of your marketing, get help from a professional.
Types of surveys
There are four ways to collect survey research data. These are:
1. Mail surveys. These tend to be the most common method of collecting data. A questionnaire is mailed out to a potential respondent, and if all goes well, the person writes in the replies and mails (faxes) it back.
2. Personal interviews. This was the most accepted way of doing market research in this business. Interviewers to go to carefully selected homes and personally interview the respondent. However, this was at a time when people were more trusting and telephones were not as common as they are now. Personal interviews are still used, but mainly in shopping malls. Personal interviews are also very useful to obtain in-depth information from key information gatekeepers or experts.
3. Telephone surveys. These are studies based on one-on-one questioning and answering between an interviewer and respondent over the phone. This is often done by professional firms with trained interviewers using direct data entry into computers. However, telephone surveys are also useful for a small firm wishing to collect data quickly and inexpensively – say from former customers.
4. Self-administered surveys. These are questionnaires left in places such as restaurants, motels, hotels, and occasionally in health and financial institutions. Respondents can pick them up, fill in, and return either in a convenient depository or by a prepaid-postage return envelope. It is not considered by market research professionals to be very useful because of total lack of control over respondents. It is often seen as little more than a public relations gesture.
Since these are the most common and potentially the most useful, we will take a took in some detail at mail questionnaires.
There are a number of advantages to mail surveys. These are:
1. Broad scope. It is possible to ask about, and get answers to, a rather amazing breadth of topics. By offering anonymity, men and women are willing to reveal many personal aspects of their lives.
2. Amount of data. If the topic is of interest to the respondent, they are willing to spend a considerable amount of time (up to an hour) filling out a detailed questionnaire. As a result, it is possible to collect large amounts of data.
3. Convenience. Respondents can complete the questionnaire whatever time it convenient. There is an opportunity for a more thoughtful response, since the person is not confronted by someone waiting. Also, they can verify their data (brand of dog food, make of appliance) before they respond.
4. Low cost per response. A large number of respondents can be surveyed at a lower cost than other methods.
5. No interviewer bias or cost. There is always the risk of an interviewer unconsciously affecting the response through voice and body language. Also, interviewers and their expenses can be a significant cost in any research project.
Mail surveys have a number of limitations. These are:
1. Sampling problems. It is often difficult to obtain a good mailing list. Problems include the list being up-to-date and reflecting the characteristics of the population you wish to survey.
2. Response problems. Generally response rates for mail surveys tend to be low. The researcher then wonders whether the results are representative of the entire population, or of only that sub-group that chose to respond. There is some evidence to suggest that those that answer surveys are the most interested in the topic, have strong feelings (pro or con) about it, and tend to be more literate.
3. Lack of control over respondent. There is no control over the how the respondent answers the questions; no opportunity to explain the questions or probe for the meanings of replies.
4. Wrong information, wrong respondent. The questionnaire may be answered by someone other than the addressee (secretary for the CEO). These self-substitutions can rarely be detected.
Here are some tips for improving the response rates to mail surveys.
1. Provide advance notice to respondent. Advance notice seems to increase the rate of return. Such notice should be in the hands of the respondent about 5 days before the questionnaire. A postcard or letter may be used telling the respondent the purpose of the survey, and how valuable their contribution is.
2. Use a quality outgoing envelope. It should convey the unspoken message of importance and need to open. The paper should be of high quality. If you previously notified the respondent as per Item #1 above, then include on the envelope in red – Here is the Survey We Wrote You About.
3. Use an effective cover letter. It should be short, on high-quality paper and motivate the respondent to complete and return the questionnaire. The letter should describe the purpose of the project, stress the importance of filling out the questionnaire, outline benefits to the respondent and include an estimate of the time it will take to complete. We also found it useful to include a name and phone number (preferably toll-free) they can call is they have any questions about the project or a question on the questionnaire.
4. Use incentives. Including a small amount of money increases response rate because many people feel guilty about taking the money and not filling out the questionnaire. Another technique that has worked well for us to hold a draw for a prize for respondents completing the questionnaire. If the questionnaire is confidential, be sure to supply a separate draw form, and allow them to fax or mail it to you separately. Mention of the odds of winning the prize. The prize should be something that your respondents want.
5. Provide a return, postage-paid envelope. This makes it easy for the participant to drop off the questionnaire in any mail box.
6. Do a follow-up. There are two types of follow-ups. One is to send a postcard to all survey respondents with a note saying something like – “If you have completed and returned our survey, thanks very much. If you have not yet done so, please complete and return it as soon as possible. Your opinions and views are very important to us. If you misplaced the questionnaire, please call this toll-free number and we will immediately send you another copy.”
The other method of follow-up is to keep track of those that have not returned their questionnaires and do a follow-up letter/card to them. If the surveys are supposed to be anonymous, then care has to be taken not to let on that you are tracking specific questionnaires. In our experience, the general follow-up is a safer, more professional route to go.
The other favoured option for collecting primary survey data is to use the telephone. This method has a number of advantages going for it that include:
1. People answer phones. There is a compulsion to answer the phone regardless of what the person is doing at the time.
2. Interviewer control. In a central facility, total control of interviewers and thus quality control is possible.
3. Better sampling. In modern societies, all businesses and most homes have telephones.
4. Higher response rates. A response rate of 80% is quite common for telephone surveys.
5. Lower costs. Since no travel time is needed, the amount of time needed for the study and interviewing costs are reduced.
6. Computer assistance. With a computerized interviewing program, the it possible to modify questions during the survey process, and the data is electronically stored for analysis.
The telephone survey also has a number of limitations.
1. Potential sample bias. The telephone list and availability of people to answer the telephone may result in a biased sample. (Many people have unlisted numbers. The way around this is to randomly select an exchange, and then randomly generate a 4 digit number).
2. Respondent resistance. People are getting increasingly annoyed at being called at home for sales pitches and surveys.
3. Limited interview length. People get impatient answering a telephone survey if it longer than 15 minutes.
4. Audio only. Telephone surveys do not permit visual aids such as ads or other display materials. Therefore, they would not be appropriate to test visual materials.
Constructing the questionnaire
A well designed, professional questionnaire will increase response rates and provide better data. Here are some tips.
1. Determine your goals, objectives and the answers you need from your research project.
2. Include only questions regarding answers that will make a difference in planning or decision making and are relevant to the project objectives. Don’t ask extraneous questions.
3. Include questions that may assist you in audience segmentation, and that will provide you with information as to possible alternatives and acceptable options (e.g., price to charge for your product).
4. Keep the questionnaire as brief as possible. If the survey will not fit on one or two pages, use an 11x17 piece of paper. The fewer the pages, the greater the response – 5 pages should be the maximum mail survey length.
5. Desk-top publishing in two columns allows you to ask more questions within the suggested maximum size.
6. Make sure the questionnaire is easy to read. Avoid jargon and complex sentence structure. Use simple plain English at about the grade 8 reading level. Test it on your children.
7. Make the questions easy to answer. Use closed end multiple choice or a yes-no checking structure whenever possible. Provide a scattering of opened end questions to allow respondents to elaborate and provide additional information.
8. Start with a general question, get into the major content areas that you want answers, and place demographic data at the end. The questions should flow in a natural and logical sequence.
9. Reassure respondents as to their anonymity (if possible).
10. Make sure that the questions are neutrally worded, i.e., they do not encourage specific desired answers.
11. Thoroughly test the questionnaire. This is extremely important for response rate and answer quality.
12. Design questions so they are adaptable for computer processing. Pre-code each question to facilitate data entry.
A cover letter encourages greater response rates to mail surveys. Make sure the cover letter looks professional – avoid the junk mail appearance. Here are some ways to design an effective covering letter.
1. Let the potential respondents know exactly who you are.
2. Describe in detail the survey and its goals.
3. Ensure the confidentiality of responses.
4. Explain how the survey may assist the respondent as well as yourself.
5. Encourage a prompt response. We have found that a specific date by when the survey is due is better than “return as soon as possible”.
6. Reassure potential respondents that a salesperson will not contact them.
7. Provide them with a name and contact information (phone, fax and e-mail) if they have any questions about the project or questionnaire.
8. Be sure the covering letter is on company letterhead. Sometimes response rates and data quality are better if an independent, third-part firm is used. Respondents feel more comfortable sending in their comments to a research firm rather than the client firm.
Sampling refers to selecting those individuals who will be asked to provide you with the data you need to your survey. Needless to say, the better and more representative your sample, the better will be your results.
There are a number of factors to consider when sampling. These are:
1. Sample size. Random sampling is a very powerful tool. It can give you amazingly accurate results from just a small number of respondents. The key is to ensure that your sample is truly random. The size of your sample should be determined by costs, degree of accuracy you need and “minimum cell sizes”. The latter term refers to analyses of subgroups and a minimum number needed for reliability of results.
2. Sample source. If your target audience is the general population, then households in a particular geographic area are probably your “universe”. If your customers are your target audience, then a random sampling of your in-house mailing lists should be used. Telephone books, business directories and business CD-ROMS are all good sources of names for sampling. In addition, commercial mailing lists can also be rented.
Be practical and realistic in your sampling considerations. Sample large enough to give you some confidence in the results. Entrepreneurs don’t worry about statistical significance – they leave that to the academics.
Analyzing and reporting the findings
The degree of analysis and reporting should be based on your needs. If you are looking for some quick answers, quick data tabulation may be all that is required. However, is the research is to be used for the basis of major corporate changes, or has significant financial implications, then a much more detailed analyses and a formal report may be required.
In terms of data analyses, here are some suggestions:
1. Use the proper software tools. Spreadsheets are okay for basic tabulations, but for more detailed statistical analyses, use programs such as SPSS (Statistical Package for the Social Sciences).
2. After your data has been entered for analyses, double-check it for correctness. Make sure that there are no obvious errors, and that the appropriate labels are assigned to the variables.
3. Run some basic descriptive statistics on your data, e.g., frequencies, means, ranges, etc.
4. Run some comparative analyses either by using crosstabs or correlation programs, or by running descriptive data on sub-samples of your data.
5. If you know what you are doing, you then may want to run some multivariate analyses such as regression analyses and factor analysis. These tools require fairly large sample sizes, but can help identify underlying dimensions and factors in your data.
6. Save, document and label your questionnaire, your data and analyses for possible later use. You may want to come back to the data to answer specific questions, or it may serve as a comparison or “benchmark” for future studies, e.g., levels of customer satisfaction with your products and services.
Here are some factors to consider when reporting the results.
1. The best report is the one that conclusively answers the questions that were asked.
2. You may want to consider preparing two reports – a technical one for the files and a shorter, non-technical version for use by your clients and decision-makers.
3. Make tables and graphs simple to read and understand. Use the appropriate graph for the data. It is not necessary to present results in more than full percentages. The data is not that precise. It is also useful for the reader if data in tables are ranked from high to low. If you are presenting tables and graphs, always provide an explanation of what is in the table. Don’t assume the results and implications are obvious.
4. A personal presentation of the research findings works well also. It allows you to highlight the key findings, the implications of these findings and to answer any questions.