4.2 The Research Process – Step 3 – 5
Step 3: Design the Data-Collection Forms
If the behavior of buyers is being formally observed, and a number of different researchers are conducting observations, the data obviously need to be recorded on a standardized data-collection form that’s either paper or electronic. Otherwise, the data collected will not be comparable. The items on the form could include a shopper’s gender; their approximate age; whether the person seemed hurried, moderately hurried, or unhurried; and whether or not he or she read the label on products, used coupons, and so forth.
The same is true when it comes to surveying people with questionnaires. Surveying people is one of the most commonly used techniques to collect quantitative data. Surveys are popular because they can be easily administered to large numbers of people fairly quickly. However, to produce the best results, the questionnaire for the survey needs to be carefully designed.
Questionnaire Design
Most questionnaires follow a similar format: They begin with an introduction describing what the study is for, followed by instructions for completing the questionnaire and, if necessary, returning it to the market researcher. The first few questions that appear on the questionnaire are usually basic, warm-up type of questions the respondent can readily answer, such as the respondent’s age, level of education, place of residence, and so forth. The warm-up questions are then followed by a logical progression of more detailed, in-depth questions that get to the heart of the question being researched. Lastly, the questionnaire wraps up with a statement that thanks the respondent for participating in the survey and information and explains when and how they will be paid for participating.
How the questions themselves are worded is extremely important. It’s human nature for respondents to want to provide the “correct” answers to the person administering the survey, so as to seem agreeable. Therefore, there is always a hazard that people will try to tell you what you want to hear on a survey. Consequently, care needs to be taken that the survey questions are written in an unbiased, neutral way. In other words, they shouldn’t lead a person taking the questionnaire to answer a question one way or another by virtue of the way you have worded it. The following is an example of a leading question.
Don’t you agree that teachers should be paid more?
The questions also need to be clear and unambiguous. Consider the following question:
Which brand of toothpaste do you use?
The question sounds clear enough, but is it really? What if the respondent recently switched brands? What if she uses Crest at home, but while away from home or traveling, she uses Colgate’s Wisp portable toothpaste-and-brush product? How will the respondent answer the question? Rewording the question as follows so it’s more specific will help make the question clearer:
Which brand of toothpaste have you used at home in the past six months? If you have used more than one brand, please list each of them1.
Sensitive questions have to be asked carefully. For example, asking a respondent, “Do you consider yourself a light, moderate, or heavy drinker?” can be tricky. Few people want to admit to being heavy drinkers. You can “soften” the question by including a range of answers, as the following example shows:
How many alcoholic beverages do you consume in a week?
- __0 alcoholic beverages
- __1–5 alcoholic beverages
- __5–10 alcoholic beverages
- __10–15 alcoholic beverages
Many people don’t like to answer questions about their income levels. Asking them to specify income ranges rather than divulge their actual incomes can help.
Other research question “don’ts” include using jargon and acronyms that could confuse people. “How often do you IM?” is an example. Also, don’t muddy the waters by asking two questions in the same question, something researchers refer to as a double-barreled question. “Do you think parents should spend more time with their children and/or their teachers?” is an example of a double-barreled question.
Open-ended questions, or questions that ask respondents to elaborate, can be included. However, they are harder to tabulate than closed-ended questions, or questions that limit a respondent’s answers. Multiple-choice and yes-and-no questions are examples of closed-ended questions.
Testing the Questionnaire
You have probably heard the phrase “garbage in, garbage out.” If the questions are bad, the information gathered will be bad, too. One way to make sure you don’t end up with garbage is to test the questionnaire before sending it out to find out if there are any problems with it. Is there enough space for people to elaborate on open-ended questions? Is the font readable? To test the questionnaire, marketing research professionals first administer it to a number of respondents face to face. This gives the respondents the chance to ask the researcher about questions or instructions that are unclear or don’t make sense to them. The researcher then administers the questionnaire to a small subset of respondents in the actual way the survey is going to be disseminated, whether it’s delivered via phone, in person, by mail, or online.
Getting people to participate and complete questionnaires can be difficult. If the questionnaire is too long or hard to read, many people won’t complete it. So, by all means, eliminate any questions that aren’t necessary. Of course, including some sort of monetary incentive for completing the survey can increase the number of completed questionnaires a market researcher will receive.
Step 4: Specify the Sample
Once you have created your questionnaire or other marketing study, how do you figure out who should participate in it? Obviously, you can’t survey or observe all potential buyers in the marketplace. Instead, you must choose a sample. A sample is a subset of potential buyers that are representative of your entire target market, or population being studied. Sometimes market researchers refer to the population as the universe to reflect the fact that it includes the entire target market, whether it consists of a million people, a hundred thousand, a few hundred, or a dozen. “All unmarried people over the age of eighteen who purchased Dirt Devil steam cleaners in the United States during 2011” is an example of a population that has been defined.
Obviously, the population has to be defined correctly. Otherwise, you will be studying the wrong group of people. Not defining the population correctly can result in flawed research, or sampling error. A sampling error is any type of marketing research mistake that results because of the sample utilized. One criticism of Internet surveys is that the people who take these surveys don’t really represent the overall population. Since a number of people don’t trust the online platform with their information, this group is less likely to participate than people who do not worry about their personal privacy online. This discrepancy can skew the data collected.
The next step is to put together the sampling frame, which is the list from which the sample is drawn. The sampling frame can be put together using a directory, customer list, or membership roster (Wrenn et. al., 2007). Keep in mind that the sampling frame won’t perfectly match the population. Some people will be included on the list who shouldn’t be. Other people who should be included will be inadvertently omitted. It’s no different than if you were to conduct a survey of, say, 25 percent of your friends, using friends’ names you have in your cell phone. Most of your friends’ names are likely to be programmed into your phone, but not all of them. As a result, a certain degree of sampling error always occurs.
There are two main categories of samples in terms of how they are drawn: probability samples and nonprobability samples. A probability sample is one in which each would-be participant has a known and equal chance of being selected. The chance is known because the total number of people in the sampling frame is known. For example, if every other person from the sampling frame were chosen, each person would have a 50 percent chance of being selected.
A nonprobability sample is any type of sample that’s not drawn in a systematic way. So the chances of each would-be participant being selected can’t be known. A convenience sample is one type of nonprobability sample. It is a sample a researcher draws because it’s readily available and convenient to do so. Surveying people on the street as they pass by is an example of a convenience sample. The question is, are these people representative of the target market?
For example, suppose a grocery store needed to quickly conduct some research on shoppers to get ready for an upcoming promotion. Now suppose that the researcher assigned to the project showed up between the hours of 10 a.m. and 12 p.m. on a weekday and surveyed as many shoppers as possible. The problem is that the shoppers wouldn’t be representative of the store’s entire target market. What about commuters who stop at the store before and after work? Their views wouldn’t be represented. Neither would people who work the night shift or shop at odd hours. As a result, there would be a lot of room for sampling error in this study. For this reason, studies that use nonprobability samples aren’t considered as accurate as studies that use probability samples. Nonprobability samples are more often used in exploratory research.
Lastly, the size of the sample has an effect on the amount of sampling error. Larger samples generally produce more accurate results. The larger your sample is, the more data you will have, which will give you a more complete picture of what you’re studying. However, the more people surveyed or studied, the more costly the research becomes.
Statistics can be used to determine a sample’s optimal size. If you take a marketing research or statistics class, you will learn more about how to determine the optimal size. It is not the size of the sample that makes the error in itself, it is if the sample is representative of the target market / population?
Of course, if you hire a marketing research company, much of this work will be taken care of for you. Many marketing research companies, like ResearchNow, maintain panels of prescreened people they draw upon for samples. In addition, the marketing research firm will be responsible for collecting the data or contracting with a company that specializes in data collection. Data collection is discussed next.
Step 5: Collect the Data
As has been explained, primary marketing research data can be gathered in a number of ways. Surveys, taking physical measurements, and observing people are just three of the ways we discussed. If you’re observing customers as part of gathering the data, keep in mind that if shoppers are aware they are being observed, it can have an effect on their behavior. To get around problems such as these, some companies set up cameras or two-way mirrors to observe customers. Organizations also hire mystery shoppers to work around the problem. A mystery shopper is someone who is paid to shop at a firm’s establishment, or one of its competitors, to observe the level of service, cleanliness of the facility, and so forth, and report his or her findings to the firm.
Earlier, the construction of a questionnaire was discussed. We may use that questionnaire in the survey process, or we may choose to not do so. Either way, survey data can be collected in many different ways and combinations of ways. The following are the basic methods used:
- Face-to-face (can be computer aided)
- Telephone (can be computer aided or completely automated)
- Mail and hand delivery
- E-mail and the Web
A face-to-face survey is, of course, administered by a person. The surveys are conducted in public places such as in shopping malls, on the street, or in people’s homes if they have agreed to it. In years past, it was common for researchers in the United States to knock on people’s doors to gather survey data. However, randomly collected door-to-door interviews are less common today, partly because people are afraid of crime and are reluctant to give information to strangers (McDaniel & Gates, 1998).
Nonetheless, “beating the streets” is still a legitimate way questionnaire data is collected. When the U.S. Census Bureau collects data on the nation’s population, it hand delivers questionnaires to rural households that do not have street-name and house-number addresses. And Census Bureau workers personally survey the homeless to collect information about their numbers. Face-to-face surveys are also commonly used in third world countries to collect information from people who cannot read or lack phones and computers.
A plus of face-to-face surveys is that they allow researchers to ask lengthier, more complex questions because the people being surveyed can see and read the questionnaires. The same is true when a computer is utilized. For example, the researcher might ask the respondent to look at a list of ten retail stores and rank the stores from best to worst. The same question wouldn’t work so well over the telephone because the person couldn’t see the list. The question would have to be rewritten. Another drawback with telephone surveys is that even though federal and state “do not call” laws generally don’t prohibit companies from gathering survey information over the phone, people often screen such calls using answering machines and caller ID.
Probably the biggest drawback of both surveys conducted face-to-face and administered over the phone by a person is that they are labor intensive and therefore costly. Mailing out questionnaires is costly, too, and the response rates can be rather low. Think about why that might be so: if you receive a questionnaire in the mail, it is easy to throw it in the trash; it’s harder to tell a market researcher who approaches you on the street that you don’t want to be interviewed.
By contrast, gathering survey data collected by a computer, either over the telephone or on the Internet, can be very cost-effective and in some cases free. SurveyMonkey and Zoomerang are two Web sites that will allow you to create online questionnaires, e-mail them to up to one hundred people for free, and view the responses in real time as they come in. For larger surveys, you have to pay a subscription price of a few hundred dollars. But that still can be extremely cost-effective. The two Web sites also have a host of other features such as online-survey templates you can use to create your questionnaire, a way to set up automatic reminders sent to people who haven’t yet completed their surveys, and tools you can use to create graphics to put in your final research report.
Like a face-to-face survey, an Internet survey can enable you to show buyers different visuals such as ads, pictures, and videos of products and their packaging. Web surveys are also fast, which is a major plus. Whereas face-to-face and mailed surveys often take weeks to collect, you can conduct a Web survey in a matter of days or even hours. And, of course, because the information is electronically gathered it can be automatically tabulated. You can also potentially reach a broader geographic group than you could if you had to personally interview people. The Zoomerang Web site allows you to create surveys in forty different languages.
Another plus for Web and computer surveys (and electronic phone surveys) is that there is less room for human error because the surveys are administered electronically. For instance, there’s no risk that the interviewer will ask a question wrong or use a tone of voice that could mislead the respondents. Respondents are also likely to feel more comfortable inputting the information into a computer if a question is sensitive than they would divulging the information to another person face-to-face or over the phone. Given all of these advantages, it’s not surprising that the Internet is quickly becoming the top way to collect primary data. However, like mail surveys, surveys sent to people over the Internet are easy to ignore.
Lastly, before the data collection process begins, the surveyors and observers need to be trained to look for the same things, ask questions the same way, and so forth. If they are using rankings or rating scales, they need to be “on the same page,” so to speak, as to what constitutes a high ranking or a low ranking. As an analogy, you have probably had some teachers grade your college papers harder than others. The goal of training is to avoid a wide disparity between how different observers and interviewers record the data.
Training people so they know what constitutes different ratings when they are collecting data will improve the quality of the information gathered in a marketing research study.
Ricardo Rodriquez – Satisfaction survey – CC BY-NC-ND 2.0.
For example, if an observation form asks the observers to describe whether a shopper’s behavior is hurried, moderately hurried, or unhurried, they should be given an idea of what defines each rating. Does it depend on how much time the person spends in the store or in the individual aisles? How fast they walk? In other words, the criteria and ratings need to be spelled out.
Collecting International Marketing Research Data
Gathering marketing research data in foreign countries poses special challenges. However, that doesn’t stop firms from engaging in international research. Marketing research companies are located all across the globe, in fact. Eight of the ten largest marketing research companies in the world are headquartered in the United States. However, five of these eight firms earn more of their revenues abroad than they do in the United States. There’s a reason for this: many U.S. markets were saturated, or tapped out, long ago in terms of the amount that they can grow. Coke is an example. Most of the Coca-Cola Company’s revenues are earned in markets abroad. To be sure, the United States is still a huge market when it comes to the revenues marketing research firms generate by conducting research in the country: in terms of their spending, American consumers fuel the world’s economic engine. Still, emerging countries with growing middle classes, such as China, India, and Brazil, are hot new markets companies want to tap.
What kind of challenges do firms face when trying to conduct marketing research abroad? As we explained, face-to-face surveys are commonly used in under-developed countries to collect information from people who cannot read/ read well, or lack phones and computers. However, face-to-face surveys are also common in Europe, despite the fact that phones and computers are readily available. In-home surveys are also common in parts of Europe. By contrast, in some countries, including many Asian countries, it’s considered taboo or rude to try to gather information from strangers either face-to-face or over the phone. In many Muslim countries, women are forbidden to talk to strangers.
And how do you figure out whom to research in foreign countries? That in itself is a problem. In the United States, researchers often ask if they can talk to the heads of households to conduct marketing research. But in countries in which domestic servants or employees are common, the heads of households aren’t necessarily the principal shoppers; their domestic employees are (Malhotra).
Translating surveys is also an issue. Hiring someone who is bilingual to translate a survey into another language can be a disaster if the person isn’t a native speaker of the language to which the survey is being translated.
One way companies try to deal with translation problems is by using back translation. When back translation is used, a native speaker translates the survey into the foreign language and then translates it back again to the original language to determine if there were gaps in meaning—that is, if anything was lost in translation. And it’s not just the language that’s an issue. If the research involves any visual images, they, too, could be a point of confusion. Certain colors, shapes, and symbols can have negative connotations in other countries. For example, the color white represents purity in many Western cultures, but in China, it is the color of death and mourning (Zouhali-Worrall, 2008). Also, look back at the cartoon-completion exercise in Figure 10.2 “Example of a Cartoon-Completion Projective Technique”. What would women in Muslim countries who aren’t allowed to converse with male sellers think of it? Chances are, the cartoon wouldn’t provide you with the information you’re seeking if Muslim women in some countries were asked to complete it.
One way marketing research companies are dealing with the complexities of global research is by merging with or acquiring marketing research companies abroad. The Nielsen Company is the largest marketing research company in the world. The firm operates in more than a hundred countries and employs more than forty thousand people. Many of its expansions have been the result of acquisitions and mergers.
