Data Collection An Eclectic Approach

We will start with the premise that data collection methods should align with research objectives. No one methodology is the best approach for all research designs. The criterion for deciding on a method should relate directly to the research requirements.

Consideration of secondary factors should influence the decision only after the primary objectives are satisfied. On occasion lower priority criteria, such as time in field, or a concern about professional survey takers, are used to decide which data collection approach to employ. These types of issues need to be addressed, but should not dominate the decision on data collection method for a study.

Taking an eclectic approach to data collection ensures the strengths and weaknesses of each approach will be evaluated in the context of the research objectives. For example, unaided awareness and preference studies are difficult to accomplish using a web-based approach, especially if the study is exploring multiple products or product types and needs to use probing questions.

Conversely, while not impossible, it would be highly undesirable to attempt a conjoint study using telephone interviews. Conjoint studies are much better suited to a web-based approach.

Web-Based Method
The web has far-reaching capabilities that allow researchers to extend the coverage of their research. More geographic coverage (larger number of countries), typically at a lower cost, is an attractive combination. The lower cost and/or more coverage can help the research team to add country coverage in emerging markets or where it may be difficult and expensive to conduct telephone interviews due to the lack of native-speaking interviewers (although this may work in reverse in some countries).

Speed of data collection is often the reason a web-based approach is selected. However, the caveat is that best practices may still require several weeks (3-4) to provide time to do waves of emails to complete a random stratified sample.

For some studies the web is the most desirable approach. It is the single best way to conduct a conjoint study. It"��s also the preferred method for collecting information when long lists of items or complex statements are part of the research instrument design (e.g., multiple response questions with either an "��all that apply"�� or k of N question structure).

The k of N question structure is a ratio, where there are N items listed and k is the maximum number of items the respondent is allowed to select (e.g., "��select up to four"��). The k of N question type requires respondents to listen to more information than they are likely to remember. Human heuristics research shows that with lists of more than 5-7 items only the first and last items heard tend to be remembered, which could bias the results (Note: list rotation helps, but cannot overcome this problem).

Another benefit of a web-based study is it allows respondents to participate when they have time, and to start-stop-and-restart if necessary. It also allows for interesting incentive programs that can combine downloadable content with other forms of incentives. In fact, it may be highly desirable to use downloadable content as an incentive, rather than a cash-based approach to avoid attracting professional survey takers. Immediately available value-added content will attract business managers, IT professionals, and other professional groups, but will have little value to the survey takers looking for cash payouts with little or no interest in the subject matter.

Telephone Interview Method
Among the most important advantages of telephone interviewing is the researcher"��s ability to control the sample. Interviewers are able to find the person in an organization most qualified to respond. They can screen respondents and use probes to find the "��right"�� respondent. Quotas for demographic strata and unique qualifications can be controlled directly with the help of a computer-aided telephone interviewing (CATI) system used by most field houses. Broad-based lists of companies, rather than opt-in email lists, can be used, which helps with randomization.

The use of unaided response questions is easier to implement with phone interviewing than it is with a web-based approach, which relies on a respondent to take the time to type responses. Interviewers can also probe and encourage respondents to consider more answers, using phrases such as: "��Are there any other companies you can think of?"��

Telephone interviewing, which is a more active recruitment method than web-based research, helps to avoid professional survey takers regardless of the incentives used. The quality of the list is clearly as important in telephone interviewing as it is in web-based data collection. However, a poor phone list is easily detected and substituted if it is not performing well.

Therefore, the ability to create highly stratified samples with multiple selection criteria and multiple sample quotas is a significant strength of the telephone approach. Data collection may be enhanced for some question types and, as stated, the type of question is an important criterion for selecting a data collection method. Not all questions are best suited to the telephone, but when the study requires probing and open-ended questions, the phone may be the method of choice.

Summary
The issue is not whether a study using a specific data collection approach is superior or inferior to another approach, but rather whether the research objectives are served better by one approach over another. Some research designs clearly align with specific data collection methods while in other cases more general strengths and weaknesses of the two methods can be used to decide the approach that will be used. In some cases, the differences between data collection approaches are less dramatic and issues such as availability of field resources, cost factors, and time in field are the deciding factors.