Methodologies

Sterling also has the capabilities to provide research using the following methodologies:


Mail Surveys

One of the most popular, time-tested and low cost approaches to data collection is via U.S. Mail. Sterling continues to use this methodology for many clients who find that it offers the best way to reach a high percentage of customers. In fact, Sterling has mailed more than 50,000,000 customer satisfaction surveys in the last decade.

Sterling works with clients to develop a fully scannable paper survey, usually printed in two colors, which can be mailed to customers whose addresses have been downloaded to Sterling. A business reply envelope, as well as an optional cover letter signed by a member of the client's upper management, is also included in the mailing. As the surveys are returned to Sterling's processing center, we check each envelope for additional letters or notes to pass along to the company (we call these "collaterals"). We provide these to the client in original form by mail or electronically by PDF. Sterling then scans the forms and tabulates the data in-house at our St. Petersburg, Florida offices.

Advantages of Mail Methodology
  • Since all customer records are downloaded to Sterling and Sterling performs the sampling (according to agreed-upon specifications), client employees cannot tamper with the guest list or with the questionnaires. This means the results reflect a more accurate picture of the client's customer viewpoint.

  • Mail is one of the most cost-effective ways to conduct this type of research since it allows the client to ask a relatively large number of questions within one easy-to-produce survey instrument.


Disadvantages of Mail Methodology
  • Mail takes more time to get results back.

  • Accurate addresses must be collected at some point in the relationship with the customer.

  • Probing guests for additional information (i.e., open-ends) can add to the cost of the survey since it requires additional processing time.


E-mail/Internet Data Collection

Email has emerged as an efficient and cost-effective form of survey research. Customers are invited to complete a survey either by email, mail or on-site instructions/invitations. These invitations encourage the customers to complete a survey by going to a website hosted on Sterling servers. In the case of email, a link to this site is included in the invitation. The website uses a password supplied in the invitation as an access device and limits each guest to one survey.

Internet surveys can use all of the good methodology techniques that are available in CATI systems such as question rotation and branching; further, they can incorporate graphics, charts, maps, photos and diagrams that enhance the data collection potential of the approach.

Since email is delivered quickly, and surveys can be completed conveniently, this approach tends to speed the data collection process. Further, since the respondent's keystrokes are recorded directly into the electronic database, rather than translated either by the data entry process (scanning) or by another individual (CATI), the data is collected more accurately and can be available to the client in real-time.

As computers become more ubiquitous in society, Sterling will urge its clients to adopt this methodology whenever possible.

Advantages of Email/Internet Methodology

  • It is cost-effective, especially for large sample

  • It allows the use of good methodology techniques

  • Tends to have fast turnaround and allows clients virtually instant access to their data

  • It is perceived as very convenient by respondents since surveys are available by a simple click, and can be completed at a moment deemed convenient

  • Works well on both long and short surveys


Disadvantages of Email/Internet Methodology
  • Email addresses must be collected consistently and in an-ongoing manner

  • Email addresses are relatively difficult to gather since characters tend to be more random than in simple names and addresses allowing for potential error

  • A large percentage of people do not have or do not readily give out email addresses, thereby potentially creating a biased sample


IVR (Integrated Voice Response)

You are familiar with IVR (Integrated Voice Response) systems, whether or not you are familiar with the terminology. IVR systems, in their most common form, are the telephone answering systems used by many businesses. These systems route you to the desired party through the use of touch-tone responses to recorded messages. IVR is a cost-effective alternative to traditional CATI (Computer Automated Telephone Interviewing) research.

In survey research, IVR can be used in various ways as a methodology for data collection.
  • Customers are sent a written invitation to call a toll-free number and complete a survey. As responses to the questions are provided, an automated process records the results of the keystrokes.

  • A telephone interviewer initiates the call to the customer and asks the respondent to take the survey. At that point, the interviewer passes the respondent to the automated survey. This approach is called "CATI assisted IVR".

  • A "fully automated dialout system" works much like the CATI assisted program, but in this case, the process does not use an interviewer to initiate the call. Instead, upon determining that the customer has answered the telephone, the automated voice does the introduction and proceeds into the survey, one question at a time.

  • A fully automated system can also be set up to work with call centers, such as hotel reservation call centers. At the completion of the call, the attendant forwards the phone call to Sterling's IVR system, which then administers a short survey. The survey responses can be traced back to attendants, which allows for monitoring of attendant effectiveness.


Advantages of IVR Methodology
  • Lower in cost than CATI system while still affording many of the methodological benefits

  • Allows for easy use of good research practices

  • Automation of voice messages assures consistent application of the survey


Disadvantages of IVR Methodology
  • Works best with very short surveys

  • May require the use of an incentive to gain participation

  • The automation factor may irritate the individual on the other end of the line



CATI (Computer Assisted Telephone Interviews)

Telephone survey research has over the years become one of the most commonly used means of collecting survey data. Today much of it is done by CATI systems (Computer Assisted Telephone Interviewing) in which the interviewer follows a script shown on the computer screen. As a result, skip patterns in the survey and item grid rotations can be easily incorporated into the survey research program. These techniques reflect good survey methodology and are unavailable with mail or on-site methodologies.

Using the telephone methodology, customer phone number lists are fed into a CATI system. Interviewers are assigned a customer, the computer dials the number automatically, and the survey is administered as a script printed on the screen. As the customer answers questions, the interviewer records the responses on the computer. Logic within the program then selects the next appropriate question to be read to the respondent.

A CATI-trained interviewer can easily and gracefully move from the survey to the explanation and back in a conversational manner. As a result, there is a good opportunity for respondent comprehension and hence better information, and less opportunity for data contamination as a result of "leading the respondent," accidentally, in the process of trying to explain a term or concept.

Advantages of CATI Methodology
  • Good survey research methodological techniques can be incorporated into the survey process

  • Quick turnaround

  • If telephone numbers are collected systematically from customers, it is easy to extract a random sample


Disadvantages of CATI Methodology
  • Telephone interviewing is relatively expensive (although IVR technology is a less expensive alternative).

  • Accurate telephone numbers must be gathered and recorded

  • Difficult to use with long surveys

  • Even with valid telephone numbers it is easy to be blocked from talking to an individual because of answering machines, caller-ID, etc.



Focus Groups

Focused group interviews, usually referred to as focus groups, are a popular technique to gather market intelligence. Focus groups are typically 10-12 people who have been screened to be of interest to the interviewer for some specific research task. For example, if a manufacturer of widgets is trying to expand sales into households that "could, but do not currently use widgets", the focus group might be composed of members of households that could, but do not currently use widgets.

These group interview sessions are typically conducted in special facilities constructed for this purpose. The facility is usually in a safe and convenient location with convenient parking. The room where the group interview is conducted often has a one-way mirror at one end, and a large table in the center where the respondents are seated. The one-way mirror connects with a soundproof viewing room where the client and the other researchers view and listen to the session. Often the sessions are video or audio taped for later analysis.

The sessions are conducted by a trained researcher who acts as the moderator of the session. Sessions often last one or two hours (sometimes more), and the respondents are recruited by offering payment for their participation. The payment incentive can be as small as merchandise, to as much as several hundred dollars in cash, depending on how exotic the recruitment specifications are.

These sessions are different from one-on-one in-depth interviews, in that the researchers want not only one person's opinion, but also the richness of a group of users/potential users interacting about the product or service. These interactions are particularly useful where new products or new concepts are being discussed, since each person's opinion is enriched by the discussion between respondents over differences of opinion. Sometimes the brainstorming that comes out of focus groups can lead to a vastly improved product or service, because it is the users who are defining the specifications.

Focus groups are not an end-all to market research. They have some very specific limitations. For example, even three focus groups on the same topic can provide at most 30-40 somewhat independent opinions about a subject. This is qualitative information, and not the kind of information that one uses to model market share or to make major financial decisions. For decisions of this nature, quantitative information is usually used-where the sample is scientifically drawn to represent the population of users and the samples are large enough to drive standard error of the sample small enough to be of comfort.

But focus groups are a powerful tool to do things like: test advertising concepts, test new product ideas, test political platforms, and the like. Sterling has trained moderators to assist its clients with properly executed focus groups, as well as trained project staff to take concepts and ideas into quantification studies after the concepts are developed.


In-Depth Interviews

Modern firms have need for all types of market research, and one type that is occasionally needed is in-depth personal interviewing. These interview sessions are usually one-on-one between interviewer and respondent, and can last from a few minutes to an hour or more.

While much interviewing is personal, that is one-on-one between an interviewer and respondent, in this brief description, we're describing a research technique that is more deliberate and more structured than an intercept interview with a passerby in a shopping mall. In-depth personal interviews usually have the following characteristics:
  • A very specific target market segment of consumers-often a small segment

  • Typically confidential and/or more complicated subject matter

  • Often of a subject matter that the respondent has expert knowledge

  • Might include use of a new product or service for which the interview is a debriefing


In-depth personal interviews are typically conducted at a central facility conveniently located with convenient parking. Respondents are recruited from user-lists or the general population and screened to meet specific criteria. The respondents are then invited to make an appointment for the interview in exchange for payment for their services. The incentive pay can range from twenty or thirty dollars to several hundred dollars or more for each respondent.

In-depth interviews are often just that-a long series of questions about a very specific topic, and in great depth. While these interviews are seldom quantitative in volume, they do provide extensive information about the topic.

Sterling Research can design in-depth interviewing research programs, and manage every aspect of the study. We have networks of personal interviewers throughout the nation, and are capable of interviewing on virtually every topic.


On-Site Distribution

There are a variety of ways to assess customer satisfaction on-site whether the establishment is a restaurant, a retailer, or even a service. One of the primary approaches to collecting data on-site is via paper surveys. Alternatively, invitations to respond to telephone or internet surveys can also be distributed on-site.

Paper questionnaires have an immediacy that is hard to rival. Unlike most other approaches, it puts the questions and the issues in front of the customer, often at the time of purchase. But this methodology also has serious drawbacks-most notably the ability of store personnel to filter the results. That is, store personnel can fail to give a questionnaire to someone who would be likely to give a bad report, or they could alternatively fill-out one or more questionnaires themselves. While Sterling doesn't generally recommend this approach, it still has some justification as noted below.

Paper questionnaires are very inexpensive to administer and to process. One of the largest costs is the business reply postage. Custom-made self-liquidators can also be pricey in small quantities but much less so for larger applications.

Self-liquidator take-one. Small cardboard dispensers positioned on or near checkouts with a supply of card-style survey forms can be made available for customers to take at their convenience. Usually the survey form has a business reply side, which allows it to be dropped in the mail for return and processing. Dispensers can be refilled for on-going programs, or they can be discarded after one use for one-time programs.

Clerk distributed. A supply of forms is provided to each clerk who provides one to each customer as the bill is paid. It is also possible to use this approach to sample from the population by only providing a survey card to every nth respondent. For example you can survey one in ten, by using a voucher number or transaction number as the basis, and then only distributing survey cards to numbers ending in the digit "2" or "7" or whatever you choose.

On-receipt. It is also reasonable to provide feedback information by printing it on the receipt. Naturally the space available is critically limited, so feedback is of a very limited nature. Much more often used is the on-receipt invitation to complete a survey using some other approach such as IVR (Integrated Voice Response) or Web. The on-receipt real estate lends itself well to this approach.

Advantages of On-Site Survey Distribution
  • The primary advantage of on-site distribution is low cost. However, while this approach is cost advantageous, it is also less likely to yield an accurate view of the customer experience.


Disadvantages of On-Site Survey Distribution
  • With on-site distribution, it is difficult to obtain a random sample of customers, which limits the generalizability of the survey results.

  • Clients who use on-site distribution rely heavily on lower level employees to distribute the surveys. Employees may be tempted to avoid handing out surveys to dissatisfied customers or only distribute surveys to those customers who are likely to give high ratings. Even worse, it is fairly easy for employees to "stuff the ballot box" with their own favorable responses. These efforts can undermine an entire survey program.