Ranking and scaling

Ranking and scaling techniques OR "Would you please rank these 50 factors in order of importance to yourself."

Spend a few minutes ranking the animals of the world in order of size:

Q         "What is the largest animal in the world?"
A         "Easy! - the whale."
Q         "What is the second largest animal in the world?"
A         "Also easy - a slightly smaller whale."
Q         "What is the third largest animal in the world?"
A         "This is getting boring! - a still slightly smaller whale."
Q         "Allright, let's look at the other end of the scale - What is the smallest animal in the world?"
A         "Not so easy! - maybe a shrew?"
Q         "What is the second smallest animal in the world?"
A         "Allright, I give up!…"

The above example illustrates what happens when a market researcher gives a respondent a choice of as few as two or three factors to rank in a known or assumed order. Beyond knowing the first and the last, the ordinary mortal is quite lost in-between and resorts to wild guessing in order to please the interviewer and not look too stupid.

We have come across market research questionnaires that put respondents through mental gymnastics of ranking anything up to 20 factors or attributes in order of importance. The average person has, in fact, great difficulty in ranking more than two attributes in any sort of order. To bring this point home with more force than may be necessary, consider the following, which affects each one of us in our work environment.

Rank the following 10 - factors in order of importance to you:

    • salary
    • status
    • pleasant working environment
    • company car
    • job security
    • a well appointed large office
    • seniority
    • office near to home
    • pleasant relationship with colleagues and superiors
    • future prospects

Were you able to rank these factors? - or did you have trouble deciding which one came first, which second and by the time you came to the third you were really guessing in order to finish the exercise. Professors Rink and Dunn state, " … in the complete rank order approach, the respondent may become confused and frustrated if he/she is asked to order, say, 20-items. A respondent's typical reaction in such a psychological state is to skip the question, if possible, or refuse to answer, or rank the most and least preferred items, ignoring the rest. "

The other problem with ranking is that although factor one may be followed by factor two, we still do not have a measure of how far the two are apart. Factor one may be far ahead of the next most important factor and factor three may come close behind, but all we have is a simple ordinal scale which tells us that one factor is more or less important than another.

Consider, therefore, the following other possibility of ranking factors, which introduces a scale against which the relative importance of each factor can be measured and does not involve the respondent in mental gymnastics.

"On a scale of 1 to 7 (or 1 to 5 or 1 to 10) where the lower figure indicates no importance at all and the upper figure the highest importance, give a score to each attribute in terms of its importance to you."

This involves no mental gymnastics of the respondent trying to remember his previous scores. He simply considers each attribute on its own and allocates a score that he thinks best expresses his feeling of importance. He could give more than one attribute the same score, indicating that they are of equal importance - and he is, therefore, not forced to place one ahead or behind another.

The numerical scale can also be verbalised by asking the respondent to allocate an appropriate phrase such as:

    • very important
    • fairly important
    • neither important nor unimportant
    • fairly unimportant
    • very unimportant

A workable but cumbersome way to rank the attributes could also be to show combinations of two attributes at a time printed on a card and to ask the respondent which attribute is the more important of the two. Clearly, if many factors are involved, the likely combinations become very numerous. 20-attributes produce 190-combinations.

Measurement scales are widely used and misused in marketing research, where the measurement of opinions is best accomplished by scaling. Remember, do not ever ask anyone to rank more than two attributes in order of importance because chances are that you will prove a shrew is larger than a whale.

Scaling of responses is an important part of measuring and understanding what consumers really mean and how important various factors are to them.

Consider the following questions:
Q         "Are you satisfied or dissatisfied with the after sales service?"
A         "Satisfied"
Q         "Are you likely or unlikely to use Squirt cleaning fluid?"
A         "Unlikely to use".

These questions allow the respondent only one of two possible replies - each being at opposite ends of a scale that could, in fact, have several positive and negative shades, ranging from:
                        Very satisfied
                        Fairly satisfied
                        Neither……
                        Fairly dissatisfied
                        Very dissatisfied

This allows the consumer expression in any one of five levels of satisfaction. The intensity of the respondent's feeling can, therefore, be measured. Although five point scales are quite commonly used, any number of expressions may be used allowing for more sensitive measurements. The use of odd numbered scale such as 1 to 5 and 1 to 7 are most common. Whereas, the 3 - and 5 – point scales may easily be converted into phrases or adjectives such as "extremely", "quite", "slightly", etc., 7-point scales are usually left in numerical form with the respondent simply knowing that 1 - signifies, say, "completely satisfied" and 7 - signifies "completely dissatisfied". He is then allowed to imagine what grades the other scores would signify between the two extremes he has been given.

The following model indicates how this works in determining levels of satisfaction for, say - after sales service:

                        Satisfied (1)  (2)  (3)  (4)  (5)  (6)  (7) Dissatisfied

The seven points indicate:

  • completely satisfied
  • quite satisfied
  • slightly satisfied
  • neither satisfied nor dissatisfied
  • slightly dissatisfied
  • quite dissatisfied
  • completely dissatisfied

The terms used are antonyms - having opposite meaning. Care must be taken to have exact opposites and not other terms that could be interpreted to mean something different. The terms can also be adjectives describing a product or service. A few examples are:

                        Sweet…………………………….bitter
                        High quality……………………..low quality
                        Inexpensive………………………expensive

            All of which are direct antonyms 
            And not:

                        Sweet………………………………sour
                        Long service life……………………low quality
                        Inexpensive…………………………overpriced
I have seen many incorrectly applied scales which have produced confused results.

When applying a scale, the market researcher must also be careful of the "halo effect" - which has nothing to do with sainthood. The halo effect occurs when all the negative terms for each attribute are placed on the same side of the scale – say, the right hand of the scale.  Studies have indicated that a respondent will become conditioned to placing his tick or cross in and around the same position for each set of terms. To avoid this, the market researcher must alternate the positive and negative poles:
                        Sweet…………………….bitter
                        Low quality………………high quality
                        Inexpensive………………expensive

When using descriptive terms we are dealing with semantics; consequently, the technique is called

Semantic differential scaling

There is considerable argument among researchers whether an odd or an even number of points should be used. Those against an odd number say that the natural inclination by the majority who have no opinion is to choose the mid point but with an even number of points the respondent is forced to give a positive or negative reply.

Those in favour of an even number argue that a respondent should not be forced into one or other opinion but should be allowed to take up whichever position he or she is comfortable with.

A model with an even number of 6 - points is shown below:

  Very Fairly Slightly Slightly Fairly Very  
wide
product
range
….….. ….….. ….….. ….….. ….….. ….….. limited
product
range
slow
delivery
….….. ….….. ….….. ….….. ….….. ….….. fast
delivery
high
delivery
….….. ….….. ….….. ….….. ….….. ….….. low
delivery

The problem of finding unambiguous opposite adjectives was discussed in the last Market Research Brief.  The problem can be avoided by using the LIKERT SCALE, Where the one end of the scale starts with a statement and the other end is left open; therefore, no problem in finding an antonym or opposite pole for the opening statement.  The respondent is simply asked to what extent he agrees with the opening statement and he is left to find an appropriate position for his level of agreement or disagreement.  The following model shows how this works:

Strongly agree-generally agree-neither agree nor disagree-generally disagree-strongly disagree 
wide product range .. ...................... ………………………… ………… ………. ………………..
product
slow delivery .….….. …………… …………………………. ………………….. ……………….
high quality ……….. …………… …………………………. ………………….. ………………..

A 5-point scale – but 6-points can also be applied by replacing “neither agree nor disagree” with “moderately agree” and “moderately disagree”.

Yet another scale may be used – called the STAPEL SCALE.  This carries certain advantages over Semantic and Likert Scales.  Advantage over the Semantic Scale is that polar adjectives are not needed and the advantage over the Likert Scale is that the statements are neutral, having neither a positive nor a negative meaning.

The following model shows the STAPEL SCALE:
+3                                                                    +3                                            +3
+2                                                                    +2                                            +2
+1                                                                    +1                                            +1
Product range                                                  delivery                                   quality
-1                                                                     -1                                             -1
-2                                                                     -2                                             -2
-3                                                                     -3                                             -3

The statement has neither positive nor negative connotations and will, therefore, not trigger a biased response in the mind of the respondent; furthermore, the problem of having to find polar statements is avoided.

The market researcher who is aware of different types of scaling techniques may be left wondering which one is the best to use.  Menzes and Elbert who studied the different types of scale as well as Luck found no clear differences in their studies but it would seem that there could be differences dependent on the education level of respondents as well as the type of industry being studied.

In the absence of any clear advantages, Luck points out the following effects of the different types of scale:

  • Leniency or tendency to give favourable answers, which they found to be reduced by Stapel method.
  • Precision or minimizing variations among respondents, which was strongest for the Semantic Differential.

Respondents’ preference among the three methods used:  Likert scale first, Semantic Differential second and Stapel scale third.

Leave a comment

Make sure you enter the (*) required information where indicated. HTML code is not allowed.