5.2.3 How a rating system works?

There are 4 steps that you should follow in the implementation of the Point Rating system in the company:

  1. Establish scaling or weighting methods. Scaling is the process of measuring or ordering the quantitative features of an object. For example, a scaling technique might involve the assessing and measuring the quality of products
  2. Obtain information
  3. Carry out the comparisons
  4. Produce the evaluation results, which can be used to make a decision

5.2.3.1 Scaling and Weighting

The features of a product, service or processes can be measured by using either a scaling or weighting method.

Scaling Methods

A) Comparative scaling

Using this technique every product/service/process is directly compared to the other.
Examples:

  • What do you like better? Wine or Beer?
  • Rate the TV advertisements on innovations from 1 to 10
  • If you had 5000 EUR to spend on ICT tools in the company, how much would you spend on hardware, on software, on hiring personnel, on NPD (New Production Development) activities etc?

B) Non-comparative scaling

Using this technique every product/service/process is scaled independently of the others.
Example:

  • How do you feel about product quality in the company?

C) Composite measures of variables

Using this technique two or more indicators of the product/service/process are combined into a single one.

Weighting Methods

With weighting scaling specific attributes or a product/services/process are weighted depending on their importance.  In this category two measures can be used:

  • Scales are used for measuring or ordering entities with respect to quantitative attributes or traits
  • indexes - they are similar to scales except that multiple indicators of a variable are combined into a single measure.1  


5.2.3.2 Obtain information (data collection)

Several methods for collecting data are used:

  • Questionnaires

    A questionnaire is a self-report data collection instrument that is filled out by research participants. Questionnaires are usually paper-and-pencil instruments, but they can also be placed on the web for participants to go to and fill out.2

  • Interviews

    Example:
    Interviewees are asked to indicate the level of agreement or disagreement (from strongly agree to strongly disagree) on a five to seven-point scale – refer to Table 1: Example of a questionnaire.

  •  Observations (i.e., looking at what people actually do)
  • Existing or Secondary data (i.e., using data that are originally collected and then archived or any other kind of "data" that was simply left behind at an earlier time for some other purpose) etc.

All methods have strengths and weaknesses which should be taken into consideration.

5.2.3.3 Evaluation of results

When the scaling and data collection have been complete, you need to perform the evaluation of the products/services/processes in question. For this we should have a reference object/process on which to base the evaluation of the features of the process/object of interest.

5.2.3.4 Carrying out comparison

Evaluation can be subdivided into categories, such as: market research, product certification, materials testing, quality management etc.

The evaluation of the results should lead to a decision. However, bear in mind that:

  • evaluation activities should meet the information needs of decision makers
  • the purpose of evaluations is to provide feedback to decision makers about how the object/process works/operates and its effectiveness.


1 http://en.wikipedia.org/wiki/Index_%28economics%29
2 www.analytictech.com/mb313/principl.htm