How to Measure Customer Satisfaction
Business Management SystemA lot of enterprise emphasis on customer satisfaction, and rightly so, as customer satisfaction is the key for improving
Business process management. A CSR(customer satisfaction rating) is often obtained through a questionnaire—the CSS(customer satisfaction survey). This method, however, suffers from the drawback of customers likely being emotionally influenced while filling out these questionnaires.
An expert on the topic of customer satisfaction, states in seminar Tales of Whoa and the Psychology of Customer Satisfaction: “People tend to rate service higher when delivered by people they like thanby people they don’t like.” Also goes on to describe what one can do to be “likable.” More often than not, the CSS rating received from the customer represents perceived feedback rather than impartial feedback.
This is not to say that enterprise do not get any value from customer-filled CSR forms. But they must recognize that responses can be emotionally based, and that the customer is not one person, but an organization—meaning multiple people.While so, only one person represents the organization and fills out the survey. Would this person consult all concerned before filling it out? Ideally, he or she should, but often, he or she will not.
This gives rise to the need for a way to compute a CSR based on internal
Business management system data—data that is free from bias and that gives a realistic metric on customer satisfaction.
Why Should We Measure Customer Satisfaction with Internal
Business process management Data?
Consider the following three scenarios:The customer is pragmatic and not swayed by influences like the recency factor and the one-incident factor, prejudices of any kind, poor judgment, or personal stake. This customer keeps meticulous records of the project execution and is expert at data analysis. While it may be rare to have such a customer, his scoring is likely a true reflection of the supplier’s performance.The customer is an average person. His scoring is influenced by some of the factors mentioned in the 1 scenario. Let us assume that he scores the suppliers’s performance as bad. If this low scoring (which is biased) were accepted, the personnel involved in the project execution would also receive low ratings in the organization as a result. They might, in turn, receive lower hikes (salary increases) and bonuses, if any at all. This would demotivate these workers, as it is possible that they in fact did a fairly good job and merit a better scoring.The customer is an average person. His scoring is influenced by some of the factors mentioned in the 1 scenario. Let us assume that he scores the vendor’s performance as high. As a result, the personnel involved in the project execution might receive better hikes and bonuses. Such a situation would further demotivate the personnel from the second scenario.
Scenarios 2 and 3 give rise to the phenomenon known as “rewarding the underperformers and punishing the better performers”—a disastrous situation for any organization. An impact even more disastrous is that the organization does not have a realistic picture of how satisfied its customers really are. In such a situation, any efforts to improve customer satisfaction would be taken in the wrong direction.
Aspects Critical to Customer Satisfaction
I have been using the following method to compute a customer satisfaction index, based on internal
business process management data, in all the organizations to which I have provided consulting services. I developed this system through reverse-engineering of the supplier-scoring index that enterprise use to score their suppliers. The method is based on the following criterion I believe are critical to customer satisfaction, which are tangible aspects that can be measured objectively.
1. Quality
Quality comes first. The dictum “customers forget the delays but not
the quality” aptly states the
value of quality. Furthermore, customers forget everything else if—and only if—
the quality delivered is superb.2. DeliveryNothing irritates a customer more than not receiving a delivery on the committed date. When a delivery is late, plans at the customer’s end have to be redrawn, resource allocation has to be shifted, and all subsequent actions have to be rescheduled, causing the customer a lot of unconvenience.
3. PriceThis refers to cost the customer is paying. It is not uncommon for escalation clauses to be built in to contracts. When the supplier chooses to apply an escalation clause and to bill more money, it greatly inconveniences the customer. The customer must obtain sanctions and approvals for the extra payout, as well as answer quite a few questions in the process. In short, price escalations irritate customers.
4. Customer feedback & Issue factorMost projects have issue resolution mechanisms (methods to solve problems). Some suppliers, in their eagerness to always interpret the specs accurately (and in their fear that they might in fact misinterpret specs), raise more issues.When valid issues are raised, the customer is usually more than happy to resolve them. But when the issues raised are trivial, the customer becomes annoyed.
5. Services, Accommodation & cooperationFew projects are ever completed without changes having been requested by the customer. When the customer requests a change, the supplier should accommodate and cooperate with the customer, and implement the change without postponing the delivery and without increasing the cost.
Quality ScoringNo project is ever perfect, and most times, failure may not be detectable immediately upon delivery. If failures are detected during the warranty period, the customer is happy. However, what is important is whether the failures fall into an acceptable range. Usually, the customer’s expectation is “zero defects,” but all professionals on quality know that “zero defects” is rarely achieved.
Sometimes, customers specify what is an acceptable defect percentage (number of failure per number of opportunities for defect); other times, the failure density is implicit. Customers select suppliers based on their certifications or market reputation. But reputation alone does not lend itself for measurement.
Using
6 sigma methodology, we can measure and specify the expected failures based on the “
sigma level” of the supplier organization.
If an enterprise is at
6 sigma level, then the expected failures from that enterprise total 3 defects for every million opportunities. If the enterprise is at 5-
sigma level, the expected failures total 3 failures for every 100,000 opportunities. At 4-
sigma level, 3 defects for every 10,000 opportunities. At 3-
sigma level, 3 defects for every 1,000 opportunities.
The expected no. of failures delivered should be contrasted against the actual no. of failures delivered.
Failures begin to be counted during the acceptance testing stage because they can be discovered by the customer just as they can be in pilot-runs, during live or production runs, throughout the warranty period, and afterward.
Normally, failures are classified as one of three categories: critical, major, and minor. I use only the critical and major defects, since minor defects can sometimes merely be a difference in perception—the customer may perceive as a defect what the supplier may not consider a defect.
The failures density is computed as defects per unit size, or conversely, as units of product per one defect. The size is usually measured as LOC(lines of code), (FPs)function points, or any other size measure used in the organization. What is important is to select 1 size and use it for all measurements.
Here is the formula to compute a QS(
quality scoring) for customer satisfaction:
QS = (actual defect density – accepted defect density) ÷ accepted defect density
If the actual defect density is less than the accepted defect density, then this index will be negative, meaning customer expectations have been exceeded. If the actual defect density is the same as the accepted defect density, then this metric will be zero—customer expectations have been fully met. If the actual defect density is more than the accepted defect density, then this metric will be positive, and it means customer expectations have not been fully met.
Delivery Scoring
Nothing is more frustrating than not receiving a delivery on an agreed-upon day. This frustration may be eased if somebody calls to tell you that the delivery is going to be delayed, but the frustration is there just the same. The funny part is, even if a delay is the result of a change that the customer requested, late delivery still frustrates the customer. It is as if the customer is thinking, “Can’t they accommodate this teeny-weeny change without postponing the delivery date?Suppliers always take any opportunity to delay delivery!”
Oftentimes, supplier prefer to compromise
on quality than to delay delivery. The philosophy is this: it will take some time for the customer to unearth the defect, but it takes no time for the customer to come down heavily if delivery is not on time. Excuses like “Sorry for the failure; here is the corrected action taken” or “In our fervent efforts to deliver on time, this failures crept in” can be quite convincing.
Customers might forget delayed deliveries, but they seldom forget bad quality. When asked for references, they normally highlight the quality a supplier provides over on-time delivery. That is the reason I place this aspect as 2 in importance when determining customer satisfaction.
To compute this index, we contrast accepted delivery with actual delivery. But which date should you use as the accepted delivery date? To compute the highest scoring possible, take the latest accepted delivery date. To derive a true customer satisfaction rating, then take the date that is on the purchase order. Some organizations use both—one for internal purposes and one for the external purposes.
The formula for computing a DS(delivery scoring) for customer satisfaction is as follows:
DS = (actual days taken for delivery – accepted days for delivery) ÷ accepted days for delivery
To determine actual days taken for delivery, use the number of calendar days between the date of the purchase order and the date on which delivery was actually delayed. To determine the accepted days for delivery, use the no. of calendar days between the date of the purchase order and the date of delivery specified on the purchase order.
If actual delivery was made before the accepted delivery date, then this index will be negative, meaning customer expectations have been exceeded. If actual delivery was made on the accepted delivery date, then this index will be zero—customer expectations have been fully met. If actual delivery was made later than the accepted delivery date, then this index will be positive, and it means customer expectations have not been fully met.
Price Scoring
Obviously, no supplier can bill the customer for an amount that was not agreed to by the customer—that is if the supplier expects his invoice to be respected in full and without issue. Why is this an important factor? Because sometimes contracts are drawn up using an hourly rate with a maximum amount, allowing some variance on either side. In such cases, the final billed amount can either be lower or higher than the specified amount.
When a cost escalation clause is implemented or an additional payment is requested against a change, some negotiating usually occurs before the customer accepts the escalation; the amount accepted might not be the same as requested by the supplier. The fact that extra price is being requested and the resultant negotiations can certainly frustrate the customer.
Whenever the customer has to pay more than the purchase order value, the customer is dissatisfied. Needless to say, the customer is certainly pleased when the supplier charges less price than the amount specified on the purchase order. To compute the PS(price scoring), use the price agreed to (before taxes) on the original purchase order and the final amount billed.
Here is the formula for computing customer satisfaction in this area:
PS = (actual amount billed – price on the purchase order) ÷ price on the purchase order
If the actual amount billed was less than the purchase order price, then this metric will be negative, meaning customer expectations have been exceeded. If the actual amount billed was equal to the purchase order price, then this metric will be zero—customer expectations have been fully met. If the actual amount billed was more than the purchase order price, then this metric will be positive, and it means customer expectations have not been fully met.
Customer complain & Issue Scoring
Issues crop up during project execution mainly because of unclear specifications or a lack of understanding the specs.Issues may also occur because of a conflict or an nonconformance in the requirements.
When the supplier raises an issue whose origin is attributable to the customer, the customer’s satisfaction is not usually affected. However, the customer’s satisfaction does become affected if the issues raised are due to the vendor’s improper understanding of the requirements. Customers expect any shortfall in exhaustive requirements specifications to be bridged by the supplier. Failure to meet these expectations cause dissatisfaction in customers.
To compute an (IS)issue scoring, use the (ID)issue density. While we can easily compute actual ID, there is no accepted measure for an acceptable ID. We also use software size for computing ID. While issues can directly relate to requirements,we cannot use the number of requirements, as the method for defining requirements can vary the number significantly.
Thus, the ID is computed as follows:
ID = number of issues raised ÷ Project size
Project size can be any project size measure, Since there is no universally acceptable ID, anorganizational standard should be defined and continuously improved.
The formula for computing IR for customer satisfaction is as follows:
IS = (actual ID – standard ID) ÷ standard ID
If the actual ID was less than the standard ID, then this metric will be negative, meaning customer expectations have been exceeded. If the actual ID was the same than the standard ID, then this metric will be zero—customer expectations have been fully met. If the actual ID was more than the standard ID, then this metric will be positive, and it means customer expectations have not been fully met.
Cooperation Rating
Most new project would not be complete without a few change requests from the customer.But since change requests are commonly implemented before delivery, how then do they give rise to customer dissatisfaction?
Change requests cause additional work for the supplier, and their impact is felt in 2 ways: revised delivery schedule and higher cost. In some cases, the supplier absorbs both, and in others, the supplier absorbs the impact on price only and passes the impact on delivery schedule on to the customer. Still in other cases, the vendor absorbs the impact on delivery schedule and passes the impact on price on to the customer. In the remaining cases, the changes are rejected.
Of course, the customer is happy when change requests are accepted without impacting the price or the delivery schedule.But since this does not always happen, that is why we compute a cooperation rating (CR), the formula of which is the following:
CR = (no. of change requests received – no. of change requests implemented without affecting delivery date or price) ÷ no. of change requests received
If the number of change requests received were the same as the number of change requests implemented without affecting either delivery schedule or price, then this metric will be zero, meaning customer expectations have been fully met.If the number of change requests received were greater than the number of change requests implemented without affecting either delivery schedule or price, then this metric will be positive, and it means customer expectations have not been fully met.
There is no way to exceed customer expectations in this scoring.
Composite Customer Satisfaction Scoring
Having computed all five scorings critical to achieving customer satisfaction, we are ready to CCSS(compute the composite customer satisfaction Scoring).
Obviously, all 5 scorings do not carry the same importance in achieving customer satisfaction. These scorings can also vary from organization to organization, and from customer to customer. Some customers may perceive quality as being the most important aspect of a product or a service, while some may perceive delivery as the most important aspect. Still for others, the highest of importance might be placed on price. Given these differences in customers’ perceptions and preferences, it is necessary to assign weights to each of the 5 scorings in order to arrive at a reasonably accurate CCSS.
The sum of all the weights must equal 1.00 in order to calculate a meaningful CCSS. Table below shows an example of how weights can be distributed.
Serial Number Scoring Weight1
quality Scoring (QS) w1 = 0.30 2 delivery Scoring (DS) w2 = 0.30 3 price Scoring (PS) w3 = 0.30 4 issue Scoring (IS) w4 = 0.05 5 cooperation Scoring (CS) w5 = 0.05 Total Weight: 1.00
The formula to compute CCSS is this:
CCSS = 5 – (QS*w1 + DS*w2 + PS*w3 + IS*w4 + CS*w5)
This formula gives the CCSS on a 5-point scale. It is possible for the CCSS to be greater than 5 in some cases. When this happens, it means that customer expectations have been exceeded.
Use of CCSS
While I do not advocate doing away with CSSS altogether (ultimately, what the customer perceives is also important), consider these facts: Only one person in a customer organization fills out CSSS, despite the fact that many people in the organization may use the product. This one person’s expectations can be managed, making it possible to calculate an accurate scoring. But the other users (some of whom could be decision makers) can certainly still unearth the defects in the product.
This is to say that perception-based scorings alone cannot be relied upon. Contrasting CSS scorings with CCSS allows organizations to improve their
Business process management.
Suppose that the internal CCSS is in agreement with a CSS scoring. This means that the customer’s perception is in sync with reality, and that customer expectations are being managed as they should be. The organization’s strengths are equal in service and expectation management, giving a realistic picture to management. In this case, the organization needs to take corrective action based on the scorng should it be bad.
Suppose that the internal CCSS is way below the CSS rating. This means that the customer’s perception of an organization’s service is better than the service is in reality. This is not of any benefit to the organization, because if it continues to praise itself based on the customer’s perception that its level of service is high, then the organization will head toward decay. Resources will continue to place emphasis on expectation management rather than on service, thus never improving services. In this case,
resources need to be trained in order to improve service.
Now suppose that the internal CCSS is way above the CSS scoring. In such a case, the customer’s perception of an organization’s service is poorer than the service is in reality. This shows that the organization is concentrating on service without any concern for expectation management. Interpersonal relations and communication with the customer are being neglected. Here, resources need to be trained in expectation management.
There is scope in the CCSS method for organization-based adaptation. Some of the 5 scorings may be dropped or substituted, or new ones may be added to suit the specific organization.
