Day Thirteen- Why customer satisfaction surveys are a waste of time and money

So I love a good survey as much as the next girl, but one thing that drives me crazy as this:

Or: “You can go online and fill out a customer satisfaction survey and you might win a chance to win a gift card…but please give me all 5’s”

Why are customer satisfaction scores a waste of time and money?

Customer satisfaction is meaningless

It does you absolutely no good to have satisfied customers if they are not loyal customers. Satisfied means meeting basic requirements, it has nothing to do with having a great experience. If you walk into a store, find what you need, pay the price you expected, and leave the store you will probably be satisfied with your experience, but nothing about that experience was special. Your goal should not be to satisfy your customers, but to create loyal brand evangelist who will talk, tweet, and facebook about you. Tracking what people are saying about you via social media, and understanding why you are getting calls to customer service are far better ways to understand the strengths and weaknesses of your brand than a satisfaction survey.

Satisfaction is subjective

I have conducted hundreds of online surveys and can guarantee that abstract terms like satisfaction are very difficult for participants to respond to. I have also found that there are a lot of philosophies about scoring among survey takers. I have observed three trends:

All or nothing- A participant will only give ratings of 1’s or 5’s (on a 5 point scale). They essentially treat a rating scale like a pass/fail option.

Middle of the road- A participant will give every question the ranking of a 3 (on a 5 point scale). Sometimes this happens because the participant is indifferent; sometimes it happens because they are racing to finish the survey. [Technical Note: If you are having problems with the later your survey is probably too long, but you could also try to include randomly placed questions that ask the participant to select a specific score. If they fail to do so you screen them out.]

Less than perfect- A participant will give almost every question a 4 (on a 5 point scale). This is the biggest reason that most companies mess up survey data. Imagine you have the experience I described earlier- typical normal experience. A lot of participants will not give that typical normal experience a score of 5 because nothing surprised or amazed them. The experience was “fine.” The problem happens with analysis of these scores and what they mean for making business decisions.

Gaming the system

Occasionally managers or employees will do things that affect who will talk your survey and how they will respond.

 “Deselecting”- In a lot of systems the employee can manually override who gets a survey. When I was a waitress at the famed T.G.I. Friday’s (I know there has to be a whole book there), survey prompts were randomly generated. However if you had to reprint a receipt the second receipt would not have the survey prompt. I knew servers who would reprint receipts so that more difficult customers wouldn’t give bad reviews, but I also saw servers reprint receipts multiple times to get a survey offer for an awesome guest.

Coaching- Once in a while you will run in to situations where employees have been coached to ask you for the score they want. In If Disney Ran your Hospital, Fred Lee gives the example of a hospital that gives patients cards when they check in explaining that they will be given a survey when they leave and they should seriously consider rating the staff and experience a “5” on all questions. The card goes on to explain that if you have any issues during your stay you should let someone know so they can fix it…but don’t give us a lower score because of it.  So coaching in general is going to give you flawed data, but the hospital example introduces another point…let us know so we can fix it so you don’t give us a lower score. While this may have been written innocently, it just isn’t how people operate. Most people tend to be non-confrontational to start with, and will probably not go out of their way to complain about something minor. This is especially true if they feel their future service will be compromised by offering up ways to improve. Finally asking customers to complain is only really useful if the customer service staff has the tools to do right by the customer. So while asking customers to tell you how you could improve their experience is okay, asking for specific scores on surveys is completely unacceptable.

Dealing with Data

One of the biggest reasons that customer satisfaction surveys are meaningless is because most people are not collecting the correct data, or they don’t know what to do with the data once they collect is.  To do proper analysis you need to understand what your goals are, and in most cases I’ve seen analysis goes a little something like this:

Top Scorers: Companies will identify the individual stores with the highest scores and give managers bonuses based on their score. This is garbage…especially if employees are asking for good scores. You are motivating your top stores to stay on top while doing little to improve your lower stores scores. Also because you are only collecting quantitative data, you will not have a clear idea of what is actually going wrong in your less successful stores.

Pass Fail: Companies will create passing and failing grades- if you score a one or two you fail, if you score a 3-5 (on a five point scale) you pass. So first of all…if you are going to analyze data in this manner you might as well be asking pass fail questions. Also if you think about the different ways people take surveys the different between a 3 and a 5 really is a big deal. In most cases a 5 is extraordinary where a 3 is “meh.” If you are analyzing data this way you are essentially saying that there is no difference between and A+ and a C-.

[as I was getting ready to publish this post, I received a customer satisfaction survey from the AMAZING Felix Hotel in Chicago. They did an amazing job of creating and easy to use and fun to take survey. They utilized a 3 point scale so they can quickly see what is working and what isn’t.]

So what can you do?

Evaluate your current customer satisfaction tools and analysis process- what are your strategic goals? Are these surveys helping you make informed business decisions? You can start to transform your surveys by adding space for comments, but know that asking for comments will mean more work during the analysis process. You can also consider breaking from a rating scale and asking things like: Was your service better that you could expect from a similar store? Could you see yourself telling a friend about the experience you had today? However, transforming your survey may not be enough.

Find evangelist and nay-sayers- these are the people who are tweeting, blogging, facebooking, and linking-in about your company and offering glowing reviews or complaining about how awful you are. Consider treating web based customer service as a real department or position. Instead of waiting for your customer to call you…probably to complain, activate some interns and smart marketers to seek out mentions of your brand and react accordingly. Also look for mentions of your competitors or your industry in general. Find authentic and meaningful ways to respond to micro feedback. For the most part I am not talking about direct marketing; I am talking about building customer loyalty. One retweet from a key influencer can be just as valuable as an ad view.

Do qualitative research- while this option is more expensive, it is also more meaningful. Create a screener to find customers that have had good and bad experiences and do phone interviews with them. Ask them what they loved and hated about your store, and what you could do to be better. Talking to the right 20 people for 30 minutes can provide you with so much more value that looking at meaningless 3’s, 4’s, and 5’s from 1,000’s of customers.

 I have a feeling that customer satisfaction surveys will not be going anywhere anytime soon. They are fairly easy, fast, and not terribly expensive. However, I urge you to consider alternative methods to understanding and reacting to customer’s experiences. Creating a hybrid approach by mixing quantitative and qualitative methodologies can truly help you understand loyalty and not just satisfaction.

Have you tried a hybrid approach to understanding customer loyalty? Have any best practices to share?