Multichannel Mystery Shopping,
and Post Transaction Surveys
A $22-billion asset bank hired Kinēsis to build an integrated customer experience monitor consisting of both mystery shops and post-transaction surveys. The post-transaction survey program started over a decade ago with telephone data collection, and eventually migrated to the more efficient E-Mail methodology three years ago. At this transition, significance testing was performed comparing the phone surveys to E-Mail. No significant differences existed. Kinēsis receives sample on a weekly basis from the client. The less frequent sample distribution was the result of the survey containing a fulfillment piece, where we ask if certain material, such as checks and debit cards, had been received by a certain date. In this case, the client was willing to allow more time to pass to allow for an evaluation of the fulfillment process. As part of the program, Kinēsis built a customized post-survey follow-up tool allowing the client to review surveys which had been flagged for follow up based on a reported problem with the customer. This tool allowed management to review the problem the customer experienced, assign action on the issue by the appropriate branch personnel, and track the issue to conclusion.
In 2015 the client expanded Kinēsis’ the program of mystery shopping, post transaction e-mail surveys, to include a receipt VOC comment survey, with plans to add additional research methods.
A $5-billion asset bank with 50 branches contracted with Kinēsis to conduct a post-transaction survey across three customer segments (teller, new accounts and loans). Kinēsis designed a customized survey instrument based on the brands cores values:
- Excellence in Action,
- Partnership in Action, and
- Service in Action.
The analysis plan consisted of content analysis of open ends, Pearson correlation of each attribute to the likelihood of referral, and gap analysis comparing performance of individual service attributes relative to the above referenced Pearson correlation, providing a frame of reference for prioritizing which areas require attention and resources. Additionally, each branch was stack ranked against each other based on their net satisfaction ranking.
In-Store and Contact Center
A provider of short-term loans needed to reinvigorate a customer experience management program consisting of both in-person and phone mystery shopping. Prior to 2004 the mystery shopping program had not been well run from the prior vendor, and issues with quality had resulted in an erosion in its credibility. Kinēsis was hired in 2004 to restore the quality of the program. We conducted a top to bottom review of the entire program, proposed changes to each in terms of the quality and nature of the data collected, and proposed a new analysis plan based on purchase intent, where Kinēsis identified specific behaviors which had strong positive relationships with purchase intent.
The program has resulted in a 20% increase in mean purchase intent ratings as a result of the customer experience, and shoppers reporting negative purchase intent as a result of the experience have decreased by a factor of four.
Regional Bank Mystery Shopping
A $10-billion asset bank hired Kinēsis primarily to perform mystery shopping, however, at several times in the past, Kinēsis fielded waves of customer satisfaction surveys.
The objectives of the surveys included a measurement of overall branch satisfaction as well as measurements of 24 attributes rolled up into a 5-dimensional model of the customer experience comprised of the following dimensions: tangibles, reliability, responsiveness, assurance and empathy. Part of the analysis plan for this survey is a technique called gap analysis. Gap analysis is an analysis technique that compares satisfaction with individual service attributes relative to their importance, to provide a frame of reference for prioritizing which areas require attention and resources. Kinēsis identified for the client clear action plans for improving the customer experience overall, as well as specific improvement opportunities among specific demographic segments.
The mystery shop program has resulted in a 25% increase in mean purchase intent ratings as a result of the customer experience.
Casino Resort Mystery Shopping
Upon finishing an exhaustive brand definition process, a casino resort wanted to align guest service to its brand personality. Kinēsis was hired to help the casino identify guest service behaviors which will animate the brand, and design a measurement process to monitor and motivate these desired behaviors.
The first part of our engagement was to map guest service behaviors to the brand personality. The client described their brand personality in terms of a set of adjectives, specifically: enthusiasm, energetic, polite, professional, and positive tone. Together we defined the guest experience in terms of six expectations: dress; welcome; promotion of play; use of name; listening, responding and acting on guest needs; and thanking with a positive parting comment. We then defined these expectations relative to the brand adjectives as well as defining a set of behaviors to animate the brand.
Next, these behaviors and brand adjectives were mapped across several components of the guest experience, including:
- Valet Service
- Table Games
- Cocktail Service
- Cash Cage
Restaurants (6 different restaurants)
- Pro Shop
To make the research actionable, guest return intent (a dependent variable) was measured with each shop, allowing Kinēsis to correlate each behavior to return intent. As a result, we were able to identify the relative importance of each behavior in terms of driving return intent. Using this measure of importance as an input into Kinēsis’ gap analysis technique, we identified the behaviors which had the most potential for return on investment (ROI) in terms of driving guest loyalty.
As a result of this analysis and constant reinforcement of these high-ROI behaviors, the average return intent of guests increased by 25% over a period of two years.
Restaurant Mystery Shopping
A restaurant management consultant with a diverse roster of clients wanted to offer them mystery shopping of their properties. They needed a mystery shop provider who could both execute high quality shops, but also provide a high degree of customization from client-to-client without adding significantly to the program cost.
This cooperative model included construction of a base questionnaire to be shared by all properties, as well as, the ability to customize portions of the questionnaire to each operator. Additionally, separate evaluation criteria were developed for different experiences: breakfast, lunch, bar and dinner – allowing for customization across all possible experiences.
Research without a clear call to action may be interesting but not very valuable. Kinēsis builds call to action elements into every program. For this particular client we elected to build in a mechanism to determine the importance of each service behavior in terms of driving guest return intent, and identify which behaviors will yield the most ROI in terms of driving return intent.
As a result of the program guest return intent improved 22% over 5 years.
Casino Resort Guest Surveys
A casino wanted to measure guest satisfaction with a recent visit to their property. Fortunately, they had a robust loyalty program and also collected email addresses of overnight guests. Kinēsis designed an email survey program where daily the client would send email addresses of guests who had visited the property on the previous day. Kinēsis cleaned these daily files removing any duplicates, any guests who had opted out of future surveys, and against blackout rules to avoid contacting guests who had recently been sent a previous survey invite.
The survey contained specific modules based on the type of experience. However, in general the survey captured guest impressions of:
- Cleanliness of the casino environment
- Friendliness of employees
- Willingness of employees to help guests
- Quality of food served
- Quality of beverage served
In addition to these independent variables, Kinēsis also captured a variety of dependent variables, such as:
- Felt Welcome
- Overall Satisfaction
- Likelihood of Referral
- Return Intent
These dependent variables not only provided a quick overview of guest satisfaction, but also assisted in making the research actionable by correlating the relationship of specific experience attributes to likelihood of referral and return intent. This analysis provided clear direction in terms of identifying the service attributes in which investments will yield the most return on investment.
Finally, unaided top-of-mind questions displayed by logic rules based on the guests survey responses captured unaided top-of-mind drivers of both satisfaction and dissatisfaction.
Grocery Trade Association
A grocery trade association made up of hundreds of members, ranging from sole proprietors with one store to larger corporations with 20 or more stores, wanted to design a mystery shop program that would allow for members with diverse needs to measure the customer experience. They wanted a mystery shop program that allowed for customization yet at the same time was affordable.
The solution: Kinēsis designed a mystery shop program made up of a number of independent modules based on the following departments:
- Overall Store Appearance
- Promotional Item Availability
- Meat Seafood
- Health and Wellness
An enrollment process was put together allowing members to select modules from the list above that were appropriate for each store. Members could select the number of shops and customized shopping frequencies for each store.
Kinēsis programmed a system to shop each store with a customized questionnaire based on the enrollment process, including customized shop counts. Allowing an almost unlimited number of variations on a store-by-store basis – in total there were over 1,000 different possible permutations.
The end result was an efficient program with a high degree of customization across hundreds of clients.
In Store and Online Grocery
A Seattle-based grocery with both in-store and online delivery channels wanted to design a mystery shop program to measure the customer experience across both channels. Kinēsis designed a program to evaluate both the in-store and online channel.
In the stores, Kinēsis drafted an audit form which investigated service attributes within five dimensions:
- Store Enviornment
- Product Availability
With respect to the online channels, Kinēsis captured measurements with respect the following six dimensions:
As a result of the program guest return intent improved 20% over 4 years.
Competitive Sales Process
A financial services client wanted to evaluate the quality and effectiveness of select competitors’ sales processes. Not only did they want to know what competitors’ were doing, but also, and perhaps more importantly, to understand what sales practices were more effective than others.
When presented with this research need Kinēsis envisioned both a wide-ranging, closed-ended questionnaire designed to capture just about any occurrence in a consistent format, as well as strategically used open-ended questions to capture the details of any possible sales presentation. In order to determine the effectiveness of the sales behaviors observed, Kinēsis used a purchase intent rating as a dependent variable from which the relative strength of the sales attribute in terms of driving purchase intent could be determined.
Six competitors were identified as targets of this study, and 10 shops were scheduled to each competitor.
The questionnaire collected observations of the following areas of investigation:
- Probing of Needs
- Questions of financial needs
- Questions of balance history
- Questions of what is important to the customer in terms of a financial service provider
- Questions of the purpose of the financial products or services
- Closing the Sale
- Follow Through
- Overall Experience Ratings
Among the conclusions of this research was that purchase intent is driven by personalized service, making potential customers feel valued by giving them personal attention, and sincerely having their best interests in mind. Do not ask for the business without first earning the right to do so, by expressing sincere personal attention.
Ask questions designed to find what is important to customer. Among these high gain questions are:
- What are you looking for in a banking relationship?
- What do you like about your bank?
- What don’t you like about your bank?
Research has determined the business attribute with the highest correlation to profitability is customer loyalty. Customer loyalty lowers sales and acquisition costs per customer by amortizing these costs across a longer lifetime – leading to some extraordinary financial results. Depending on the industry, a small increase in customer loyalty (5%) translates into a 25% - 85% increase in profits.