Reply “Please” to Unsubscribe?

Last week, we wrote a blog post discussing SMS signatures and doing some analysis to see how common they are.  This week, we decided to continue with the theme of text message processing and analyze opt-out requests.

Background
According to the Mobile Marketing Association’s best practices, text messaging providers are required to look for special opt-out keywords and automatically unsubscribe users who text them in.  The published list of opt-out words are STOP, END, CANCEL, UNSUBSCRIBE or QUIT.  Our customers are all required to include opt-out instructions (e.g. “reply STOP to unsubscribe”) in their first interaction with end-users and encouraged to include instructions on subsequent messaging whenever possible.

Perhaps due to the conversational nature of many of our customers’ programs, people often don’t realize that their messages are initially processed by an algorithm. As such, many people do not reply with carrier-recommended opt-out keywords, but rather ask to opt-out in plain English (“Please remove me from your list. Thanks!”).

Early on, we realized that for the best possible user experience, we needed a way for organizations to opt-out users who send in plain English responses.  The Mobile Commons platform provides an SMS Inbox that our customers can use to view all their incoming messages.  We added an “Opt Out” button to the Inbox, allowing our customers to read incoming messages and opt-out users as needed.

Over the past two years, we’ve accumulated a ton of data about users’ mobile behavior.  Specifically, by analyzing the list of manual opt-outs, we are able to determine the most common words and phrases that people use to opt-out that aren’t recommended by the carriers.

Experiment
We pulled 50,000 random, anonymous  text messages from our database from the past year that included both automated and manual opt-outs.  We then did a frequency analysis of the bodies of the messages to determine which words and phrases were the most common.

Results
As expected, STOP by itself is by far the most prevalent opt-out word, accounting for 80.4% of all opt-outs.  The word STOP followed by more text (SMS signatures or an entire phrase) was the second most popular, with 9.3% of all opt-outs.  In total, incoming messages starting with the word STOP represented 89.7% of all opt-outs!

The requisite carrier opt-out words rank as follows:

• STOP: 89.7%
• CANCEL: 1.0%
• END: 0.8%
• UNSUBSCRIBE: 0.7%
• QUIT: 0.1%

92% is obviously a huge percentage and almost all opt-outs can be captured just by checking for these words at the beginning of a message!

This still leaves about 8% of all opt-out requests going unprocessed.  For systems like ours that process millions of text messages, 8% represents a huge number of people who would like to stop receiving text messages but are confused about the exact procedure to follow.  At Mobile Commons, we strongly believe that mobile is a highly valued communication medium and we should go the extra mile to make the best possible end-user experience.  Keeping mobile communication spam-free and nuisance-free is critical to our success!

As with all good algorithm design, we began by analyzing the thousands of messages in our database.  This would allow us to write regression tests using real-life messages.  We’d like to share some of our findings with you.

Much to our surprise, the word or phrase that started the majority of the non-standard opt-outs was PLEASE, as in “please stop”, “please remove me”, or “please only send emails. texts cost money.”  It’s nice to know people out there are still polite!

The following table represents the percentages of the most common words that are not part of the Mobile Marketing Association required list but resulted in an opt-out:

• PLEASE: 21.7%
• STOP2: 8.3%
• REMOVE: 7.9%
• F*CK: 4.6%
• TAKE ME OFF: 2.4%
• DO NOT: 2.9%

Just to be clear, we do not mean to imply that every message beginning with a “please” should be an opt-out.  In fact, the majority of them are not opt-outs. It means that of the opt-out messages that do not use standard opt-out words, 21.7% of them began with the word “please”.

The word STOP2 was somewhat surprising, but after further investigation, we realized it was a very common response to the often used phrase: “Rply STOP 2 opt-out.”

Conclusion:
Over the past year, we’ve worked very hard on our message processing algorithms.  The algorithms often go unnoticed by end users and customers – after all, it’s pretty tough to see an algorithm =)

We have a number of goals with our message processing:

• Correctly identify the incoming messages, whether they be opt-ins, opt-outs, help requests, replies, phone call requests, search queries, donation, and more
• Automate as much as possible, reducing the amount of extra work and manual intervention our customers need to do to keep their campaigns running smoothly & their lists clean
• Keep the number of false positives as low as possible (accidentally opting-out a user is a terrible thing to do!)

Our engineers have done a great job of all three, including processing opt-outs.  We automatically catch an extremely high percentage of them correctly with almost no false positives.  This makes for a great user experience:  a “Please take me off your list” text message will be followed by an immediate “You have been removed. Thanks for your support” response (as opposed to the “Unknown keyword. Try again.” response you often find on shared short code services).

Of course, our job is never complete. We continue to update our algorithms as we learn more and more about user behavior and our natural language processing improves.  We’ll keep you updated on our blog!