Improving conversational interfaces with data
Welcome back to What’s NEXT, the podcast exploring the future of technology. In this episode, I talk with Dashbot CEO and co-founder Arte Merritt about how his company gives clients a better understanding of user behavior and engagement with actionable bot analytics.
Ryan Lawler: Arte, tell me, what is Dashbot?
Arte Merritt: Dashbot’s an analytics platform for conversational interfaces and when we say that, we really mean anything that could be more text-based like a Facebook chatbot, or web-based chatbot, or voice like Alexa or Google Home.
Ryan Lawler: What was the origin of the company? Why did you get started?
Arte Merritt: My background is primarily in mobile analytics, in fact, I had a mobile analytics company that sold to Nokia. So a couple of years ago, I was playing around with Slack and realized that analytics were missing, and I was curious what Jesse was up to. I thought this could be a neat idea to work on, so I reached out to him just to see what he was up to and if he wanted to grab a coffee. So I met up with him and I said, “Hey, what do you think of this idea for analytics for Slack bots, Facebook type bots?” And then he started to smile because come to find out, he and Dennis were working on a Slack chatbot, and one of their to-dos was find an analytics platform. They knew it was missing. So we just started building the thing and a few weeks later, we launched the first version.
Ryan Lawler: So you mentioned Jesse and Dennis, your co-founders, what’s your background with them? How did you get to know them?
Arte Merritt: I went to MIT with Jesse. We were actually roommates out here for a while, so I’ve known him well over 20 years because, do the math, getting close to 25 years. So that’s how we knew each other. And then he and Dennis were working on this Slack bot together. And Jesse’s background is he had a social gaming company that he sold to the Game Show Network. And then Dennis went to Cornell and had a vintage marketplace that he sold to eBay. So all three of us had some success in the past, so that helped too.
Ryan Lawler: Right. So coming from the mobile analytics world, what was unique about conversational interfaces, and what got you interested in that?
Arte Merritt: For starters, the data is much richer and more actionable than either web or mobile. And so that’s what really got, I think, all three of us excited. It’s all unstructured data. Users can say or send whatever they want to these devices. They may not respond, but you can do whatever you like. So that ends up being quite a bit more actionable. If you had one more user in your mobile app, or your website, knowing that in real-time, would that matter? Probably not, but in a conversation, if someone was stuck or frustrated in the middle of chatting with your chatbot if you can identify that in real time, you can actually do something about it. Maybe you do want to put a live person in to help lead them through to conversions. So that’s part of it, yeah.
Ryan Lawler: How big is this conversational interface market right now? And how has it changed just over the past few years?
Arte Merritt: So the interesting thing is it’s all relatively new. I think that’s one of the things that actually helped up. Facebook opened up in April of 2016, and that’s, basically, when we started, so it’s only a few years old, at least these kind of chatbots. So initially, there was a lot of interest, and maybe hype is the better word in Facebook chatbots, so that was for the first year or so. And the beginning of 2017, we started to see more interest in voice and then that was really popular through the middle of 2017 into ’18. And what we see now is, well, there are folks still doing those, and I think voice is the future. There’s some things that need to be put in place for brands to monetize in that space. But where the interest really is right now is in customer service chatbots, like web-based customer service chatbots. So those kinds of things, especially with live agent on the websites, those have been around for a while, it’s just for brands and enterprises, that’s where we see a lot of opportunity.
Ryan Lawler: I think one of the things that most interesting about your business is, it doesn’t matter whether it is a Facebook bot, or whether it’s Slack integration, or whether it’s some voice interface, you work with all of them.
Arte Merritt: Yeah, definitely. We’re Switzerland in this one, so we work with any platform and any conversational interface. So you do see voice, Alexa, Google Home related things like that, all the variations of those Facebook style chatbots and WeChat, and Line, and Telegram, and Kick, and web-based ones, and home-grown ones. You see everything and the data is really fascinating how people use these things and how they’re interacting. So it’s an interesting position to sit in because we see so much difference.
Ryan Lawler: Tell us a little bit about how Dashbot actually works. What do you do with your customers, and how are they using you today?
Arte Merritt: As I mentioned, we’re an analytics platform for these conversational interfaces, so we enable brands, developers, enterprises to increase engagement, monetization, user satisfaction. It’s a little bit of what their use cases are. At a high level, the way it works is a copy of every message in or out from the chatbot or voice skill gets sent to us, and we just take care of the rest. So it’s meant to be super easy. And part of the reason for this is there’s a lot of additional metadata that’s sent in, it’s unstructured data. If you use Google Home and said, what’s the weather, as a developer, that developer gets the raw text, what’s the weather, but they also get the intent is check weather, what the context is, the action to take. There’s a lot of additional metadata there that’s useful to have, and those are the things that we end up reporting on
Using that same example, if you built a chatbot to answer the question, what’s the weather, you might think that’s all people would say, but then you’ll quickly find, they’ll ask is it hot out, is it raining, do I need a jacket or an umbrella. There’s so many different ways to ask that, you need to look at the analytics to see what’s happening. Are you giving the I don’t know message back? Are you giving them something totally unrelated? So that’s where we come in, we can show this is what the user said or wrote this in, this is how you responded, you might want to improve those responses. And it goes a bit to what you’re underlying use case is. Is it more to increase engagement, or are you doing customer service and you want more for the user satisfaction and all. So those are some of the areas where we help.
Ryan Lawler: Got it. So this allows developers to, basically, provide better answers because first of all, they might have an idea of how people will use their platform but they don’t actually know until it gets put into practice, right?
Arte Merritt: Definitely. So there’s a popular pizza company I can’t mention, but you can order pizza through seven different conversational interfaces. And you think they would know how folk order pizza or ask for a pizza. But then, as you see the analytics, there’s all different ways that people ask for a pizza and the different variations of toppings and all these things. So it’s really important to look at the data to see what are users saying and how are you responding, so you can improve that. And that’s, basically, what we do, whether it’s developer, brand, or enterprise, we’re helping them improve the user experience and hopefully, lead into whatever your goal is with that interface.
Ryan Lawler: Okay. When you talk about the metadata aspect of it, what sort of data are you collecting, or are you seeing? Are there examples of things that people, consumers, might not know they’re sharing that actually helps inform the brand or the company you’re working with?
Arte Merritt: It depends a little bit on the platform for what the additional metadata is. If it’s a voice skill, like an Alexa or Google Home, in that case, they’ve caught a raw utterance. Amazon is a little bit different because they actually even hold this information back. But the message itself that the user said or wrote, you get the NOP data around it, which is the intent, what that message would, basically, translate into, what the context was. So if you said, what’s the weather in San Francisco versus what’s the weather in New York, what action to take. That’s some of the information you get there.
And then, depending on the platform, you might get things like user locale information. In the case of Facebook, you do get some of the profile information like first and last name, profile photo, gender. It’s not the full profile. And then in the case of, say Facebook or a similar platform, if they’re sending in images or audio files, you actually get those too. So there’s quite a bit of data. That’s why it’s all unstructured.
Ryan Lawler: You mentioned a few examples. You mentioned commerce, you mentioned customer service, you mentioned brands. Can you give a few example use cases in each of those categories and maybe some that I didn’t mention?
Arte Merritt: Yeah, we actually see quite a wide variety. Even from the Facebook chatbots and Alexa type skills, in general, a lot of the use cases are very similar to what you’d see in mobile. There are people doing, believe it or not, there’s dating ones, there’s religious ones. And then you have all the media, news, and travel, and everything else. Where we started focusing a bit more is on brands and enterprises that are doing something in the customer service space or commerce. And there’s something overlap there because commerce is sometimes the pre-post sales, which is, basically, customer service. So the industries that we like at are like software, basically, high tech software, retail, travel, any sort of communications data is Telecom or your Comcast kind of companies, and then finance and insurance because these are the types of folks that are doing stuff there.
Ryan Lawler: So I mentioned a few different industries that use Dashbot, how do most customers find out about you?
Arte Merritt: Yeah. We try to make a conscious effort to be a thought leader in this space. And the previous startup I had was a mobile analytics company, and we had a competitor that pretty much destroyed us. I actually thought that what they did on the marketing side was brilliant. And so when we started this, I’m like, you know, I’m going to do some of the same stuff. I learned I guess, the hard way there.
So, as I mentioned, we got the first version fairly quickly going. So in May of 2016, we launched the first version of the product. We immediately started publishing insight reports and data, what we were seeing, how case studies … so how did a chatbot, in this case, a Slack chatbot use our analytics to improve engagement and retention and all those things. And then we immediately started hosting monthly meetups. They started at NSF. We expanded to New York.
And then when we were publishing all this data, it got to be at a point that other people were picking it up and running with it, VentureBeat, in particular, which basically runs the majority of the stuff we have published. In addition to all those meetups and writing all the reports and sharing data and all, we’ve hosted two conferences already. We have our third conference coming up. And the reason I’m bringing all this up is most of this stuff becomes inbound. The brands, and enterprises, and even the startups have heard about us from these efforts.
We do also do outreach, as well, and we try to participate in other people’s communities, whether that’s online to offline, as well. So it all goes back to trying to be a thought leader in the space.
Ryan Lawler: Right. I think it’s really fascinating, a lot of the research that you put out. Can you share just some of the more interesting or surprising findings from that research?
Arte Merritt: We get a lot of data on the Facebook side, so this is a little bit more Facebook-specific. Do you know the most common message sent to a Facebook chatbot?
Ryan Lawler: I don’t.
Arte Merritt: It’s just ‘hi,’ so ‘hi’ and ‘hello.’ And it makes sense. You start conversations with hi and hello, so people start the chatbot conversations with hi and hello too. But a significant portion of these don’t actually respond. If I remember off the top of my head, about half of them don’t respond with an appropriate greeting back. In fact, one time we did a meetup here at Samsung and someone in the audience said pull up my Facebook bot. Can the panel critique it? So we pulled it up, and the very first thing I did just to see how it worked, was I typed hi. And the response was I don’t know, I have no idea what you’re asking me. So that one was kind of interesting.
And then we pulled a bunch of data on the images and all folks sent into these Facebook chatbots. And do you know what the most common image someone sends?
Ryan Lawler: I have no idea.
Arte Merritt: It turns out it’s a selfie. So people are sending selfies, not to other people, but to these chatbots. Because at Facebook, you can separate male versus female, we were looking a little bit deeper and male, you started seeing they were sending images of motorcycles, and cars, vehicles. And then women still skewed pretty heavily to selfies. But then this one tag showed up interesting top types of images women sent in, and it was electronics. And do you know why, by any chance?
Ryan Lawler: No.
Arte Merritt: So it turns out when women take selfies in the mirror, the mirror captures the phone and this image recognition search is describing that as electronics, so it’s pretty interesting.
Ryan Lawler: That’s fascinating. And just curious, what’s the context for someone sending a selfie, or some other image to a chatbot?
Arte Merritt: That’s a strange thing. It turns out, as we talked to some of the developers and brands that had these, folks are treating the chatbots as if they are their friends. So you’d send your friend an image, then they’d send this chatbot an image. There was a popular weather chatbot on Facebook for a bit, and they even created personalities around this. First, they’re ignoring the images, and then when they saw on the analytics, people are sending a lot of images in, they would come up with a response for it just to make it kind of fun and increase engagement for them.
Ryan Lawler: I wonder how many people are trying to abuse the chatbots in this way. I imagine you have a lot of off-color language being thrown at them. What are you seeing from that standpoint?
Arte Merritt: Yeah. There’s a women’s lingerie brand retailer. And so we had this thing we launched a while back, it was just effectively word clouds. It was clustering individual words. And the interesting things is it was a great way to segment the audience because you could see the people that were legitimately shopping for the clothes and then all these other folks that were just trying to abuse the bot, just based on the words and all that they were using.
There an enterprise tech software company that they … you can set up alerts in our platform to be notified of different conditions. They set up the alert to be notified of all of the profanity. So just all the profanity coming through, they look at. Related to the image one, one of the things that was surprising to me, when we pulled those images, I at first thought the top image might be something a little bit different than what I was just mentioning. So it turns out, people do send naked selfies to chatbots. Luckily it’s not a lot, it’s about 1% or 2% of the images, similarly of the users. But the thing is, when people do it, they do it a lot.
So a normal image would be sent to a chatbot just once, basically, on average. The average naked selfie is sent in five times. And then, there was one person in this one month period that sent his naked selfie about 266 times. So there’s some weird hurdle you must get over where you feel comfortable doing it and you just keep doing it. And it’s the chatbots.
Ryan Lawler: I really am curious what that guy was hoping to get back from a chatbot.
Arte Merritt: Yeah, I don’t know.
Ryan Lawler: How does the chatbot respond in that instance?
Arte Merritt: Oh, I didn’t look at that one. But that’s what I say, some of the companies, they just would respond with some sort of response to the image just to give it a personality. Like stickers were another thing, the thumbs up and happy faces and all those. People do communicate in emoticons. So if you go back to that pizza company, when they first integrated with their Facebook bot into our analytics, there was drop-off and abandonment all throughout the users experiences. So we looked at this behavior for our report, and why is there all this drop off and all. And it turns out, it was the thumbs up.
So these users would get really excited. They’re in the middle of ordering their pizza, they do the thumbs up, the company never got the order because it would sort of break the bot. So the company never got the order, the user, obviously, never gets their pizza. No one had any idea. Oh, the thumbs up, it wasn’t that you were ignoring it, it actually just broke the bot. And the thumbs up is one of the top three messages sent inside Facebook messenger.
Ryan Lawler: Okay. So just registering or recognizing that someone is trying to say, okay, that sounds great would break the bot.
Arte Merritt: Yeah, exactly, in this particular case. That’s why we often would say to folks, look at those messages coming in, the top messages, look at your I don’t know, or fall back intent or response and see what’s triggering that so you can decide whether you want to add support for those things or not.
Ryan Lawler: Got it. What’s the biggest challenge to adoption?
Arte Merritt: I don’t want to sound like an egomaniac, but I think we do know can adoption just because it’s self-serve, and it’s meant to be fairly easy to integrate and free and all those kind of things. Just the timing of the space because it’s still relatively early, we’re not up against the not invented here syndrome at enterprises yet because it’s so early. I think the challenges that we’re facing is not the initial adoption, it’s now we want to, obviously, monetize. So how do we monetize our platform when this is still an early stage for brands and developers? And sometimes it’s the innovation teams that are developing these things and they don’t quite have the budget. So that’s more where our challenge comes in.
Ryan Lawler: Got it. Once someone has adopted the platform or is testing it out, how do you ensure successful engagement and keep them coming back?
Arte Merritt: Yeah. Obviously, we make use of analytic too, on everyone’s site to see how people are using it, what are the more popular things, where is drop off occurring, what’s the retention and all that, and how can we keep optimizing these things. It is self-serve. All you have to do is sign up with an email address and password. You get a little snippet of code that you put inside your chatbot, and it’s just sending a copy of all the messages in or out to us. And it’s asynchronous, not blocking and that kind of thing.
And then there’s a bunch of premium features and functionality that if you’re interested in, we’re happy to show folks demos and help them out with those things. We’re pretty much there to help people, so if anyone has any difficulty with either integrating, or they want to ask about one of the premium features, or they just want a demo … sometimes they just want to know what they should be looking at.
Ryan Lawler: One of the things that comes to mind as you talk about all of the data that you’re collecting in these conversations is just consumer privacy concerns. And I’m curious how you think about that, especially as people are sending in receipts or naked selfies, for instance, making sure that data exists solely with the organization or brand that they’re sending it to and that it’s not in some centralized store where a third-party like Dashbot could see it.
Arte Merritt: First, we don’t sell or resell anyone’s data, it’s their data. Everything is encrypted in trans internet rest. We’re GDPR compliant. We work with some pretty large enterprises that are doing some financial related things, insurance, and retail that have put us through different security audits that we passed every single one of these things. It’s something that we’re quite cautious of, and we understand when someone might have a concern around that.
One of the things that we implemented was a way to redact any of the PII before it even hits it. If a brand or individual developer is concerned about this, we don’t want them to even send the data to us, like don’t send anything that’s PII. So there’s a flag that you can set that will remove common PII, whether it’s first and last name, email address. If by some chance, you have a chatbot that someone might actually put a credit card in, it’ll remove those kinds of things. So we’re very cautious of that stuff and we want to make both the developers and the enterprise and brands comfortable.
Ryan Lawler: What’s one controversial opinion you have that’s very strongly held?
Arte Merritt: I would say just if I bring it into this space, this context, if you look at what happened with web and mobile when the platforms first started taking off, a lot of the folks get shaken out. There’s a lot of people that start building that and there’s excitement there and different parts of the ecosystem that get built out at different platforms. And this is the part, it’s not meant to be strongly held or controversial, but it’s something just to know in the space. We were friends with these folks so it’s sad to see, but the sum set of these platforms that we’re working with that are in the space that might not be there in the next year or so. And that’s the part of this that, I guess, maybe is a little bit, I don’t know, a little bit on the controversial side.
Ryan Lawler: I think it’s only controversial if you start naming names.
Arte Merritt: Oh, yeah, yeah. Otherwise, I’d say I like cats, I’m fond of my cats.
Ryan Lawler: Okay. So if you weren’t doing this, what other areas of tech, or other areas, in general, that you would be interested in working on?
Arte Merritt: Yeah, it’s a good question. One of the areas that I find fascinating and interesting and I don’t really have much experience in, is all the driverless cars and all that. That seems pretty exciting, especially at CES, it was interesting to see what those concept cars look like. You’re so used to seeing there’s the steering wheel and the whole dashboard and all this. And the ones that I saw are, basically, like boxes, like train car boxes that people were kind of looking at each other and there might be a table in between. And they’re just kind of fascinating.
And then I ran into someone and I brought that up how it just looks different, and they said, “Well, probably the first time people saw cars without a horse in front of this thing seemed different too.” And it was just kind of interesting.
Ryan Lawler: I’m curious, are you seeing adoption in the auto space? Because I know that conversational interfaces are slowly making their way into vehicles, but right now, it’s kind of like you’re making requests to your phone which just happens to be connected via Bluetooth to your car. But is that advancing.
Arte Merritt: Yeah. One to the things that was there was Alexa had the whole Alexa for autos. We see more integrations of Alexa and Google Assistant into different car brands. Then Microsoft is trying to do some form of almost a white label one for them. So for us, the auto industry is huge. That’s one of the ones that we go after, and it’s for a couple of different things. There’s, obviously, the voice inside the car. What are people doing and saying, and how can we show all the analytics around that. There’s the customer service pre and post sell, folks looking up and doing research on a pretty good car and they want to chat with someone. And maybe after you purchase the car, you want to book your repair service, all of that through that.
And then the other area is more the marketing. We do see some of these car manufacturers have Alexa skills or Facebook chatbots that are more or less marketing tools that tell me about this car, and what are the features, and where can I test drive it. So there’s two different areas where we can work with those folks, that’s what we’re trying to do with some of them.
Ryan Lawler: Got it. How will things be different is Dashbot becomes ubiquitous?
Arte Merritt: Well, hopefully, we’re doing a lot better. But I think that would generally mean that all these interfaces, these conversational interfaces, have taken off quite a bit more. And hopefully, that translated into better experiences for consumers and your day to day life too if you can get some sort of answer to something relatively easy and quickly, or painlessly without having to go through a whole customer service channel. That could be quite helpful. We’re big believers in conversation, in general, like conversation is the future of human-computer interaction.
If you remember all the videos of two-year-olds swiping the iPhone and the iPad, the same thing happens with devices like Alexa and Google Home, kids know how to interact with those. So the more our stiff takes off it means there’s more interest and more energy being put into these interfaces. And those are getting better, which will help all of us, whether it’s in your personal life or work, or anything.
Ryan Lawler: I think what’s interesting to me about this space is just how part of the hype cycle that we went through was partly defined by the fact that the promise of the technology was way further along than what it was actually able to do. And by that I mean, people are trying to speak to their devices and ask for things, and the bot would break or it wouldn’t understand, which led to a really bad experience.
Arte Merritt: Yeah, definitely. Yeah, yeah, yeah. If you go back to just when iOS opened up for apps, a lot of the apps were the crummy ones, or fart apps people talk about. So that’s what you kind of saw initially with Facebook, it’s a lot of these ones. It sounds mean to say, but they’re kind of the crummy ones. So I think on that side, we still haven’t quite seen the greatest user experiences. Voice, you’re starting to see some neat stuff. Even then, it’s still really early in this. And it’s just had to build for these. Users can say whatever the heck they want to them. It’s hard to know all the different things someone might say and how to respond to it. That’s why I often use that weather extraordinary, if you think as something as basic as just boxing it in, what’s the weather, but there’s all different ways people are going to ask for the weather.
Ryan Lawler: Since you’re the expert and you see all of these bots, what are some of the more interesting or exciting experiences that you’ve seen deployed?
Arte Merritt: Yeah. There’s a company called Volley that makes a lot of games that are pretty interesting like more of the voice trivia games that are pretty good. And then, I do actually think the pizza delivery one is pretty interesting too because it’s one of the early ones where you actually do commerce. So that’s, I think, one of the other promises that’s still in the works is will people actually purchase things through these different interfaces and some of them are set up a little bit better for that you already have your credit card or you have some kind of account there to make it a little bit easier. So that’s kind of interesting to see.
And then there’s one of the enterprise software companies that has customer service chatbots that works with us. It’s pretty impressive what they’re doing. They want to provide 24/7 support. It’s expensive to do that with live agents, so they’re trying to make that chatbot as effective as it can be and will, one, understand what the user asks, and then responding appropriately. And then hopefully, responding with something that the user is happy with. So there’s a couple of different areas to improve. And just seeing that over time, how they keep improving that is pretty fascinating.
Ryan Lawler: Alright, well, this was great. Thanks for being with us, Arte.
Arte Merritt: Yeah, I really appreciate this opportunity. Thanks a lot for having me.