Episode 18
· 36:11
Hey, this week we have Jason E. Vanish on the show. He talks about moving from Boston to San Francisco, as well as how he got a job at Kissmetrics doing product. Before we get started, any of you are working on software teams and you've ever wondered, how much of our time are we spending on bugs? How much of our time are we spending on maintenance?
Speaker 1:And the all important question, how much of our time do we spend on actually building and shipping features for our customers? If you've ever asked any of those questions, you need to try sprint.ly. Sprint.ly is project management software for agile teams.
Speaker 2:I'd like you to try sprint.ly for free.
Speaker 1:You can sign up for a thirty day trial at www.sprint.ly.
Speaker 2:Hi, I'm Justin.
Speaker 3:And I'm Kyle.
Speaker 2:And this is Product People, the podcast focused on great products and the people who make them.
Speaker 3:And today is the Product Manager episode. We have Jason Ibanish from Kissmetrics on the show today. So hello, Jason.
Speaker 4:Hi, guys. Thanks for having me. I'm excited to talk product today.
Speaker 2:Right on. Now, Jason, why don't we start with your story? All three of us are actually product managers, and product management is an interesting job. How did you get into product management?
Speaker 4:So it was definitely a very circuitous path. I didn't have like a direct path knowing like when I was a freshman in college, it's what I wanted to do. I actually started out as an electrical engineer at a school in Boston called Northeastern. By the time I got my degree, I realized I didn't want to be an electrical engineer. And so, decided to go straight into a master's program my school had called technical entrepreneurship.
Speaker 4:And what it was is take people with tech backgrounds and give them business backgrounds to launch companies, because I knew that my passion was really entrepreneurship and startups. But when I got that degree and started trying to get a job, because I was like, Oh, I should totally get some experience in sales and marketing or something. Everyone looked at me like, Who the heck are you? Go be an electrical engineer and make lots of money. Like, go away.
Speaker 4:But what ended up happening was I found that the Boston startup community was really disorganized and messy, and so I ended up starting a company called Greenhorn Connect to solve that problem, and so I kind of made my own first product role, Basically trying to get that site off the ground and, you know, designing what we needed and trying to solve the community's kind of customer problems. And after doing that for a few months, I basically had kind of shown people, oh, this electrical engineer can do internet startup stuff. And I got a part time job with a guy named John Prendergast, who took me under his wing and taught me customer development. Conveniently, after working with him for a couple of months while I was building Greenhorn Connect, he ended up being the independent board member of a startup called one hundred forty, which was the app store for Twitter. He was like, well, that company needs some customer development, so you should hire Jason to do it.
Speaker 4:And so I worked with the VP of product there on what our direction of the company should be from a product perspective. And so I did that for a year. Fast forward, to December 2011, and I was kind of working on my own projects, and, I visited the Valley. And one of my mentors when I was at 140 was a guy named Hetan Shah, who's CEO of Kissmetrics. And his product person had left about two weeks before I showed up in the Valley.
Speaker 4:And after a lengthy conversation about a lot of things, he said, Why don't you come work for me? And three months later, March 2012, I joined Kissmetrics as their product manager because they really wanted to fill a gap they felt they had in having someone who could really be customer driven, really understand customer problems, and get the rest of the team on board doing the same thing.
Speaker 3:Wow!
Speaker 2:By So the time you joined Kissmetrics, how much experience did you have in product management?
Speaker 4:It depends on how you count, but I would say that I've been managing the side project of Greenhorn Connect for about two and a half years at that point, so I would count that as, like, about a year of real experience doing that. And then, at one forty, I didn't get to make the decisions on, like, what a future would look like, but, you know, the stuff I did in talking to customers and I was really the analytics guy at one forty. So I kind of was was apprenticing almost there to try to understand how product worked, but I wasn't getting to necessarily do the full gamut. So KISSmetrics is really kind of my first rodeo and getting to do everything and having it all kind of on my head instead of maybe someone else's.
Speaker 2:Yeah. Well, seems like a pretty big role. Kissmetrics is a well known company. What what do you think, Heaton Shaw saw in you that made him wanna, you know, take you on?
Speaker 4:I think a lot of it was a lot of it's the network. So, he knew the founder of one forty. He knew this guy, John Prendergast, that taught me customer development in the beginning. And, you know, the biggest thing was that he and I were both mentors at Lean Startup Machine in New York City in the April before we met in person in in December. So April 2011, he and I were both mentors at Lean Startup Machine, and, like, seven or eight months later, I was meeting him in person in the valley again.
Speaker 4:And we were both mentors at Lean Startup Machine New York. I can tell you that if you get the chance to mentor at an event, it is the best networking you can do because the other people that mentor are like really smart people who know a lot about whatever topic you're mentoring on. And so basically half the time I'm helping people with the things I've learned and being in the trenches doing customer development. And then they're half I'm hanging out, you know, Patrick Laskovitz, who wrote the Entrepreneur's Guide to Customer Development and Heaton's job. So that was really cool.
Speaker 4:And so like, went out for drinks afterwards and basically I spent like probably ten hours with Heaton. So he got to know me pretty well and like figured out how quickly I picked up skills and the things I was learning on the job. So I think it was basically that it wasn't about whether I knew how to do everything already. It was that he was confident I'd figure it out.
Speaker 2:Yeah. Yeah, that's amazing. Maybe talk a little bit about what it's like to move from Boston to San Francisco.
Speaker 4:Sure. So it's like two different worlds. Boston has seasons and San Francisco doesn't really, unless you count foggy versus not foggy. So it's been a really amazing experience because while the cities have a lot in common, like they're often considered sister cities and a lot of people live in both cities, different parts of their lives. Okay.
Speaker 4:Culturally, they're completely different. Like when people ask me, like, do you prefer? Like, Boston or San Francisco? It's like apples and oranges to me. Like, it's really hard to say, like, one or the other, because every everything you do and everything that's approached is totally different because the cultures are so different.
Speaker 4:And, of course, in the tech scene, it's like, this is Mecca, this is Hollywood, this is whatever you'd like to call it. This is the epicenter of everything. And so, you know, that's been a really cool experience, just to be in a place where working in tech is the norm, not this weird exception, like, Oh, you work at a startup? I've never heard of is what I would hear in Boston. And here it's like, Dude, everybody does that.
Speaker 4:What are you talking about? Like, you know, you go and you play Ultimate Frisbee and somebody on your team is, you know, the co founder of Sidecar and you're like, Holy shit, that's so cool. Everyone else is like, Oh, you'll get used to that.
Speaker 2:So would you advise maybe people that are wanting to build products? Because you've done both. You've done product in Boston and product in San Francisco. Actually, maybe there's two parts of this question. First, if you were going to build your own product, where would you build it?
Speaker 2:And second, if you wanted to get experience in product management, which city do you think is better?
Speaker 4:So to me, I don't think you have to be here. The world is flat and information is everywhere. I mean, Justin and Kyle, you're both in Canada right now. And we're talking and sharing ideas and stuff. Like, between, like, Twitter, LinkedIn, Quora, blog posts, like, there's so many areas where information is being shared and, like, there's so many people I've connected with that are in different cities that, like, you can learn a lot about doing great product anywhere you are.
Speaker 4:I do think if you have it in your head, like, I want to be in the major leagues or I want to be in Hollywood and do the whole thing, the Valley has a lot to offer. But also keep in mind that it is the major leagues. There's a really high bar here that I wouldn't have been able to start my career fresh out of grad school here and gotten into product. I needed the experience I got in Boston first to really be able to get a foothold here and get the chance at Kissmetrics that I have.
Speaker 3:Yeah. I think like, I mean, where is maybe less important than who you work with if you're starting out. You kind of mentioned that you had good mentors when you started at one hundred Yes. And I would say that I'm the CEO of Granifi, Jeff Lawrence, he's got a pretty solid background of building successful companies and he's amazing as a product manager and it's kind of my first kick at the can with product management. And I would say that working alongside him has been like hugely valuable.
Speaker 3:It wouldn't matter to me really where the experience happened. It's just through chance, I guess, is in Edmonton and I'm in Edmonton and I got the opportunity to work with him. And I think that the opportunity to work with him is probably the most valuable part rather than, like, being located somewhere on a map.
Speaker 4:Yeah. So that's the thing is, you know, going to the second part of your question, Justin, about if you're starting your own thing. I think whether you're starting your own thing or you're looking to work with someone else on product, it's all about mentors. Like, surround yourself with some really smart people and, like, try to learn from them. And, you know, ideally, it's great if you can work day to day side by side with someone like that or get them in as an adviser that you can meet up with in person.
Speaker 4:But, I mean, whenever I worked at one hundred forty, my two biggest mentors were Dan Martell and Heen Shah, and those were once a month phone calls where they just beat me up pretty good. And, you know, I learned a lot from that and they were not in the same room as me and I still got a lot of value out of it. If you're starting your own thing, definitely look to try to find people on the web that you admire and you are learning a lot from by reading their blogs or whatever they share, podcasts they do. And that'll really help you improve your skills no matter where you are. I know I've Skyped with people in a lot of different places asking me about how to get into product and stuff.
Speaker 4:There's a guy in Germany I've helped. Again, the world's flat, you can totally take advantage of it. You don't have to be here, but there are advantages where people like me wouldn't keep moving
Speaker 2:And what for someone that you know, we have a lot of people who listen to our show that are building their own thing, and they're just solo developers, solo designers. How do you think they could benefit from getting some product experience, like product management experience or experience doing customer development?
Speaker 4:What could they do, you mean?
Speaker 2:What could they do and how would that benefit a solo person? If you're a solo developer, just saying, I just want to build my product and release it, why should they care about customer development?
Speaker 4:Oh, well, if you want people to actually use what you make, you may want to look into customer development. No, that's the thing. It's really fun and it's really cool to build stuff. I know some of the most enjoyable conversations I have is when Heath and I are just spitballing about random ideas of the direction GuestMetrics could go. And those are really exciting, but then we have to ground those in reality of, alright, where are we now with the product?
Speaker 4:What technology is really enabled right now? What can our team do? And most importantly, what do customers want right now? Because there are things I know that we could do that are gonna be awesome, but I don't think our customers are ready for it yet. And luckily, we're not really ready to give it to them.
Speaker 4:But basically, if you're building your own side separate project, you need to think a lot about what your goals are with it. You just want to build something cool for yourself, like, hey, more power to you. You're your own customer, just do it. But if you actually want to get other people to use it, then it'd be a good idea to hone the skills of talking to people and understanding, like, you know, how does this solve their problems, or why doesn't it solve their problems, and what is the core use for them? If you get a thousand people using something you make, are they all using it as you intended, or are they actually using it for different things?
Speaker 4:You'd be surprised how people will try to put a square peg in a round hole, And often there's a really cool opportunity if you make that round hole a little more square like for them.
Speaker 2:Yeah. I think we'll get into some specific tactics about customer development a little bit later on. Before we get to that, I just want because product management is an interesting area and we were talking a bit before the show. You know, there's a lot of different definitions of what product management is. So what's your definition?
Speaker 2:How do you describe what a product manager does?
Speaker 4:So to me, the product manager is juggling a lot of things, but your goal is to be the bridge between the voice of the customer, the interest of the business, as in we're in the business of making money at some point in some way, and the technical, you know, manifestation of that solution. Because in the end, you're basically balancing, like, what does it seem like the customer really needs, What will help us make the most money as a business? And what is, like, the technical debt and the technical hurdles that we're facing to bring that solution to bear? Like, to me, the product manager is the one who has to try to figure that out. Because in the end, the designer is trying to make it look great and easy to use, and the developer just wants to build it so it works.
Speaker 4:So you have to bring those other areas to the table. Otherwise, those two guys waste their time. Yeah.
Speaker 3:Right. That's a that's a pretty, like, good description. Think people I've talked to have kinda described product management as, in a lot of ways, being the hub of the entire company or kind
Speaker 4:of Yes.
Speaker 3:Synthesizing information from all these different sources and at the end of the day, turning it into a shippable piece of software that hopefully customers want to pay you money for.
Speaker 4:Yes. And the other challenge is, like, figuring out, like so I have to be a filter. I have so much information coming to me in different areas, and I know I can't inundate everybody else with that same information. So a lot of what I try to do is also figure out, like, okay, of the stuff that I'm hearing, what do I need to filter to other people so that they feel in the loop and in the know and understand why I'm suggesting doing something? Because I don't know, I find good engineers, good designers wanna know this stuff and wanna be engaged and involved in it.
Speaker 4:If you just tell them do it this way, they won't understand the why and then you'll struggle to get the right solution. But if you can help them understand the why without pouring too much information on them, then they build something really awesome because they understand the motivations behind it and what they're really trying to accomplish.
Speaker 2:Sorry. Go ahead, Kyle.
Speaker 3:I was just gonna ask, because that's, like, you're definitely talking my language now because that's something that I'm finding a little bit difficult to kinda get the hang of as a new product manager is I'm like bombarded with information from all these different sources and like I kinda have a development and design background so it was kinda like toss on the headphones, chew through tasks and Yeah. I mean, you're kind of still multitasking, you're for the most part, you've got these like specific things that you can just kind of shut everything out and focus on. But now that's not at all how I can work. Like, I've got Yeah. A steady stream of information coming at me from, like, our business development team, from our engineers, all over the place.
Speaker 3:So so how do you deal with with all of that information coming at you? Like, who is it coming from and how do you triage it and make sense of everything?
Speaker 4:Okay. So first of all, I would say one of my best skills is pattern matching. And I think that's a skill every product manager has to hone. And so what happens is I take this data in from all over, and I just like, it just chews in the back of my mind all the time looking for new patterns to emerge. And so that's part of it.
Speaker 4:The other part of it is, like, to get signals. So I talk to my support team all the time. I talk to the sales team all the time, and we have a feedback box at the bottom of every page on Kissmetrics. And with that feedback box, we get customers telling us stuff all the time about the product. So I read through all those, and so I have all this, like, passive information constantly coming into me, and I'm keeping an eye on it.
Speaker 4:And a lot of it, I just mark up and organize so that whenever we get to a point where we're going to act on it, I can call up a whole bunch of it right away, and then really try to, like, crunch on all of it at once and figure out, like, alright, here's the pattern over here and over here. Like, for instance, at Kissmetrics, we're about to fix up our daily emails, which we send to customers. We've never iterated on them ever, and they're pretty terrible to be quite Yeah. So we're gonna finally fix them. And so I have, like, 30 emails from customers in our feedback channel about wanting them to be better.
Speaker 4:I talked to one of our top sales guys for his feedback on, like, what do customers really want? And I asked support, like, hey, what do you guys keep hearing on support send about this? And then I've been crystallizing all of that into, like, one document that now I can pass and share with the design team and the engineering team to say, like, hey, here's here's what's going on. Here's motivation behind doing this. So, you know, I always try to tell them, Why are we working on this right now?
Speaker 4:In addition to, What are the core use cases and problems and opportunities we have? So coalescing all that into one document, which we call the thesis, has helped a lot in sharing the right level of information. And then, you know, if I go and, like, we're gonna talk to customers specifically about this, maybe I follow-up with guys, I'll try and get the design and developers to maybe either listen in on a call that I've recorded or literally sit in on a a custom dev meeting or two, just to give them a little flavor. Like, we're working on a mobile app, and our mobile engineer is actually leading the customer development on it. I'm in the room with him helping him, but he's actually leading a lot of the discussion, and because of that, it really removes a lot of those barriers where it's not Jason's opinion, it's actually what the customer is saying, you know, and that feels a lot better.
Speaker 4:Even if, you know, I think I've done a lot to build trust that I don't have an opinion, I'm the voice of the customer. It's there's nothing better than them hearing the customer, like, going through our app and being like, I don't understand this. I'm stuck. How do I do this? And he goes, oh, alright.
Speaker 4:We gotta fix that. Yeah. You know, that's so much better than me just saying, hey, Will, you need to put a triangle on the top little button there so people know it's a button. Like, he'd be like, alright, fine, I'll do that. But when you see someone actually struggling and feels that pain directly, it changes everything for how it affects an engineer or designer's opinion on something.
Speaker 3:Right. Because half the half the battle with talking to designers and engineers is getting them to buy into something. Right? Like, why it
Speaker 4:should
Speaker 3:be done a certain way.
Speaker 4:Oh, yeah. Yeah. We had a feature a while back. So we have this thing called live in Kissmetrics, which is like a stream of current activity on your site. So we just say, like, hey.
Speaker 4:Justin is on the homepage, and Kyle is looking at his metrics dashboard and, like, stream through. And so we did a massive iteration on it that made it easier to filter the information and stuff. Our designers thought it'd be awesome to make it like a more Twitter like experience. So we put all kinds of rich data in each one of those entries on those people. Our customers hated it, like venom, like pure just vicious, like, I hate you guys, like like curse words, come on, what's wrong with you?
Speaker 4:This is terrible. And what's great is I, was lucky enough to get our designers to agree to read all the feedback coming in, and they just like it bruised their egos so much that like, boom, twenty four hours, we had a brand new design ready to be deployed, to fix, the fact that everyone hated the new stream. And they would have like, they really thought it was gonna be great, they convinced me it would be great. But then our customers told us how wrong we were, and our designers seen the Venom firsthand. It made them wanna, like, fix it really fast.
Speaker 4:And it works great because then they were seeing customers are like, oh my god. I yelled at you guys twenty four hours ago. You already have a solution. You guys are amazing. Yeah.
Speaker 4:And so, like, we have the tweets, and it was like a huge emotional win for the whole company then because everyone's like, oh my god. Look what we could do when we're really motivated. Look what we can do in, a day or two, and look at the response we get from customers. Like, people who tweeted they hated us 24 ago are tweeting they love us. Yeah.
Speaker 4:You can't put a price on that. Like, our our designers went through like the full range of emotions in the span of like one week.
Speaker 3:Yeah. Yeah. You used an interesting word there, ego, which Justin, earlier when you asked what maybe a single founder could learn about product management and customer development, I think that that's probably something that everybody who starts their own product needs to kind of go through is this ego crush that you actually don't know best. Like you could be a super talented designer or engineer but your assumptions are not necessarily true because you know how to design or code. It's pretty humbling to release something that you think is perfect and then, like you said, gets slammed.
Speaker 3:People hate it and they refuse to use it. Getting caught like that is a pretty valuable lesson for for anyone to learn, I guess.
Speaker 4:Yeah. I mean, look at I don't know if you guys are Evernote users like I am, but I use Skitch a lot. Yeah. And when they release the new Skitch, everyone's like, this is awful. I hate you so much.
Speaker 4:Yeah. Yeah. So it got to the point where they had to release the old version, and I still use the old version because I still think the new one's terrible. Yeah. And and what's funny is, I saw Phil Liven got interviewed by Jason Kalakanis a couple weeks ago out here, and one of the things he talked about is the Evernote philosophy is that you don't talk to customers before you build it.
Speaker 4:You only talk to customers afterwards to see what they hate, and you fix the things they hate, and the rest you don't worry about. And so it's like, Oh my God, Skitch makes so much sense now. They built it because they thought these things would be great to better integrate with Evernote and stuff, not realizing that, Dude, you're totally jacking my workflow. I make Skitches really fast, and I don't want to save them all to Evernote automatically.
Speaker 2:Yeah. I think for a lot of product people actually a lot of people in general this is a huge paradigm shift. The idea of getting away from your idea that you love and becoming more people focused. And so maybe this is a good time to talk about customer development. You're becoming known as an advocate for customer development.
Speaker 2:Could you describe for our listeners what customer development is?
Speaker 4:Yeah. It's like a series of tactics or, you know, kind of a methodology to get to the core of what a customer really needs. Like, goal is, like, what is their problem? And then you create a solution that, you know, can really delight them. And the real challenge is that customers don't flat out say, Justin, my email newsletter tool sucks.
Speaker 4:Will you help me solve these things that your newsletter solves?
Speaker 2:Exactly.
Speaker 4:Come out and say that, so you need a process for how to, like, draw it out of them. And, like, I know one of the hardest things is I don't know if you guys have ever had to talk to product managers as, like, part of your, like, part of your user base, but they're the worst people to talk to because we always have we're like, dude, I'm a product manager. I know the perfect solution for what I want. Yeah. And as a fellow PM, you have to be like, no, dude, just tell me what your problems are.
Speaker 4:I have to try to solve this for many customers, not just you.
Speaker 2:Yeah. Yeah.
Speaker 4:And so it's a it's customer development is that process, and I think the most valuable thing is, dude, you wanna get politics out of your company. Like, one of the fastest way to do it is become a data driven company, and that means, like, buying into analytics, and it means talking to the customer because there's no argument anymore whenever everyone just says, well, this is what a customer told me, and this is what a customer told me, and here's what the numbers say. Like, once that happens, all the opinions and the hippos, you know, the highest paid person in the organization, all that goes out the window, because now you just have the numbers and the data. And so, like, that's one of the biggest reasons I love this process, is that in addition to being better at building stuff people want, it gets rid of all the bullshit.
Speaker 2:Now, maybe talk about that a little bit more, because a lot of people think that customer development is highly subjective. So I'm talking to human beings, and I'm a human being, and what I'm interpreting, I'm then saying, Okay, well, this is where we're going to take the product based on this. But it's a human conversation. Sure. And so how do you deal with that element, the subjective element?
Speaker 4:I guess you have to trust the interpreter. I mean, just like, I guess, you know, if you had someone who was interpreting Chinese to English in an important business meeting, you know, they have a pretty important job, and, like, there could be things lost in translation. And that's why I talked about the importance of when doing customer development, getting your developers and designers on the front lines is actually hugely valuable. Because if I'm removing myself from the translation at times, I think that can help a lot in removing any biases that I may have, because we all have our biases in how we interpret information. But between that and just pulling data from enough different places, I think it removes a lot of it.
Speaker 4:So the fact that I'll go and talk to 10 customers specifically about something new we're doing, and I'll also pull in things from our feedback channel, and I'll talk to support on what they're hearing, and I'll talk to sales, and I'll share all of that. Like, it basically means that, well, we're hearing it from so many different places. It's like after a while, you hear the same thing everywhere, and then it's like no longer a bias because you're like, it's it's just so loud. It's totally the signal is so much higher than the noise now that there's no room for interpretation.
Speaker 3:Yeah. So so you kind of you mentioned data and analytics and stuff.
Speaker 4:Mhmm.
Speaker 3:What's what sorts of things do you track? Like, what kinds of data do you do you look at?
Speaker 4:So I look for product adoption a lot. So, you know, what features are people using most? Like, I find you can learn the most from your power users because often just because someone uses a feature a lot doesn't mean they're in love with it. You know, often they use it a lot because, again, it's a square peg round hole thing. It does it good enough, and then they end up using it a lot because it's not quite perfect in what they were looking for.
Speaker 4:And so one of the things I often do is Kissmetrics tracks people down at the individual level, and so I'm not trying to sound like an infomercial, but it actually really helps me do my customer development because I can do a search based on, like, who's using this feature a lot, and then go talk to them and find out what's really going on. Also, there's a tool called Qualaroo, which actually we made, but we sold to another company that's now taking it to another level. Qualaroo does little pop up surveys on pages, and so what I love to do is pick a page on my site in the product and basically send it have a little pop up, ask them a question, and then based on the answer, you follow-up with them. And so yeah. So that gives me a lot of opportunities to talk to them.
Speaker 3:Cool. So so is it are those primarily the two, like, pieces of software that you use? Like, two tools is Kissmetrics and Baru?
Speaker 4:Yeah. I mean, we'll do surveys too, but I don't like to do surveys until I kinda know the answers I'm gonna get. Like, I don't like asking open ended questions on surveys very often or very many of them because it reduces the response rate a lot. So what I like to do is if I talk to, like, 10 people and I do a couple things with, like, a koaluru pop up, we'll then create a survey based on what we know now the answers are gonna be and, you know, send out it to our wider customer base to get, you know, essentially quantitative data to go to qualitative data. Because in the end, I think to me, development, good product, and being a data driven company is all about having a firm balance of qualitative data as well as quantitative data.
Speaker 4:Quantitative data tells you what the larger base is thinking, and the qualitative data helps you understand the why behind it. So, qualitative data is going to be able to tell you, say, 45% of customers say that live is the most important feature to them, and the qualitative data will then tell you why 45% of customers say that live is the most important feature for them. Because you won't you just won't know. And even if they fill out the why in the comment box, often you need to talk to someone for five minutes to really get to the core of what they're really trying to do with it. And the one sentence they write in a survey isn't gonna inform you as well.
Speaker 3:Yeah. I think like talking about sort of the importance of both qualitative and quantitative Mhmm. Especially in like a lean sort of process is you basically need both equally because, you know, the whole idea of building hypothesis, building a hypothesis is based largely on qualitative data. Like you talked about like this incoming information, use earn recognition and you kind of build up a You test it out and then you use the quantitative results to to say like what the outcome of that hypothesis test was.
Speaker 4:Yeah. Yeah. That's exactly it. The other thing I find is that the one risk I think in customer development when you're just talking to customers is edge cases. So if I talk to 10 people, there's an outside chance that with our, like, over 1,000 person customer base we have at Kissmetrics, if I talk to 10 people, that might not actually be representative of the entire population of Kissmetrics.
Speaker 4:And so a survey and quantitative data can be a good way to say like, Oh, three of those people I talk to actually represent that 1% population. So I need to pay a little bit more attention to what those other six people said because they're representative of 70% of the population. So it helps remove any biases you have because you can't talk to all 1,000 of your customers every day. Yeah.
Speaker 2:Well, this might Oh, be sorry. Go ahead, Kyle.
Speaker 3:I was just going
Speaker 4:to say that it kind
Speaker 3:of got me thinking of another thing where you talk about product adoption. So when you guys say you have a new feature that you've decided you're gonna build and test, do you kind of roll it out incrementally and then deal with like you sort of talked about how Evernotes, they release it and then they just deal with the things that people hate. Do you kind of incrementally roll it out and iterate and gradually go up to larger amounts of customers or how does how does the feature rollout work in Kissmetrics?
Speaker 4:It depends on the feature. If it's an iteration on a current feature and we feel pretty confident about it, we'll usually do so we have a preview site that we'll put all of our product on that our team can play around with it before we release it. And sometimes we'll let a couple of customers play with it too, and then we'll just release it to everyone. But if it's a brand new feature, generally, what the way we've been approaching it is that we'll release it to, like, 10 to 20 people who are, like, really seem passionate and interested in it to get their feedback and see, one, do they actually use it? Because if they said it was a burning problem and then they don't actually use it, that's a pretty strong indicator that might be trouble.
Speaker 4:And then two, like what are problems you have with it? So if you release a new feature to a handful of these people, they can give you the feedback you need to make it good before you put it in front of everyone. I know I would much rather release a skits that people hate to a dozen people and find out why they hate it and fix it, than I would like to roll that to our whole customer base and have them freak out. Yeah. And so we've done that with a number of things, like our analytics app right now.
Speaker 4:We have in, essentially, a private limited release. We have, like, a few thousand people on a mailing list that signed up on a landing page, and now we're rolling out like a 100 at a time, giving them access to get feedback on what they do and don't like about it. And there's definitely a lot of things that we need to fix, and what's great is we only have one mobile engineer working on it. So he's like cranking and just working like crazy, And this buys him time. Like we have enough, we can put it in people's hands, but it's not like feature rich yet.
Speaker 4:And already because of feedback we've got, it's completely changed what he thought he was going to be building next week. And so like, can't put a price on like the feedback we got that prevented us from wasting his time building a feature that would take him a couple of weeks to build.
Speaker 1:And a big thanks goes out to Jason Evanish for being on the show this week. You can check out Jason on Twitter evanish. We have Jason back next week to talk about the specifics of customer development. He also lays out this really helpful step by step plan for launching an app and testing out the idea before you write any code. Now it's time for shoutouts.
Speaker 1:Shoutouts are a chance for you to advertise your Bootstrap product, a job opportunity, or your side project to our audience of product people. It doesn't cost that much. You can purchase your own shout out by going to productpeople.tv/shoutout. Let's get to the first shout out. The first shout out goes out to Sam Baumgarten.
Speaker 1:Check out Sam Baumgarten's blog to learn about creating web apps and get updates on his projects such as Nathan Barry's Convertkit. You can check that out at sambaumgarten.me. That's Sam, b a u m g a r t e n dot me. And our second shout out is to subscribe to our newsletter. If you want to get the updates and resources we only send to our mailing list members, then you need to be on our list.
Speaker 1:You can sign up by going to productpeople.tv/newsletter. Again, if you want to purchase your own shout out, you can go to productpeople.tv/shoutout. As always, if you have any feedback, you can get us on Twitter at productpeopletv or email us at productpeoplebizbox. Ca. We will see you next time.
Listen to Product People using one of many popular podcasting apps or directories.