Tuesday, October 21, 2008

Results for the Q3 Quarterly Customer Survey--First Look

After some thorough initial analysis, I am happy to share the preliminary/non-locale specific results of our first Firefox Quarterly Customer Survey. In general, it looks like this data confirms a lot of our "hunches" about our users and gives us more information about how we are doing when it comes to customer satisfaction--what some people view as the most important metric of this survey. At the same time though (surprise, surprise), there were some results we didn't expect. For this time around, I'm going to focus on our expected results--I'll go through the unexpected ones in the next posting. Before I get going, let me emphasize that although these results are interesting, we won't really have a good idea of how valid these results are until we've gone through a few rounds of quarterly testing. Also, I have not included question #4 for analysis ( Which of the following browsers have you used in the past week? : Internet Explorer, Safari, Firefox, Opera, Other) because we got conflicting answers from people, probably due to the wording of the question. We will reword the question for the next survey.

Here's a quick overview of the survey:
This survey was released on the 23rd of September and was available to 10% of FF3 updaters of both 3.02 and 3.03 in 10 locales, including: Japan, China, France, UK, Spain, Italy, Brazil, Poland, Germany and the United States. The basic goal of this survey was to create a standard the Marketing Team can use to periodically measure our users' awareness, usage habits and attitude towards the Firefox, a goal that I think we achieved.

Below you will find the six questions and some analysis. Others in the Marketing Team will also post some of their opinions, and in a future post, I will do more analysis of the responses to these questions by locale.

To begin, the six question survey was completed by 30, 272 people (the results are statistically significant)--a much higher number of responses than anyone expected. The distribution of respondents is here:

Locale frequencies


1. How did you find out about Firefox?:
Recommended by a friend or colleague
Offline newspaper or magazine article, review
Online banner ad
Search engine listing
Online article, blog, review
Other

If "other," please specify. Most popular answers were: "Cant remember how they started using it" or they've been using it for so long, they have no idea", "they've been using it since the beginning".

55% of responders found out about FF3 because of a peer recommendation, a finding that I expected in that I thought most people found out about Mozilla products by word of mouth. I did not expect that such a large majority of people would find out by word of mouth, imagining response rates closer to 40% and the other responses to be more evenly distributed. At the end of the day, seeing that the majority of people learn about Firefox through word of mouth is an excellent sign. For people to make the effort to recommend Firefox to another person means that they believe in our product enough and are satisfied enough to want to share it with others. As long as our satisfaction rates remain high, the rate of people finding out about Firefox from a peer will remain high--they are inextricably tied.




2. What is the primary reason you downloaded Firefox 3.0?:
Security
Customization
Performance
Open source
Recommendation
Other

If "other," please specify. Most popular answers were "Automatic update" or "All of the above".

This first graph shows that overall, performance seems to be the biggest reason people are choosing Firefox. Security, however, is not far behind--results we expected. I did not expect that customization to be so low down on the list, probably because I've always seen the ability to customize as the main difference between Firefox and the rest. At the same time, however, to have performance as the most popular reason people downloaded Firefox means that we're doing a good job--it means our product is doing what it's supposed to do and doing it well.

One reason customization may not have performed so well is that people don't know how to understand the term "customize"--i.e. they don't know that an add-on means customizing their browser. It also may be possible that people just don't know anything about add-ons, but that seems unlikely if you look at the results for question 3. I'll look into this more deeply for the next post.













3. Have you installed an add-on (including themes) to customize Firefox 3.0?:
Yes
No
Unsure
I do not know what an add-on is

The good news about this results to this question is that the majority of our users know what an add-on is and have installed one. At the same time, I think that for +20% of our users to not know what an add-on is seems a bit high. The next question to ask then is why? Is it an awareness issue? Is it an issue with the AMO site? Maybe people don't understand that installing plug-ins is the same thing as installing an add-on?

The next step here is to compare this data to our user profile information to try and figure out how a user's profile may explain this lack of awareness of add-ons--ie the people that don't know what add-ons are the ones who are on the Internet the least.

5. On average, how much time do you spend on the web?:
More than 8 hours per day
Between 5 and 8 hours per day
Between 2 and 5 hours per day
Between 1 and 2 hours per day
Less than 1 hour per day

This graph is pretty self-explanatory and the results are representative of what we expected. Most of our base is made up of moderate to heavy users, although there is a good number of light users. I think that the light internet user segment of the population is where the most of our growth needs to come from in the coming months, but its nice to see a good proportion of them responding to this survey. (see D Bottoms' Comscore data).














6. On a scale of 1-5, with 1 being 'not likely' and 5 being 'most likely', how likely are you to recommend Firefox to a friend?:

This last graph is the one that I think most people will end up thinking about and following the closest. To clarify, we got this result by combing the answers of people who responded with 1, 2 or 3 into the "unlikely" to recommend column and combining the answers of the people who responded with 4 or 5 in the "likely" to recommend column. Taken together, we have an 87% likely to recommend rating which can also be understood as an 87% customer satisfaction rating. This is an incredible number, one that few other products reach. As such, we will continue to monitor this rating and to see how it continues to trend over the coming quarters.

Be on the look-out for more posts in the next few days. I'll be posting a complete pdf version soon.


2 comments:

The Jack said...

Thanks ever so much, very useful article. Great information! Very smart, thanks for kindly sharing it with us. Very well done indeed. If you do not mind, please visit my article related to pandeglang district in Banten, Indonesia at Kenali dan Kunjungi Objek Wisata di Pandeglang or Kenali dan Kunjungi Objek Wisata di Pandeglang and information about district Banten Indonesia Banten dan Kabupaten Pandeglang

Daniela said...

Interesting post. I have been wondering about this issue,so thanks for posting. I’ll likely be coming back to your blog. Keep up great writing.

Travel News | Free Song Lyric | A Love to Kill | Learn Indonesia Language | cari rumah | jual rumah | rumah dijual | Laptop Price List | New Blackberry | Health & Jewerly