Quarterly Survey Results, December 2008--84% Satisfaction Rating

The second installment of the Quarterly survey was released on December 14th with a total of 15,823 responses in ten locales. The survey was hosted for a week and a half by surveygizmo.com and displayed to 10% of users on the Firefox 3.05 "What's New Page".

Overall, this survey received half as many responses as the survey released in September (30,272). We also had some trouble getting responses from both Chinese locales, even when increasing the display percentage from 10%-50% of all Chinese-language users. Regardless, we have a confidence interval of 3.3 for zh-cn and 2.45 for zh-tw with a 99% confidence level.
All the other locales have between .5-.9 confidence interval and 99% confidence level. (I removed Question 6 from this overview because of a data recording error).

Responses have remained pretty consistent in the two surveys. About 52% of people found out about Firefox through a recommendation in this installment compared to 55% in the last survey. The consistency in responses to this question shows that the user's experiences with Firefox and their positive interpretations of these experiences drive further awareness of Firefox.

The next question "How long have you been using Firefox" was added to this installment of the survey in order for us to understand the backgrounds of the people answering the survey.


70% of the people who responded to this survey have used Firefox for more than a year, with the majority of that 70% (44%) having used it for more than two years. This may be because our longer term users (because of their high level of satisfaction) are more likely to take a survey about Firefox. It may also have something to do with the timing of the survey, with our hardcore/long-term users (who also tend to be online most of the day) more likely to be active on the Internet in mid-December than one of our newer/less intense users. Regardless, there may be a slight skew towards answers we would associate with the longer-term Firefox user.

















The primary reason for download of Firefox was performance followed by security and a recommendation--consistent with last quarter's results. I still don't really understand why customization is so low on the list, especially considering the responses to my add-ons question (#5 which shows 56% of respondents saying they have installed an add-on).We may have gotten such a high awareness(I'm calling awareness a positive response) because most of our long-term users know about add-ons--and with more than 70% of our respondents using Firefox for more than a year, this could be a very plausible explanation.















Looking at the cross-tab analysis of questions 3 and 5, it appears as though those people who have used Firefox for more than 2 years are most likely to have installed an add-on (53%). Users who have used Firefox for 1-2 years (green highlight) are almost equally likely to have an add-on as to not have one or to not know what an add-on is. What's even more interesting (at least to me, anyway) is that the the users who have had Firefox for 6 months or less are more likely to not have an add-on or to not know what an add-on is. As the length of time of use continues to go down, users are even less likely to have an add-on. This data suggests and confirms what many of us at Mozilla already know--that we have to do a better job of introducing new users to add-ons and their benefits during each new users first session(s) with Firefox.







In last quarter's results I broke down Internet usage into light, medium and heavy usage. From now on, we're not going to combine the different "usages" in order to try and learn the most we can from the information we get. It does look like the results from both quarters, however, are consistent, with mos of our users falling in the "moderate Internet usage" range (2-5hrs).















Lastly, and most importantly, we have our Quarterly Customer Satisfaction numbers...

We get this number by following the methodology of Net Promoter--Combine the 4s and 5 and subtract the 1s and 2s from that total and ignore the 3s. In this quarter we would add 65 and 21 (86) and subtract 2 for a grand total of 84% satisfaction rating. Last quarter our satisfaction rating was 89%. So far, the number seems pretty consistent--the real test will be with next quarter to see how we perform.



















You can find more in-depth results in the following files hosted on mediafire.com. Feel free to check out.

http://www.mediafire.com/file/ywz343zdxdo/report12-30-08 Report 12-30-2008.pdf (Original report)
http://www.mediafire.com/file/mlma2wykaj2/ResultsSurveyQ42008.xlsx (In depth for every locale)

Landing Page Progress and Analysis

As Pascal and Seth have blogged in the past month, we have been working on a landing page project to bring better visibility to the newer locales and to improve the user experience for non-English speaking Firefox users. We currently have 27 landing pages complete (you can read Pascal's blog entry here) and still have 10 remaining.

Since some of the pages have been running since early November, I wanted to share the page view numbers we're seeing (the locales in yellow do not yet have landing pages).


The first columns shows the number of page views from the relevant country of the en-us version of mozilla.com. I'm hoping this will give a better idea of how many people these landing pages may be able to help. The second column shows the number of page views of the relevant landing page for November, while the third column shows the page views for December. (If the second column has an "x" in it, the landing page was not ready in November).

These pages are generally accessed by users typing in "download Firefox" and their locale--for example, "Firefox Arabic" in either English or the native language. The landing page should display first organically on whichever search engine the user uses.

From looking at the spreadsheet, you can see that every locale that had a landing page available in November had improved page views in December. Although some locales have very low page views, like occitan (oc) only had 44 page views in December, we need to take into account the size of the population and when the page was available. I suggest that the page views for December are lower than an average month because of the holidays--we'll only really know that after another two months of tracking, but until then I do think the number of page views of the landing pages will continue to increase as more people find out about these landing pages. Although we cannot be sure of exactly where this traffic is coming from and how much of it is coming from newer versus older users, we can guess that the people hitting these pages are hitting them for a reason--because the language and the content of these pages are relevant to the users.

Regardless of how these pages continue to perform, I do think that creating these pages has been both helpful in improving the UE of mozilla.com and the existing Firefox and Thunderbird community sites that are linked to these landing pages. Even if Occitan continues to average 44 page views a month, I feel like the improved UE for those 44 users was worth the effort and work of the marketing and localization team.

I will report back about page views in January and periodically there after, but for the time being, I'd like to tentatively call this project a success. We will continue working on the remaining ten languages and start working on more locales this coming quarter.

Happy New Year!