Starting July 1st, 2009, every download you generate from your Firefox 3.5 Affiliate buttons gives you an opportunity for being rewarded as an active member of the Spread Firefox Affiliates Program.
Everyone with more than five downloads a quarter will be entered into a reward pool. We will randomly award ten individual Affiliates each quarter with incentives like a Flip Video Camera, an iPod Touch, an iPod Nano, Amazon Gift Certificates and exclusive Top Fox T-shirts!*
Awardees will be notified in the beginning of October 2009 for activities between July 1st-September 30th, so make sure your email in Spread Firefox is up-to-date! You can learn more about the program here and make sure to check out the Affiliates Terms of Service.
Last but not least, any Affiliates buttons are eligible for this program, but why not update your buttons to our latest and greatest dynamic Affiliate buttons and help us "Upgrade the Web"?
Thanks to John Slater, Rhonda Spencer, Elise Allen, Alex Buchanan, Stephen Donner and the WebQA team for all their hard work--I'm really excited to see how this program continues to grow!
Happy Firefox 3.5 launch! Note*: Once you have been chosen to receive a reward (not including t-shirts) you will be ineligible to receive another reward for a full year.
One of the projects I have been working on for the Firefox 3.5 is a new dynamic messaging campaign for the launch called "Upgrade the Web". The idea first started as an IE switch campaign but morphed into an upgrade campaign for all browser users.
There are three main touch points for this campaign: -Affiliate Buttons -Word Press Plug-In -Google Gadget Ads
The idea is that each one of these Firefox promoters dynamically displays a different message to the viewer depending on what browser they are using. We made an effort to make the tag lines cute and cheeky (thanks Elise!) and to make the graphics snappily emphasize (Thanks Sean!) the "new and improved-ness" of Firefox 3.5.
Here are the buttons and their respective messages:
These same messages are also being used in the Word Press plug-in and the Google Gadget Ads.
Put these buttons on your blog or website and help spread the word about Firefox 3.5!
[You can get the Affiliate buttons or the plug-in from the "Upgrade the Web" program homepage. To download the buttons (and get credit for the downloads you generate) you need to be a Spread Firefox member. ]
Thanks to Elise Allen, Sean Martell, John Slater, Alex Buchanan, Mike Hostetler, Chris Blizzard, Stephen Donner, Krupa Raj for all their hard work--Awesome job!
Join Tara and I for a fun workshop where we talk about how you can use your creative skills to help "Spread Firefox" and promote Firefox 3.5! Here's what we're covering: * Why Creative / Visual Design is important * How to Contribute * Design Guidelines * Community Store / T-shirts * What are Affiliates * Why are they important * Existing buttons / Creating Buttons
Thursday, May 7, 2009 9:00 am PDT
Dial-in Info: +1.650.903.0800, followed by 92# and then 7391#
Or you can use our toll-free number: +1.800.707.2533, followed by 369# and then 7391#. If you’re outside the US, use Skype to call in with our toll-free number.
If you can’t join the call — but want to ask questions — you can join us in #marketing on IRC (irc.mozilla.org).
Please sign up on Spread Firefox. We’ll be archiving the Air Mozilla episode and sharing the presentation if you can’t make it. We’re looking forward to tomorrow!
The much awaited results of the all.html redesign test are here!
What were we trying to do?:(You can skip to the Results section if you already know about this ;))
In September of last year, I started looking at the current all.html page to see if there were ways to optimize the page. We had a high bounce rate (40%) and I felt that the page was visually confusing and that there was a better way to display the information listed.
I then asked The Royal Order (an amazing design firm) to create three designs to address the problem of trying to display 60+ Firefox localizations and their platforms to a worldwide community of people with different needs (ie, language spoken, ability to read English, various levels of tech-savvy, country of download, etc).
These were the designs we received:
All is the current all.html page. (This picture is of the older page because we had already taken the all.html page out of circulation by publication).
All-1 was our most "designed" of the pages that included a map. ((This picture is of the psd because we had already taken the all-1.html page out of circulation by publication).
All-2 was in the middle, with some geographical images and expandable lists.(This picture is of psd because we had already taken the all-2.html page out of circulation by publication).
All-3 was the most like the current all.html with some changes to some details like # of columns.
After looking at them, we were unable to decide on face value which one should be the new replacement. We all had plenty of opinions, but nothing based on verifiable data. Consequently, we decided that we would test all three designs (and the current design) to see which one performed the best.
The Test: We set up a week long test that ran from March 21-29, 2009. We rotated the pages evenly so each page showed for 15 minutes of every hour for the entire period. We defined success or "best performance" as:
1) Best User experience on the page comparing to all.html
2) Best download rate on the page comparing to all.html
(You can read more details about the test/process here and here.)
The Results: The results are organized by version of all#.html and broken up into page views and downloads. The conversion rate was found by dividing download clicks by page views. The baseline we used for all comparison was the all.html page. The baseline, therefore, is that 52.7% of people who hit that page downloaded a version of Firefox. Better Performance would then be defined as anything above the average of 52.7%.
All-1.html performed much worse than expected, with 32.4% of people downloading Firefox after hitting that page. All-2.html, performed at 56.5% which was on par with my expectations.
Lastly, All-3.html was comparatively stellar, getting about 60.9% of the people who hit that page to download. This basically translates to an 8% improvement over our current all.html page.
Now, before we get TOO excited about these results, resident Metrics Guru, Ken Kovash, asked me to do some further digging. Basically, he was concerned because "download clicks" represent a snap shot of a site visitor’s experience at a single moment in time and it doesn’t factor in what actions a person takes after that moment (e.g., a download occurring when that person subsequently navigates away to other pages within Mozilla.com).A Firefox user could hit the all.html page, then move on to the mozilla.com homepage, and then click download.In this example, the action of a download is not counted in our numbers and results above.
To figure this out, I looked at a funnel report in our web analytics system for all-3.html and all.html to try and figure out where people were going after the all.html pages to download (if not from the actual page) and to integrate these numbers into our results above.
All.html
First column you see below is total number of page views.
Second column is the total of downloads from the all.html download buttons (all of them).
Third columns is the total of downloads from the entry page (mozilla.com)
Fourth column is the total of downloads from download buttons that are not on one of our top pages (eg customize).
Fifth column is total after subtracting colunms 2, 3, 4.
Sixth column is the number on the total downloads/page views which results in an average of 67.6% conversion rate.
All-3.html First column you see below is total number of page views.
Second column is the total of downloads from the all.html download buttons (all of them).
Third columns is the total of downloads from the entry page (mozilla.com)
Fourth column is the total of downloads from download buttons that are not on one of our top pages (eg customize).
Fifth column is total after subtracting columns 2, 3, 4.
Sixth column is the number on the total downloads/page views which results in an average of 68.4% conversion rate.
Seventh column is the difference in conversion rates, with the average difference between all-3 and all.html being .89% (in favor of all-3.html).
Synopsis All in all (pun intended) all-3.html was the best performer of the four pages. The 8% difference in user experience and .89% improvement in download rate (comparing to all.html) shows that even a small change in a page's design can have an effect on download numbers. And even though 8% improvement in ease of use or a .89% improvement in downloads may seem small, considering the number of people who see this site every day (70-90K page views a day) the effort and hardwork that went into this redesign and test were well worth it.
All-3.html has now taken the place of the old page and I'm looking forward to tweaking and testing some details and re-checking its' performance in the next few months.
Thanks to everyone who has put in so much hardwork these past two quarters: The Royal Order, Steven Garrity (Silver Orange), Wil Clouser, Jeremy Orem, Stephen Donner, Krupa Raj, Ken Kovash, Seth Bindernagel, Mike Morgan and John Slater--couldn't have finished this project without you!
After looking at last quarter's survey results, I started to get concerned about the validity of our high "customer satisfaction score". Although our responses have been rather consistent, the drop in satisfaction rate between Q3 2008 and Q1 2009 got me a little worried--mainly because it was a 4% drop.I started to re-examine the data and it appears as though there are many factors that should be considered when looking at the “drop” in “satisfaction”.
The first flag was what appears to be a drop in survey participation over the past few quarters. During the first installment of the survey we had 30,272 respondents. That number has dropped by nearly 50%, with 15,823 respondents in Q4 of 2008 and 15,682 respondents in Q1 2009.
Here are a few possible explanations for this “apparent” drop in participation over time:
1) The novelty of the first survey encouraged more participation.
-Even though we have 100 million people seeing the “What’s New” page with every update and we only allowed the survey to appear to 10% of that population, there is a self-selecting bias in any opt-in survey.
-The further we move away from Firefox 3 launch, the lower the enthusiasm for the product. I'm not saying that people have stopped loving Firefox 3, I'm just saying that the further we get away from the launch, the less likely people are to be willing to take a survey to tell us they love the product. (Its’ awesomeness, in essence, has become the status quo).
2) The first survey was released with 3.02 and 3.03 (released a day later), which may have inadvertently increased the number of people who saw the survey link, and consequently, may have artificially doubled our participation with the survey’s first release.
I think option #2 is really the only one that makes sense in this context. It’s easy to see how (self-selecting) individuals would have taken the survey twice and thus inflated our initial “customer satisfaction score”.And since the past two surveys have been consistent in participation and responses, I think that it may be worth completely throwing out the data from the first survey.
Other concerns about the survey came from looking at the responses to certain questions.
For example: "How long have you been using Firefox?"
About 50% of respondents to this question say they have used Firefox for more than two years. That response rate doesn't quite make sense to me if we are trying to get responses from a cross-section of all of our users. Those numbers would make more sense coming from our user-base or long term, loyal, customers. Other survey questions that were similarly concerning: "Have you installed an Add-On?" (50% said yes), "On average, how much time do you spend on the web?" (2-5 hrs a day). The responses to both of these questions again seem to indicate that a large number of the respondents to our survey were part of our core fan-base as opposed to part of a more general cross-section of users. (You can take a look at the Comscore data Dave Bottoms gathered last year if you want to know about the average Internet user.)
Another red flag was the number of people who started but did not complete the third installment of the survey. The survey released in December ’08 had a total of 37 “abandoned” surveys. The most recent installment of the survey was “abandoned” 5419 times. Why the huge jump in abandonment rate? I think it comes down to two changes we made to the survey:
1) We increased the number of questions from 8 to 10.
2) We reformatted the survey so that the questions were split equally on two pages.
I think both of these changes, though well intentioned, really took us away from our main goal of this program—to create a simple, short survey that would efficiently allow us to get a better picture of all of our customers.
To review, here are the takeaways:
1) Q32008 Results and “Satisfaction Score” are probably inflated and should be disregarded.
2) The surveys tend to be taken mostly by long-term, loyal Firefox users. We need to find a way to get responses from a more diverse cross-section of Firefox users.
3) We need to pay attention to the design of our survey to ensure we don’t scare users off with the number of questions/layout of the survey.
So, what next? I think our next steps are to step back and try and get a better understanding of what the data we have collected in the past nine months *really* means. That also means trying to figure out what we have previously called our "Customer Satisfaction Score" is really measuring. Is it a customer loyalty score? Is it an enthusiasm score? At this point, I'm not really sure, mainly because I don't know the extent of the biases in our data.
It would be great to hear your thoughts and suggestions: what can we do differently? How do we get a better sample of all types of Firefox users? How can we try to minimize bias?
The third installment of the Quarterly Satisfaction survey was released on March 6, 2009 with the 3.07 update. The survey was displayed on the “What’s New” page to 10% of the incoming population and was completed by 15,682 people in eleven locales (Hebrew is the newest locale to join the survey distribution).
(For the results of the last two quarterly surveys, click (Q308 or Q408)
Overall, I’m happy to say that the responses to our survey have remained consistent from last quarter, with our current customer satisfaction coming in at 82.6%. Although this is a drop from last quarter (84.3%) the difference between the two is 1.7 percentage points, which in essence, almost identical from last quarter. A comparison to and discussion of some differing results from last year will be explored in a follow-up post.
Looking at the rest of the results, we've pretty much stayed consistent with our results from Q4 of 2008.
Recommendation/Word of Mouth is the main way people find out about Firefox, with PR outreach (online and offline news & blogs) coming in second.
This next question was added in Q4--results have remained about the same, with most responders (75%) using Firefox for more than a year.
Performance is still the primary reason for download with security coming in as the second most popular reason.
Question 5 is interesting to me because it indicates that most people are aware of the ability to customize Firefox, but Performance is still the stronger reason to use Firefox.
I included a good portion of the results for question 6 so everyone could see what other browsers our users are using. I was also surprised by the low number of responses that named websites as opposed to browsers--I was expecting a little more terminology confusion.
This is a new question attempting to better answer why people use other browsers beyond Firefox. The main reason people use other browsers, seems to be a lack of compatibility--some websites users want to use are not behaving properly. The next major reason is because people aren't able to use Firefox at their place or work or their school. Both of these reasons were very much expected and we are continuing to work hard to be as compatible as possible.
This question was also pretty consistent, with most responses coming from the mid-usage range.
Here is where we usually calculate the "Customer Satisfaction Score". We do this by adding up the % of people who answered 4 and 5 and subtract the number of people who responded 1 or 2. For this quarter, we have a score of 82, which is about 1.5 percentage points less than last quarter. (Read this blog post to learn more.)
This blog was my attempt to memorialize my trip to Europe, particularly the interesting bits. Now, it will be used to document what's going on in the Mozilla Corp Marketing World.
