Change is Bad? Mistakes are Worse

CNET ran an article about a month ago about how Microsoft Live Hotmail – the upgrade/replacement of Hotmail failed to win a lot of converts. Microsoft ultimately had to deliver a “classic” version of the new code base to mimic the behavior of the old software. CNET’s analysis, while accurate, was not particularly insightful. The analysis was done by a staff-writer / journalist, not a product manager. The real problem is that Microsoft tried to do the wrong stuff – and failed anyway to meet their own goals.

CNET’s Conclusion

The CNET article, while interesting news, with good background stats and interviews of the right people, seems to lead readers to what I believe to be the wrong conclusion. The article points out that the new ML Hotmail took a long time to load and presented a lot of changes for users. The article then goes on and on about the “people don’t like the changes, therefore people don’t like change” angle. Even one of the bloggers we read got caught in this snare – amplifying the “change is bad” argument.

The whole change is bad argument is a red herring. The issue isn’t with people’s reluctance to change. It’s with what they were being asked to change to – and why.

Our Analysis of Goals

A little digging discovered a blog entry from Richard Sim, a senior product manager on the Microsoft team, where he outlines the original goals for the “new” Hotmail.

Here is some background… When we decided to build a new web mail service several years ago, we had two major goals:

1. Build a faster, simpler and safer mail service for our Hotmail customers
2. Create a service powerful enough to meet the needs of a broader set of customers (for instance, did you know our mail is also the mail behind the new Office Live service?)

Richard Sim

OK – those might be good goals.

Lets assume that the internally identified goals were to grow or retain market share for Hotmail. Lets also assume that research led the Microsoft team to conclude that speed, ease of use, and security were the user goals that most needed to be addressed to achieve Microsoft’s corporate goals. If that’s the analysis that Microsoft performed, then the first goal would in fact be aligned with that research. If their research also indicated that they needed a more capable interface to attack another segment of the market, the second goal would also appear to be a good goal.

The CNET article glosses over the stark contrast between what Google and Yahoo were doing at the same time. Google and Yahoo were focusing on chat / instant messaging. Apparently they felt like their efforts were best spent helping people communicate more effectively. IM has been very effective with tech-savvy users over the years – and integrated chat may be a killer feature for email. The middle-schoolers I know have all migrated to GMail over the last year, because the embedded chat client allows them to communicate and engage more effectively. This same intuitive “its just there” interface may also help people on the “big” side of the chasm too.

So maybe better, faster, stronger isn’t the right set of goals at all.

Google took an approach of creating differentiated innovation when they integrated chat with their mail client. Yahoo is now playing me-too. Microsoft, meanwhile, established goals around a “more is better” strategy, avoiding innovation. As a side note – the “full” interface, the one struggling with adoption rates, according to the CNET article, is a “make it more like Outlook” interface. Is there a word for “kinda innovative?” No one else has Outlook on the web, so that might be a source of innovation – but it isn’t likely to change the way people communicate. Adding chat was disruptive.

Our Analysis of Execution

The Microsoft team had a problem early in development – the new feature-laden email solution was too slow to load. With fully a third of their users on dialup, this was a deal killer. Research is important. The team should have known in advance the environments that they needed to support. As an example, here’s the platform analysis we did while defining nexus. With over 75 million users per month on dialup (extrapolated from ComScore Media Matrix data provided in the CNET article), they should have established some measurable criteria for “fast enough.” And they should have known immediately the kinds of designs that this constraint would eliminate. As it happens, they had to bolt on an “alternate interface” for those users. And late in the game, they had to make that the default interface – allowing the “fatter thin client” option to only those who asked for it.

So, they blew it with “faster.”

What about “simpler?” Well, the only version that was fast enough is “essentially the same” as the old Hotmail. Can’t be both simpler and “the same.” Oh fer two.

Safer? Probably. Will that matter? Only if there was a perception of a safety problem – or a risk that one would arise. They may have had to do security work just to “keep up” with Google and Yahoo. If they spent a lot of time making it safer “beyond the point of relevance” that would be bad – but we really have no way to know without having the same data that they have.

What about goal number two – “more power?” Yes, an Outlook-style email system would certainly be more powerful. And if that’s what people want, it will work for them. But people aren’t going there in droves (at least yet). Based on the feedback that the Microsoft team quoted, most users wanted the “old Hotmail interface” – so goal number two probably shouldn’t have been a top priority.

The CNET article focuses on the demoralizing effect on the Microsoft developers of spending less time on the “power interface.”

This hints at poor internal communication by the product / program managers. Developers like building cool stuff, sure – but if you can’t motivate your team to be excited about building the most desirable stuff, you aren’t doing your job. Developers can easily be rallied around an “own the market” mission. Don’t make a widget that is impractical to use (due either to user confusion or design constraints). Make something intuitive and impossibly fast. You can’t convince me that the developers on that team would not be up to (or at least motivated by) the challenges. They just need to understand the context, and have some guidance.

Conclusion

So, execution on the goals that were chosen seems poor. Faster and easier were stated goals. The new application is slower and/or the same. Missed on both counts.

More strikingly, the goals were “inside the box” goals – evolution, not revolution. Both Yahoo and Google are busy changing (and arguably improving) the way that people communicate while Microsoft was trying to make email “a little better.”

The CNET article completely missed this angle, and could easily lead you to the conclusion that change is implicitly bad. Irrelevant change is bad. Having the wrong goals is worse. Failing to achieve those goals – worse still. You can’t succeed by missing the wrong goals.