Is your market competitive, or concentrated? What’s the difference, and how can you be objective about it and not just subjective? The United States government uses a measure called HHI – the Herfindal-Hirschman Index – as an objective measure of how competitive a market is. They use this measure to determine if a company is operating monopolistically, or to determine if a merger needs to be explored from an anti-trust perspective. The ask the question: would the merger create an anti-competitive market?
You can use this metric to get insight into how competitive your market is, and to gain a better appreciation for trends in your market. Read on to review the competitive landscapes and trends for email, search engines, browsers, and internet service providers.
Email Client Competition
It is hard to compete in the market for email clients (the applications people use to read their email). Usually people will describe this market as “very competitive” – meaning that there are some products that dominate the market. What people really mean is that it is “very hard to compete” in the email client market. Take a look at this market data from fingerprint.
[The graph shows the percentage of emails that are read in each email client by consumers (blue, on the left) and business users (red, on the right). Yahoo! Mail, Outlook, and Hotmail clearly dominate this space. Almost three million emails were analyzed to produce this market-share data. The emails were opened by people, and fingerprint determined which email clients people used to read the emails. It would definitely be hard to compete in this market with the introduction of a new email client – but is this market competitive?
Definition of Competitive and HHI
The United States government classifies markets based on degrees of competitiveness – competitive, moderately concentrated, and concentrated. They use an objective measurement, the HHI (Herfindal – Hirschman Index, sometimes called the Herfindal Index), to determine the proper classification of each market.
Calculating HHI
To calculate the HHI for a market, you take the market share captured by each company and square it. Then you add up those values to determine an HHI value.
A pure monopoly market is one where a single company has 100% market share. The HHI calculation for that market would be 100*100 = 10,000. That is the highest possible HHI value, reflecting the complete absence of competition.
A market with 100 competitors, each holding 1% of the market would have an HHI = the sum of 1*1 for each of the 100 competitors = 100. That is an incredibly low HHI value, reflecting an almost purely competitive market. [At this point, you are probably wondering, “How do I find out the market share of the 99th largest company?” – don’t worry, you won’t need to. We’ll cover that later.]
The US government has declared that markets be classified based on the following ranges of HHI values:
- HHI below 1000 – a competitive market. There are no dominant competitors in this market.
- HHI between 1000 and 1800 – a moderately concentrated market
- HHI above 1800 – a concentrated market. There are one or more dominant competitors in this market. Higher HHI values translate into fewer, more dominant competitors.
Anecdotally, the HHI values for the consumer and business email client data (shown in the graph above) are 2254 and 2650, respectively. While Yahoo! Mail, Outlook, and Hotmail collectively dominate both markets, Outlook and Hotmail dominate even more (relative to Yahoo! Mail) in the business email client market. The HHI shows this distinction with a 400 point (20%) difference between the two markets.
The US government uses this data to establish context – is a company operating in a concentrated market – one that promotes collusion, or is the company operating in a competitive (as in “free market”) market? The government also uses HHI values as a litmus test for determining if a proposed merger might be anti-competitive. If combining the market share of two companies results in an increase in the HHI calculation for a market of more than 100 points, then the move will raise anti-trust concernts. I am not an anti-trust lawyer, but my interpretation is that if a merger does not raise the HHI value by more than 100 points, that would be evidence that could be used to argue that the merger is not anti-competitive.
You can get useful insights about a particular market from the HHI value, that may guide your strategy for growing your market share, and apply them as part of your prioritization strategy. Since it is an index that ranges from 0 (for an infinite number of competitors with effective zero market share) to 10,000 (for a pure monopoly), a difference of a few points between two HHI values does not yield a lot of insight. A difference of hundreds or thousands of points does yield insight.
Avoiding Analysis Paralysis
One challenge you face is knowing when to stop gathering market share data, in order to calculate the HHI for a market. I gathered a couple hundred data points in a dozen different market analyses, and determined that there is a reasonable stopping point, where additional leg work does not yield additional insight. I’ve combined those findings in the following chart:
[As you record market share data, each new competitor adds to the total market share already analyzed. That value is captured on the vertical axis of the chart above. The horizontal axis shows how much the HHI value for any particular market will increase by gathering one more data point. When adding the data for one more competitor has little or no impact on the resulting HHI calculation, you can stop gathering data.
There are two distinct curves present above – the lower one represents a competitive market (HHI = 223), and the upper one represents multiple concentrated markets (HHI values ranging from 2254 to 6753).
[By the time you’ve addressed more than 50% of the market share, additional analysis increases the calculated HHI value by less than 0.1% (per additional competitor). In one market I analysed, the top 20 competitors combined had just under 50% market share. Including the next 50 competitors in the analysis increased the HHI by 2% (from 219 to 223). You can’t gain any insight from that difference in value. A good rule of thumb is to only look at the top 10 competitors, or however many it takes to capture 50% market share.
[In a concentrated market, once 90% of the market share has been calculated, the incremental increase to the HHI value by adding additional competitors is also negligible.
Looking at Trends
One powerful way to apply HHI measures for a market is to look at the trends of the market – is it growing less competitive or more competitive? Specifically, are dominant competitors gaining market share, or losing market share? Having this insight can tell you who’s products are succeeding, so you can focus your analysis effort. The following diagram shows how competitiveness has been trending for the last five years in the web browser and search engine markets.
[The lines represent the HHI values (downward sloping blue line is for web browsers, upward sloping red line is for search engines) that indicate the competitiveness (or concentration) of the two markets over the years 2005 – 2009. The inset on the left shows market share data for web browsers from 2005-2009. The inset on the right shows market share data for search engines from 2005-2009.
The HHI values for both markets indicate that they are both very concentrated – with a small number of companies controlling each market. From reviewing the HHI values, you can see that the web browser market is gradually becoming more competitive and the search engine market is becoming less competitive. Both markets are dominated by a handful of companies.
A Competitive Market – Internet Service Providers
The internet service provider (ISP) market, unlike the others we’ve looked at, is very competitive. The HHI value for the ISP market is 223. A pie chart of the major competitors shows that there is no clear dominance by any one provider.
[As a column chart (to be consistent with the other charts here), the same ISP Mar 2009 market share data looks like the following:
[There is not a set of clearly dominating ISPs – this is a competitive market – based on just looking at share data.
Danger
However, there’s a big danger in just looking at the numbers. Internet service providers, while competitive in aggregate, generally have dominant market share for any one physical region of the country. Rogers does not offer internet services in Texas, and Comcast does not offer them in my neighborhood. So, from an end-customer’s perspective, this competitive data is almost meaningless. It still serves as an example of a very competitive market, in contrast with the email client, web browser, and search engine markets we also reviewed.
Conclusion
Understanding the problems that plague the customers in your market is critical to being a successful product manager. Understanding your competition (and their solutions to the problems) in your market is also very important. The HHI value provides a metric for measuring competitiveness in particular markets. Your approach to competing in a particular market will be different, depending on the nature of the competition in (and competitiveness of) your market.
Note that this is a primarily red ocean analysis – a blue ocean analysis is a focus on finding ways to redefine your market so that competition is irrelevant. You can read about the differences in Blue Ocean Strategy, which is this month’s Smarter Product Managers book club book, created by Cindy Alvarez on the Booksprouts site (you may need to create a free account to view the page). Analysis of the existing market is still required in order to define a new market – you just apply your insights in different ways.
Scott, can you provide some more tangible examples of how you’d apply HHI knowledge? Ries and Trout recommend positioning products based on the competitive landscape. For example, a product that is second in its category should be positioned relative to the biggest strength of the category leader. But maybe there are different rules when the HHI is extremely high or extremely low.
@Roger [1] – Thanks for kicking off a great discussion! I’m not a positioning expert by any means, but here are a couple insights I think you can gain from HHI, that may drive different “competitive positioning patterns.”
A High HHI market is one that is dominated by one or a few players. They have “defined” the market. It is likely that they have used a small set of capabilities to establish what they hope will drive purchasing decisions. If they do dominate, it may also be that they have effectively established perceptions (with analysts, buyers, etc) that those are the factors that drive the market. I think this increases the likelihood that there are opportunities to identify persona (with unaddressed problems) or unaddressed problems for customers otherwise in the market.
When we talked about Blue Ocean Strategy last night, we felt that the book read like a “reverse engineering exercise” of previous market successes. We also agreed that creating a “market map” felt like a novel and effective way to visualize the problems being faced by customers in the market (and the effectiveness of solutions being offered to those problems). The authors suggest that discovery of solutions to the un(der)-solved problems was the way to redefine the market. When a market is dominated by a few large players, I think this strategy is more likely to succeed, because (1) there are more likely to be unaddressed problems, and (2) you are more likely to be able to position your solution as a novel approach with a distinctive value.
In a low-HHI market, with many active companies, I would expect to see a much broader array of “problems being solved” thus making the Blue Ocean Strategy approach less likely to work (or harder to execute). It may also be a reflection that this strategy is not effective in this market (for one reason or another). In that case, I would be inclined to pursue a Red Ocean Strategy with a focus on “more is better” capabilities – building a better mousetrap, not a does-more-than-traps-mice strategy.
Again – I’m not an expert, so these may be naive conclusions to reach, but I do think there is something there. HHI values would at least establish a hypothesis for me, and then as I engaged the market, I would focus on market-responses to course correct.
Roger, Under Moore’s TALC, the number two competitor would be positioned by the market, not the firm. Once the market structures a category, the top five really have no reason to worry about positioning. Those beyond the top five will find themselves competing more on a promo spend basis, so positioning matters.
The market establishes a structure that does not change. Until the M&As consolidate the market, or the market leader exits the market. I’ve run into far too many marketers that really believe that #2 can become #1. That only happens once #1 has decided to let that market go and focus elsewhere like when the category becomes commoditized.
Scott, thank for bringing the HHI to my attention. HHI provides another tool that a market leader can use to help them avoid being a monopoly. It’s hard to know where you stand in a market that you are creating.
Where would the data come from? It’s easy to do retrospectives, because the annual reports are available. How would you do the analysis in a forward-looking manner.
@David [comment #3] – First, thanks for engaging this very interesting discussion. I’m skipping a few of the comments until I can ‘catch up’ with you and Roger.
On HHI data – in an ideal world (populated with unicorns – thanks Justin Burrows for the imagery), you would just access freely available market research data like I did for this article. Or have access to for-a-fee data. Barring that, you have to estimate. You can identify competitive products and their companies. By researching the company announcements, press releases, securities filings, etc, you can create an estimate of their sales. You can ask customers who they feel your competitors are (and why). I think those are the old-school “who plays in your market” approaches. I’m not an expert on this.
What makes a lot of sense to me is to identify the problems you solve for your customers, and then identify other companies and products that solve those same problems. That seems like a much better way to populate the list, while gaining insight into your customers’ needs. For example, if you are going to start an airline, your competition is not just other airlines – it is bus lines, trains, and people who choose to drive. The problem being solved is getting from A to B, not getting from A to B in a plane.
It was a good read on HHI. Can HHI concept explain the frequency of new product introduction, as a strategy to gain market share, in high as well as low HHI market?
@Santosh – Thanks for commenting, and welcome to Tyner Blain! HHI shouldn’t have a causal impact on frequency of product introduction, and I guess I would be a little surprised if it had a correlation. Very competitive (low HHI) markets have many competitors and therefore many products. So there would be frequent product introductions into the market (even if each competitor released infrequently). However, one or more dominant companies in a very concentrated (high HHI) market could still be introducing new products / capabilities very rapidly. In Keeping Up With Change, there are a couple anecdotes about the rate of change of markets. I also wrote about using agile to establish a competitive advantage last August. So a dominant player (or someone trying to become a dominant player) can be using that strategy to rapidly introduce new capabilities into a high HHI market.
David, admittedly being only slightly versed in Moore’s TALC, I am very surprised by your contention that “the top five really have no reason to worry about positioning”.
Any competitor, no matter what its rank, can concentrate its features and messaging on certain sets of problems, certain sets of users, and certain sets of buyers. These concentrations, relative to those of competitors, constitute the product’s positioning. The decision of where to focus is deliberate and not just a complete accident of existing market perceptions.
Perhaps you mean something different by positioning?
@Roger [5] – I use basically the same definition of the word “positioning” that you do. Given that, I would have the same challenges to @David’s [2] comment. Still catching up on the discussion…
Roger, you may be in a different market. You may be web-based, so you skipped over the TE. EA, V, EM, and went straight to late market, straight in to the red ocean. You may be a complementor of someone elses product. Complementors always compete to be the market leader of the week based on last week’s promo spend. Late market/red ocean companies position themselves relative to their competitors. They brand. Oh, well.
But, if you fight it out in the tornado, when you do have to think about your positioning, and you come out on top of the category structure, you are set. Then, the market sell you. The market buys you, and everyone else gets your crumbs. You do have to play nice and not take the whole market. But, you have a market, not customers, and if you chose to be nice to customers, it’s a goodwill thing, and not a process necessity. The next event for that market leader is entering the late market, a bad place to be avoided, and the place where every web startup starts. Market leadership still matters in the late market, but you better start looking around for the next biggest thing.
As for those that find themselves in the late market, red ocean, sure act like a consumer goods company, market yourself that way, do what they do, but for a technology company, and not a technology-based company, there is a lot more money to be made focusing elsewhere.
If you had the choice of innovating or branding, what would you do? Yes, that is the choice, and for most techs, the answer seems to be branding, and commodity marketing.
@David [6] – I think my choice is “innovating” not “branding.” Of course I’m a product manager, not a marketer, so I lean towards “solve bigger problems better” before I look to “help more people understand why my solution is already better.” As a company, you have to do both.
One thing that confuses me in your second paragraph – you said “The next event for that market leader is entering the late market, a bad place to be avoided, and the place where every web startup starts.”
I really don’t understand where you’re going with that. I see Twitter as an example of someone who created a new market as a web startup – asynchronous real time communication. Real time was a red-ocean because of instant messaging. But it required symmetric relationships (people became “friends”). Many asynchronous communication mechanisms already existed (publish-subscribe model), but none of them were real time. The second movers (Pownce, Identica, etc) entered a “red ocean” here. I guess, by definition, most companies enter red oceans, since only one company can be first in any given ocean. Anyway, I probably just missed you point – if so, sorry.
David, you’re defining “positioning” differently and much more narrowly than I do. One of the most common mistakes that product managers make, in my view, is to assume that positioning is merely an outbound, promotional activity.
You asked what I would do if I had a choice between innovating and branding. The choice you pose is false. Innovating is part of positioning. It’s positioning when a product manager chooses to focus on a particular problem and go to market with an innovative solution to it. It’s positioning when a product manager chooses to focus on a market segment. Positioning is not just what you do to promote your product.
Positioning is the process of establishing identity and perception in the mind of the customer. To be sure, a part of establishing these perceptions is through outbound promotional activity. But the set of problems you choose to solve with the product also has a substantial impact on the perceptions.
@Roger [7] – great explanation and distinction between positioning and messaging!
Positioning arrises in the mind of the customer even if you do nothing.
As for a product being positioned by a product manager, I’ve never seen that. The product manager is positioned by the product, because they do not get hired until the product has already been positioned.
The product is positioned initially by the early adopter client, not by the vendor. Going into the early market, the vendor does need to position its offering. But, your prior positions from the bowling ally have already positioned you, or you will lose any advantage and risk reduction you earned by doing the bowling ally.
I read Trout and Ries back in advertising class and fought it out with a professor that had us reading Oglyvey and stuff like subliminal persuasion, all of which has been debunked in subsequent years. I wrote the best copy by surveying my potential customers. I undestand the orthodoxy. It used to be that a software vendor advertised only for investor communications. So how did they position? Certainly via means other than advertising. Solving the problem posititions. We agree on that. But, for me the client and only the client solves the problem, not the founder, not the team, not the product manager. The client understands their problem, their vertical, and their solution. A client is the solution for most software vendors. The posititioning remains anchored in the client visualizations.
@David [8] – You said “As for a product being positioned by a product manager, I’ve never seen that. The product manager is positioned by the product, because they do not get hired until the product has already been positioned.”
I don’t have to look any further than my engagement with my current client. They defined a product that solved a specific problem for a set of users. I worked with them to add/change the problems being solved, and the people for whom the problems are being solved. The product is evolving. The addressable market is evolving. The solutions being provided are evolving. The positioning (using Roger’s definition) is evolving.
You might argue that as a consultant, I’m using a loophole since I wasn’t “hired” as an employee. I’d argue that the company founder was the original product manager, even if he has other responsibilities. He’s still the “top” product manager, and my decisions are really just “suggestions” :). Is that any different anywhere?
Positioning in the mind of the customer – ok, maybe, but I think it is a semantic distinction. While every customer (or potential customer) has their own perspective on where a product is “positioned,” every company should be defining what they want that perspective to be, and working to achieve it. I’m not willing to take a passive perspective on positioning (e.g. we are what our customers say we are) – that’s a valid static analysis. I’m going to engage my markets and help drive our customers and prospects to adopt the perception that my product is where I can compete most effectively.
David, someone playing the product management role does position the product. Someone decides which market segments to target, what problems to solve, and what perceptions to instill in the mind of the customer. Unfortunately, it’s true that in many cases the person(s) playing that role is not trained in, or skilled at, product management.
The client, product, and company team up to solve problems. The company makes deliberate decisions on which clients and problems they will target and what they will put in the product to help solve the problems, so the client, early adopter or not, has a limited role in determining positioning.
This debate isn’t merely semantic. It has many practical implications. After market research, positioning drives just about every activity a product manager performs.
@Roger [9] – I wonder if we’re also wearing the same color shirt today :). Completely agree!
Good stuff. Any considerations for calculation or application of HHI when market share stats available are revenue based instead of quantitative?
@Andrew – Thanks for the great question! The PC industry is a really interesting one, in that market share is reported both in units (shipped) and revenue (collected). The revenue share data, though, is primarily giving insight to the investment community. HHI is useful when considering “how many (of the) customers do you serve?”
If I only had revenue data, I would develop an estimate for average selling price (ASP), and back out an estimate of “units sold.”
Scott,
On problems solved, the underlying concept that drove Christensen’s stating the issue in that way begins with what sales reps call the FAB framework. FAB is features, advantages, and benefits. In software, benefits are task performance, or doing a job. Advantage claims come from weak differentiation.
On marketing vs. tech, see “War in the Boardroom” by Al and Laura Ries. It talks about the way left-brain managers and right-brain marketers. Being both a manager and a marketer, I see the need to do both technological innovation and branding, so no, I’m not passive. Being a value merchant, I know that I have to address a wider range of messaging that just innovation and brand.
As for category emergence with its market leader, and the market leader’s next challenge, I go back to Moore’s technology adoption lifecycle. Entry into the late market kills companies. You either arrive reaady and conscious of it, or you go out of business. It is also in the late market that you have to brand, as in advertising, because you are in the red ocean. Frankly, I’ll come out with another disruptor, and leave the end of life to a product manager. Entry into the late market is the end of organic growth for that technology and the products based on it.
Innovation is a wide word that I tend to use narrowly and others use broadly. In “Living on the Fault Line,” Moore talked about the need to limit differentiation to being that that matters. I’ve worked in places that changed the red background art to green. Call that an innovation if you must, but if you follow the technology adoption lifecycle, there are times to innovate the carried, the benefits, the job being done, and the rest is innovating the carrier. Most of the discussion by product managers is about the carrier, or churn to the real business value of the carried.
As for Twitter, it is currently in the geek/technical enthusiast (non-)market, a non-market, because you can’t monetize this market, rather you leverage it to reach people other than yourself who have a real job to do. Enabling this you end up with a product to sell into a vertical. Then, you repeat. And, then …. The place where we really innovate is pretty distant from the everything is innovation crowd. Some of that crowd never ever visit that place.
The real place to compete is in the construction of a category, and the winning of the market leader spot in that category. Then, oddly enough, it is the market, rather than customers, that gets the buying done. But, this only lasts so long. Microsoft branded quietly for years, and for investor communincations, rather than product. This beyond the geeks. They also provided free technical support, but not because the market made them do it. You might say that they spent money on tech support, documentation, etc. to create a market barrier for other software startups. We mimiced them at our expense, but since we were not the market leader, customers demanded this from us.
All processes are phase specific in the technology adoption lifecycle. What I will do in one phase, I may not do in the next. I may wait to do something that everyone else things of as being constant and always.
On the proprietary nature of HHI, I’ve run into the profit pool concept, which is very interesting, but when you want to create profit pool representations for your industry or category, or potential blue ocean competitors, you run into the problem of finding the data. Historical data is a matter of public record, but forward looking or real-time data would involve collusion. You might be able to do that if you have a standards body approving the standard before members code competing products. That standards body could gather the data without legal issues, but then you have a best practice situation where everyone is doing it, so it provides no competitive advantage.
I’m also reading “Hype Cycle.” It presents another proprietary data situation.
@David [22] – You said “Frankly, I’ll come out with another disruptor, and leave the end of life to a product manager.” Personally, I think someone who has only “changed the red background art to green” is really a product steward and not a product manager. I approach every product management assignment as an opportunity to discover valuable opportunities to solve new problems. In the “innovation” framework, this is developing / leveraging distinctive competences for companies, and providing valuable differentiation in products. That can happen by extending a company’s reach into new markets, or into solving new problems faced in existing markets. I don’t believe those behaviors are limited to emerging categories.
Thanks again for the great discussion!
Interesting analysis. You can also look at power law distributions. For example I find that most stats for my website follow the Pareto 80/20 power law – e.g. 80% of visitors come from the top 20% of ISPs or countries. However search engines results are noticeably different with >80% of traffic coming from Google alone.
@Andy – thanks for the insight. I wonder what levels of HHI are consistent with Pareto? Perhaps it depends on how you count. Also, I suspect Pareto only applies “on average” – for example, I get three times as much search engine traffic from Yahoo as I do from Google. If you combine our stats, it probably gets closer to the 80/20.
Hi Scott,
Thanks for this article. Very interesting. Is this ISP Household market share or Revenue market share? Would you mind sharing the source of your ISP data?
Thanks!
Michael