The power and limits of transparency


Figure 1

Internet Explorer (IE) has been Microsoft's web browser for over 20 years, delivering many important innovations over this timeframe. In a feature by feature comparison of web browsers), IE has always done well. However, as Figure 1 depicts (source: NetMarketShare) Google Chrome has steadily been capturing IE's market share since Chrome was introduced in 2008. While the specific ways of measuring usage are subject to manipulation, the overall trend is obvious. Over this timeframe, Microsoft's development team has simply found it harder and harder to crank out new features; their original architecture has become more and more outdated, making it harder and harder to add new technology-driven functionality necessary for fielding a competitive product.

Web browsers fall into the software category of content and communications software. Since dynamic web pages were first introduced through the DHTML web standard in 1995, hundreds of follow-on technologies have been introduced. These technologies have become increasingly sophisticated, enabling web applications to be built which have gradually eroded Microsoft's traditional dominance in computing applications. These changes were built from a commodity-based hardware infrastructure known as cloud computing. This transformation unfolded as these technologies collectively represented a disruptive innovation for Microsoft's traditional operating system platforms, displacing them with cross-platform alternatives. This evolving platform exposed holes in Microsoft's traditional platform lock-in strategy which new platform entrants, such as Android and IOS could exploit. Despite continuing to offer a ton of new features, IE's market share was eroded by Google Chrome, and it became clear that IE would never recover.

Competition among browsers has always been cutthroat, since browsers compete for page views on different sites and across different devices, and through that means, create opportunities to capture bigger slices of advertising revenue. Each competitor in the browser wars strives to be the first to introduce novel technologies that will unlock new customers by operating at the frontiers of emerging standards, just to defend their position in page rank  (let alone grow it). Classic examples of these technologies in the past have included Active X, Microsoft Silverlight, and Adobe Flash; upon release, each offered advantages over other platform offerings available at that time.

When Internet Explorer 6 was released back in 2001, it had a market share of over 90% of the internet's web traffic at that time, but was widely criticized for its limited support of modern web standards available at that time. Since web sites optimize their performance for the browsers their end users use, they benefit from network effects, as complementary sites attract more and more traffic. These forces shaped IE into the defacto standard for that era. This dominance rarely depended upon compliance with what each IETF specification said about how an ideal browser's behavior should operate. These network effects in turn provided opportunities across the ecosystem for creating and adopting a particular combination of server-side technologies, which exploited common patterns their user community collectively traversed, by codifying them into the form of best practices. Note this virtuous cycle provided a lock-in whether or not they were compliant with a particular set of web standards. This reinforcing cycle created both significant first mover advantages and barriers to entry for follow-on competitors.

Unfortunately, by 2004, PC World was calling IE "the least secure software on the planet", just about the time that Firefox was introducing a credible alternative. Firefox supported many more standards than IE, and was believed to offer better security features (though the data just may have been reflective of how much bigger a target Microsoft was). It took Microsoft nearly 5 years between their IE6 release and their subsequent offering, IE7 (which was published in late 2006) to begin attempting to address this situation. When Internet Explorer 8 was finally released in 2009, it was the first to pass the Web Standards Project's Acid2 test, which had been designed to detect rendering problems of web pages relative to these standards. But IE then flat-out failed the follow-on Acid3 test, which hints that Microsoft may have 'coded to the test' for Acid2,  rather than taking the requirements seriously, and changing their architecture accordingly.

To attract new customers, and retain those still loyal to IE, Microsoft realized they needed to offer a new value proposition that would appeal to both types of stakeholders. They chose to pursue two areas of focus: speed and security. Microsoft decided to take a data-driven approach to planning their development efforts. In essence, the team built a model of the internet's most visited content by crawling the top million web sites, and analyzing which features were supported from each relevant standards. They then used that data to prioritize an implementation roadmap for their 'next generation' offering, such that they could maximize their chances in competing to regain some of the market share which they had lost over the previous decade.

Figure 2

Microsoft also made a real commitment to describing their reference architecture in well documented form, including traceability to the requirements and test cases they use for verification. In this way, they identified a pattern that other browsers in the industry tend to follow... which is partially driven by the functions and interactions implied by the various protocols for published standards such as html, css, and javascript. They invested heavily in tooling to produce this information and highlight the specific approaches they used for benchmarking their competition. They also promised a new focus on avoiding the classic delays that have troubled their prior major releases, by shifting from bundling the browser with the OS to making it a download in their new Microsoft Store, so that their historical tie to major OS releases could be cut. Their biggest challenge is that they had to somehow prod their partners to follow them in this transformation, even though such customer groups have been suspicious of Microsoft, and notoriously resistant to change.

Because of concurrent evolution occurring in their business model, Microsoft also had to make these changes as interdependent Windows teams were in the middle of a unification of their many code bases which had fragmented across the list of Microsoft Windows versions over the years. In parallel, thousands of partners and Microsoft's internal development teams were also lining up product releases to a common cadence that embraced weekly builds (and sharing those with 10 million new windows insiders), in preparation for rollout to hundreds of millions of users within the first month of new features entering service.

It has always been tricky to demonstrate a browser's compliance with web standards in a consistent fashion, especially since web sites routinely use a user agent string so that each web site can invokes special code tailored for each browser implementation. This means that to make a site work properly, its rendering must mimic the user agent's actual behavior, rather than whatever those pesky standards said, and no one really followed in the same way. Internet standards are often only based in a negotiated peace across the industry, rather than on a clear specification for implementation. This means that compatibility is less about what the standard says, and instead requires an understanding of how each browser's team had chosen to interpret each standard.

Peter Thiel explains how such dynamics unfold once companies can exert monopoly power:

In the real world outside economic theory, every business is successful exactly to the extent that it does something others cannot. Monopoly is therefore not a pathology or an exception. Monopoly is the condition of every successful business. Tolstoy opens Anna Karenina by observing: “All happy families are alike; each unhappy family is unhappy in its own way.” Business is the opposite. All happy companies are different: each one earns a monopoly by solving a unique problem. All failed companies are the same: they failed to escape competition.

A monopoly is good for everyone on the inside, but what about everyone on the outside? Do outsized profits come at the expense of the rest of society? Actually, yes: profits come out of customers’ wallets, and monopolies deserve their bad reputation—but only in a world where nothing changes. In a static world, a monopolist is just a rent collector. If you corner the market for something, you can jack up the price; others will have no choice but to buy from you. Think of the famous board game: deeds are shuffled around from player to player, but the board never changes. There’s no way to win by inventing a better kind of real estate development. The relative values of the properties are fixed for all time, so all you can do is try to buy them up. But the world we live in is dynamic: it’s possible to invent new and better things. Creative monopolists give customers more choices by adding entirely new categories of abundance to the world.

It was within this context that Project Spartan was launched in 2014. The Spartan project quickly introduced a newly architected, proprietary layout engine known as EdgeHTML, which replace Microsoft's long-toothed Trident engine. The Spartan team forked from the IE source for MSHTML in early 2014, leaving behind a branch that Microsoft warned all its customers would only be updated for high priority security and reliability fixes. This allowed Microsoft to make a clear break from the previous architecture. This new EdgeHTML engine served at the core of their new browser, which Microsoft bundled as the default browser with their flagship Windows 10 release, and which was branded as Microsoft Edge. This pivot included Microsoft declaring IE11 to be the last major release that would be offered for that browser, though it would continue to support that version to retain the compatibility necessary for the huge number of legacy applications that had not yet been able to make the shift to HTML5. To date, the Edge team has been highly productive, producing major releases in 2015, 2016, and 2017. With their focus on its performance, Edge has been able to achieve top benchmark results when compared with its competitors, dramatically improving the performance of Javascriptoptimizing battery life, and improving scrolling performance, to highlight selected examples of improvements to the product.

Figure 3

But in our modern world, as security intrusions have come to dominate the evening news, and even affecting our political system, how exactly should a determination be reached on whether a web site is secure? Putting a security group in charge of the web browser's design might be the last thing that would make sense, as it concentrates the knowledge, and creates a bottleneck.

Security is at once a system-level requirement and a set of properties that must emerge from the interactions of many components for a system of interest, especially for an ecosystem of platform providers, developers, end users, and the underlying operational environment in which work is performed. Like safety, the emergent behavior we call security can only be realized as a result of a complex set of derived relationships, few of which are intrinsic to a particular feature set of components of the system itself. Rather, such non-functional requirements depend upon characteristics of these roles and how they interact with these components within a changing environment. Did you click on that link that was in that mail message from someone you knew? Do you trust the source that sent that PDF file, to the degree you are confident their security practices are sound 100% of the time?

A much better security strategy is to work with experts across the industry to define measurable policies that produce a robust multi-layered security architecture, and provide information, tools, and data to enable their rapid implementation. A good example in Edge's implementation has been the strengthening of their sandbox. Implementing such a layered strategy requires collaboration across Microsoft's design groups themselves, and their entire ecosystem, rather than trying to empower some watchdog agency trying to pick winners and losers. The reality is, for Chrome and Edge, when browsing sites that a typical user visits, there isn't much difference in security today, though others may try to scare us by claiming otherwise. Google has been spreading lots of information about Microsoft's vulnerabilities for PR purposes. They need to learn that it is in everyone's interests to deal with such vulnerabilities through working together relationships across browser teams, rather than trying to score points in public forums by attaching other implementations. The aviation industry learned this long ago in dealing with safety, once they realized that the next accident may be your own.

Facts and data about a platform's status are thus crucial to awareness of an ecosystem's timing and implementation status to respond quickly to emergent situations. Quality management is equally essential for emergent situations to occur at declining rates over time, especially as page views continue to expand exponentially across the globe. Microsoft has made a significant commitment to providing such status transparently, and calls this behavior 'working in the open'. Such transparency is indeed necessary for them to re-establishing trust with their ecosystem partners, and offering a credible alternative for their future efforts. They must demonstrate a record of continuous improvement trends, since no one is capable of perfection out of the gate. While developers don't want to have to code around temporary situations, they also need reliable information to make informed decisions themselves.

Figure 4

As an example of this information, Figure 4 provides a summary of Edge's support for just a single feature, the brotli data compression algorithm, which was introduced into Edge in built 14986. That feature offers roughly a 20% improvement over deflate, the prior standard that 90% of browser users don't even know existed. In this snapshot of Edge's roadmap, Brotli is described as being introduced in a preview version, while the Box alignment feature above it is described as 'under consideration', having received 1002 votes for it to be included in a future release. This is the real power of transparency - being 'user driven', which means worrying first about the features affecting the highest number of users.

Edge's platform status also includes a searchable API catalog that summarizes all available specifications and identifies which browsers implement these specifications (not just Microsoft's); for example, here is a list of all the specification-driven APIs which only Edge supports, which indicates that in only 3 short years, Microsoft has made sufficient progress to full API implementation that they are able to begin leading the implementation of new standards, rather than chasing niche implementations that others have already introduced. All for a product that doesn't earn revenue itself, but enables Microsoft's other products to achieve that goal more quickly and effectively.

Edge has also committed to maintaining excellent performance as they implement these standards-based features. Their need to demonstrate this capability has resulted in a number of impressive demos. One of my favorites is a classic depiction flight simulator, written in Javascript, and exploiting WebGL. It demonstrates how all of this technology comes together to provide value. But while these demos are impressive, there is nothing quite as fun as following the design choices Microsoft has been making and found necessary for modernization of their document object model implementation.

Despite these impressive accomplishments, Edge is still in its infancy, and has only been able to grow their market share by a few percentage points since the Windows 10 Anniversary Update was released. I suspect many tried Edge when Windows 10 was first released, a benefit of being the default browser after each major OS roll. However, it doesn't take many trips through a site that works in Google but not in Edge, before even Microsoft fanboys throw up their arms and switch back. They have to not only get better, but get better at getting better. Their real challenge may be the Window's market share itself, which only has a market share of 25% on devices. One might assume once Windows 10 has displaced 100% of earlier Windows versions, Edge's share may be able to capture as much as 80% of the IE market. But Microsoft's dismal showing in the mobile space imposes additional constraints on the page views Edge can reasonably hope to achieve in the near term. That's why it seems unlikely that even with all their recent impressive hardware innovations, they'll need to get back in the mobile space somehow.

For those of us on weekly Insider Preview releases, one is never sure if a problem encountered on those releases is with the build or with residual defects that have been around since IE. I have tried to stick with Edge, but have run into 2 issues (failure to print return labels from Amazon, and freezes while watching a video from Construx) on the recent Creator's update. Last month it was inability to pay bills from my banking site. Each of these weakens my resolve. While Microsoft's commitment to working in the open is fantastic, Edge remains surprisingly brittle, even on production releases. Meanwhile, both Mozilla and Google are working on their own rearchitecting, behind the scenes. Any gains Microsoft has achieved with performance and security are likely to be temporary, as each development team is motivated to differentiate itself from their competition. While Edge's support of standards much improved and for which they've received good press, they are going to have to match Google's support of progressive web applications step by step, to avoid another box-in by Chrome.

When you first log in to IE11 in the Creator's update, you are now met with a message which dangles the idea of writing on web pages as a major motivator to cause a browser switch. Yes, I like the idea of writing on my web pages. Sadly, many of the web pages I've written web notes  on insider builds in the past have disappeared without warning, as have the ability to set tabs aside (which is really so they don't need battery-sucking attention). But on the whole, my primary expectation of a browser is that it 'just works', regardless of the correct implementation of a standard is used or not. As a user of web technologies in general, I need that functionality to be robust before features like pen-based input get much attention, especially when that particular feature still has a ways to go to achieve its potential in user acceptance. I notice Microsoft doesn't publish feature-driven metrics - how frequently are people writing on their web pages today? The raw numbers of page views matters little to Microsoft; revenue from search matters, as does the trend over time, and whether new features attract enough attention to motivate 20 million people or so to switch - because that's what it will take to make even a noticeable change in the numbers for the usage share of browsers. Such is the challenge of scale.

The above was written shortly after the Creator's update entered service. In the short time since the Creator's update, Microsoft has made substantial changes to the browser, and each of the problems has been resolved. More importantly, I have now discovered situations in which writing on web pages is amazingly useful, especially on a surface device that allows side by side windowing. I hope everyone will continue to give Edge another look, as it is now my preferred browser. Their next challenge will be to convince enterprise customers to migrate off of IE. The recent security issues with Meltdown and Spectre can only accelerate this acceptance, especially as Microsoft continues to mitigate the inherent risks.


Savvy businesses
Consumer mass market

Article rating: