Intermittent Issues:
HD and the Format Wars (2002-2005) Part 1
By Ben Gruchow
July 16, 2015
BoxOfficeProphets.com

Days of BOP's Future Past.

Part 1 of a 4-part series on the advent of HD cinema.

“People thought it would come down to pixel rate or refresh rate, and they’re pretty much the same. What it came down to was a combination between gamers and porn. Now, whichever format porno backs is usually the one that becomes the most successful. But, you know, Sony, every PlayStation 3 has a Blu-ray in it…”

“You talkin’ to me this whole time?”

--Tropic Thunder, 2008

Full disclosure: I was that guy throughout most of 2007 - the one who gave everyone around me a lesson in HD-DVD/Blu-ray comparisons and tech specs, whether they wanted one or not. The character’s logic has precedent, by the way; despite Betamax being a superior format to VHS on the A/V end, the adult-film industry did back the latter, and VHS did win the format war. Of course, the main reasons why the adult-film industry backed VHS are some of the same reasons why it might have won anyway: VHS tapes could hold longer recording times, and VCRs were far less expensive to own at the time.

That concerns the first major format war for home cinema, though. What we’re going to do here is take a look at the second major format war, between two different pieces of high-definition (HD) optical technology. This format war went on in public for about two years - from 2006 until early 2008 - but it started from a conflict that started brewing in 2002 and built up through 2005. This will be the first entry in a series concerning HD’s advent and its place in modern theaters and home setups, up to 2015. For those of you in the near or far future who happen to be reading this: your interest in history is much appreciated, and I hope this article is displaying appropriately on whatever wearable holographic technology you’re using to view it.

The HD Puzzle

In 2002, two different consortiums began to pave the way for a high-definition standard. HDTVs were a rarity at the time, but they wouldn’t be for long; unlike the initial experiments with high-definition broadcast technology in the 1970s and 1980s, which petered out after disputes between different lobbying firms and the underlying difficulty with broadcasting an HD signal, the new HDTV standards were fit for mass production; sets capable of high-definition images had been available on the consumer market for several years. With a set aspect ratio (16:9, influenced chiefly by the 1.85:1 spherical aspect ratio of theatrical films) and two different sets of “HD” resolution (1280 x 720 and 1920 x 1080), the stage was set for some kind of delivery format to be developed. On each consortium’s mind was the knowledge that’s arguably not too far out of mind for anyone in the technology industry: If we don’t do it, someone else will, soon.

Upping the challenge was the imperative not to go too far astray of what the market would accept for a physical delivery system. DVD’s introduction had gone off fairly smoothly, owing to a coordinated effort by the DVD Forum (one of the two consortiums covered here); however, memories of the VHS-Betamax format war were still fresh enough to inspire caution when it came to designing a new A/V format. In addition, the smoothness of the DVD launch, and the degree of its success, presented its own limitation: customers were familiar with the DVD’s size and appearance, which was physically very similar to the size and appearance of a CD. Form factor was relevant here; the LaserDisc was a technologically superior option to VHS at the time of its release, and arguably superior to DVD in its initial days (owing to no compression being needed for LaserDisc video). Expense did the LaserDisc format in, but it’s hard to imagine that the physical size of the discs - so large as to be unwieldy to transport over any decent stretch - didn’t play a role in the format’s demise, too.

Adding a final layer to the HD puzzle was the challenge of conveying HD video and audio to the consumer’s TV and audio system. DVDs had a maximum line resolution of 720 x 480, and the MPEG-2 video codec used for DVD content had a maximum bitrate of 10.08 Mbit/s - enough to carry compressed video and audio over the two dominant connection methods at the time: composite (the yellow, red, and white cords you remember from your childhood, or your infanthood, or [insert appropriate age-related classification here], and component (a more advanced connection method utilizing three separate cables for each of the red, green, and blue color channels prevalent in the NTSC standard). Component video had the ability to carry an HD signal, but it was far less familiar to the average consumer than composite, which was unable to carry HD. It was also more expensive to produce.

From this complicated pool of expectations and limitations, one consortium would seek to re-conceive the optical disc format in a way that would be ideal for storage space (for reasons we’ll get to), while the other consortium would endeavor to build upon existing infrastructure while ensuring consumer familiarity wouldn’t suffer. The first party was Sony, making another venture out into new technology after their Betamax lost to VHS. The other party was the DVD Forum, an organization of multiple technology companies that had developed the DVD standard and sought to succeed it here.

Sony and Blu-ray

For Sony, 2002 was a banner year. The final tallies wouldn’t come in until its annual report in March of 2003, but the corporation’s year-over-year increase in net income was significant. The Sony Pictures division had dominated a good chunk of 2002 with the financial successes of Spider-Man, xXx, and Men in Black II. Its PlayStation 2 gaming console had shipped its 10 millionth unit, with another 8.5 million units projected for the year’s holiday season; it was handily winning its own console war. In short, it was a bad year to be a competitor.

Blu-ray was already named and under development by the time the annual report came out in 2003; its initial announcement came in February of 2002, so its first corporate mention was actually in the previous annual report. The goals for the format in both reports were lofty: Sony had nothing less on its mind than domination of every direct or tangential market segment. Key to the development of Blu-ray was a piece of laser technology, partially responsible for the format’s name: a blue-violet laser diode, developed by Sony and operating on a wavelength of 405 nanometers (nm), capable of reading and extracting much denser blocks of data than the red-laser diode used to extract data from CDs and DVDs. The blue-violet laser diode was conceived and developed specifically for high-density optical recording and playback, and the initial anticipated results were impressive: between 23 and 27 gigabytes of storage space on a single-layer Blu-ray Disc (by comparison, a single-layer DVD stored 4.7 gigabytes, and even the dual-layer DVD only hit 8.5 gigabytes).

At the time of the blue-violet laser diode’s creation, Sony actually chaired the DVD Forum. The diode’s creation spurred a dilemma that would eventually form a rift: whether to use the diode in a production capacity, now that it was shown to be feasible. There were two main issues that needed to be resolved: one, using the blue-violet laser diode was inherently more expensive than attempting HD content using red-laser diodes. The other issue involved form factor.

Due to the sensitive nature of the blue-violet laser light, the protective layer that was utilized on CDs, DVDs, and other optical discs up to that point was determined to be too thick. Blu-ray Discs were going to need a protective layer thinner than anything provided, which introduced its own set of problems. Namely: damage. A thinner protective layer meant that, as easy as it was to scratch a CD or DVD beyond playability, a Blu-ray Disc would be even easier to scratch. Considering the greater density of the data, a simple scratch could easily be catastrophic to the disc’s integrity.

Sony’s solution to this was a sturdy plastic disc caddy, one that operated in a similar fashion to the plastic shell of a floppy disk: a small portion of the caddy would be open, exposing the Blu-ray Disc to the laser, while the rest of the disc would be sheltered behind a plastic wall thick enough to withstand mishandling, drops, being dragged behind a moving bus, anything.

Personal note: A production company that I briefly worked for back in 2006 used Sony HD camcorders, and the cameras utilized the caddy-based Sony Professional Disc, which had a very similar form factor to these early Blu-rays. I’m barely exaggerating as far as the dragged-behind-a-bus thing; those suckers could take a lot of abuse. Not that I know from personal experience, or anything. Sometimes you don’t have to drop something off of the roof of a building to see if it survives in order to know that it would.

The problem with the caddy was twofold: it was expensive to manufacture (those Professional Discs I talked about just now cost the company roughly $25 per disc, and that was at wholesale prices), and it wrecked the form factor. The Blu-rays were housed like floppy disks, and they looked and felt about that cumbersome.

This was Sony’s entry into the optical HD world. Blu-ray possessed a staggering amount of promise - a dual-layer disc could theoretically hold six times the amount of data as a dual-layer DVD - and a couple of major obstacles to mass-market implementation.

The DVD Forum and HD-DVD

The entry of what would become HD-DVD onto the playing field was initiated by the two main motivations that brought the Blu-ray specification its problems: cost and form factor. Having witnessed the considerable expenditure needed to adopt infrastructure for the blue laser diode, and the early form factor of Blu-ray discs, the DVD Forum was initially exploring a solution that involved pressing HD content onto dual-layer DVDs. Given what we already know about the space constraints of DVDs, this approach was problematic from the beginning; one suspects it only gained any traction at all as an emergency fallback in case Blu-ray absolutely fell through. Compression techniques and the resulting video quality (both of which we’ll talk about in the next section) were improving continuously, but dual-layer DVDs were already being packed to the gills with SD material: the feature, different audio tracks, making-ofs, deleted scenes. Pandora’s Box had been opened as far as what digital media could provide, and trying to press HD content onto the same storage space would have, one suspects, allowed for the film alone and little to no special features.

The potential solution here was something called low-bitrate encoding, which involved a greater degree of compression and thus a smaller file size for each relevant segment of HD content; however, it also involved two new and different methods of encoding. This was the only feasible way to squeeze HD content onto a standard DVD with the appropriate quality; however, the development of low-bitrate encoding had ramifications far beyond this particular initiative.

An initiative is all it turned out to be; despite the planning and considerable engineering work taken to essentially provide HD product in what was an SD container, the method of encoding was never settled upon, and the DVD Forum ultimately decided to go with the same blue-violet laser diode that Sony chose, which gave them the same 405 nm wavelength to play with; the format that was ultimately adopted by the Forum was christened as HD-DVD, in an attempt to maintain continuity with the existing DVD branding. There was one big difference in their approach, though: the Sony Blu-ray spec utilized a lens with a wider numerical aperture. In plain terms, this meant that the system’s laser stayed more uniform in radiance as it passed from the lens to the surface of the Blu-ray Disc; the effect of this was that the laser could focus on a smaller area, and move with more precision. HD-DVD did not have this wide-aperture lens, and less data could be read from a similar surface area. As a result, each layer of an HD-DVD had a maximum storage capability of 15 gigabytes - significantly below Blu-ray’s 25 gigabytes per layer. There were additional complications with this comparison, which would only show themselves later (we’re getting there, I promise). Fifteen gigabytes was still a healthy distance ahead of DVD’s 4.7 gigabytes per layer, and there was now a blue-laser HD format that fit the expense and form factor of DVD to a closer degree.

All of this took place in 2002; HD optical media was years away from mass production, and yet some of these details had already been relatively public by early 2003. Information was sufficiently both available and complicated enough so that publications began to get some of their wires crossed as to what was really possible with the new format; one particularly memorable forecast utilized the concurrent release of Peter Jackson’s Lord of the Rings films to claim that the format would permit all three Extended Editions to be pressed onto a single disc, in HD. The thing that stuck out the most, though, was the presence of more than one standard. The specter of a format war loomed on the horizon by 2003, and there was pressure on both Sony and the DVD Forum to come to some kind of deal, consolidate their approaches, and avoid a conflict. One was not forthcoming, due to two main reasons.

To isolate the first, we’re going to very briefly step all the way back to the 1990s here, and the establishment of the DVD standard. It was an incredibly smooth launch by comparison, but even here there was some initial disagreement, and two formats were initially created: Multimedia CD (pioneered by Sony and Philips), and Super Density CD (pioneered by just about everyone else). There was the possibility of a conflict and potential format war here, too, except IBM stepped in and mediated the discussions. A war was averted and DVD was launched, but in the process of discussions Sony was made to give up Multimedia CD’s disc structure; this proprietary aspect of the format cost them a large amount of money in royalties. With the HD format, Sony held firm to their disc structure (involving the wide-aperture lens mentioned above), and would not give it up. This was the first major point of contention.

The other one was centered around the gaming aspect of the format. Console and PC gaming technology accelerates in complexity as quickly as anything else in the industry we’re covering, and optical HD was going to be a big part of any future gaming plans. From day one, it was a given that Sony was going to link its HD format with the next generation of its PlayStation gaming console. HD-DVD, on the other hand, had Microsoft as one of its early and constant backers, and Microsoft had just jumped into the console gaming market with the XBox several years before.

Sales weren’t a patch on Sony’s PlayStation 2, but they were successful enough to warrant a follow-up console. Blu-ray’s disc structure utilized Java for its multimedia, and Microsoft had its own multimedia framework, known as .Net; it was unwilling to give up its own kin to adopt another framework. For this reason, Microsoft became one of the biggest proponents of HD-DVD during the series of negotiations and meetings, between Sony’s newly-named Blu-ray Disc Association and the DVD Forum, that went on between 2003 and 2005.

Their offer to Sony was relatively simple: You put everything about Blu-ray on the table; we’ll put everything about HD-DVD on the table; our engineers will assess the entire table, pick out what works best, and discard what doesn’t. This included Blu-ray Disc’s layer structure, and Sony rejected the proposal. The DVD Forum took this as inflexibility on the part of Sony, and both parties basically packed up their toys and went their separate ways. This was in mid- to late 2005, and no further negotiations were attempted. Both Blu-ray and HD-DVD were being prepared for a 2006 launch.

Thus concludes Part 1 of this Intermittent Issues series. Check back for Part 2, where we go into the actual format war, the developments, and the resolution. Thanks for reading!