Wednesday, 3 May 2006

Removing the Middleman, Part 3: Books

"The sums involved are so minuscule, they're not even insulting: they're *quaint* and *historical*, like the WHISKEY 5 CENTS sign over the bar at a pioneer village. Some writers do make it big, but they're *rounding errors* as compared to the total population of sf writers earning some of their living at the trade."
--Author Cory Doctorow on the financial realities of being a writer


Most people agree that the Internet and technology changes will make it possible to replace the pipe companies – the carriers, publishers, and networks that deliver the world's information, entertainment, and communication. But that change has been predicted for years. Will it really happen? If so, when? And what business and technology infrastructure needs to be created first?

In Part One, I gave an overview of the situation. Part Two discussed emusic. This time I'll cover ebooks.


Summary

This is a long post, so here's a quick summary of the highlights:

1. The traditional publishing business is incredibly inefficient. It absorbs most of the dollars paid for a book before they get to its creator. That reduces our favorite authors' incentive to write more books. Consolidation and other changes in publishing are limiting the availability of books and short stories. Ebooks could theoretically solve all of these problems.

2. People have been predicting the imminent takeoff of a mass market for ebooks since the late 1970s. It didn't happen then, and it's not happening now.

3. Unlike emusic, there are major structural barriers to the broad adoption of ebooks. The problem is not just technological. The most important barrier is a chicken and egg situation: People won't buy an ebook reader until there are millions of ebooks available for it, but there won't be lots of ebooks available for an ebook reader until it has millions of users.

4. Until we break that logjam, the mass market for ebooks will never develop.


My life as a publishing enthusiast

I should start with a little background. I'm passionate about publishing, and have been since I was a kid. In high school I hung out in the print shop, doing everything from hand-setting type to running a clanking old Linotype machine that cast lines of type from molten lead. I think the Linotype machine was one of the most impressive purely mechanical devices ever created. It had to deliver molds for every letter as quickly as the operator typed them, space them properly, send the finished line of text over to the caster, inject the lead, eject the finished line of text, and automatically sort all of the molds back into their proper positions for re-use. None of this was computerized – the operator typed on a mechanical keyboard and yanked a lever to start the caster, whose actions were then automated by a fantastic array of gears, pulleys, and levers. The molds for each letter had a unique set of notches on them, which allowed them to be sorted automatically by a series of gravity-fed chutes and cogs.

A Linotype machine in full operation was a symphony of noise – the clacking of the keyboard (the best operators typed fast), the clank and whir of the caster, and a fantastic tinkling sound as the little brass molds ran through the sorter.

You can see video of a Linotype machine in operation here. Man, I wish I had one of these things in my garage.

The most dramatic thing about running a Linotype machine was that if you sent over a line of text that didn't have enough letters or spaces in it, the machine might shoot hot lead straight out between the molds, where it would splash all over the place, including the operator's left arm. This was called a "squirt." I remember diving for the floor more than once when I realized I had accidentally sent out a line of text with the wrong spacing.

It was a pretty cool school district that let a teenager run a complicated and dangerous machine like this.


This boy isn't smiling, but I guarantee he's having fun with his Linotype machine.

In college I spent five years on the campus paper as it transitioned from typewriters to CRTs, and after graduation I started a desktop publishing company that made the first downloadable PostScript fonts for the Macintosh. In 1999 I spent time as the acting VP of marketing for SoftBook, a company that was making electronic books. And now here I am publishing my own writing online.

The most exciting thing about all of this technological change (other than the part about flying molten lead) has been the promise that publishing would become more democratized and more accessible to everyone. In some ways it has happened – desktop publishing paired with Kinko's and the photocopier made it much easier and cheaper for people to print on paper. And the Internet has spawned an unimaginable explosion of commercial and self-published media.

Yet in many important ways the revolution is incomplete. Books are still generally published through the same business processes used a hundred years ago. The distribution system is so inefficient that it absorbs almost all of the revenue paid for a book, and it's still a struggle for first-time authors to get paid for their work.

For decades it has been obvious that there's a better way to publish – deliver books electronically to a screen where the reader can enjoy them. Almost 30 years ago, Alan Kay and Adele Goldberg talked about using a computer to read digitally-stored books in their visionary paper, Personal Dynamic Media. Management guru Peter Drucker made a similar prediction at about the same time. Ever since, the computer industry has produced periodic waves of exuberance for ebooks. Donald Hawkins reviewed the forecasts of major industry analysis firms and found:

"IDC said that the U.S. e-book market would grow from $9 million in 2000 to $414 million in 2004, and Forrester Research predicted a revenue increase from $383 million in 2000 to $7.8 billion (!) in 2005. These are large numbers indeed, and the ... market research firms companies making them are well known in their field... To be fair, these numbers may apply to print-on-demand as well as e-books. (But) it is now apparent that the e-book shot missed its mark, and the e-book revolution has fizzled. Indeed, it never really got off the ground. The marketplace did not develop as originally predicted."

He wrote that in 2002, but the situation isn't much different today. Despite decades of effort, and numerous failed startups like SoftBook, we're still generally using an inefficient old publishing model that would feel familiar to Ben Franklin.

How could so many smart people all be so spectacularly wrong?

My belief is that the takeoff of ebooks is not just a matter of time. Unlike the e-music situation, where the march to electronic distribution is now inevitable, e-books are blocked by technology and market barriers. Until they're overcome, I think the revolution will be delayed indefinitely. Here's why...


The economics of book publishing today: "You don't write a book to make money"

That's the consensus from everyone I know who's involved in the publishing business. When you look at the economics of book publishing, it's easy to understand why they say that. For hardcover books, an author typically gets 10% to 15% of the retail price, depending on how many copies are sold. For paperbacks, the royalty rate is 4-8%. An agent will take about 15% of those royalties off the top. There are good summaries here and here.

That means if you write a hardcover book that sells for $30, you'll probably get about $3 to $4.50 per copy sold. If you write a paperback that sells for $7, you may get about 42 cents per book sold.

This ought to make it very attractive for an author to sell e-books direct rather than going through a print publisher. Let's assume the cut taken by an e-book reseller would be similar to Apple's iTunes cut – 30%. If everything else is equal, you'll make the same amount of cash selling 1,700 e-books as you will selling 10,000 print books. So the tipping point in book publishing comes when about 17% of the book-buying public purchases e-books rather than paper.

But that's just a first pass. The real situation's a lot more complex, for several reasons:

Advances. In most cases, an author gets a non-refundable advance when the book is accepted or published, and no more royalties are paid until the advance has been covered by sales. The advance can range from a few thousand dollars for a first-time author to much more for an established author whose book sales are more predictable.

This means that in many cases the royalty percentage is merely an academic exercise – what really matters is the size of the advance.

Publishers sometimes use the advance as an investment mechanism. They may deliberately pay an advance greater than the expected royalties on the first couple of books by an author they believe has great potential, in the hope that they'll eventually get a very profitable bestseller (there's a very good discussion of the process here). On the other hand, if a book is expected to do well in sales and fails, the disappointment over that might get the author blacklisted throughout the industry.

Price of the book. This one works against e-books. Most people buying a book, especially a hardcover, feel they're getting an object of tangible value. A hardcover book has heft to it, the pages have a nice texture, it has a pretty color, and it makes you look educated when you put it on a shelf (unless it's a book by Yann Martel, in which case you just look mildly befuddled).

An e-book is ephemeral. It's just a bundle of bits. You might erase it accidentally, and you can't display it on a shelf. Although it can be more convenient to carry an e-book than a paper book (you could hold thousands of books on a single USB drive), that isn't enough compensation for most people. Based on what I saw at SoftBook, no one other than an enthusiast will be willing to pay hardcover pricing for an e-book. Paperback pricing, maybe – but that's the upper limit.

This changes the tipping point for some authors. If you're successful enough to be published in hardcover, chances are you're getting the high end of the royalty range, and your book might sell for $30 a copy. You probably get about $4.50 per book, or about the same as your total revenue per book if you self-distribute an e-book at paperback prices. So for these authors, the tipping point is harder to map. I start to wonder about weird hybrid distribution models, in which successful authors publish first to hardcover paper books, and then the ebook takes the place of the paperback.

Investment by the publisher. This one's less of a factor than you might expect. The public image of publishers is that they carefully select and nurture authors, editing and marketing them like a jeweler carving a diamond and setting it in gold. That does happen sometimes, but the reality is that the big publishers deal in volume. They publish thousands of books a year (ie, tens of books every single day). How much time is available to coach and edit all of those authors? And how much marketing budget is available?

The answer to both questions is, "not much." Publishers don't actually market books, for the most part. They look for books that will market themselves.

The reality of modern book publishing is almost like digging for gold – you toss a lot of dirt and gravel into the sluice, hoping to find an occasional nugget. Since it's hard to predict where the gold will be, it doesn't pay to be picky about what you shovel. Just throw in as much dirt as you can and trust that the law of averages will eventually reward you.

Translating that metaphor back to publishing, the bookstores are the sluices, and the dirt is the flood of new books that publishers shovel through the stores. If a book starts selling well on its own, that's gold and the publisher makes money. If not, there's another ten books coming out tomorrow.

A publisher who believes in a new book may pay a little extra to give it a fancy cover, or place a small ad for it somewhere. But that's about the limit; otherwise, the market rules. The book and author have to sink or swim on their own.

There's a fantastic discussion of book publishing economics here (thanks to Lee Fyock for pointing it out to me). The most important takeaway from that discussion is that the publishers aren't getting wildly rich off this system. Most of the money is absorbed by various inefficiencies and expenses unique to printing: writing off the production cost for returned or unsold books, editing and typesetting, shipping, and so on.

So most of the value added by publishers is logistical – they manage the complicated production process, get the book placed in stores, handle returns, and so on. If the printing and bookselling process is bypassed, the value-add of most publishers goes away.

And so, perhaps, does the publisher.

Where's the tipping point? Netting out all the pluses and minuses, my guess is that the tipping point comes when about 20 percent of the book-buying public switches to e-books. Once we hit that point, ebooks will take over the majority of new book sales quickly. I want to emphasize the 20% figure is just a guess, though. There are so many variables that we won't know exactly when we've reached the point until we're already past it.


E-book publishing today: Slow growth

When will we hit magic 20% adoption rate for e-books? It could be a long wait.

The fragmentary statistics available on ebooks indicate that the market's still extremely small and is not growing explosively. A trade organization called the International Digital Publishing Forum (formerly the Open eBook Forum) compiles statistics from about 15-20 ebook publishers, and issues occasional press releases about the results.

I created the chart below based on IDPF's press releases. I had to extrapolate the totals for 2002 and 2003 because the forum released only first half statistics. The other drawback of these statistics is that the base of publishers reporting sales has changed over time, but the numbers were not revised to account for that.

The numbers show publishers' ebook unit sales and revenue growing about 30% each year. About 1.6 million units were sold in 2005, and revenue was about $12 million for the year. Unit sales were flat from 2004 to 2005, but revenue increased because the average selling price per ebook rose from about $5.70 to $7 (remember, that's publisher's revenue, not necessarily what the user paid for the book). Curiously, the number of new ebooks published has remained steady since 2002 at about 5,000 per year.


Units and dollars in thousands.

That sounds impressive in the abstract, but compare it to the printed book business. There were about 175,000 print books published in 2004, according to a trade group called the Book Industry Study Group. About 2.3 billion books were sold. Publisher revenue from book sales was about $43 billion, ($28.6 billion from major publishers, and $14.2 from small and specialty presses). You can add to that about $800m to $2.2 billion in used book sales, depending on which authority you believe.

Don't get me wrong, I know some people are having a great time reading ebooks, and that's wonderful. I was a big promoter of ebooks for Palm OS when I worked at Palm. But the question I'm asking right now is when they'll displace paper books and transform the publishing industry. From that perspective, the reality is that ebooks are about three one hundredths of one percent of the book market today. They're a hair on a wart on an elephant. At the current growth rates for both markets, we'll hit the tipping point for ebooks in spring of the year 2032. That's well past the date when Ray Kurzweil says we'll all transcend to cybernetic superhuman form. At current course and speed, we'll all be reading books by direct absorption before ebooks hit the big time.

[*Note that total ebook sales are undoubtedly higher than what the IDPF reports, because not every publisher submits its numbers to them, and because a lot of ebooks sold are old titles that are now out of copyright (and therefore wouldn't be counted in the publisher statistics). I wasn't able to find any firm figures on those sales. Nevertheless, the basic situation for ebooks is very clear.]


Other troubles in the book industry

Before I get into a discussion of why ebooks aren't growing quickly, I want to mention several other troubling trends in the book industry that electronic publishing might help to address.

Consolidation: Reduced diversity. Like the music industry, book retailing is being hit by consolidation as discount retailers like Wal-Mart, and chains like Barnes & Noble, skim off the most profitable books and sell them at discount prices. This drains the profits out of independent bookstores, which many publishing observers believe play an important role in discovering and promoting promising new authors.

Publishers have also been consolidating, reducing the diversity in the industry and therefore the opportunity for new authors to be discovered.

Pat Holt, former book reviewer for the San Francisco Chronicle, wrote passionately about the situation. I've excerpted part of the essay below, but it's worthwhile to read the whole thing here:

"From the 50 or so independent publishers in the mainstream industry about 30 years ago, when I started work in publishing, today about 7 conglomerates publish the bulk of mainstream books. This awesome consolidation of power is having a disastrous effect on the kinds of choices readers are offered... Not only are gimmicky, high-turnover, "no-risk" books replacing serious literature, an invisible censorship is taking hold of the industry as more and more decisions about what Americans can read fall into fewer and fewer hands....
"At the same time, the appearance of chain bookstores, beginning in the 1970s, has been devastating to independent booksellers.... I'm also concerned about the significance of independent booksellers in the larger scheme of things - how they discover new books that the chains often miss; how they support unknown writers; how they cultivate audiences of readers and writers, how they aren't comfortable with the kind of "paid placements" that bring Amazon.com, Barnes & Noble and Borders hundreds of thousands of dollars; how they continue to play a key role in the preservation of literature, even though they are routinely dismissed by mainstream publishers as powerless.

"Well, how powerless is this: Taken together, independent bookstores offer a wider range and variety of titles than do all chain stores put together."

The decline of the short story market. There has been a lot of angst in literary circles over the declining market for short stories. Short stories were once the training ground for many new authors, and some authors specialized in them. But since at least the 1980s, magazines have been reducing the space they devote to short stories, and short story collections don't sell as well as novels (according to a study conducted in the UK, short story collections sell about 1/3 to 1/4 the volume of novels).

Author Quinn Dalton summarized the situation nicely here:

"The conventional wisdom is that magazines are bowing to the realities of necessary advertising revenues, which put pressure on sections that don't perform based on reader feedback. As a result, the inclusion of fiction in commercial magazines has become an exception to standard procedure. The number of available outlets for short fiction has been steadily shrinking, from The Saturday Evening Post to Redbook to Cosmopolitan, which still publishes excerpts occasionally, to Mademoiselle, which ceased publication in 2001. More recently, Seventeen dropped regular stories in 2003, and Jane discontinued fiction during its redesign in 2004, though both run an annual fiction contest."

Editor and commentator Kelly Jane Torrance adds:

"Today short fiction is confined to an elegant ghetto: the New Yorker, the Atlantic Monthly, and Harper's. There still exist many literary magazines, but these publish seldom and have few subscribers. Moreover, outlets like the New Yorker publish mostly established names—John Updike, Alice Munro, Joyce Carol Oates, etc.—regardless of whether they are past their sell-by dates. Some joke the New Yorker would publish a grocery list if it had Updike's name on it."

I've watched the effect of this market change in science fiction (hey, this is a technology blog, of course I'm going to talk about science fiction). There were once a large number of relatively successful science fiction magazines that specialized only in short stories. A couple of them still exist, but they're not as numerous and vibrant as they once were. There are online efforts to take up the slack, but most of them pay authors nothing or a very close equivalent. One exception was Sci Fiction, an online forum of short stories funded by cable television's SciFi Channel. Unfortunately, the parent channel pulled the plug on it at the end of 2005.

In the period before Sci Fiction was shut down, Ellen Datlow, the site's editor, commented on the market for short story magazines:

"Until a workable business model is created for webzines independent of large corporations funding them as part of a larger entity (like mine), it's going to be a tough sell. But what I see in print magazines isn't any better. They start very small as semi-prozines, the best get bigger, more ambitious, come out more often and then the publisher/editor (usually the same person) burns out and/or runs out of money. Flourish? I don't know. Limp on over the years, yes definitely."

We're losing a lot of great fiction as a result. Science fiction authors like Arthur C. Clarke (2001: A Space Odyssey) and Robert Heinlein (Starship Troopers), are known for their novels, but in my opinion their best work was their short stories. And then there's Ray Bradbury (although he prefers to be classified as a fantasy author). I think we're missing out on much of the next generation of short story creators because they can't get paid for their work.

Out of print: The amazing disappearing author. Although books from the most popular authors often remain available long after their deaths, many others fall out of print rapidly. The figure I found quoted most often on the Web is that 90,000 books go out of print every year. Barnes & Noble reports that there are about 5.6 million books in its database, but only 1.5 million are readily available from publishers.

The result of this is that even well-known, award-winning authors are disappearing from view. I'll use science fiction as an example again. The late Clifford Simak is considered one of the grand masters of science fiction (he was the third author officially recognized as such by the Science Fiction and Fantasy Writers of America). He published a total of 27 novels and 141 short stories, and several of his stories won the industry's highest awards, and are considered classics of the genre. Yet only two of his novels are currently in print, plus one partial collection of his short stories. And those are available only because a small publisher has recently done a limited run of them.

To be fair, I should acknowledge that there isn't a huge market for some of Simak's books (his quality was pretty uneven). But a lot of excellent stuff is available only through used bookstores, if you can find it at all. Similar situations exist for other famous authors. Galen Strickland, who has written a lot of online commentaries about famous science fiction authors and their best-respected work, searched for copies of all the books he had referenced in articles on the website Templeton Gate. He found that of 996 books mentioned, almost half were out of print. Strickland pointed out that even living authors have problems with books going out of print – for example Arthur Clarke's The Fountains of Paradise, which won the field's two highest awards in 1979, is not currently in print.

Some people in the publishing industry complain that used book sales, especially those facilitated by Amazon.com, are undercutting the sales of new books. It's hard to quantify the real impact of used books (estimates of used book sales range from $800m to $2.2 billion a year), and some studies say the cannibalization of new book sales by used books is low. But even if used books do affect the demand for new books, until the industry does something about books going out of print I don't think it'll be in a position to say anything negative about the used book economy.

Generally, authors go out of print because there isn't enough demand for their work to justify a print run. It's uneconomical to do a print run for a small number of books, and besides most bookstores won't stock older books, especially if their authors are dead and no longer generating new fans (remember, the point of the system is to shovel fresh dirt into the gold sluice).

One way to address the print run problem is print-on-demand technologies, which produce short print runs of books that have been encoded into electronic files. The printing cost per book is about double that of traditional printing, but the shorter print runs mean less inventory risk. One of the leaders in this space is Lightning Source, an on-demand printing service that claims to have more than 100,000 titles in its library. Authors can self-publish via print on demand through services like iUniverse.

Although print on demand helps with availability, there's still the problem of getting books onto bookstore shelves. And the economics of publishing continue to eat up most book revenue – for example, an author selling through iUniverse gets 20% of the net proceeds for the book (after the costs of printing, shipping, and bookstore discounts are taken out). That's not much different from the payback from traditional publishers, which at least pay royalties based on the gross revenue from the book, not the net. There's a good discussion of the whole print on demand situation here.


Summary of the situation: This airplane won't fly

Add it all together, and here's how the publishing situation looks. Book publishers are not feeling as much financial pressure as music publishers (major publishers' revenue was up about 2.8% year over year in 2004, although book unit sales were down fractionally). But the diversity of books is suffering as channels and publishers consolidate. There's a serious problem with books going out of print, and the short story marketplace is moribund. And hanging over it all, the gross inefficiencies of print publishing prevent most of the money that you and I pay for books from ever reaching the people who write them. Our favorite authors don't have as much incentive, and freedom, to write as we'd like them to.

The situation seems ripe for ebooks. Not only would they allow a much higher percentage of book revenue to reach authors, but once ebooks are a standard no book need ever go out of print again. The only restraint on diversity will be our ability to sort through all the ebooks that are available. And I personally believe that a properly functioning ebook marketplace could revive the short story market, by making it possible for people to purchase short stories directly at low cost. (In fact, Amazon is experimenting with paid short story downloads).

However, most of these problems have existed for decades. People have been talking about ebook-like products for at least 30 years. And yet while sales of emusic are exploding, sales of ebooks poke along at a rate of growth far too low to change the industry. The ebook market simply isn't taking off.

Why the heck not?


Why ebooks haven't taken off

There are two main reasons, one technological and one structural.

The technical problem is that, for most people, reading long documents on a computer is unpleasant.

I know, you read weblogs and news online. So do I. But most of us do so in relatively short sessions, taking breaks and doing other things in between. Unlike those short reading sessions, the ergonomics of reading a book on a computer are uncomfortable for a lot of reasons.

The first problem is the screen. A piece of paper is reflective – it's only as bright as the room around it. A computer screen is transmissive – you read it by the light that shines through it. To be read clearly, a computer screen has to be substantially brighter than the ambient light around it (that's why most computer screens are hard to read in direct sunlight). This excessive brightness produces eyestrain when you're reading text for a long period of time. If you're like me, you can't read a computer screen nearly as long as you can a printed book.

Contrast ratio is also a problem for computer screens. If you look closely at the on-screen text in a word processing document, you'll see that it's not really black and white. The text is fairly dark gray, and the background is a bluish-slate color that you'd probably call light gray if you saw it on a piece of paper. Speaking as an old printing fanatic, it's been well established that people read most easily when they have the highest contrast ratio between white paper and black ink. Dark gray text on a light gray background is not at all ideal for reading. And the computer's readability gets even worse if there's any reflection off the screen, which is pretty common unless your lighting conditions are perfect.

Screen resolution is another important issue. Speaking from my printing background again, most people can read English (and other Romance languages) most efficiently when text is written in a serif font like Times Roman or Palatino, as opposed to a sans-serif font like Helvetica or Arial. The pattern of thick and thin lines in a serif font, and the little hooks at the edges of letters (called serifs) help our eyes quickly recognize letters and words.



To accurately reproduce the serifs and line thicknesses, you need about 300 dots per inch, or the resolution of an early laser printer (600 dpi is even better, and the printing standard is 1,200 or 2,400 dpi). Most computer screens are about 80 dots per inch. Text is readable on them, but not nearly as readable as it would be on a piece of paper. The result, once again, is more eyestrain and slower reading speeds.


Which one do you think is easier to read?

Despite all the drawbacks of computer screens, I think they're not the biggest barrier to ebook adoption. When I was at SoftBook, people sometimes complained about the screens, but other issues were a much bigger problem.

You can't use a computer the way you use a book. There are general ergonomic problems with reading on a computer. If you're using a desktop computer, you have to hunch over a desk in an office chair, rather than sitting back in an easy chair or couch the way most people do when they read a book.

You can read from a laptop, of course, which makes it easier to sit in a comfortable spot. But laptops have their own problems. The batteries don't last all that long, so your laptop may hibernate just as a book comes to a thrilling climax. Laptops are much heavier than books, so they are hard to hold comfortably. And they get disturbingly hot when used for hours (like the one that's trying to scorch my thighs right now).

Goldilocks and the two tablet devices. Then there are the two leading alternate mobile devices – tablet PCs and handhelds. Neither of them make great e-book readers for the average person. Tablet PC is basically a PC with the keyboard sawed off. It's still quite heavy compared to a book, so much so that you can't hold it by the base the way you would a book (your wrists will wear out rapidly if you try). Instead, you have to prop it against something, which puts you back into the wrong ergonomics for most readers. And it has the same battery life and screen problems as a portable computer.

I had hopes that Microsoft's new shrunken tablet design, code-named Origami, would give a better experience. But press reports say it's going to cost $800, weigh two pounds, and will have short battery life. To me that's just another tablet PC.

At the lighter end of the hardware scale, ebook reading software has been available for years for handhelds, and it's now moving to smartphones. There are people who read ebooks on handhelds (I'm one of them), but unfortunately it's not the right platform for most people. The screens are just too small for a comfortable reading experience. It's great if you're trapped on an airplane or bus and have nothing else to do, but there's no way it substitutes for a printed book for most people.

There's a nice discussion here on the problems of reading on a small screen.

So I find myself feeling like Goldilocks – the Tablet PC is too big, and the handheld is too small. I need something with the thickness and cost of a handheld, but the screen size of small tablet.

Chicken and egg. But even if that hardware were available, it wouldn't be enough to make ebooks succeed. After all, the Rocket ebook reader came pretty close to the right size in the year 2000, and it went out of business.


The Rocket ebook reader

The biggest problem facing ebook adoption is market structure. It's a classic chicken and egg dilemma, and it works like this:

--1. Most dedicated book consumers, the core customers for ebooks, won't buy an ebook reader device until there are millions of affordable books available for it, including a lot of current titles. I saw this one first-hand when I worked at SoftBook.
--2. Most publishers won't make their books available for an ebook reader until it has an installed base of millions of units, and they won't price them at paperback price points unless forced to.

This dilemma broke SoftBook and Rocket Book and every other attempted ebook reader launched in the last decade. And I think the new ebook readers being developed by Sony and Philips are at grave risk for the same reason.

Even a world full of PCs and handhelds and smartphones equipped with ebook software is not going to change the situation. If they were acceptable reading devices for most people, the revolution would have happened already.

Emusic vs. ebooks. Looking at all the barriers to ebooks, it's interesting to ask why emusic succeeded. A lot of people try to draw parallels between the two markets, but in fact some critical factors are very different.

1. Content was available for mobile music devices from the start. There were two sources of emusic: pirated MP3s, and CDs owned by the customer. Remember Apple's old "Rip, Mix, Burn" campaign? Even before the iTunes store was fully stocked with songs, you could use an iPod to do something interesting and useful with your music, just by ripping your CDs and transferring the music to the iPod. That helped to jump-start the iPod market.

2. Fear of piracy made the record companies deal. The record companies were so fearful of piracy that Apple was able to talk them into selling singles for 99 cents each – something they never would have agreed to if customers hadn't been stealing the songs for free. This allowed Apple to assemble a very large library of affordable songs.

Because of the special conditions in the music market, Apple managed to give birth to both a chicken and an egg at the same time. Unfortunately for fans of ebooks, the conditions Apple took advantage of don't apply in books. There are no book CDs that can be ripped and burned to an ebook reader, and there is no large-scale book piracy problem that would convince the publishers to make their books available at paperback prices. In fact, they have a strong incentive to keep the ebook market just as weak as it is today.

I'm not trying to paint the publishers as villains. Some of them, such as Baen Books, have been experimenting with ebooks as a promotional tool. And the O'Reilly publishing organization has been doing some very interesting work with hosting technical books online. They're also very active as promoters and communicators of new technology ideas. I think they realize that their core business is the marketing of ideas, not slabs of paper. But Baen and O'Reilly are both smaller, more flexible publishers willing to take risks. I think it's unrealistic to expect the major publishers to convert their catalogs en masse into ebooks, and sell them at paperback prices, when doing so will undercut their current channels and marginalize their companies. I think we'll have to work around the publishers, not through them.**

Our challenge. If we want to see ebooks really take off, we have to find some way to get a well-designed ebook reader to into the hands of about 20% of book buyers, before it has a critical mass of first-run ebooks available for it.

I have some ideas on how to do it. That'll be the subject of my next post, in about a week.

Next time: Desperately Seeking the InfoPad

________________
** This is why I think Google's Book Search program, formerly called Google Print, was one of the most strategically bone-headed moves made by a Silicon Valley company in the last five years. By asserting the right to scan books and post excerpts from them without prior authorization, Google managed to do the seemingly impossible – it got the authors to make common cause with the publishers. That's like getting the National Rifle Association to partner with People for the Ethical Treatment of Animals. Way to go, Google – in pursuit of the niche of searching scanned library books, you screwed up your ability to move the entire publishing industry to electronic format.

_________________
More reading

Although it's several years old, this is a splendid essay by Berkeley professor Clifford Lynch on ebooks and all their implications.

An interesting ebook commentary by author Cory Doctorow

Mobile Read (a discussion forum on general mobility topics, but with a special focus on ebooks).

Weblog of Bill McCoy, who works ebook issues for Adobe. He sounds more optimistic about the short-term prospects than I am.

_________________
Thanks to the latest Carnival of the Mobilists for linking to my post on technology predictions. This week's Mobilist edition had a number of interesting articles in it; it's worth checking out.

Tuesday, 25 April 2006

Thank-yous to two mobile forums

Material from Mobile Opportunity is being featured in two online forums, and I wanted to acknowledge them.

The Carnival of the Mobilists is a group of mobile-related bloggers who take turns hosting a collection of the week's best mobile-related posts. They recently featured my commentary on CTIA.

Wireless Watch, one of my favorite websites for Japanese mobile news, has just added a new community service in which it's consolidating the feeds from about 20 mobile-related weblogs around the world, including mine. There's a lot of material, but it's an interesting way to quickly see what the buzz is in the mobile community.

Saturday, 22 April 2006

The futility of high tech forecasting

In my other blog, I took the authors of a famous business book to task for making bad projections about future technological change. But it's not fair to single them out -- the same sort of problem happens to experts making projections inside technology companies. I've seen a lot of those projections over the years. The usual pattern is that technology predictions with a two year horizon are pretty good, because a technology has to be almost in prototype stage now in order to appear in high-volume products two years from now. Five year predictions are moderately useful, but subject to embarrassing errors. Ten-year predictions are almost useless, and twenty-year predictions are best used as plot outlines for science fiction novels.

This came home to me the other day when I stumbled across an old document from Palm that I had put aside years ago. It was the company's projection, created in 1997, of what future handhelds might look like in five, ten, and twenty years.

Keep in mind that the folks at Palm were, at the time, the world's leading creators of handheld technology. It's humbling to see how many things they missed. Here's what the experts forecast, with some comments from me:


Five years in the future (in other words, predicted for 2002):

-320x320 display. Okay, that came along with the Tungsten T at the end of 2002. And it was in other companies' handhelds before that.

-56 kbps wireless data. Didn't happen in a Palm-branded product until the Tungsten C in 2003.

-Built in cell phone. Early smartphones were appearing by that time, although they were pretty clunky. I'll give this one only a partial correct score because I think the people making the prediction were looking at what would be built into a typical handheld, and smartphones weren't typical by then.

-Multi megabyte secondary storage (hard drive or flash). OK.

-MicroCD player. No, a clear miss. Rotating removable media turned out to be much more delicate, expensive, and power hungry than we'd all like it to be.

-Voice synthesis. Didn't happen with the level of quality that was needed for it to become commonplace.

-High speed serial for desktop connectivity. I think USB 2.0 qualifies here, although the authors thought it would be FireWire. But I don't think any handhelds were shipping with either USB 2.0 or FireWire in 2002 (please correct me if I missed one).


Ten years in the future (in other words, 2007):

-Size and thickness of a sheet of cardboard. If only. Part of the problem is that it's hard to make something that thin also rigid enough to resist breakage.

-Foldable screen; can be used in folded or unfolded mode. Not going to happen next year, alas. Maybe in another five years, although personally I'm even a little skeptical about that.

-24-bit color. Okay.

-Voice recognition and synthesis. Not with the reliability that would let you use it as your primary interface to the device.

-10 mbit/second wireless data. They didn't specify whether they meant local wireless or cellular. We won't have cellular that speed in most places, but some companies have announced 802.11g modules for handhelds, so maybe...

-Built in cell phone. Okay.

-Built in TV/FM receiver. This is possible but there doesn't seem to be huge demand for it, at least not in the US.

-Built in video camera. Okay, but it'll be a pretty low-res one.

-Built in GPS. Okay. But let me point out that no one I'm aware of is doing GPS + TV + FM + video camera + cellphone + 802.11g in a single device, which is what was predicted.

-Body heat powered. Uhhh, no.


Twenty years in the future (ie, 2017).

This prediction still has a long time to run, so maybe we can revisit it if I'm still alive and blogging in 11 years. In the meantime, here's how the prediction looks now:

-Device is surgically implanted into brain. I doubt it, more for cultural reasons than technical ones. Even if we can find a way to do this, I think the current generation will be uncomfortable with implanting anything into their brains. I think people will have to grow up with that idea in order to be comfortable with it. Kind of like nose rings.

-Displays images to optic nerve. There have been some interesting experiments in stimulating the optic nerve to produce synthetic sight for people with retinal damage. The research is still very early, though. I don't doubt that this will happen someday, but I think that in 11 years it won't be something that you'd use with anyone other than a medical patient.

-Talks into auditory nerve. Similar to the vision situation.

-Input through voice, muscle input, or thought. I am completely confident that we won't have handhelds that can read thoughts eleven years from now. Reading gestures is more interesting; there has been a lot of research on reading eye movements and such. The challenge seems to be more on the software side than in hardware – how would a gesture-driven interface work, and how would you train people to use it?

-Powered by blood movement. I guess it would have to be if you're going to put it in someone's brain. You wouldn't want to have to plug your head into the wall to recharge every night.


Twenty years seems to be the magic horizon at which we think anything is possible. Maybe that says something about how we all think. Or maybe it's just a sign of the pending Singularity.

Why Google got hammered for its China policy

There's a long and fascinating article from the New York Times describing Google's policy in China. As usual, the real story is more subtle and nuanced than what was first reported in the press. The article is great reading on several fronts – it talks about the cultural differences between the US and China, it discusses Chinese Internet usage patterns that are significantly different from other parts of the world, and it attempts to explore the thinking of the average Chinese Internet user.

The last bit rang a little false for me – the article's implication that many Chinese people welcome censorship of the Web reminds me of the articles during the Cold War that found Russians appreciated being oppressed by the Soviet government. When the government is powerful and punishes misbehavior, people quickly learn what they can and can't say. China is by no means a police state, and there's enormous diversity of opinion there, but I think it's dehumanizing to say that Chinese people somehow desire or deserve less of their rights under the UN charter than do people in other countries.

But all of that stuff aside, the most striking thing about the article to me was the description of how Google took enormous criticism in the press and in Washington for its China policy. The article describes much more egregious compromises made by Google competitors, and concludes: "Against this backdrop, the Google executives probably expected to appear comparatively responsible and ethical. But instead, as the China storm swirled around Silicon Valley in February, Google bore the brunt of it."

The article implies that timing and unrealistic expectations caused Google's problem, but I think there's another explanation: it's Google's own fault. When people misunderstand your intentions, when they blame you for things that you didn't do or didn't intend, it's a sign that you are not communicating properly. It's not their fault for misunderstanding you; it's your fault for failing to explain yourself properly. For a company, the process of explaining yourself properly is a thing called "marketing." Google's contempt for marketing was cute when the company was younger, but now it's just an embarrassing handicap. Unless Google comes to terms with that, and empowers some senior marketing people, I think it will inevitably get hammered again.

Thursday, 20 April 2006

American Idol and the future of online market research

A commentary on my other weblog, Stop Flying Blind. It's not mobile-related, but I wanted to mention it here in case you're interested in the subject.

Tuesday, 11 April 2006

Simmering at CTIA

Last week I walked the busy halls of CTIA, the main mobile phone trade show in the US. The show's very large. Not as big as CES, but the exhibits filled the main hall at the Las Vegas Convention Center, plus a big side hall. The aisles were filled with an overwhelming variety of everything phonelike, from phone holsters to RF components to those big fake plastic trees used to disguise cellphone towers. And of course there were mobile phones. Lots of mobile phones, from every major vendor and many of the minor ones.

When I go to a trade show, I try to look for both interesting details and the big picture, to see the trees and the forest. I had trouble doing it this time; there was so much detail and so many different things to look at that I was kind of overwhelmed by the trees. I list some notable ones below. But as I collected brochures and watched demos and poked at display models, I eventually found myself coming back to one basic thought, over and over:

Gee, the user interface on this phone really sucks.

That was the common denominator across almost every mobile phone I saw. The phone hardware was interesting, sometimes beautiful. But the phone software was generally unworthy of the effort and creativity being put into the hardware. It was gaudy, overfeatured, and extremely difficult to navigate. The more sophisticated the phone's hardware, the more impenetrable the user interface generally was.

This was intensely frustrating. It was like being served a beautiful five-course meal, and then finding all the food was made of wax. Or sitting down by a crackling fire to read a great book, only to find that all the pages had been glued together.

The interface problems clustered in two general areas: looks and navigation.

Looks: Tart me up, baby. Las Vegas is a good spot for CTIA, because the interfaces of many mobile phones seem to be designed according to the same principles as Las Vegas casinos – the glitzier the better. Why use easily understood words to label a function when you can substitute a cute color icon? And why use just a static icon when you can have it bounce around and animate when the user selects it, or better yet have several more icons pop out of it to the accompaniment of a sproingy sound effect?

On many phones, the cumulative effect of all this graphical clutter was much like what happens when you stand on the Las Vegas Strip at night – visual overload. It's pretty, but you also have trouble figuring out where you're going. On the Strip, the effect is intentional; the casinos want you to feel a little disoriented so you'll stay longer and lose your inhibitions. I'm not sure what benefit the mobile phone companies get by making their customers feel lost and overwhelmed.

Navigation: Help, I've opened a sub-menu and can't get out. My general test of a phone interface is to see if I can confuse myself in one minute. I start clicking around, trying features, to see if I can lose track of where I am or get stuck without any obvious way to get back to where I was before. On almost all the phones, it was depressingly easy to do this. Typically I'd get stranded in a sub-menu or dialog with no way to get out without canceling everything and going all the way back to the main phone interface. The more sub-menus I had navigated through to get to that point, the more frustrating it was to have to start over.

Usually the culprit was missing or inconsistent use of navigation buttons. Often different applications on the same phone will use different buttons inconsistently. For example, in one app Back may take you back a step, while Cancel may do the same thing in another app. On some phones, the button for Menu and OK are the same. If you want to get back to the main menu you'll push the Menu button, only to find that you've just said OK to a function that you didn't want to select.

Another common culprit was phones with "soft" buttons, the two buttons under the screen on the left and right side, without labels on them. In most cases, one of these buttons is normally used for Back, but some applications and functions change the button's meaning, leaving the user with no way to go back at all.

Backspacing in text when I made an entry mistake was another nightmare. Even after a lot of experimentation, I wasn't able to find the backspace function on some phones, or I accidentally selected OK or Cancel instead.

These inconsistencies and problems have the effect of punishing someone who tries to experiment with the phone. Once you've memorized a few basic functions, you're afraid to try something new for fear that you'll screw things up. I saw this a lot in early DOS users, pre-Windows. They used only a small percentage of the PC's features because they were afraid that they'd accidentally erase the disk drive if they did something wrong. Software usage rose substantially with the Mac and Windows, because people could re-use interface tricks across multiple applications, and because they were more protected from tragic mistakes.

The phone world hasn't reached this point yet. That isn't a critical problem today because most phones are used only to make calls and send text messages. But I think adoption of more sophisticated data services is going to be held back tremendously unless the interfaces get a lot more transparent and consistent.

Phone user interface vs. data user interface. What the phone industry doesn't seem to understand is that the interface needs for a mobile data device are very different from the interface needs of a traditional mobile phone phone. A phone that does just voice and texting can have a fairly playful interface because it's only doing a couple of things and the user isn't too likely to get lost. But the goal of a data device interface is first and foremost to be usable. If people can't figure out how to find the data service, or how to use it, the whole purpose of the device is defeated. That means as the phones get more sophisticated, the interfaces need to become more simplified. Right now the interfaces are going in the opposite direction.

My favorite phone interface at the show was the SonyEricsson Walkman phone. To play music, you press the Walkman button. Your commands are listed as text (rather than dancing icons) displayed in a list that scrolls up and down. It's very easy to figure out what you're doing, and very easy to go back if you make a mistake.



Sony's not exactly known for making great user interfaces, and I know from working with the Sony Clie handheld team that they have a fondness for 3D icons and strange graphical effects. So you might wonder where SonyEricsson got the inspiration to create something so simple and easy to understand.



Yeah, it's pretty clear they copied the market leader.

I don't have a problem with that. At least Sony was willing to learn from someone else's success. And I think the iPod has a lesson for all mobile device companies. Do you see any dancing icons on the iPod's screen? Do you think that has hurt iPod sales at all? The customer evaluates the iPod on how well it does its job, not on how groovy its screen looks. To paraphrase an old Macintosh commercial, "sometimes the most powerful devices are the ones that people can actually use."

Looks over usability. At the other extreme was my poster child for an overdesigned user interface, the vaunted ESPN phone.

I love the idea of MVNOs (mobile virtual network operators), small operators who buy air time from the big players and sell phones and service plans specifically tailored for the needs of vertical markets. Mobile ESPN is an MVNO focused on sports fans, leveraging the brand and content of the ESPN cable network that dominates sports television in the US.

I played with the ESPN phone at CTIA and felt both captivated and dismayed. I was captivated by the idea – a phone for sports nuts, saturated with stats and video and alerts that tell you every time your favorite team scores. I was dismayed by the implementation. The ESPN phone uses an extremely high-resolution screen that's incredibly sharp. ESPN took advantage of this resolution to cram the screen full of extremely tiny icons, and text that's literally about five points high. Think I'm exaggerating? Below I've reproduced a photo from the phone's screen (left), plus two screen shots from the Mobile ESPN website. They have all been scaled to the size of the phone's screen. The images look fuzzier here than they do in real life, because your computer's screen isn't as high resolution as this phone. But the size of the fonts should be about right. (To check, look at the service and battery icons at the top of the screen image at left. They're standard sized.)



Maybe ESPN's target demographic is 19-year-olds with the eyesight of eagles, but I don't know how many of them can afford the phone's monthly service charge, which can run to $60 or more.

My other issue with the phone is its navigation scheme. The phone uses an innovative icon menu called the Sideline that pops out of the left edge of the screen. You can scroll up and down in it, and then the menu pops back out of sight after you make your selection. I think ESPN could have achieved the same effect with greater usability if it had used text labels instead of icons (preferably nine-point text), but I can live with that because the number of icons is relatively low. What bothered me most was the sub-menus you get after choosing an icon. These menus don't operate the same way as the Sideline itself, and I couldn't figure out how to get out of some of them.

To me, the ESPN phone is a great example of glitz over functionality. But out of fairness I should also let you know that it has gotten some enthusiastic reviews here and here and here and here. Maybe I'm out of touch.

Or maybe a lot of reporters are nearsighted.


Brew: Great technology, wrong business model

Part of Qualcomm's booth was given over to small demo stations for Brew developers. If you're not familiar with Brew, it's Qualcomm's answer to iMode and Java (Brew...Java...get it?) Brew is a software development platform paired with a very nice billing and software installation engine. Qualcomm's trying to get it installed on a lot of phones. Software developers are being encouraged to create Brew apps, which they can then sell to users through operators that adopt Brew.

Unfortunately for the developers, Brew is not set up to be an open garden. Instead, the operators choose which Brew applications they want to offer to users. This gives developers very little control over their own fates – they have to go begging to operators to carry their products. This favors larger developers who can afford the investment and risk associated with marketing to operators.

The downside for Qualcomm is that the most innovative developers are usually the smallest ones. They're the people who are least likely to be able to jump through all the business hoops associated with developing for Brew. As a result, most of the applications I've seen for Brew are pretty unimaginative, not the sort of things that would compel an operator to feel like they had to offer Brew. I think Qualcomm's very nice architecture is being stunted by a bad business model.


Palm and Microsoft, sittin' in a tree...

Palm is doing an extremely good job of leveraging its relationship with Microsoft. The Palm booth was in a very visible spot, right next to Microsoft, and they were featuring each others' products. Meanwhile HP, the single largest licensee of Windows Mobile products, was stuck some distance away in an obscure booth overshadowed by another pavilion. Even after I found HP's spot, there were almost no mobile devices on display. CTIA underscored how far behind HP has fallen in the phone market.


LG: Love my heinie, love my phone

Most companies that step up from regional player to global brand go through an awkward phase while they learn how to talk to people from other countries without coming off as peculiar. LG is in the middle of that process, and earned my award for the most embarrassing booth at CES. As far as I can tell, LG wants desperately to convince the world that it's not a square Korean electronics company. Unfortunately, to prove this it put on a floor show in which very leggy girls rapped badly with supposedly edgy hip-hop DJs who looked slightly like refugees from the Cotton Club circa 1940.


Photo by PC Magazine. There are a lot of great phone photos from CTIA on their site.

I guess the theory was that if you liked a dancer's big hair you'd also like her phone.

The show was hilarious, but probably not in the way LG intended. Rather than making the company look cool, it looked out of touch and insincere. That's marketing poison in the US, where companies are falling all over themselves to appear genuine.

LG makes very creative hardware. Its clamshell "V" keyboard phone (formerly called the VX9800) would be formidable if it were paired with a RIM client instead of an MP3 player, and supposedly the LP4100 Sobriety Phone with built-in breathalyzer is a hit in Korea. (I'm not making this up.)


Yes, that's a driving game on the screen of the Sobriety Phone. I guess the idea is that if you're too blitzed to drive for real you can make believe while you wait for the cab.

LG is a sharp, fast-moving, incredibly intense company, but almost none of its culture or history involves tall blond women, and a quick check of the website of LG CEO Kim Ssangsu revealed no mention of hip-hop at all.

Here's hoping that LG will grow out of its very awkward adolescence quickly.

Monday, 10 April 2006

(Re)Announcing Stop Flying Blind, my second weblog

I'm very happy to say that I've finally re-launched my second weblog, Stop Flying Blind. It's now hosted at a new location, using a different blogging tool, with a cool design. The design is by Michael Rohde, and the programming was done by Michael Ashby. They do great work, as you'll see if you check it out.

Stop Flying Blind is a book I'm writing on how to plan corporate and product strategy, using market and technology information. It's the summation of what I've learned through almost 20 years of work in strategy and product planning roles.
It's not focused on tech the way Mobile Opportunity is, and I hope it'll be useful to people in any industry who work in marketing, product management, strategy, and general management. I will continue to post all my tech commentary here, while general comments on management and business strategy will be in Stop Flying Blind.

I'll be posting a new section of the book about once a week (the first part is already posted). The idea is to get the first draft out there, collect feedback, and then hopefully get it into print in the future. Your comments and suggestions will be incorporated, and are encouraged.

If you're interested in the subject, I invite you give it a look.