Lies, damned lies & statistics
July 16th, 2010, 11:09pm by Jake
Today, watched with some amusement the latest edition of Applepalooza. Been pretty amusing watching the coverage the first week, especially when the complaints centered on the number of bars displayed. That was humorous to me, for a couple reasons: 1) In digital, if you got a connection you got a connection. Although w/ CDMA you use more power with a weak signal; 2) With previous phones (see: Treo 700p), the number of bars was pretty much completely unrelated to the likelihood of dropping a call. I’d drop calls with 5 bars, and have flawless calls with one bar. The “more bars in more places” a campaign certainly killed it as a useful indicator of, y’know, signal strength as it turned into a purely marketing ploy.
And while it was pretty funny to see how Apple handled the situation so poorly (Step 1: hold different; step 2: DO NOT OFFER FREE BUMPERS; Step 3: It’s a bar display issue, suck it up or return it… we DARE you), almost as silly was the massive coverage, leading to letters from Senators (no joke), Consumer Reports withdrawing a “recommended” rating & front page coverage on the New York Times. So let’s just get out of the way.
Now, take a look at the chart above. We’ll start at the bottom & work our way up. The iPhone 4 does drop more calls, but per Steve Jobs it’s less than 1 for every 100 calls. We’ll just assume it’s an extra 1 for every 100. Now, if AT&T users really do drop 4.5% of their calls— which is epically bad– that’s only a 22% increase in dropped calls. On the other hand, if the iPhone is so super that it only drops 2 calls out of every 100 (Verizon’s numbers), that means that there are 50% more calls being dropped. That’s a pretty big deal.
Next, we see the return rate for the iPhone is 1/3 of the 3GS return rate. That’s pretty impressive, but… I wonder how much that may have to do with people returning phones at the end of the 30 day window, which no one has reached yet. But unless we know that it’s a real Apples-to-Apples comparison (return rates at day 21), it’s not terribly useful.
Finally, the big one. Only 0.55% call AppleCare about antenna issues. Well, that’s great. But how many people purchase AppleCare w/ their phone, how many know about it, and how many have called AT&T instead? Sure, AppleCare is free for 90 days, but I don’t know how many are aware of that. Another statistic without context, which makes it really difficult to understand the meaning of any. That’s really what the so-called RDF is– the ability to provide context-free data and have the public or press repeat it back credulously. Maybe the press will learn and start ignoring numbers that are provided w/o explanation.
July 17th, 2010 at 1:27 am
It took ten writers & editors to put together this regurgitation of the Apple press event, breathlessly repeating Apple’s numbers as if they had substantial meaning. Also, BusinessWeek is now on the iPad.
http://www.businessweek.com/news/2010-07-17/apple-sets-up-cots-for-engineers-working-butts-off-on-iphone.html
July 17th, 2010 at 7:21 am
While the numbers themselves might not have the weight that Apple is ascribing them, I have no problem with that part of Apple’s message– that the real world impact of the antenna flaw is exaggerated by the bars and the media. I’m surprised that the 4 has actually dropped more calls than the 3GS at all. Everything I’ve heard (from actual users) says that it’s reception is phenomenally better than it’s predecessor (which I’m sure, is one of the worse performers on AT&T, not one of the better ones). It is true, I’d estimate that 99.9% of 3GS’ I’ve seen in the wild are in a case, while I’ve never seen anyone use a case with an iPhone 4. Not that it’s the least bit relevant– Anandtech showed that a case had no bearing on the 3GS’s signal strength (obviously not true for the 4). Apple should have just put the microphone in between the antenna segments to keep people from holding the phone that way. That’s where it is on the Pixi, and no one is calling that a flaw.
What I thought was disingenuous was the contention that the new exposed antenna design is not any more susceptible to signal loss than any other antenna, when Anandtech and CR showed the opposite pretty clearly. I love how Nokia and RIM have respond by calling out Apple’s Rookie Mistake. Maybe Ed Colligan was right when he talked about how PC guys won’t just walk in and know how to make a phone :). Apple’s antenna room looked awesome though– straight out of the X-Men.
July 17th, 2010 at 12:31 pm
The antena room is pretty awesome. Reminded me of another bit. Given that Apple had this secret lab no one knew about, do you really think we can take a face value the claim that, According to Phil, what we were looking at was “the most advanced lab for doing RF studies that anyone in the world has.” Seems that Nokia, Samsung & others are also pretty likely to have their own secret labs. Let’s see if that one slips into the mainstream media without the “Phil says” qualifier.
July 17th, 2010 at 2:35 pm
I’m sure they can provide some statistics that prove that it is.
July 19th, 2010 at 6:07 pm
this article seems so off to me, like the tech press are trying to convince themselves that everyone else was affected by the RDF. Thoughts?
July 20th, 2010 at 1:09 pm
Is there a “munged” link? Not sure what article you’re referring to.
July 20th, 2010 at 1:32 pm
Probably forgot to paste it… hang on.
http://www.tipb.com/2010/07/19/apple-moved-conversation-iphone-4-deathtouch-industry-deathgrip/
July 20th, 2010 at 5:08 pm
Good observation– As RIMM said themselves, Apple is trying to divert everyone from their specific design issues (detuning) to the industrywide problem of hand attenuation. Hey, at least John Gruber seemed to buy it– I skimmed his blog and saw post after post of videos showing other smartphones dropping bars.
July 20th, 2010 at 6:48 pm
Heh. Great catch about Gruber. It’s funny how the Macophile press seems so willing to believe Apple’s understandably biased interpretation of the problem while ignoring effectively unbiased sources like Consumer Reports. One week, they’re going on about how the bars aren’t important and in real life, the signal attenuation is a non-story. The next week, the same writers (literally) are going on about how every smartphone suffers from the same problem because of… wait for it… the number of bars, and detailed testing is irrelevant.
It’s sad, in a way. I’ll have to check in at Macintouch, they always seemed pretty straight about bad news.
July 20th, 2010 at 6:51 pm
On the other hand, I love Apple for stuff like this:
In a bid to avert the looming problem, a team headed by senior vice president Kris Rinne met with Apple to ask for help. Of course AT&T was planning to upgrade its network to handle the increased demand, Rinne’s team told Apple executives, but that was going to take years. In the meantime, would Apple take measures to help throttle back the traffic? Perhaps Apple could restrict its YouTube app to run only over Wi-Fi. Maybe the iPhone could feature a smaller, lower-resolution videostream or cut off YouTube videos after one minute. Rinne, who had already met with Apple’s iPhone team at least half a dozen times, fully expected the company to play along. After all, manufacturers agreed to such restrictions all the time. It didn’t make sense to build phones and offer features that carriers couldn’t support.
But in meetings with Apple engineers and marketers over the subsequent year, Rinne and other AT&T executives discovered that Apple wasn’t playing by traditional wireless rules. It wasn’t interested in cooperating, especially if it meant hobbling what had quickly become its marquee product.
http://www.wired.com/magazine/2010/07/ff_att_fail/all/1
July 20th, 2010 at 8:22 pm
Nice story, but I believe the iPhone does serve up a lower res Youtube version over 3G vs. over wifi.