Why is Netflix is sending out e-mail surveys? Aren’t they tracking our usage?
Over the weekend, I got an e-mail from Netflix, entitled “We’d like to hear your opinion,” asking me to compare my experience with Internet downloads (PC and Netflix player) and DVD rentals. It was an innocuous survey with fewer than ten questions, but I’m surprised to learn that Netflix knows so little about me.
And it was lame. Honestly the lamest survey I’ve ever seen.
Surely, a company that has recently challenged mathematical modelers and statisticians to refine their recommendation engine….would have been able to find someone (in this job market!) who had some semblance of a clue about how to do an online survey…and what questions to ask.
What Was The Survey About?
I think the idea was to measure the relative merit of online downloads in comparison to DVD rentals through the mail. But the survey only ineptly measured this with what should have been a 100-point scale slider. Whomever coded this survey clearly had no idea about the existence of Javascript and the technologies available to them.
And there were no questions about features and capabilities — the relative strength of those. Examples include:
- Selection. A question could compare the selection of movies available on DVD versus those available for Internet download. How do people feel about the movies and TV series available. I’d say the selection for DVDs is far better than for download which I’d categorize as “hit or miss.”
- Subtitles. I find the ability to turn on subtitles a valuable feature that I often use on my DVD player. Subtitles are not available on my Roku player.
- Forward/Reverse. How often do I use forward and reverse? How does the DVD compare to the download? I’d give the DVD the nod over the Internet download for this capability as well.
- Quality. Have there been times when the quality was unacceptable? What about the quality of the stream vs. the quality of the file vs. the quality of the DVD. I’ve had two download files with critical errors in them. And I’ve had far more DVDs
- Immediacy. Is it more valuable to be able to download something immediately, or is it no problem to wait 2-3 days for a DVD to arrive in the mail.
And so on and so forth. A decent survey would have defined the contours of the relative strength of the platforms. And it would have clarified the relative value of these products, including willingness to pay.
The Questions Netflix Shouldn’t Have Asked
- Age, gender and income.
Netflix then wasted time asking me about who I am. These are questions that are easily correlated in other manners…with far better accuracy. I already have given the company my credit card number, and they have my address — that should be enough.
- Viewing patterns.
Furthermore, they asked about viewing patterns, including something about whether anyone in my household had downloaded a movie to watch instantly. Note, there was no question about recency or frequency (e.g. “in the past 30 days, have you or anyone in your household…”), it was a blanket “have you ever…” question.
I bring this question up, because I’d figure that Neflix would already know how frequently we use our Roku player, how many DVDs we rent a month, and how many times we’ve download to our computer.
From there, they should have sent a far more detailed survey to the heavy downloaders
The Likely Answers
I wouldn’t be surprised to learn that most Netflix subscribers do not see the download service as a pure substitute for DVDs. The biggest reason for this will be title selection, followed by features (DVDs have better features).
The electronic programming guide (EPG) factors will play heavily into the overall Netflix platform, though heavy users will have pointed critique of the Neflix platform for video search and discovery on the television (Xbox, Roku, etc.) platforms.
But the practical upshot is that Netflix should be using data mining to answer many of these questions, and they should rely on the surveys for the more practical questions of features, relative strengths, and willingness to pay.
Downloads are a very different product than DVD rentals. If anything, the questions Netflix should be asking are about Pay TV services, VoD and so on and so forth. Because those are the real competitors to the services that Netflix offers today. And without an understanding of those dynamics, Netflix has no way to value potential EPG partnerships and distribution deals with cable MSOs and satellite providers.