You may have noticed that CableLabs is focused on innovation. One of our goals is to be recognized as the leading industry innovation lab in the world but talking about our innovation can be a bit tricky. Our job is to deliver innovation for the worldwide cable industry, but we can’t really talk about what we are working on now. We need to keep that secret for our member companies (cable operators) until the technology is ready to launch.

Our CEO, Phil McKinney has talked about how innovation is messy. Where you start may not be where you end up. I want to tell you about the path that led to one of our most important innovations--and part of our 10G platform. Low latency.

Our Low Latency Journey

We started on this journey over four years ago, with a challenge question (Focus in the FIRE methodology): What applications will drive a need for 60Mbps+ of sustained Internet bandwidth? That led to ideation sessions that unearthed the usual suspects: Internet of Things (billions of sensors, but each with such low bandwidth that they still don’t add up to much), 4K streaming video (good try, but still only 15Mbps or less), “Big Data” (sorry, not really a candidate for consumer households). Those applications didn’t quite answer the question.

But the emergence of 360° immersive video looked promising. Experiencing some of the earliest 360° video at the beginning of 2014 (shot on 6 Go-Pro’s, manually stitched) on a low-resolution Oculus Development Kit Virtual Reality headset got us thinking about where the technology might lead. Six 4K videos, streamed to the headset met the challenge of over 60Mbps, although compression gains would reduce the bandwidth and resolution increases would increase it.

Rather than “geeking out” on the technical possibilities, we followed advice from Phil: “Talk to consumers!” In February of 2015, we did primary research, bringing 50 varied members of the public into CableLabs to try out “immersive video content.” Rather than just focusing on virtual reality (VR) headsets, we constructed some other ways of consuming the content, such as immersive multi- 4K TV displays, ultra-wide projectors, tablets and regular TVs. We needed to understand whether “regular humans” (not geeks) would like these technologies.

The consumer research was massively informative. We shared the insights with our member companies at the time and realized that this ecosystem was likely to take off. We stepped back and tried to work out other mass-market use cases for VR.

Digging Deeper

We pivoted. We started to look at the possibilities of transforming how people communicate, and the ability to have holographic telepresence using digital human technology to perform digital headset removal. We don’t really want to talk to another person and see that person with a headset on; we want to see other people eye to eye and have them see us eye to eye. To prove the point, later in 2015 and into early 2016 we developed eye and mouth tracking capabilities that we added to a wireless VR headset and developed a digital human avatar of one of our staff.

We linked the head, eye and mouth tracking to real-time control of the digital avatar.

And in May of 2016 we demonstrated this to our board of directors.

What we found was a compelling application that others companies such as Microsoft, High Fidelity and more recently Facebook were starting to identify and investigate.

We also found that realistic digital human avatars take LOTS of compute to render in real time, and that required a tethered PC. Even as mobile processors get faster and more capable, PC graphics will always be faster and more capable, due to their power budget. Phones get hot when you try to render realistic humans. To get to mass-market adoption, we need to go wireless and move the PC out of the home.

No Less Than a Revolution

VR needs incredibly low latency between head movement and the delivery of new pixels to your eyes, or you start to feel nauseated. To move the PC out of the home, we need to make the communications over the cable network be a millisecond or less round trip. But our DOCSIS® technology at the time could not deliver that.

So, we pivoted again. Since 2016, CableLabs DOCSIS architects Greg White and Karthik Sundaresan have been focused on revolutionizing DOCSIS technology to support sub-1ms latency. Although VR is still struggling to gain widespread adoption, that low and reliable DOCSIS latency will be a boon to gamers in the short term and will enable split rendering of VR and augmented reality (AR) in the longer term. The specifications for Low Latency DOCSIS (as a software upgrade to existing DOCSIS 3.1 equipment) have been released, and we’re working with the equipment suppliers to get this out into the market and to realize the gains of a somewhat torturous innovation journey.

Low latency is a key component of our 10G initiative. You can read more about the importance of latency here, and gain access both to a technical brief (members only) and to a detailed report (members only) on Wi-Fi latency in retail Wi-Fi routers.

This is the second of a series of posts on Virtual Reality technology by Steve and others at CableLabs. You can find the first post here.

In the recent study we performed at CableLabs, we asked what would stop people from buying a virtual reality headset. High on the list of items was availability of content. Setting aside VR gaming, people didn’t want to spend money on a device that only had a few (or a few hundred) pieces of content. So there has to be lots. Who is creating content, and how quickly will content show up?

Leave it to the Professionals?

Three companies have emerged as leaders in the space of professional content production:

• JauntVR who just secured another $66M investment

• Immersive Media who worked on the Emmy award-winning Amex Unstaged: Taylor Swift Experience

• NextVR who seemingly wants to become the “Netflix of VR.”

Each company is developing content with top-tier talent, each working to establish brand equity and to develop critical intellectual property. Another thing they have in common is that they have developed their own cameras to capture the content, yet they all say they are not in the business of building cameras.

JauntVR Neo Camera 

The new Jaunt Neo camera seems to have 16 sensors around the periphery, which we might expect to yield a 16K x 8K resolution. This is far in excess of what the Samsung GearVR will decode and display (limited to 4K x 2K 30fps, or lower resolution at 60fps), but shows the type of quality experience we might expect for the future. The NextVR camera uses 6 4K RED professional cameras, each of which is a rather expensive proposition before you even buy lenses.

But there seems to be a problem of scaling up. To get to thousands of pieces of professional content we will have to wait for years for the three companies and any other new entrants to scale up their staff, equipment, and get engaged in and produce enough interesting projects.

Or will consumer-generated content win the day?

On the flip side, we have seen the rate at which regular video is created and uploaded to YouTube (300 hours uploaded per minute).  All we need is for affordable consumer-grade cameras to show up. The first cameras are already here, though their resolution leaves something to be desired. Brand-name cameras like the Ricoh Theta and the Kodak SP360 are affordable consumer-grade 360° cameras that are around $300, but they only shoot HD video. When the 1920 pixels of the HD video are expanded to fill the full horizon, all that can be seen in the VR headset is less than a quarter of that - or around 380 pixels. Most of the content we used in consumer testing was 4000x2000 pixels of content, and even that showed up as fairly low resolution. There are some some startup names entering the scene (like Bublcam and Giroptic), but these are not shipping yet, and don’t advance beyond HD video for the full sphere of what you see.  Perhaps more interesting in terms of quality are cameras coming from Sphericam and Nokia that promise 4K video at 60 frames per second, and an updated Kodak camera that supports 4K. These cameras reach a little beyond what the Samsung GearVR headset can decode (which is 4K 30 fps) but are a little beyond the consumer price point, entering the market at $1500. Oddly missing from the consumer arena here is GoPro, but I think we can reasonably predict a consumer-grade 360° camera from them late in 2015 or early 2016 that is a little more manageable than the 16-GoPro Google Jump camera, or the 6-camera rigs that hobbyists have been using to create the early content.

YouTube and Facebook 360 are already here

Facebook launched support for 360° videos in timelines just last week. YouTube launched 360° support in March, and in six months already has about 14,000 spherical videos, with few mainstream cameras available. I strongly suspect that user-generated content will massively out-number professionally produced 360° content. Out studies suggested that while consumers guessed they would be willing to wear a VR headset for 1-2 hours, short-form content would be favored. This matches the predominant YouTube content length well. While Samsung has created their own video browsing app (MilkVR), a YouTubeVR application is sure to emerge soon, allowing us all to spend hours consuming the 360 videos in a VR headset.

What Content Appeals?

Shooting interesting 360° content is hard. There is nowhere to hide the camera operator. Everything is visible, and traditional video production focuses on a “stage” that the director populates with the action he or she wants you to see. We had a strong inkling that live sports and live music concerts would be compelling types of content, and might be experiences people would value enough to pay for. We tried several different types of content when we tested VR with consumers, trying to cover the bases of sports, music, travel and “storytelling” - fiction in short or long form. We were surprised to see the travel content we chose as the clear winner, leaving all other content categories in the dust on a VR headset. We gained lots of insights into the difficulty of producing good VR video content (like not moving the camera, and not doing frequent scene cuts), and the difficulty of creating great storytelling content that really leverages the 360° medium. Let’s just say I am not expecting vast libraries of professionally produced 360 movies any time soon.

The ingenuity of the population in turning YouTube into the video library of everything is likely to play out here too, with lots of experimentation and new types of content becoming popular. The general public, through trying lots of different things, is most likely to find the answers for us. For that to happen, mass adoption of VR headsets has to take place, and consumer 360 cameras have to become cheap and desirable. Samsung announced the “general public” version of their headset at the Oculus Connect 2 last week, with a price point of $99 over the cost of their flagship phones. Oculus expects to ship over 1 million headsets in their first year of retail availability. Cameras are already out there, inexpensive, and getting better. In 12 months I expect over 100,000 360° videos to be published on YouTube.

Is the killer app for VR… TV?

Netflix just launched their app on the GearVR. Here at my desk I can watch movies and TV episodes on a VR headset, sitting in a ski lodge with views out the window of ski runs. I’m sitting in front of a (virtual) 105” TV, enjoying an episode of Sherlock. The screen quality is pretty grainy, but I quickly forget that and become engrossed in the episode -- until my boss taps me on the shoulder and tells me to get back to work. I thought this was a pretty dumb use case for the technology, but I was wrong. Dead wrong. Only problem is I can’t see the popcorn to pick it up and try to get it into a mouth I can’t see. There is lots to ponder here.

Steve Glennon is a Principal Architect in the Advanced Technology Group at CableLabs.

In the Advanced Technology Group at CableLabs I get to play with all the best toys, trying to work out what is a challenge or an opportunity, and what is a flop waiting to be unwrapped. 3D is a great case study. I first got to work with 3D TV in 1995 (in a prior life), with an early set of shutter glasses and a PC display. Back then it faced the chicken and egg problem - no TVs, and no content. Which would come first?

I still remember the first 3D video I saw in 1995. It was high quality, high frame rate, and beautiful, but not compelling. I still didn’t find it compelling in 2010 when the 3D TV/content logjam was broken.

I remember the first time I saw Virtual Reality video at CES in January 2014. My reactions were starkly different. The first VR Video I saw was grainy, blurry, and a little jerky. And it was incredibly compelling. I was hooked.

You can’t talk about VR - you have to experience it

Lots of people talk about VR - they “get it” in their heads. You see video playing, and you can look all around. And you can look stupid with a headset on.  But you don’t really get it until you experience some good content in a good headset.

This doesn’t convey what it feels like see elephants up close. Everyone loves elephants, and with a headset you feel like you are meeting one for the first time, in their natural habitat.

And this doesn’t convey the reaction of stepping back to avoid getting wet feet, with icy cold Icelandic water.

So we have been doing a couple of things. We have been showing the headsets to people at the CableLabs conferences, where we got almost half the attendees to try on a headset and experience it first hand. We also wanted to put some market research behind it, so we designed some consumer panels to get regular consumer feedback across a representative demographic cross-section.

VR Is Only For Gamers. Or Not.

A lot of the early focus for the Oculus Rift (and Crystal Cove) headset has been around gamers and gaming. It is the perfect add-on for the gamer, where every frame is rendered in real time and you can go beyond “looking around” on the PC screen to just looking around. I had the pleasure of getting a tour and demo of the HTC Vive headset at Valve’s headquarters in Bellevue, and there you can walk around in addition to looking around. It is the closest thing to a Holodeck that I have ever experienced. I think we can reasonably expect these to go mainstream within the gaming community.

But the consumption of 360° video has much more importance to cable, as mentioned by Phil McKinney in this Variety interview, because of the need for high bandwidth to deliver the content. We’ll look at that in more detail in a future blog post.

Rather than use our own tech-geek, early-adopter attitudes to evaluate this, we wanted to get a representative sample of the general population and ask their opinions. So that’s what we did. With some excellent 360° video content from Immersive Media, a former Innovation Showcase participant, we asked regular people, with regular jobs. Like teachers, nurses, students, physical therapists, personal trainers, and a roadie. We tried to come up with some different display technologies to compare against, and showed them the Samsung GearVR with the content. Here’s what they told us:

57% “Had to have it”. 88% could see themselves using a head-mounted display within 3 years. Only 11% considered the headset to be either uncomfortable or very uncomfortable. 96% of those who were cost sensitive would consider a purchase at a $200 price point (the GearVR is a $200 add-on to the Samsung Galaxy Note 4 or Galaxy S6 phone).

So this seems overwhelmingly positive. There is the novelty factor to take into account, but we were surprised by how few expressed any discomfort and how positively regular people described the experience.

“VR is the most Important Development since the iPhone”

I had the distinct pleasure of spending time on stage with Robert Scoble during his keynote at the recent CableLabs Summer Conference. We discussed the state of the art in head-mounted displays, 360° cameras and content (we’ll talk more about that in later blog posts) but he expressed this sentiment (paraphrased): “Virtual reality is the most important technology development since the original iPhone”. I hadn’t thought about it that way, and now I agree with him. This is not just hot and sexy, a passing fad. It has massive potential to transform lots of what we do, and we can all expect incredible developments in this space.

I want to meet Palmer Luckey

Palmer Luckey created his first prototype head-mounted display at age 18. Four years later he sold Oculus to Facebook for $2 billion, having dropped out of school to build more prototypes. I didn’t see the Facebook deal coming, and didn’t understand it. Now I get it. I want to meet him and thank him for transforming our lives in ways we cannot imagine. We just haven’t witnessed it yet. We will explore more in the next few posts.

Steve Glennon is a Principal Architect in the Advanced Technology Group at CableLabs.