Encoding 360 video in 6K for uploading

TL;DR: Use ProRes for your upload, or transcode to H.264 on the command line using ffmpeg to get >4K into H.264 outside of standard level specs.

If you’re trying to upload 360 video content greater than 4K, this guide might be useful. Here are couple possible scenarios:

• 4096 x 4096 top/bottom stereoscopic 360 video in equirectangular projection
• 6144 x 3072 mono 360 video in equirectangular projection

This will be a theoretical discussion for most people because not many people can create content greater than 4K content. As of Feb 2017,  greater than 4K support for 360 videos on Facebook hasn’t been rolled out to everyone, but large pages should have it. YouTube has announced support up to 8K.

4K video capture is somewhat standard these days (all high-spec smartphones are 4K-capable), but in traditional video consumption on the web, 4K is still rarely streamed. When embedded in a Facebook timeline, for example, video resolution is nowhere near 4K, and when streaming from YouTube and Vimeo, 4K is only available for people who have monitors that are high enough in resolution. One might wonder why people still like to capture video in 4K? High-end productions need as much resolution as possible. I do it because I am both a photographer and a data hoarder. I imagine that one day I’ll go back and look at old videos, and I’ll be glad that I captured them in 4K (or greater) rather than in 1080p.

360 NEEDS MOAR PIXELS!

In 360 video, things are different. Videos are spread across a 360° field of view, and in the typical ~95° FOV in a mobile VR head-mounted display (HMD), a 4K UHD (3840-pixel) video is now left looking more like an XGA video (~1000 horizontal pixels in the field of view). Since a HMD like the Gear VR is capable of showing more than 1000 horizontal pixels per eye, video greater than 4K in resolution is useful, and since we can see 3D content in headset, we’re really interested in using two >4K videos (e.g., two 6K videos for omnistereo 360 panoramic videos—one 360 video in equirectangular projection for the left eye, and one for the right). This presents a problem not only because it would require a lot of bandwidth to stream 2 x 6K back to a headset in realtime, but also because the Samsung S6 or S7 behind a Gear VR can’t decode >4K video. At Facebook this lead to the development of “adaptive-viewport” streaming technology, which is used to (effectively) deliver >4K stereoscopic (3D) content to Gear VR headsets in the Oculus Videos app. The Facebook Code blog discusses this in great detail, but the point is that uploading 6K video to Facebook could be useful if it’s for 360 video.

HOW DO I ENCODE AND UPLOAD IN >4K?

The problem with trying to encode and upload videos in >4K is that it’s hard to encode >4K video using highly-compressed codecs that are both upload friendly and Facebook compatible. H.264 is currently the standard video codec used to compress video for Internet sharing, and its highest performance leveldoesn’t support video resolutions higher than 4K. Encoding in H.265, which supports video up to 7680×4320 in resolution, is becoming more popular, but Facebook doesn’t support H.265. Here are two possible solutions:

1. Encode >4K videos using Apple ProRes in Quicktime containers
2. Encode >4K videos in H.264 using ffmpeg on the command line 

ProRes movies can be huge, and I usually prefer option 2.

If you’re unwilling to use the command line, there is no need to read any more of this article.

If you’re new to ffmpeg download and install it. If you’re on a Mac, you might prefer to install it using Homebrew. Here’s the ffmpeg transcode command I use (assumes that the original video is >4K):ffmpeg -i input6k.mov -c:v libx264 -preset veryfast -crf 18 -pix_fmt yuv420p -c:a aac -b:a 256k -movflags faststart output6k.mp4

A constant rate factor (CRF) of 18 is considered to be “virtually lossless or nearly so.” You can adjust the CRF or ask ffmpeg to encode at a target bitrate if you need to stay under a specific file size.

If you injected the required 360 metadata, your resulting MP4 will be recognized as a 360 video when you upload it to Facebook. If you didn’t inject metadata, you’ll have to run Spatial Media Metadata Injector to inject the proper metadata into the file before uploading.

>4K APPLE PRORES ON A MAC

If you want to export in ProRes, here are settings in Adobe Media Encoder that will work, assuming you’re working with a monoscopic 360 video.

>4K H.264 IN WINDOWS (OR MAC)

1. Export master video using DNxHR in a Quicktime container (I’m not using ProRes on my Windows machine because of security issues).

2. Transcode to H.264 using ffmpeg on the command line (see above for the command).

3. Inject metadata using Spatial Media Metadata Injector.

Note that encoding in ProRes and DNxHR yields gigantic files. I’m not using high-end cameras, so I typically encode in ProRes 422 (LT) or DNxHR SQ 8-bit, and even these “light” and “standard” settings result in files that are often too big to upload. For example, I encoded a 2.5-minute 6K video in DNxHR SQ 8-bit, and it was 24GB. Using the ffmpeg command above to transcode to H.264 at a CRF value of 18, I ended up with a 276MB file. I injected metadata and uploaded successfully as a 360 video to Facebook.

Here’s the example I used in this post, which was uploaded as a 6272×3136 H.264 MP4. Click through, “save” the video, and check it out in Gear VR! You can also download the original videos to watch them locally (instructions in the original post).