Along with the growing demand for virtual reality (VR), inquiries about not only pre-recorded 360-degree videos, but also VR live streaming, which is a relay of 360-degree video, have increased. In this article, I will mainly explain the technical aspects of live streaming using a wide-angle lens (not only 360 degrees, but 250 degrees, etc.), centering on commonly heard items.

What is VR live streaming

In order to view images on head-mounted displays, the use of 360-degree video (omnidirectional video), which can change the area to be displayed according to the user's head movements, is increasing. In VR live streaming, by distributing such 360-degree videos in real time, it is possible to experience all current ambient conditions on a head-mounted display or browser. The following hardware and software are required to realize this VR live streaming.

Compatible cameras and compatible lenses

Cameras that enable VR live streaming have at least the following 3 conditions.

I want to shoot at a wide angle

In order to take advantage of the wide viewing angle on a head-mounted display, images to be streamed live must also be shot at a wide angle. Since it is reflected in 360 degrees, images can be displayed in all directions even on a head-mounted display. Also, if the image is 360-degree, even the shooting staff and equipment such as the cameraman will be shown, so dare to do things wrong250 degree lensUsing it would be a good idea.

I want to output images to a transmitter such as a PC through the output interface

However, for simple shooting, there will be no problem if you record it on the built-in memory card, but in the case of live streaming, it is necessary to output the video from the camera in real time and input it to distribution equipment such as a PC in order to distribute it to the internet. Therefore, it is necessary to be able to output images via terminals such as HDMI or SDI.

Want to drive for a long time

I think most cameras used for normal shooting are battery-powered, but in the case of live streaming, there are many cases where streaming continues for a long time, and in such cases, electricity must be supplied from an AC power source etc. for two reasons: power supply and measures against camera heat generation.

VR live streaming transmission set

Cameras that meet the requirements for VR live streaming as described above are as follows.

250 Degrees 4K: Entaniya 250 + Blackmagic Design Micro Studio Camera 4K

By attaching a 250-degree lens compatible with the Micro Four Thirds mount manufactured by Entaniya to the Black Magic Design Micro Studio Camera 4K and outputting images from that camera to a PC with an SDI terminal, it is possible to distribute images in 4K. The area shown is not 360 degrees, and since it is 250 degrees, photographers and the like can hide in that blind spot. By using a PC equipped with a GPU for distribution, high-resolution images can be distributed at a high bit rate. At our companyThis set will be made to order and will be soldThat's it.

360 Degrees/2K: Ricoh R Development Kit

A 360-degree live streaming camera announced by Ricoh and is currently accepting pre-orders and scheduled to be shipped after 2017/5Ricoh R Development KitIt will be. This is a model with a 360-degree camera called Theta that is more advanced and specializes in live streaming, and although it has full HD resolution, it is characterized by being inexpensive (59,800 yen), compact, and highly functional (real-time stitching and zenith correction).
Note that in order to use Ricoh R for live streaming, it is provided by PC or Cerevo, which performs distribution to the internet in the latter stepLiveShell XIt uses a combination of distribution devices such as.

360 Degrees/4K: Insta 360 Pro

Cameras have evolved rapidly, and products that can shoot at 360 degrees and in 4K, have also come out with an Ethernet cable stuck inside the camera body and even a built-in distribution function.
The first one is manufactured and sold by Arashi Vision in ChinaInsta360 ProThat's it. Not only can this camera do live streaming in 360 degrees and 4K, but the images are compatible with stereo vision, and it is also compatible with up to 8K in recording mode. It is scheduled to be shipped after the spring of 2017, and it is likely to be around 420,000 yen in Japanese yen. Hakosco is the distributor in Japan, so please contact us if you wish.

360 Degrees/4K: Orah 4i

The second one is sold by French company Videostitch, which is also the developer of Vahana VR, which is real-time stitching software for multiple camerasOrah 4iThat's it. This is also compatible with 360 degrees and 4K, and the price is expected to be around 420,000 yen (3,595 US dollars+shipping) in Japanese yen.

communication protocol

For live streaming, communication protocols are used according to the application. There are options such as RTMP if you want to distribute to a large scale (multiple users) even if there is a slight delay, HLS for viewing mainly on smartphones, etc., and WebRTC if you want to watch with low latency even if the number of viewers is small. Here, we will introduce the 3 types we have used in web services and experiments.

RTMP: Real Time Messaging Protocol

The protocol used by most of the above cameras for transmission is RTMP, which is a protocol suitable for large-scale live streaming. Compared to WebRTC described later, delays occur from a few seconds to tens of seconds, but it is a protocol suitable for simultaneous access etc. by multiple users.

HLS: Http Live Streaming

HLS is a protocol mainly used when viewing live video streams on smartphones, and streaming in HKS format can also be viewed on iOS and Android. In streaming servers, although delays of tens of seconds occur due to the mechanism of buffering video data, they are often used in combination with RTMP, making them suitable for large-scale distribution.

WebRTC: Web Real-Time Communication

If RTMP and HLS are large-scale distribution/medium latency, then WebRTC is a small-scale distribution/low latency protocol. Since it was originally a protocol for messaging such as live chat, etc., it is often used for one-on-one video chat etc. without going through a relay server, and by making it compatible with 360-degree video, live streaming for a small number of people can be realized with low latency.

Compatible websites

Here, we will introduce a web service that users who have actually obtained a camera or distribution set like the one described above can use when broadcasting on their own.

youtube

First, YouTube is a video distribution site, and since around 2016/4, it has also been compatible with receiving 360-degree live streaming. You can use it for free,Delivery instructions are also described hereThere is such a thing, so I think it's the easiest service to use.

Facebook and Twitter

As of 2016/3, it has not yet been opened to Japanese users, but it has been announced that Facebook and Twitter will also support live streaming of 360-degree videos in the future.

PanoPlaza Live

It is also compatible with PanoPlaza Live, which is a 360-degree video distribution and relay service provided by our company. As of 2016/3, it can be used for free in the same way as Youtube, and apps are also provided for iOS/Android/GearVR, so confirmation on a head-mounted display is also prepared. How to useListed hereThere is.

‍

Stereo/3D compatible (added)

What is described above is a 360-degree or 250-degree image with a monocular lens. In order to support stereo with a three-dimensional effect, the camera must first be compatible (only Insta360 Pro is compatible with the above), and the website being viewed must also support stereo 360-degree live streaming. According to our research, as of 2017/3, I don't think there are any sites that are compatible on the website side yet. However, it's not technically difficult, so as the number of stereo 360 cameras increases, the number of compatible sites (including our company) is likely to increase.

‍

Delivery method

We will bring cameras and lenses suitable for VR live streaming, and equipment for distribution to the site. Onsite requirements are a stable power source and a wired internet connection. Once you have them, all you have to do is distribute the video to the compatible website (reception address). It can be handled by using a switcher when using multiple cameras at the site, and by using a distributor or the like when there are multiple distribution destinations.