Author: Aoki Takayuki

　The Apple Vision Pro, which went on sale in the US on 2024/2/2, went on sale in Japan on 2024/6/28 with a delay of about 5 months, and it arrived at our company today 2024/7/8. I tried using it for an hour and was impressed, so I wanted to write this essay. The first half is full of folktales, and if you start to want to talk about folktales, people are likely to say it's old age, and I'm going to spit it out here so I don't show it out at drinking parties or meetings.

‍

Me and the history of VR head-mounted displays (HMDs)

We (Kadinche Co., Ltd.) first used an HMD with a model called the Oculus Rift, and that was in 2013/12. This image was posted on Facebook at the time, and my comment “Frankly Speaking, I Felt Motion Seduced Seeing the Demo Contents of Oculus Rift.” was attached. At the time, we were working on creating 360-degree images by using a combination of multiple cameras. When working on the input side called a camera, it is inevitable that people are interested in the output side for displaying it, and I remember that time bought the first Oculus Rift unit called the Development Kit (DK).

After that, viewers for viewing 360-degree images with an HMD were developed, and were also provided to VR video distribution companies, and Oculus and HTC Vive HMDs were purchased immediately when they were sold, and other manufacturers also bought and tested main products such as Varjo, Pimax, HP Reverb, Pico, etc., and are looking forward to messing around with them when they have particularly characteristic features. For example, the Varjo had its high resolution, the Pimax had a wide viewing angle, and the HP Reverb G2 Omnicept Edition was equipped with various vision sensors intended for use in the medical field. Although I wasn't personally immersed in it enough to use it as a leisure or hobby, the possibilities of VR have been greatly exploited, and it has become the axis of the company's business.

Why have you felt that VR (HMD) has so much potential?

As a new graduate, I worked at an image signal processing laboratory for an electronics company. Since I joined the company in 2003, it is an era where smartphones, let alone HMD, are not popular (the first iPhone was released in 2007). Research results were mainly mounted on televisions and projectors. It was a time when televisions were the king of AV (Audio Visual) devices, and it was a time when the transition from cathode ray tubes to liquid crystals and plasma was underway. I've been working on “displays” since I was a new graduate, and the root is that I've been interested in visual systems and video systems since before.
I became passionate about VR when I developed a 360 video viewer using Oculus Rift from around 2016 and provided it to client companies, but after that, around 2019, the magazine “WIRED” Japanese Vol. 33 featured “Mirror World: The Next Giant Platform Created by AR,” and I was excited about AR and MR along with VR. This mirror world was also called AR Cloud or digital twin at the time, but in 2024 style, I think it would be spatial computing. There is a description of “80% of the global workforce doesn't have a desk” in one of the articles in this magazine, and I was moved that XR (VR, AR, MR generic name: extended reality) is a revolutionary medium that can provide computers to people who cannot use desktops/notebook computers at work. It says that HMD has great potential as a wearable computer.

‍

As a company's business?

I had a feeling that the VR boom began around 2016. For the company, the 360 camera boom and VR HMD boom occurred at the same time, and new products came out one after another for both input and output systems, and it was a fun and difficult time just chasing them. When a certain level of boom occurs, both consumers (consumers) and enterprises (companies) think about how to use it, and everyone has been working on various system development, such as entertainment applications for consumers, training for enterprises, and remote control systems. However, over the past 2-3 years, the consumer market has come to an end, and I felt that there is a sustainable need for the enterprise market. I don't use it every day, but there are experiences, etc. that are difficult to realize without VR for training and specific occasions, and it is used as a means to achieve that.

What should I evaluate for Apple Vision Pro

By the way, to be fair, I like Apple for smartphones as much as iPhones and Macs for computers. It was Windows until my master's degree, and Solaris at the company I joined as a new graduate, but since then it's been Mac ever since.
When such iPhone and Mac users used AVP, the usage procedure was as follows. When you take the AVP out of the box and connect it to the battery, it starts up immediately, and if you have an iPhone, do you set the AVP to the iPhone side just by bringing the iPhone close? The message is displayed, and if you answer yes, account information registered on the iPhone will also be carried over to AVP. Then, if you bring the AVP and Mac closer after starting up, you can now display the Mac screen on the AVP. It only took a few minutes to connect the three devices. What a seamless experience.

By calling itself spatial computing, you can browse not only 2D still images and videos, but also 3D spaces and objects. Once you place the Mac screen around your desk, then stand up and walk around the office and come back to your seat, the Mac screen is perfect where you placed it earlier, isn't it? Is this spatial computing?
Until now, when using an HMD, for example, in order to look at a smartphone or PC to enter a password, there was the hassle of removing the HMD and processing every time there was a notification on those other devices, and then putting on the HMD again, but since the pass-through camera has sufficient resolution in AVP and the Mac screen can also be displayed in mirroring, there is no need to remove the HMD one by one. Wearable computing that you can keep wearing all the time!

Not suitable for short term rentals

Incidentally, when purchasing an AVP, there is a process called fitting using an iPhone camera. Thus, the size of the headband and light ceiling is determined, external light is firmly blocked, and something suitable for the shape and size of one's head can be delivered. Also, as a calibration (adjustment) at the time of initial setting, there is a process of registering the movement and position of one's own eyes to the AVP by following an icon arranged in a circle with the eye, and this takes 2-3 minutes. On top of that, it syncs with my iOS account and displays a Mac screen, so there is thorough personalization of all hardware, software, and content. Therefore, even if you are casually told to “lend me a little bit,” I think that the true value of AVP will not be conveyed until you go through this process that takes about 30 minutes to 1 hour. There's also a guest mode, but for that level of VR experience, I think the cheaper MetaQuest 3 etc. would be fine

Future possibilities for XR HMD

However, there is no denying that there are just a few challenges. First, the Mac screen cannot be multi-displayed. AVP can only be displayed on a single display, and the experience of using multiple desktop displays cannot be realized so far. Also, although the pass-through camera has a high resolution level that does not interfere with life, motion blur occurs when reading fine print in newspapers or looking at the surroundings while walking. The weight of the AVP main unit is about 600g, and the included battery is about 350g for a total of about 1 kg, which is not as heavy as 1.24 kg of the Macbook Air, but it is heavy. In the photo when I applied AVP without thinking about anything, it was shown that wrinkles were on my cheeks due to the weight of the AVP.

However, even so, I felt that AVP had the potential for human expansion. Until now, I felt that VR and MR HMDs also had potential due to their sense of immersion and spaciousness, but none of them were as stressful for everyday use as AVP and could not be linked to other existing devices. The current pass-through camera is a stereoscopic camera with RGB+ depth, but for example, by linking it with AI, automatic recognition and analysis of what is being projected can be performed, invisible light that humans should not be able to see can be seen, or ultrasonic waves can be heard through microphones, so I feel the possibility that it will further expand the human head where many senses are concentrated.
This is getting fun, isn't it?

‍