The solution sounds simple – transmitting the human body and face in a 3D picture, but it is much more. In Košice, the second largest town in Slovakia resides a small innovative company called Matsuko. Just a dozen or so experts from the video game industry, machine learning and mixed reality bring into life something, which up until now was seen in science-fiction movies: they bring humans into 3D. They aim to make a revolution in remote communication between people.
“IMAGINE TALKING TO YOUR FRIEND OR COLLEAGUE IN 3D AS IF YOU REALLY WERE WITH HIM AT THE SAME PLACE”
Using just a single camera and a pair of mixed reality glasses, Matsuko delivers the most realistic experience of being physically anywhere, while zeroing down the environmental impact. You can chat with your partner or employee through Skype, but it is not so personal. People live in 3D worlds, so they prefer to communicate with 3D people. Using a human hologram could be a new experience.
Mária Virčíkova was a researcher in the field of artificial intelligence and machine learning at Technical University in Košice. She met Matúš Kirchmayer, Matsuko CEO when he became a tutor for her research project Emomime which was a 3D avatar (virtual character), that was meant to replace mannequins in shop windows. Kirchmayer’s background is in the development of video games where he used virtual, augmented and mixed reality.
Trained network
Their technology is based on a developed method of how to train neural networks to capture and reconstruct the 3D head and body from a single 2D image. Neural networks work like the human brain, it is software with artificial intelligence that can learn new things from experiences. Matsuko experts developed software and trained it, to learn how to quickly extrapolate the shape of a face from a camera stream. If the camera does not see all your head, it can image how it should look based on experience. It is like a machine learning process.
Commercial applications
The human holograms can be used in many areas, and they can be monetized as well. Images in 3D are suitable for video games, remote coaching or medicine, or for telepresence in companies. Holograms can become a tour guide in museums or shop assistants in retail. In cosmetics, it could work as a great marketing vehicle. The make-up company can recommend some treatment just according to your picture. Hologram or 3D face will be made from your picture, and professionals can present to their customers how their face will change. A similar approach can be used in plastic surgery. Matsuko has successfully integrated its high-quality 3D face reconstruction technology into the cosmetics segment.
