Back in IBC - Cutting Edge Session 1&2

It was almost on all lips during IBC this year, you have to improve the immersive experience for the viewers. Not only the story or TV program have to be original and addictive and if possible an endless source of money generator, no, they also have to be more immersive.

But what is an immersive experience? Different people, different interpretations. The idea is to offer to a spectator - while he is at the cinema or in front of its tv - an experience the more realistic it can be, the viewer should forget he is watching a display (e.g. watching a football game on your sofa and feeling you are in the stadium). The question becomes how to improve the display to make it more immersive? That's what the conference speakers (#ibcconf hosted during the show #ibcshow early September in Amsterdam) were trying to answer in their presentation. And especially in the sessions "Cutting Edge 1 & 2" new technologies were discussed. I took part in session 2 and we are all presenting what are the things comings, which technology will be predominant in a near future and hopefully the one we propose will be.

Starting from a rectangle display with a fixed resolution (e.g. HD), a fixed frame per second rate (e.g. 25fps), a fixed color dynamic (e.g. each color channel pixel coded on 8bits) what do we do? How do we introduce new "things" without destroying completely the existing workflow (from image acquisition, encoding and compression, diffusion and display)?

Increasing the resolution to 4K was written everywhere this year, increasing the resolution almost 4 times the one corresponding to HD (which is the one for Blue-Ray, DVD is way smaller), but so far there is not real content or affordable customer product for home. Professional movie cameras (or the latest GoPro) can offer this resolution, but only movie theater could follow on that (also I don't remember what is the offcial/recommended resolution for film distribution). Comparing to some years ago, the switch to full digital is a reality in many movie theaters and even so the quality in full digital is said to be lower than with film, two things are interesting: there is limit for an average viewer to see/perceive a difference in quality above 2K (so do we need to go so high?) and depending where you sit in a theater you will perceive different resolutions. In case you don't know this already, the human eye is performing filtering on the signal he receives, details will be perceived in an image depending of the viewing distance (this is use in many image compression algorithm where you remove what is not seen anyway).

Researchers at BBC where showing their latest work on fps, which fps is achievable in order to increase the viewing comfort. They said that above 100fps it's better, for you to appreciate the difference they were showing the same sequence (athletes doing high jump) with "normal" fps and higher fps side by side.

About hdr, the display market is almost reduces to one company (Dolby). The technique for taking hdr images (high dynamic range images) is known, basically you have to combine several pictures taken at different exposures. One can see the issues when you want to record movie file and not a single image. Researchers at Fraunhofer ISS proposed a very neat solution where sensors with different sensitivities pseudo-randomly distributed where used. And taking into account sparsity they could in one single shot obtain an hdr image. A brilliant friend did suggest me the following "but why don't they read continuously the captors to get the values at different exposures and create the hdr data on the fly", hmm why? And hdr must not be misunderstood with RAW file format despite data in that format possibly available in higher that 8bits per channel.

And there is our approach which consists in surrounding physically the viewers: the screen is curved (e.g. cylinder or digital dome to spherical display). To do so we combine several projectors to project images on the whole curved surface. This year we proposed a study case where 3D stereoscopic content was streamed to a digital dome. The installation we described use two fish-eye cameras and a 2 diameters dome tilted at 90degres such that while standing you are looking straight in the middle of the curved screen (the field of view proposed is bigger than your natural field of view, you should feel immersed). 3D and immersive experience go together and research is actively done to make the 3D experience more seamless to the user, meaning no glasses to wear, auto-stereoscopy and multi-views. Here as well the colleagues at Fraunhofer HHI and ISS have a strong presence. The matter of 3D and immersive display (or surround cinema) is tricky, the whole 3D thing is tricky (which makes it very interesting), because looking with one eye in real life doesn't make you see a flat image, you still perceive depth. And coming back to immersive display/surround cinema, the way you generate 3D images has to take into account where the viewer may look at, the freedom you are offering (possibility to look in every direction) is a challenge for the generation of proper stereoscopic effect. And I probably lost 98% of my friends after these lines.

All these large, curved, multi-blended displays need to be accurately color calibrated on the top of their geometrical calibration (we also do that at Fraunhofer FOKUS in our department VISCOM). The viewers need to have the feeling to look at one single light source only. The more you forget about the technique during the show you are watching/attending, the more immersive is the experience.

Last but not least, sound is of course in the game. You can't talk about surround cinema without surround sound. That wasn't too much mentioned in the sessions, but both image and sound are working together, beautiful isn't it... That is not my specialty, mine is color and image processing and more, I let the sound to my colleague specialists.

In the Amsterdam harbor

I can see a little improvement over the years coming back to Amsterdam, after almost 8 times I get less lost which is always nice. But to look like a local demands some practical organization, you don't want to not give the impression you are a tourist (by staying in front of the Van Gog museum or asking where is the red light district or example) or one the numerous IBC visitors (they are easy to spot because they often keep their badge around the neck, or they are packed in the tram line 4 and 11,  men dressed in suits, some are struggling to check in and out in the public transport...) they are simple rules to follow: avoid tram line 4 and 11 during the dates of the IBC event, visit other museums than the one mentioned above or simply walk from place to place and try to get lost.

Going to IBC give you only a few hours (or to any fairs or conferences) to explore the city where the event is hosted. And after the whole day standing up, running to this or this session or talking in front of the audience of the "Cutting Edge Session 2" about your work on 3D stereoscopy camera for digital dome you want to find a place who will treat you well (I'm not talking about massage with good ending), a place with a decent coffee or a good beer who tastes different than Heineken are welcomed.

Being a spotter since this summer for Berlin, I use the spottedbylocals city guide for Amsterdam which have the great feature to be usable off-line on your smart electronic device (the other features being a list of regularly updated spots in the city you are visiting or just living). So, you are lost, turn the app on, check on the map the closest spots depending what you are looking for "et voilà".

And for my second evening in town I had the pleasure to meet the people behind the spottedbylocals city guide. Of course we met in one of the local Amsterdam spots, the stadscafe Van Mechelen which offer a large choice of beer and a delicious burger. Nice evening it was.

La 3D à Berlin

Petit tour chez les collègues au 3D Innovation Center. En partance de mon bureau, s'éloigner de Mierendorfplatz, descente vers le sud, traversée du Spree, le tout à vélo et 5min plus tard arrivée à destination.

Qu'est-ce me direz-vous? C'est un show-room qui expose tout ce que sait faire le Fraunhofer à Berlin avec la 3D. La 3D pour le cinéma, la tv, applications médicales, tout. Le pipeline de production de contenu 3D comme on dit est ici montré: acquisition avec caméras, comment récuper les informations de profondeur avec deux caméras ou plus, comment générer des images multi-view (donc pas uniquement stéréoscopie dans une direction), comment encoder un flux d'image 3D (codec tout ca). Ils ont aussi un studio tv tout vert et plein de caméra partout, une régie pour travailler les vidéos, une configuration pour tester la diffusion (broadcasting donc) vers différents supports (TV, cinéma). Il y a aussi une salle de cinéma pas dégueux avec au choix deux systèmes de son immersif. Plutôt bien fourni donc. Et j'allais oublié les écrans 3D sans lunettes, donc auto-stéréoscopique.

L'idée est d'une de montrer ce que l'on sait faire, mais aussi de pouvoir organiser des workshops et démontrer ainsi la faisabilité de création de contenu en 3D. En particulier qu'un workflow HD pour production 2D est parfaitement utilisable pour la 3D (pas de coût supplémentaire excepté celui de se former). Et enfin pouvoir orienter les gens vers les personnes compétentes et éviter de travailler avec des charlatans de la technologie.

Du lien social:
- 3D Innnovation Center