Friday evening in LA

in approximated English below

La conférence finissant en milieu d'après-midi et n'ayant pas de logement à San Diego ce soir je me décide spontanément de rallier LA quelques heures plus tôt que prévu. Valise fermée, sac sur l'épaule, au revoir aux collègues de la couleur, je file au terminal Greyhound de San Diego.

2h30 de bus en soirée après que le soleil se soit couché, un arrêt à Long Beach et j’arrive dans mon hotel Standard booké à l'arrache avant mon airbnb. Mon voisin de greyhound est méxicain et s’en va à SF parti de Tijuana en début de journée, on parle politique américaine - c’est rapide - et européenne mais surtout football. Ce qui confirme une chose, Merkel est connue jusqu'au Mexique, par contre Hollande c'est pas encore ca.

Cette arrivée plus tôt à LA me permet de rencontrer une locale - nous l'appellerons Betsy - amie d’un ami à Paris. On se retrouve vers 10:00pm sur le rooftop bar de mon hotel pour à priori un drink de bienvenue, il y en aura plusieurs. Dans le ciel les hélicoptères patrouillent, les manifestants manifestent et la police veille - pas une impression de fin du monde mais un peu quand même.

On redescend, direction un autre rooftop bar. Queue à l’entrée, nous bougeons parce que trop bougie, quelques mètres plus loin un autre bouge, pas de queue, des lumières rouges, bonne musique, un jardin en arrière. La musique se mêle aux sirènes ou le contraire, avant d'arriver ici on a du attendre que la fin d’un cortège passe suivi de voitures remplies de robocops, ambiance - pas l'impression d'être dans un film de zombis mais un peu quand même.

Ma guide n’est pas complètement fatiguée et moi non plus, ca tombe bien, autre endroit deux blocks plus loin au bar du Regent theatre The Love Song. Bar à l'atmosphère cosy old school - cocktail délicieux, bar woman à l'accent australien, tête rasée, robe couleur peau et énorme tatouage dans le dos elle se fond dans le décor - et discussion toujours intéressante sur LA qui dévie sur Bukowski et autres.

Pretty cool - l'impression que chaque phrase est ponctuée d'un it’s so bougie! ou quelque soit le propos d'un as a fuck mais toujours dans une tonalité charmante - pretty cool disais-je.

Le über de ma nouvelle amie arrive, je file vers mon hotel et ce sans me perdre, un miracle.

A photo posted by jeremie Gerhardt (@mrbonsoir) on Nov 10, 2016 at 10:09am PST

a google-ified translated English version

The conference ended up middle of the afternoon and having no place to stay in San Diego tonight I decided spontaneously to hit the road to LA. Suitcase ready, goodbye to the colleagues, direction the Greyhound bus terminal of San Diego.

2h30 of bus ride after the sunset, a long stop in Long Beach and I reach my Standard hotel booked in emergency a few hours ago, I only have an airbnb from tomorrow. My bus neighbor is Mexican and traveling from Tijuana to San Francisco. We talk about American politic - it's quick - and European but mostly about soccer. This confirms one thing, Merkel is known until Mexico, for Hollande it's another story.

This earlier arrival in LA allows me to meet a local - we will call her Betsy - friend of a friend in Paris. We meet at 10:00pm on the rooftop bar of my hotel for in theory one welcome drink, there will be many. In the sky the helicopters patrol, the demonstrators protest and the police monitors - not an impression of end of the world but not far from it.

We go down, aiming for another rooftop bar. Queue at the entrance, we don't wait because it's too bougie, a few meters away we dive into another dive, no queue, red lights, good music, garden in the back and beer in our hand. The music is mixing with the sound of the police, before reaching this place we let a cortege of policemen dressed like robocops following demonstrators passing before crossing the road, spooky - not the impression to be part of a zombie movie but not too far neither.

My local guide isn't totally tired and me neither, good for us, next stop is two blocks further away in another place, the bar of Regent theatre called The Love Song. Atmosphere is cosy old school, the cocktail delicious, the Australian bar woman fits with the scenery, shaved head, white dress skin color and a huge tattoo on her back. Ongoing discussion with my partner in crime still interesting about LA, Bukowski and many more.

Pretty cool - I have the impression that each sentence is punctuated with a it's so bougie! or whatever the topic is by as a fuck but always in a lovely key - pretty cool did I say.

The uber of my new friend arrives, I left to my hotel without loosing my way, a miracle.

Vision de San Diego

in lost English below

CIC se termine sur un rythme connu, éprouvant mais toujours réjouissant, discussions et présentations intéressantes entre autres à propos de display, l’arrivée d’outils d’optimisation, l’extension des CMFs [mes quelques notes sur la conférence CIC 2016 ici] et bien sûr la fatigue due à la distortion journalière du temps grace aux inévitables et indispensables bières d'après les sessions officielles.

16 ans après avoir suivi les élections américaines de Montréal - donc presque localement - je renouvelle l'expérience, cette fois ci de la Californie et dans le dernier fuseau horaire des résultats où l'issue fait peu de doutes. Chaque soirée à partir du mardi soir était animée de manifs anti nouveau président dans centre ville  de San Diego - où se déroule ma conférence couleur cette année - avec l'impression depuis de se réveiller chaque matin dans la mauvaise time line de Retour vers le Futur.

and now in lost English

CIC is getting over soon under its usual rhythm,  demanding but always gratifying, discussions and presentations were interesting on various topics such as display, new optimisation tools, the extension of CMFs [you can read here my few notes about the conference CIC 2016] and of course your body getting more tired every day because of the time distortion... time distortion mainly due to the inevitable and vital evening beer sessions after the official sessions.

16 years after having followed the US election from Montréal - so almost as a local - I renew the experience this time from California. The local time zone makes the doubts about the final results. Each evening from Tuesday demonstrations were happening in downtown San Diego - where my conference is taking place - with the sensitive impression to wake up each following morning in the wrong time line of Back to the Future.

Berlin Amsterdam New-York San-Diego

maybe in English below

Voyage un peu long mais ca va, de la place pour mes jambes donc je ne peux pas trop me plaindre et de plus tous mes avions ont réussi leur atterrissage. J'ai gagné pas mal de points sur Foursquare avec tous ces aéroports ainsi que quelques taches de sauce tomate sur mon t-shirt blanc... A retenir pour les futurs voyages: mettre un t-shirt noir et éviter les plats en sauce.

L'arrivée à San Diego ressemble à celle que j'ai eu à San Antonio ou Albuquerque. L'aéroport est vide, il fait chaud, il fait nuit, je ne sais plus dans quel fuseau horaire j'évolue. Mais rapidement je trouve mon bus, 2$ et 12min de trajet et j'arrive à mon hotel. Là je fais la connaissance de mon voisin de chambre, Alexandre prof brésilien comme moi venu écouter et aider sur place la conférence CIC. Il temps de s'effondrer, la session "short course" dont je suis un peu responsable débute dans 7h.

in live translated English

Journey a bit long but still ok, I had room to squeeze my legs so I can't really complain, besides my numerous airplanes did succeed in each of their landing.  I won a few points in Foursquare with all these airports as well as a few tomato sauce stains on my white shirt... Notes for the future trips: wear a black shirt and avoid pasta with tomato sauce.

The arrival in San Diego reminds me those I had in San Antonio or Albuquerque. The airport is empty, it's dark, it's warm, I don't know in which time zone i'm evolving. But quickly I find my bus, 2$m and 12min later I'm in my hotel. I meet my room neighbor for the time of the conference, Alexander a Brazilian professor coming like me to help and listen this week at CIC. It's about time to collapse, the session "Short Course" I'm involved in is starting in 7h.

Algo in mobile phone camera

Smartphones are becoming our primary camera and our primary way to consume images. The technical differences between those devices are shrinking. How do you differentiate between them?

Considering all devices coming with approximatively the same hardware from sensors to optic, the differences should come from the software. if you take Android OS, despite its presence on most of the smartphone, the manufacturers using this OS can still pre-load their devices with advanced options.

My point here is not to list or say which one is better than the other, but to highlight all the options now available. What I think is interesting here is to see what is considered to be granted from the user point of view: auto-focus, white balancing, hdr, face detection, low light condition, panorama and to wonder what people really understood of these tools. The marketing department surely has something to say about the last point.

To know a bit about the production workflow around this problematic, I really see the the battle of new algorithms/automatic image as the way for company to market their image (see the last Iphone campaign). A bit like the battle between film manufacturers (Kodak, Agfa, Fuji...) back in the days, where for the same ISO sensitivities (for me my favorite was 400 color by Fujifilm) the same color signal will appear slightly different on paper. The camera allows you to record the scene and the film/digital camera chosen will print the look of the final image.

So what can you really do? A lot will answer the emergency color scientist. It is super important and very interesting to study how our visual system is functioning, how it adapts with the light condition (eg. how to improve an image when observe on your phone in low light condition). All sounds pretty cool to me, color scientists can be useful.

La suite dans le futur.

Personal insights on color science

Context
In a previous post I talked about the last event in my field I did attend. Now I want to talk about my perception of this domain which is called color science. I'm pretty sure it can be applied to other fields of research as well.

From the first time I joined this community, from article reader, article contributor to reviewer, committee member and session chair my understanding of what is color science has evolved. One important thing is to stay humble, especially with the new comers. I have been one them, it was impressive. Impressive because you meet the people, authors of research articles that are part of the foundation of you work. You can add a person, a voice to written words, it's actually pretty cool.

There aren't thousand concepts to understand/enter the world of color science. Like in every fields it's about observation and trying to explain what's happening. But here it's all about light - its spectral properties - how we perceive this signal - a single light source to an image in the visible spectrum - and how can we develop robust scientific/engineering "stuffs" around it. What I find interesting is to witness what is the new thing coming each year, how a technical improvement can open a door for further applications.

Color trends
Among the research sub-fields presented at CIC this year I want to come back on four of them.

There is the recurrent discussion about color metrics, from a purely mathematical/geometrical approach to a more perception-wise approach trying to add an average human appreciation of the difference between two signals. Having a good metric is always helpful to evaluate your algorithm/experiment. Over the years the metrics are evolving, context is important (from display calibration to color textile differences...).

There is the what I call "purely geometrical approach" discussion where having a signal as vector of n values - for n wavelength -  a group of sensors - basic configuration made of three basis like RGB basis - you want to know the value of this signal once projected on the known basis/sensors. From that you can jump into optimization, addressing various problems such as finding the scene illuminant/white point, study metamerism. It seems obvious but it's not.

There is printing and 3D printing - there I meant color 3D printing. Just think of how to design a color test-chart for such printing system. HDR display is also coming stronger than ever. What is interesting with these two examples is that they both require to know your workflow, they are the "end" of a process chain: you need to understand the acquisition process to do a good reproduction. Understanding the use of the technology is obviously required.

On the last paragraph one can add the understanding of gamut mapping and how you "move" into your color space as something very important. For printers you have multi-inks system changing the shape of the color space available. For high resolution TV and HDR screen the color gamut shape may not change a lot - almost - but the variability of screen size, intensity scale, technology available make it difficult - to be understood as something cool and challenging for me - to offer a comfortable experience to the user among the different platforms.

Now that I'm a bit more in control with the tools/concepts in my field and sub-fields I have the tendency to prefer the projects combining several concepts - like high quality printing and movie post-production - and I always appreciate to hear how the authors are presenting their projects, which story they are telling us.

CIC visiting Darmstadt

What is CIC you may ask yourself? It's stand for Color Imaging Conference, a conference about color and imaging. This year it took place in Darmstadt DE. The last 22 editions always took place in the US, last year it was in Boston MA, two years ago in Albuquerque NM, three years ago in Los Angeles CA, four years ago in San Antonio TX and that's it for my involvement. Next stop is San Diego CA in November 2016.

I'm a regular attendee, I joined this community already ten years ago alternating between CGIV, AIC, EI and CIC. Depending of the event you will meet a slightly different crowd or so to say different crowds will meet allowing to go deeper in the various fields represented. But for sure it's about imaging, color, perception, printing, archiving, image acquisition, color management, camera and display calibration, gamut mapping and more.

This year almost 200 persons were attending the event in Darmstadt. There is a kind of routine in such event and being part of the committee allows you to see the people interaction with a special look. It's very special to see the attendees - former colleagues, friends, known members of this community - arriving from everywhere almost - from North America, Europe, Asia, Australia... - and being all jet-lagged. Even if you are traveling in the same time zone you will end up jet-lagged. First of all the schedule is tide and you have to use the "free" time to talk with everybody. Sharing a meal or a beer is usually very appropriate. As a result you barely have time to rest, but the kind of adrenaline you get from meeting the crème de la crème of the color scientists keeps you awake.

Clash of the titans - optimization vs. machine learning

In the beginning 

What is important to know about machine/deep learning problems? First remark to myself is "what are we trying to solve in general?" and then "which method/technique do we choose?" or "which approach is most appropriate to answer a given problem?".

Optimization for the people

Optimization is widely used to solve complex problems that don't have an analytic expression. But this doesn't mean that problems that have an analytic expression couldn't be solved using optimization.

Optimization relies on a provided model that "model" with reasonable efficiency a phenomenon (e.g. find the colorant combination of cyan, magenta, yellow and more for a give red, green, blue pixel) or anything you want. You may hear about derivatives, gradient, local minimum, cost function, quadratic form, linearity, non-linearity, iteration and more when you start messing around with optimization.

And it's completely possible to use optimization techniques as applied mathematics tools without knowing exactly how they work (e.g. you provide your model and the tools will perform the derivatives for you). In an engineering world you are connecting boxes, each one trying to solve a simple task taking for starting point what the previous is having for output.

Deep learning for the people

Deep learning and neural networks let you do something clever with the way to solve your problem. First of all your problem has been defined and described, but optimization techniques did not provide expected results: it's not fast enough or it's simply not working. One possibility is that your model isn't good enough or way to complex.

The simple idea is to let a system to learn about an ecosystem. To do so we let the algorithms mimicking how our brain is working. The concept of learning is very important here because it is really what we want to achieve. We want that our algorithm learns in a first step by obtaining representative parameters/weights before giving us a result. Then once the learning is finished, for a given entree and with the help of the parameters the algorithm can give us answers. For example is this image an image of a car, an elephant and this with different degrees of confidence.

A big part of the learning is to prepare the training sample. You can't just give images to the algorithm. Applied to computer vision, deep learning methods try to extract features from images in a similar way of how we human recognize information in images. This step of features extraction goes by applying multiple filtering on the images and the resulting filtered images, using convolution and tile approaches. At the end you obtain classes of features and it's very similar to the filters used for face recognition. Only difference is the features that describe a human face are now almost standard and doesn't need to computed or extracted again.

There exist competitions where for a given large database full of images and  keywords, people can submit their algorithms. Pretty interesting results are obtained and as in sport faster solutions are appearing often coming with new tools to handle large databases.

Breaking the machine

Hopefully there is always something to improve. Because images can contain more than one object, you could have a bike and an elephant in the same picture. In that case what should reply our algorithm first? There is room for subjectivity here.

These algorithms have to deal with the constant stream of information we are processing, meaning that we are always learning - in theory of course because the world is full of lazy bastards which keeps the marketing and sales people happy making us predictable and therefore easy targets but I digress - and we have to find a way to give this ability to our algorithms or there is the risk for them to over-learn. I really like the metaphor of trying to make an algorithm able to forget part of what he is deep learning to be able to adjust its judgement.

The interesting problems are those that overcome the first limitations encountered. You could try to distinguish what are the elements in a picture (e.g. there is an elephant and a bike) or "simply" give to the image a score. If you take an artist, he will have the tendency not only to make the same picture but to add to its images something that defines the way he perceive the world around him, something pretty unique. A similarity factor or score can be very helpful when you are browsing a large image databases or "just" the internet.

Deep learning (ou deep learning in French)

What was your question already?
How to explain deep learning to your friends, family members, neighbors, random stranger, dog? A very good question indeed. Rather than going deeply into neural networks and other festivities let's start with describing the problem(s) we want to solve. Or least let's give an example of what we are trying to do here.

Over the years I had to come with strategies if I wanted to explain what I do for living. Giving keywords such "color science", "computer vision", "image processing", "digital photography" is usually not enough or saying "I do work with images" neither. I always found interesting to answer the question "why to you want to do that?" or "which problem do you want to solve?". So to explain what I can do I try to give an idea of the tasks I have to solve.

What is the problem you are trying to solve already?
In some way asking these questions is already machine learning/deep learning-ish approach of solving a problem. In theory if someone asks you to solve a problem he knows the kind of results he want to obtain for a given input or starting point. What he doesn't know is what is happening between these two stages. Applied mathematics and optimization are a reasonable standard solution: you develop of model that recreate more-less accurately what is happening between these two stages, then for a new entree point your model will predict what an output will be.

I'm sure "big data" is an expression you have heard in the past years or months. It has of course different meaning depending who to is giving a definition. But, coming back to images and the incredible amount of images we are producing daily there is a need to develop solutions, tools to be able to interact with these images. You have in your hand an extremely large image database and using keyword as a search query isn't enough anymore. So here is the problem: how to navigate, how to browse into large image database in a more natural way? There is a bit of database here but that is not the main point of my article, check my past post on graph and database if you are interested.

Face recognition to recognition of everything
Working with images is fascinating, you see one image and automatically you extract some of its  information. Of course there is a long learning curve, when you see a tree, a car, a known object in a picture you don't even realize it, you know, you have learned over the years you spent on earth to recognize, categorize, organize the continuous stream of visual information that come to your eyes and is later processed in your brain.

If you think of face recognition, the mathematical tools are now pretty standard. We can with high probability find out faces in images, classification comes after the recognition. And if you train your model you will be able to recognize semi automatically in a database faces of different persons as the tools/filters can be tuned for a given target. It can be scary of course if the threshold that decide for a true recognition/classification isn't verified by a real human and that action lead to a rocket launch. Actually any automatic action issued from an algorithm decision having impact on a human being is pretty bad (hello mass surveillance and hello Terminator). You want help from robots not to help robots or it's too late anyway.

An idea behind deep learning is to be able to learn what are into images - in a similar way as we human do - to extract features and to perform tasks on other images based on a trained neural network. I'm making shortcuts but that's the idea. To understand and to later mimic how information is circulating into the brain has been a dream of many researchers. Neural networks go into that direction. If a few years ago the algorithms were limited because of computer power the global picture is different now.

What is also interesting is that new strategies had to be developed to overcome the overload of data. In a way the system were "over learning" and people talked about over-fitting the data. And it makes sens. If I'm not too mistaken our brain is not indefinitely expandable, meaning we are sorting information continuously. One big part of these tools is to perform drop-out which can be explained as "now that our system can learn we have to teach him to forget part of what he knows in real time".

Cross disciplines 
A chance I see - for me - is the need in some industries for expert being not only expert in one field. Specially for this kind of large scale problems involving images, computer vision, real time and fancy applied research projects. To know only about machine learning or statistic is not enough, to know both about computer and machine learning tools is better.

[We talk later about existing and possible applications.]

Warsaw - troisième jour

A last full day full of resolution. I crossed the city by tramway, entered its many parks, visited the POLIN Museum of the History of Polish Jews and later in the afternoon met the Oslo team again.

4:00pm we are all waiting for our guide for a scheduled tour of the city. Then he arrives with its old cool mini yellow van. No need to worried about safety belts as there are not safety belts. We just need to relax and grab the red and black fur on the seats to not fall at every turns or bumps on the road.

The path followed by the mini van brought us to many places I have seen the last two days, but the tour guide was entertaining. To conclude our exploration we crossed the river and had a snack in what we can call a witness apartment of old Polish communist time. The vodka was good.

8:08pm despite the aperitif we start looking for a place to have diner and we end up in a kind of semi classic restaurant called Folk Gospoda. To give an image it's all in wood, a bit German style, waiters and waitresses are wearing "traditional" costumes. In comparison to the German version, the waitresses are wearing white socks, semi short skirts revealing her knees but without "décolotté pigeonnant" Bavarian style.

About the food is was way too big, good but way too big, but the beer drinkable. And sadly I could not see the couple making out in the other restaurant room in my back, but at some point two of my friends said "I think the woman on the back is giving an hand job to the guy next to him, noooo" we all said turning synchronously our heads. That was fun.

11.34pm the day isn't over and I left my Norwegian crew at its hotel and continue my city exploration. I have not idea where I'm, but I'm. I met again the Polish and French pair of yesterday with more Polish people. A very nice end of my short trip in Warsaw. I never thought this trip will have ended up with a long and passionate discussion about color imaging and ACES, but life is full a surprises.

I will have to go to bed at some point, my Polskybus is leaving the city in a few hours.

Cyborg time

About two years ago I started with a friend - who happen to be a color scientist as well - a series of research experiment having for goal to study projector displays and wondering if we could evaluate some of theirs properties - such as response curve and color gamut - without proper measuring device - meaning without expensive measuring devices.

Said like that I admit it doesn't sound very sexy. But that were the fun come in, it is not because it doesn't sound über cool that the approach can't be fun. Inspired by the work of other researchers where instead of using measuring devices experiments were conducted using a panel of human observers. A series of images were presented to the observers. Based on their decision, to say if pairs of color patches were identical in brightness/intensity - one made of a single gray continuous value and the second made using halftone technique - the famous response curve could be described. There is so much things you can do with the response curve but I will not pursue in that direction today.

Our idea is simple and we are not the first to have had this idea: to replace the observer by a camera, a webcam. Three simple reasons to do so: for having been several time one of the observer I know it is super annoying to be part of such experiment, mimicking an eye with a camera is a nice challenge and webcam are cheap. And not to mention that opencv is there to avoid you to loose to much time in accessing the video stream of your webcam and programming in Python is pleasant.

To be short, we are re-doing a pair single color patch comparison. One patch is a continous patch and the second halftoned.  The project is called devForWebCam, it's programmed in Python and located here. The code should help to setup an experiment in order to gather data.

Monday in NYC - Going to the Queens

I didn't bring my jogging equipment for nothing. After one week of non-activities I woke up early and went jogging in wild Upper West Side. I went up from the 97st and enter the Riverside Park along the river, ten turned right after reaching Columbia University, crossed the Morningside Park with a beautiful view on Harlem and Upper East Side in front of me, then joined Central Park by its upper west corner. At the level of my street I left the park and came back home.

Jogging in NYC and Central Park, checked.

The journey of the day continued by walking downtown where all street corners look like an album cover of the Beastie Boys Paul's Boutique. We went back where we went out last Saturday, thinking ahhhh it was there... I left Jean-Baptiste and Sarah in Washington park and went to the Queens. There I enter the Moma studio where its main imaging division is. For the connoisseurs I'm not taking about PS1 but another location into the Queens. I can't say it's a secret location as it's written in big Moma Studio in front of the light blue building. It is in this place that they will shoot pictures for their collection from paintings, sculptures, multimedia installations, cars, everything. To be short it was very cool. A tour made possible after meeting the right people in charge of this building some months ago during the Archiving conference 2014 who took place in Berlin.

Going the Queens, crossing a bridge above railways with the sun light going down through wire fence with the view on Manhattan in the background, checked. Never felt so hip hop.

I left the Queens, made a short detour by Manhattan and arrive in Williamsburg. I had a tacos from a tacos truck while waiting. JB and Sarah arrived, we had another delicious Asian fusion snack and went to a roof top bar. From there we enjoyed our beer with the NYC skyline by night.

First time for me in two areas of NYC, the Queens and Williamsburg. The last one reminded me a lot of SF, a bit of Montreal and Berlin. And now I know where the Vice headquarters are, there are facing the Brooklyn brewery.

Back in 96st after some food shopping I'm getting ready for cooking a risotto. It wasn't legendary but very eatable. Another great day.

CIC 22 in Boston

For its 22th edition the Color Imaging Conference or CIC for the connsoisseurs took place in Boston. Comparing to last year in Albuquerque (NM) the color LUT did change a bit: yellow and blue for the desert and the sky to red and about 50 shades of autumn for the buildings and trees.

If as the previous editions the crowd of attendees remains similar - la crème de la crème of the color scientists - you can detect a slight dominance according to the conference location or with whom is organized the event. This year the keynote sessions where for both color scientists (CIC) and medical imaging scientists (IADP), then sessions were ran in parallel.

The joint keynotes brought interesting discussions. They were highlighting another practices of imaging system - like microscope - in the medical world. That is not a surprise that the imaging system described can provide very different visual results, from one microscope to the other the data do not appear identical to human observers but the extraction of information should. In one word there is no reference images or colors when the practicians is looking at samples, but they know that each patient is different and can base their diagnostic on their experience. In comparison to the CARS event, the IADP was much less about computer assisted radiography and more about image analysis (at least for what I have seen).

The day before the real opening of the conference is the short course day. Professionals, experts in academia or industry are giving lectures on color science topics. After chairing this session last year in Albuquerque NM I could follow two courses in Boston: one on color display given by Gabriel Marcu from Apple and a second on Color Rendering Index and Lighting given by Wendy Davis from University of Sydney. It is always refreshing to take part into these courses, to re-learn about display technology, display calibration, lighting technology and equally important how our comprehension of color science has evolves over the years and how new technologies force to change our approaches.

One thing I remind in particular is the new range of LEDs available - for some years already - and the changes that go with them. It is possible to design the properties of your lights - its spectral power distribution SPD - at very narrower wavelength bands. What does it mean? It means that you can really shape the gamut of your display or light installation, this in term of triangle shape in the color space of your choice. But being able to define a larger, wider color gamut on a sketch is not or should not be the only goal. We also have to think of the internal color space distribution, after all the light source we are most of the time in contact - the sun - has a very different SPD. The good side of it is the door open for us color scientists to continue exploring these new spaces.

It was good to see how my field of studies during my PhD - multi-spectral color reproduction, multi-colorant printing - has evolved. How multi-ink printing systems that were semi-experimental some years ago are now the basis of further experimentation. Further more it always a pleasure to hear someone telling you "I have read your thesis" when they see your name and your face for the first time.

Boston 13 years later

Second time in Boston, about 13 years after my first and only time in 2001 when I was an exchange student in Montréal. For the first trip a car full of Frenchies and Belgies was going South, leaving Québec behind us for the a long weekend. As far as I remember it was a really good weekend, not legendary but still very ok.

But back to our actual trip. I'm arriving from Berlin via Amsterdam. Luckily the Delta Airlines plane is pretty empty - I have two seats for me alone - and equipped with fairly new personal screen, I can actually see something - ce qui n'est pas toujours du luxe. I can say I was efficient in term of watching videos.

Landing was smooth, so was the custom and retrieving my luggage. In the plane I met a fellow going to attend the same conference as me. The reason I'm travelling to Boston is to attend the Color Imaging Conference (CIC22 to be precised). For once we left the South states of New Mexico, Texas or California, no excursion to the desert this time.

But back to Boston as I'm diverging again. The city isn't too big and the public transport is actually existing. I jumped in the subway and reached shortly after landing my AirBnB apartment in South End, somehow the South. Neat. Especially neat if you have seen the average price for an hotel room this week.

Thanks to the conference organisation I'm able to attend the event another time. In a smaller city all attendees are usually staying in the same hotel. This time not. Meaning I will have to wait tomorrow to meet the color scientist fellows, except one coming from France with whom I'm sharing the flat. According to plan and his plane he should arrive soon.

Color science for beginners

I’m a color and imaging scientist who does photography, but sometimes I’m a photographer who does color and imaging research. Depending of the moment and the project one or the other will be predominant.

The practise of photography is always interesting to remind how the light is captured, how images are made. Know the acquisition process in details - light condition, lens used, subject or content - will always help when it is time to work on the images - be it for displaying the resulting images or for extracting automatically information from them.

These few lines are only a glimpse of what color science is. In my many attempts to explain what a color scientist does I came up with this little definition: a color scientist deals with light, its acquisition, preservation and reproduction, of course he also works with images. The term “color” refer to the visible spectrum - a color scientist is a multispectral imaging scientist with limited spectral boundaries - and by adding the word “visible” - visible to the human eye of course - we just extended the range of possible activities such as studying how a human eye does function, how do we perceive light signals, read images. From physics we come to philosophy.

Engineering projects which involve to work with images are often straightforward: you have an image, you need to detect some information, you use a define metric and it is done - almost of course. A color imaging project which involves art, artists and their work is different. Artists and scientists do not speak the same language, they may use the same tools but with different guidelines for sure... But that’s where the fun comes in.

Archiving conference 2014 in Berlin

This week I had the pleasure to attend the Archiving conference (AC) 2014 in Berlin. The event was hosted at the kino Arsenal in Potsdamer Platz in one of the cinema room with extremely comfortable red seats. This conference belongs to a group of conferences run by the Society for Imaging Science and Technology. I usually attend the Color Imaging Conference (CIC) or the Electronic Imaging conference (EIC), both in the US, so when I saw that the AC was taking place this year in Berlin I jumped at the chance and joined the event.

Comparing to CIC the topics of AC are much more applied and not only dedicated to imaging problems. The variety of represented fields makes this event very interesting. Starting from archiving itself, we can understand it by all problems related to collect documents, from papers, books, audio files, video files, electronic documents for e-government, art collections, website, internet. Once the databases are built you have to think of how to access the documents knowing that the amount of information grows without stopping, having documents digitized do not mean they don't need to be processed anymore (e.g. how to save scanned pages of an old book with its hand-written annotations). In relation to the scanning task, the one day industrial exhibition gives us the possibility to see different book scanners with special features to be able to manipulate fragile documents.

A never ending challenge in this domain is the continuous evolution of technology. If the tasks remain identical - to archive documents whatever they are - the people in charge of these tasks - working in library, museum, archive department - have to save the documents and keep alive the technology to actually store these documents. Archives being public they have to be accessible to the public. This also asks the question of the founding, the responsible people have to continuously argue with politicians to keep their founding at a reasonable level.

To be give a very simple image, in the past all documents were saved on solid media (ie stone, paper, film...), something physical with a long life duration. Then the time is speeding up, we accumulate more and more information, we need more space to archive it and we need to reduce the physical size of the archiving media. Data centers full of servers are available but the life duration they proposed is not as long as the physical media: you enter the configuration where these centers are continuously copying and migrating information from server to server to be faster than the archiving media life, therefore our global archives are always moving: a global power outage could mean the end of our archives.

Archiving film documents
An important parameter is the constant evolution of technology. This is something the movie industry is facing also continuously. In a way the challenges of restoring and archiving very old film material are very similar to those faced by during a movie production when different sources (ie contents from different cameras, different softwares) have to be combined: there are different workflow that need to be merged in order to produce the final document/film. There are discussions that propose to use the DCP film distribution format to store new films and old films. One advantage it is understood by many people but it brings the matter of DRM in the document. One problem is the projection system and the encryption are linked (if I'm correct), what happend when the projectin technology changes? All these aspects were tackled in the opening keynote given by a fellow from Fraunhofer IIS.

Crowd sourcing and database browsing
Crowd sourcing was to me a very interesting topic. It did appear in oral presentations and in posters. I joined together the database browsing because I think these two points are related. The amount of available information - as I mentioned above - is increasing and if you remember big data we have both feet inside. To process the documents the archiving departments have (eg in the conference program from the BNF in Paris, in the Netherland or in Germany) started campaigns where they ask the public to achieve tasks to validate digitization (I'm simplifying the problem here) where the task can't be perform fully automatically without errors. It follows the web 2.0 model where the user is also the one creating the content, except here he is helping/working on archive media which concerns everybody, it's our memory after all.

Graph database was mentioned and I could like to hear more about it. Especially how database query languages are developed to ease the access to the information in those databases. But this also implies to think how to build these databases. Pretty interesting in any cases.

First impressions
To be short there were mostly good. Different people, different backgrounds but similar challenges as archiving, digital preservation, curation are. Multidisciplinary as I like it.

Evening talk in ARTsLab UNM

First day of the scientific program of the 21th Color Imaging Conference and this year it's taking place in Albuquerque, NM. A very funny name to pronouce in french. But enought with the jokes and back to the serious business. Being involved in color science and immersive display I try to follow what people are doing in both fields all over the world. And going to conferences is always a good excuse to knock on the door of the local lab or research department. So this year I contacted some weeks in advance Jane Crayton which I did meet at the Imersa event in Denver three years ago.

After some e-mails exchanged it was planned for me to give a talk entilted "Color Science for Immersive Displays" at the ARTsLab. The ARTsLab (Arts, Research, Technology and Science Laborator and check the link below) is a nice research lab where people from various fields can meet and colaborate, and if you have read the lines above it may involved immersive display... One of their permanent installation is a digital dome of 6 to 7m (?) and a studio which can be used for various tasks.

I wish more people could have attended the talk, but quality is better than quantity. And I was served. The pre-meeting before the talk at a local restaurant was delicious and I could see that some people take their coffee seriously. Hoping to see again the already known contact and newly made here or there in the future.

Some links:
- ARTsLab
- Color Imaging Conference
- Imersa

Color Imaging Conference, first day and Short Course day

It is the third time that I'm able to join the Color Imaging Conference (CIC) after Los Angeles, CA last year and San Antonio, TX in 2010. This time I attend it without article, the one we submitted wasn't exactly finished and did not deserve to be accepted. It is always good to have your paper rejected from a good conference, it only pushes you to improve your work and you end up publishing later a better article. And enough of stating the obvious.

So this time I joined the conference as co-chair on the short course program, or short course session, I never know... Anyway this conference day offers - not for free - a panel of courses given by experts from academia or industry. As co-chair we had, based on last year program, to propose new courses and select submitted courses, they should reflect the latest development in a field and/or give an overview of a given topic (such as multi-spectral imaging or gamut mapping). Interesting and important to me, the courses are also showing connections between research and industry, basic and applied research.

I had the chance to attend three courses: "Normal and Defective Color Vision: from Theory to Simulation" given by Caterina Ripamonti from UCL Institute of Ophtalmology, "Color Grading and Color Management for Motion Pictures" given by Stefan Luka from Walt Disney Animation Studios, and Peter Postma from FilmLight and "What Color is 3D?" given by Matt Cowan from RealD.

The first course was very interesting, because before going to the point of simulating how people with defective color vision see color, state of the art was presented. But more important it shows how color and vision are treated depending of your field of study or depending of your application and which assumptions you are stating to build up your theories and explanations (e.g. which functions are used to approximate how a signal is captured by the eye). We are the border of physiology and computer color science.

The two other courses were related. And the instructors from the color management for motion picture joined the color and 3D lecture after giving their lecture, which ends up with an interesting discussion. A nice aspect of color science in the field of motion picture is the very applied nature of it: how do you capture light, how to process it, how to you compose, create with the various available tools and how do you reproduce and display your work.

And I end up having diner with "la crème de la crème" of color scientists based in UK, Australia, Spain and Germany. Fun fact, none of us four are working in our respective country (whatever it means). As usual, the minimum number of language talked per person was 2.4 . Take that narrow minded and obsessed with boundaries people.

Perdu dans le désert

Très belle lumière au réveil, tôt le réveil, décalage horaire tout ça. Peu de choses
se passent aujourd'hui au niveau de la conférence (Color Imaging Conference 21th pour les curieux) et j'en profite pour me joindre à la paire franco-allemande Steven-Christophe en provenance de Darmstadt qui est prête à aller faire un petit tour dans le désert. Breaking Bad quand tu nous tiens.

Aux USA les routes sont droites, ce n'est pas un mythe. Nous sortons de la ville assez rapidement, deux virages pour rentrer sur l'autoroute puis tout droit en direction du sud pendant un paquet de miles. Première étape une réserve d'oiseaux en provenance de l'Alaska pour certains. Il fait chaud, le ciel est bleu, les arbres sont jaunes et c'est le seul signe que la saison est bien l'automne.

Nous revenons vers Socorro pour déjeuner et c'est l'instant cuisine mexicaine. On réalise, enfin mes partenaires de voiture réalisent que nous ne sommes pas loin du very large array (ou VLA) radiotélescope. Le repas fini nous prenons donc la direction plein ouest pour une cinquantaine de miles, là encore tout droit.

On monte sur un plateau et on doit pas être loin de 2000m d'altitude. Et à part le village Magdalena, sa station service et le panneau indiquant réserve indienne à droite, il n'y a rien. Il y a bien quelques vaches mais c'est tout.

Puis des points blancs apparaissent, même forme que les antennes paraboliques sur certains balcons, sauf que ceux-la font au bas mot 20m de diamètre et qu'elles sont espacées d'au moins 500m et ce sur plusieurs km, le tout en forme d'étoile à trois branche. On gare la voiture au "visitor center". Le temps d'acheter un sticker ou deux, lire les quelques posters expliquant le bazar, traitement du signal, transformée de Fourier tout ca ou comment observer les étoiles en les écoutant.

Retour par le même chemin. Tout droit plein Est pour virage à 90 degrés plein Nord. Une petite centaine de miles et on est dans les embouteillages à Albuquerque. La prochaine fois on ira plein Ouest, dans l'autre direction se trouve Roswell pour les connoisseurs.

Back to IBC 2013 - Movie Making Colour Management, Best Practice and more

The last two days of the IBC 2013 conference have seen two interesting sessions on movie post-production, pipeline, standards and more important than everything else: color (or colour (ou couleur (oder Farbe))).

The session "Movie Making Colour Management, best Practice" was on many aspects my favorite. The panel discussion was covering all the movie pipeline steps: from real acquisition people (camera system with ARRI), to display people (Dolby), to big studio (Walt Disney) and post-production (Fluent Image Ltd). Comparing to a few years ago - when the full digital wave was hitting the cinema world, first scanning then editing, filming and now displaying (?) - it seems that people are getting familiar with the new equipment, digital cameras do not appear as magic devices being able of technological feats only... but cameras with their given properties (how they capture light and color). On that aspect, the ARRI person as well as the Fluent Image person were saying the same (it was two months ago so I may not be completely accurate): try the camera before starting shooting your movie, once on schedule for shooting you do not have the time for experimentation (this is what reasonable post-production companies do, I have seen that in Berlin by ARRI post-production people). In the past you were choosing this or this film-roll (i.e. fuji or something else) and your lab was then processing the film, now you choose a camera and its captor/sensor properties.

What is interesting when a new technology is introduced in an established pipeline is how the cards are redistributed (i.e. power, influence). Before the director of photography (DP and not TP which is a French basketball player) was deciding the light, the film-roll for the next production and in a way fixing the color "limit" of the production and kind of imposing its choice, vision. Now it is a bit different: the color can be modified anywhere along the movie production pipeline (which doesn't mean it should be done that way!). And this give a nice transition regarding the input to the discussion from the Big Studio person who was talking about standard and ACES and simplifying the workflow. Starting from color consideration (extremely important considerations say the color scientist) with ACES, it is more guidelines than absolute recipe that are introduced to the pipeline.

Of course the importance of display calibration was mentioning. All and all pretty standard comments but reasonable comments on what people should do or not do: be prepared in advance, know your tools. Danger of processing the dailies too much for the important people, it means you need a grading team on set and you take the risk to fix the color "mood" of the film to early.

I also do remember, or I think I remember that this panel of professional said they need color scientists, good for me, or I dreamed it? In any cases they need color scientists.

During the last day and maybe the last session it was about compression (for the Hobbit movie), especially jpg2000, and color artifacts that can occur. The problem of archiving was tackled by people from the EYE center in Amsterdam: what do you do when you have to scan old film material with very different fps.


And more political, how the standards are pushed from each side of the Atlantic and which company, studio, government, open-source project and so are the main players in these battle/discussion.
 

Back in IBC - Cutting Edge Session 1&2

It was almost on all lips during IBC this year, you have to improve the immersive experience for the viewers. Not only the story or TV program have to be original and addictive and if possible an endless source of money generator, no, they also have to be more immersive.

But what is an immersive experience? Different people, different interpretations. The idea is to offer to a spectator - while he is at the cinema or in front of its tv - an experience the more realistic it can be, the viewer should forget he is watching a display (e.g. watching a football game on your sofa and feeling you are in the stadium). The question becomes how to improve the display to make it more immersive? That's what the conference speakers (#ibcconf hosted during the show #ibcshow early September in Amsterdam) were trying to answer in their presentation. And especially in the sessions "Cutting Edge 1 & 2" new technologies were discussed. I took part in session 2 and we are all presenting what are the things comings, which technology will be predominant in a near future and hopefully the one we propose will be.

Starting from a rectangle display with a fixed resolution (e.g. HD), a fixed frame per second rate (e.g. 25fps), a fixed color dynamic (e.g. each color channel pixel coded on 8bits) what do we do? How do we introduce new "things" without destroying completely the existing workflow (from image acquisition, encoding and compression, diffusion and display)?

Increasing the resolution to 4K was written everywhere this year, increasing the resolution almost 4 times the one corresponding to HD (which is the one for Blue-Ray, DVD is way smaller), but so far there is not real content or affordable customer product for home. Professional movie cameras (or the latest GoPro) can offer this resolution, but only movie theater could follow on that (also I don't remember what is the offcial/recommended resolution for film distribution). Comparing to some years ago, the switch to full digital is a reality in many movie theaters and even so the quality in full digital is said to be lower than with film, two things are interesting: there is limit for an average viewer to see/perceive a difference in quality above 2K (so do we need to go so high?) and depending where you sit in a theater you will perceive different resolutions. In case you don't know this already, the human eye is performing filtering on the signal he receives, details will be perceived in an image depending of the viewing distance (this is use in many image compression algorithm where you remove what is not seen anyway).

Researchers at BBC where showing their latest work on fps, which fps is achievable in order to increase the viewing comfort. They said that above 100fps it's better, for you to appreciate the difference they were showing the same sequence (athletes doing high jump) with "normal" fps and higher fps side by side.

About hdr, the display market is almost reduces to one company (Dolby). The technique for taking hdr images (high dynamic range images) is known, basically you have to combine several pictures taken at different exposures. One can see the issues when you want to record movie file and not a single image. Researchers at Fraunhofer ISS proposed a very neat solution where sensors with different sensitivities pseudo-randomly distributed where used. And taking into account sparsity they could in one single shot obtain an hdr image. A brilliant friend did suggest me the following "but why don't they read continuously the captors to get the values at different exposures and create the hdr data on the fly", hmm why? And hdr must not be misunderstood with RAW file format despite data in that format possibly available in higher that 8bits per channel.

And there is our approach which consists in surrounding physically the viewers: the screen is curved (e.g. cylinder or digital dome to spherical display). To do so we combine several projectors to project images on the whole curved surface. This year we proposed a study case where 3D stereoscopic content was streamed to a digital dome. The installation we described use two fish-eye cameras and a 2 diameters dome tilted at 90degres such that while standing you are looking straight in the middle of the curved screen (the field of view proposed is bigger than your natural field of view, you should feel immersed). 3D and immersive experience go together and research is actively done to make the 3D experience more seamless to the user, meaning no glasses to wear, auto-stereoscopy and multi-views. Here as well the colleagues at Fraunhofer HHI and ISS have a strong presence. The matter of 3D and immersive display (or surround cinema) is tricky, the whole 3D thing is tricky (which makes it very interesting), because looking with one eye in real life doesn't make you see a flat image, you still perceive depth. And coming back to immersive display/surround cinema, the way you generate 3D images has to take into account where the viewer may look at, the freedom you are offering (possibility to look in every direction) is a challenge for the generation of proper stereoscopic effect. And I probably lost 98% of my friends after these lines.

All these large, curved, multi-blended displays need to be accurately color calibrated on the top of their geometrical calibration (we also do that at Fraunhofer FOKUS in our department VISCOM). The viewers need to have the feeling to look at one single light source only. The more you forget about the technique during the show you are watching/attending, the more immersive is the experience.

Last but not least, sound is of course in the game. You can't talk about surround cinema without surround sound. That wasn't too much mentioned in the sessions, but both image and sound are working together, beautiful isn't it... That is not my specialty, mine is color and image processing and more, I let the sound to my colleague specialists.