Vision de Jérémie - Tech Tourism for new rich people

Je retourne à Infinite Loop aujourd'hui pour y déjeuner. Toujours Apple mais un autre restaurant que lundi dernier. Attendant un peu Nicolas j'observe les gens visitant le magasin de la Pomme où l'on peut acheter des goodies uniquement vendus sur place.

12:12am
Des cars remplis de touristes chinois se succèdent dans un bel effet de chenilles processionnaire: sortir du bus, poser devant le sigle pommé, se prendre en selfie, rentrer dans le magasin, regarder, toucher et répéter how much? how much? devant des vendeurs au calme digne de moines tibétains en retraite méditative.

A photo posted by jeremie Gerhardt (@mrbonsoir) on Mar 16, 2016 at 1:52pm PDT

Venu de mon hotel en uber je rentre à pieds. Préférant être au calme dans ma chambre pour un entretien téléphonique dans le même fuseau horaire. D'Oculus, de vision, de perception, de computer graphic, de réalité virtuelle, de système immersif, d'interaction homme machine, de décomposition spatiale et fréquentielle nous parlerons avec mon interlocuteur.

10:01pm
dehors à l'entrée de mon hotel, assis sur un banc je regarde, les gens sortant pour la St-Patrick, les couleurs vertes sont bien présentes et les lumières de la ville nocturne aussi.

A photo posted by jeremie Gerhardt (@mrbonsoir) on Mar 15, 2016 at 9:17pm PDT

Meetup for the human machines

I finally managed to attend the Shadow ML - for Machine Learning - meetup in Berlin yesterday evening, hosted by Amazon in their Computer Vision division in Berlin. Two talks were scheduled, one with images and a second with words. I explain.

Before pizza time
Here we learn about soft shadow removing. I liked this talk because it combined computer vision (CV) and machine learning (ML) and it's a problem I'm aware of as I'm regularly facing it when I'm post-processing my spherical panorama pictures taken under the sun - you can see my shadow in the picture.

What I remember from the hard shadow problem description is that a big part of the solution is to be able to isolate in the picture the shadow areas. What is a shadow area you may wonder? It's a part - or parts - of an image where the brightness has been drastically reduced such that they appear almost grey but there is still some color information available. Saying that we almost solve our problem: we need to find the color information in the shadow area and adjust its brightness to match the non shadowed neighbor area. You may have to operate in a different color space than RGB to keep the chromatic information undamaged and to change only the pixel brightness/luminance. A good image segmentation is an inevitable step.

For hard shadow the segmentation is an "easy" task as the transition between shadow/not-shadow areas is fast/brutal, in another word not soft. The problem with the soft transition is that is required a lot of human inputs to mask the image - in the sens of creating a mask that isolate the shadow areas from the others -  and we want to automate this task.

A solution proposed yesterday was to use machine learning in order to make your system learning about the difference image with and without shadow. The speaker talked about the problem of getting data - which is a recurrent part of machine learning problem modelisation and any other scientific problems - and how he did create his data-set: computer generated images with Maya where he could get two sets, one with shadow and another without for the same scene.

After that I got a bit lost of on what the author does when he found out where the shadow areas were. But assuming the areas have been well discriminated you still need to adjust the brightness level. From that two solutions at least: if the area is homogeneous then a simple scaling factor/function should do something, treating the background - or the area - as a texture can be helpful too especially if you plan is to use in-painting techniques. But the chosen solution is of course linked to what you want to do: preserving information in the image - then I will say no in-painting - or tricking the eye/human brain such that the image appears nice without shadows - then go for in-painting.

After pizza time
A complete different topic to follow but not less interesting. It was about text and word analysis. For an introduction you can check WordNet to have a glimpse of what that field is. But back to the second speaker, his problem was to see if we can predict an affiliation to a political party based on text analysis.

As the speaker did mention it this is/was a work in progress where the first task was to establish a usable data-set for building the classifier. The text of each party manifesto was employed for that purpose.

Once you have your classifier what you want is to evaluate it. All the interventions, talks given by the government members, parliament members are the perfect data sources to be used for that as well article from different newspapers could be feed to the system.

This work goes as well into the direction of sentiment analysis and a temporal parameter is something you want to have in such problem. Depending of who is running the country, who has the majority at the parliament the roles, the words play/use by the people representatives evolve. It might be obvious but this kind of tool can tell us how much we perceive the words, talks given by our politicians and how much they or we interpret/dream/hallucinate about different situations.

Building such system wasn't too complicated - if I got it right from the speaker(s) - and the main challenges were/are to get clean data. As for all machine learning you need clean data, in every basic or applied research actually.

Deep learning talk @Zalendo Tech Event

First Zalendo Tech Event at their Tech HQ nearby Alexanderplatz yesterday evening. To open their series of Meetup event Zalendo invited Professor Sepp Hochreiter of Johannes Kepler University in Linz to talk about deep learning. 

attentive crowd

About the talk
The talk was good but not adapted to an academic audience. If you are familiar with the topic you probably wouldn't have learned something new. But the talk did lead to interesting - and often expected - questions around and about deep learning. Sadly - to me - it was more where does it work?, what are the best parameters? than how does it work actually? 

As the speaker did remind to us, neural networks (NNs) aren't new on the market. They were discoveries years ago, it was promising and then nothing, other techniques were used, leaving specialists in their niche. I do remember courses during my master in image processing about 15 years ago [in Pierre et Marie Curie Paris VI] where the person teaching and introducing KNN and NNs sounds both excited and disenchanted. This until computers got faster (thanks to cpu, gpu, many-core, cluster, graphic card programming "et j'en passe") and suddenly it was possible to use NNs, to get results, to reproduce them and to beat classification challenges by far comparing to the expert of the field.

For every new promising technique there is the temptation to use if for everything in a brute force manner. But it doesn't work all the time. One remark given by the speaker is these solutions work when you are overloaded with data, when you immersed into data. It's not a surprise that big players such as Google, Facebook, Amazon and more are heavy on growing their deep learning team.

About automation, AI and drugs
You hear and see more and more presentations about deep learning, artificial intelligence (AI) where people are dreaming of AI being able to put words on a given image in a similar way a human will do. It's kind of working but there is no magic. It made me remember about an experiment where the researchers claimed to be able to produce images/video corresponding to the images we see in our dreams. Often people fear - and they can - about computer taking control over us, making decisions for us until we start working for them.

It is interesting to understand why pharmacy companies - those making drugs - are so big into deep learning. Bio-Informatics offer the perfect environment for developing big data solution. Here I'm not talking about the phase where drug need to be tested and evaluated on human but what happen before. Biology and chemistry (or computer chemistry) can be simulated using pretty accurate models, meaning you don't need to run an actual biological or chemical experiment. You can simulate the experiment, generate a huge amount of data and let your algorithm do the analysis. And guess what, computer vision, machine learning, deep learning - not to mention optimization - are part of the solution. And the faster you get your results, the faster you have a new drug to potentially introduce on the market hopefully before your competitor. I'm not sure "normal" people got a glimpse on that side of research, in that field it's actually the biological/chemical experiment that will validate a virtual experiment (remember to watch Terminator 4 or 5 at leas the last on screen...).

About the big brain project and graphic cards and evolution
Research is cool. It's very interesting to see how connections/links between highly specialized fields are happening to build a new framework for research. The big brain project (not sure about the name but there is the US and the EU version) is the perfect example, different fields from neuroscientists to computer graphics and hardware manufacturers need to collaborate to build this virtual brain model.

One of the last comment from the speaker yesterday had a pertinent echo in my head. This comment illustrates perfectly how technology is evolving and frameworks are crossing their paths. He told us that graphic card manufacturer (such as nvidia to not name them) are now developing hardware dedicated to run deep learning process, once again the hardware architecture helping to fasten a programmed algorithm. But until when and is it a good approach? 

Years ago and not so long time ago when computers were already getting faster, people were designing hardware to run image processing/computer vision algorithms. This because the computers in their at-this-time state weren't fast enough. Like the brain was too small and needed to grow or modify its physical body to evolve. But then computer became faster and those special design weren't adaptable enough, too specialized. I feel that we are living a similar state with deep learning. The question will be is hyper-specialization of computer hardware the solution - momentarily for sure - for deep learning or not?

About the future
We are all doomed. Soon computer will be smart enough to redesign their body when they will reach their limits to overpass them. I haven't any spoiler about how and when, out Mayan friends had a big fail about it three years ago, we have to be patient.

  

Clash of the titans - optimization vs. machine learning

In the beginning 

What is important to know about machine/deep learning problems? First remark to myself is "what are we trying to solve in general?" and then "which method/technique do we choose?" or "which approach is most appropriate to answer a given problem?".

Optimization for the people

Optimization is widely used to solve complex problems that don't have an analytic expression. But this doesn't mean that problems that have an analytic expression couldn't be solved using optimization.

Optimization relies on a provided model that "model" with reasonable efficiency a phenomenon (e.g. find the colorant combination of cyan, magenta, yellow and more for a give red, green, blue pixel) or anything you want. You may hear about derivatives, gradient, local minimum, cost function, quadratic form, linearity, non-linearity, iteration and more when you start messing around with optimization.

And it's completely possible to use optimization techniques as applied mathematics tools without knowing exactly how they work (e.g. you provide your model and the tools will perform the derivatives for you). In an engineering world you are connecting boxes, each one trying to solve a simple task taking for starting point what the previous is having for output.

Deep learning for the people

Deep learning and neural networks let you do something clever with the way to solve your problem. First of all your problem has been defined and described, but optimization techniques did not provide expected results: it's not fast enough or it's simply not working. One possibility is that your model isn't good enough or way to complex.

The simple idea is to let a system to learn about an ecosystem. To do so we let the algorithms mimicking how our brain is working. The concept of learning is very important here because it is really what we want to achieve. We want that our algorithm learns in a first step by obtaining representative parameters/weights before giving us a result. Then once the learning is finished, for a given entree and with the help of the parameters the algorithm can give us answers. For example is this image an image of a car, an elephant and this with different degrees of confidence.

A big part of the learning is to prepare the training sample. You can't just give images to the algorithm. Applied to computer vision, deep learning methods try to extract features from images in a similar way of how we human recognize information in images. This step of features extraction goes by applying multiple filtering on the images and the resulting filtered images, using convolution and tile approaches. At the end you obtain classes of features and it's very similar to the filters used for face recognition. Only difference is the features that describe a human face are now almost standard and doesn't need to computed or extracted again.

There exist competitions where for a given large database full of images and  keywords, people can submit their algorithms. Pretty interesting results are obtained and as in sport faster solutions are appearing often coming with new tools to handle large databases.

Breaking the machine

Hopefully there is always something to improve. Because images can contain more than one object, you could have a bike and an elephant in the same picture. In that case what should reply our algorithm first? There is room for subjectivity here.

These algorithms have to deal with the constant stream of information we are processing, meaning that we are always learning - in theory of course because the world is full of lazy bastards which keeps the marketing and sales people happy making us predictable and therefore easy targets but I digress - and we have to find a way to give this ability to our algorithms or there is the risk for them to over-learn. I really like the metaphor of trying to make an algorithm able to forget part of what he is deep learning to be able to adjust its judgement.

The interesting problems are those that overcome the first limitations encountered. You could try to distinguish what are the elements in a picture (e.g. there is an elephant and a bike) or "simply" give to the image a score. If you take an artist, he will have the tendency not only to make the same picture but to add to its images something that defines the way he perceive the world around him, something pretty unique. A similarity factor or score can be very helpful when you are browsing a large image databases or "just" the internet.

Deep learning (ou deep learning in French)

What was your question already?
How to explain deep learning to your friends, family members, neighbors, random stranger, dog? A very good question indeed. Rather than going deeply into neural networks and other festivities let's start with describing the problem(s) we want to solve. Or least let's give an example of what we are trying to do here.

Over the years I had to come with strategies if I wanted to explain what I do for living. Giving keywords such "color science", "computer vision", "image processing", "digital photography" is usually not enough or saying "I do work with images" neither. I always found interesting to answer the question "why to you want to do that?" or "which problem do you want to solve?". So to explain what I can do I try to give an idea of the tasks I have to solve.

What is the problem you are trying to solve already?
In some way asking these questions is already machine learning/deep learning-ish approach of solving a problem. In theory if someone asks you to solve a problem he knows the kind of results he want to obtain for a given input or starting point. What he doesn't know is what is happening between these two stages. Applied mathematics and optimization are a reasonable standard solution: you develop of model that recreate more-less accurately what is happening between these two stages, then for a new entree point your model will predict what an output will be.

I'm sure "big data" is an expression you have heard in the past years or months. It has of course different meaning depending who to is giving a definition. But, coming back to images and the incredible amount of images we are producing daily there is a need to develop solutions, tools to be able to interact with these images. You have in your hand an extremely large image database and using keyword as a search query isn't enough anymore. So here is the problem: how to navigate, how to browse into large image database in a more natural way? There is a bit of database here but that is not the main point of my article, check my past post on graph and database if you are interested.

Face recognition to recognition of everything
Working with images is fascinating, you see one image and automatically you extract some of its  information. Of course there is a long learning curve, when you see a tree, a car, a known object in a picture you don't even realize it, you know, you have learned over the years you spent on earth to recognize, categorize, organize the continuous stream of visual information that come to your eyes and is later processed in your brain.

If you think of face recognition, the mathematical tools are now pretty standard. We can with high probability find out faces in images, classification comes after the recognition. And if you train your model you will be able to recognize semi automatically in a database faces of different persons as the tools/filters can be tuned for a given target. It can be scary of course if the threshold that decide for a true recognition/classification isn't verified by a real human and that action lead to a rocket launch. Actually any automatic action issued from an algorithm decision having impact on a human being is pretty bad (hello mass surveillance and hello Terminator). You want help from robots not to help robots or it's too late anyway.

An idea behind deep learning is to be able to learn what are into images - in a similar way as we human do - to extract features and to perform tasks on other images based on a trained neural network. I'm making shortcuts but that's the idea. To understand and to later mimic how information is circulating into the brain has been a dream of many researchers. Neural networks go into that direction. If a few years ago the algorithms were limited because of computer power the global picture is different now.

What is also interesting is that new strategies had to be developed to overcome the overload of data. In a way the system were "over learning" and people talked about over-fitting the data. And it makes sens. If I'm not too mistaken our brain is not indefinitely expandable, meaning we are sorting information continuously. One big part of these tools is to perform drop-out which can be explained as "now that our system can learn we have to teach him to forget part of what he knows in real time".

Cross disciplines 
A chance I see - for me - is the need in some industries for expert being not only expert in one field. Specially for this kind of large scale problems involving images, computer vision, real time and fancy applied research projects. To know only about machine learning or statistic is not enough, to know both about computer and machine learning tools is better.

[We talk later about existing and possible applications.]