I can do Python, I can do data analysis (stats, modeling, machine learning, analytic and so on), I can do data visualization, I can learn fast, I can create, I can communicate, I can translate (or report information to different departments), I can write, I can connect people, I can invest myself a lot, I can team play. Basically I can do a lot and I want to do a lot. I also run pretty fast.
What I can't do is to come with miracle solutions to every problems before knowing the problems... Or it's pure luck and I should really start playing lottery then.
And I'm looking for a job.
Affichage des articles dont le libellé est data science. Afficher tous les articles
Affichage des articles dont le libellé est data science. Afficher tous les articles
24/11/2015
Some data about a data conference
I had the chance to attend the Data Natives 2015 event last week in Berlin. A first time event having for topics FinTech, IoT and of course Big Data. I heard some interesting talks but also less interesting ones. You can still hear people having the dream of forecasting anything with the help of more data, but with often the feeling it's only in order to sale more stuffs. I haven't got the life improvement that suppose to go with Big Data (e.g. mass surveillance doesn't work obviously).
But here and there you can sometimes hear someone talking about a project that hold your attention. For me the most interesting aspect is the inter-disciplinary or multi-disciplinary aspect of the data. To achieve something relevant or meaningful with all the available information, you need to be able to define first what is the problem you are trying to solve (and yes I have a degree in opening open door).
Four presentations are still in my head, one using NLP to pre-sort a lot of CV (from HitFox during the first day of the program) and a second using computer vision to automatically give feedback on webpage design (from EyeQuant last talk of the second day). Actually for the last one their talk was much wider than this single problem.
About IoT and FinTech is wasn't really impressive. Actually the only striking aspect is that the same tools are used whatever is your field of work (like data science / analytic / finance): you accumulate data, you trying to find information and pattern into them, this in order to derive model to make prediction. And without surprises the most interesting talks about FinTech came from the people involved in the Bitcoin economy / technology (Blockchain and ascribe Gmbh). Maybe the banks have some cool stuffs to talk about, but they weren't really present.
But here and there you can sometimes hear someone talking about a project that hold your attention. For me the most interesting aspect is the inter-disciplinary or multi-disciplinary aspect of the data. To achieve something relevant or meaningful with all the available information, you need to be able to define first what is the problem you are trying to solve (and yes I have a degree in opening open door).
Four presentations are still in my head, one using NLP to pre-sort a lot of CV (from HitFox during the first day of the program) and a second using computer vision to automatically give feedback on webpage design (from EyeQuant last talk of the second day). Actually for the last one their talk was much wider than this single problem.
About IoT and FinTech is wasn't really impressive. Actually the only striking aspect is that the same tools are used whatever is your field of work (like data science / analytic / finance): you accumulate data, you trying to find information and pattern into them, this in order to derive model to make prediction. And without surprises the most interesting talks about FinTech came from the people involved in the Bitcoin economy / technology (Blockchain and ascribe Gmbh). Maybe the banks have some cool stuffs to talk about, but they weren't really present.
02/11/2015
Data conference coming!
I'm interested in knowing, observing how my field - applied research, technology, imaging, innovation... whatever you call it - is evolving. Working full time on one project is of course a good solution to see what's going on, but it's also taking the risk of being stock in daily routines. In that sens it's always wise to have a look of what your neighbors, competitors are doing, how they try to solve the same problem you are working on.
From my own experience I know that we - let's call us/me applied/data scientist - are very fast categorized in sub-fields, as experts and that it is sometimes difficult to extract yourself from the prism of how people are perceiving what you can do. Having said that, to be able to attend events, meet a new crowd, hopefully interesting people, exchange information, re-present yourself, feel how an industry is growing is something vital.
A few days ago I did spot an event Data Natives 2015 scheduled in Berlin the coming 19-20 of November. The keywords combination used to introduce the program is almost too perfect: IoT (internet of thing), FinTech (Financial Technology and not tech from Finland which sounds pretty cool too) and Big Data of course. Needless to say that I'm pretty excited to attend this conference!
From my own experience I know that we - let's call us/me applied/data scientist - are very fast categorized in sub-fields, as experts and that it is sometimes difficult to extract yourself from the prism of how people are perceiving what you can do. Having said that, to be able to attend events, meet a new crowd, hopefully interesting people, exchange information, re-present yourself, feel how an industry is growing is something vital.
A few days ago I did spot an event Data Natives 2015 scheduled in Berlin the coming 19-20 of November. The keywords combination used to introduce the program is almost too perfect: IoT (internet of thing), FinTech (Financial Technology and not tech from Finland which sounds pretty cool too) and Big Data of course. Needless to say that I'm pretty excited to attend this conference!
30/09/2015
About not being an expert as a data scientist and other tech stuffs
Last evening I did attend a joined Meetup from the Python User Berlin (PUB) and the Zalando Tech Event hosted by Zalando and offering talks on Natural Langage Processing (NLP). Both talks went well and gave two views on the topic: one on the state of the art of the tools for NLP using Python and a second more applied.
The discussions I add after while enjoying a club mate - la boisson des champions - were equally interesting. First of all I started discussing with a expert of NLP trying to explain why I joined this event and what was my link with NLP. In my very recent job experience at EyeEm I just touched the surface of NLP preparing data for Machine Learning (ML) using nltk together with WordNet, ImageNet. Actually I didn't do much of text analysis but batching word definition. In that experiment the text analysis will have come after this step and that's where semantic is jumping into the discussion. Because working with the word dictionary is one side of the problem: you have one word with its definition and often - at least with scientists or engineers - you are in the inverse configuration which is you having words when actually you want to extract a definition, an idea, an information... And I let you google automatic image tagging, deep learning.
After exchanging ideas and experiences about NLP I did continue seeping the offered mate with one Zalando employee. I was curious - as usual - to understand what it means to be a data scientist here. Because if the definition is very general - a data scientist works with data, we are not expert - it's interesting to see how many fields we - I'm one of those people - cover in our daily work. Using the same language - e.g. Python - we can go from signal processing, computer vision, image retrieval, NLP, how to deal with Databases - a year ago I wrote on the topic Databases and natural Langage graphs en stock - how to present your results to non expert by doing nice visualization and many more... So if we are not expert we need to be pretty fast I acquiring skills from various fields and/or use the appropriate tools.
08/09/2015
Meetup for the human machines
I finally managed to attend the Shadow ML - for Machine Learning - meetup in Berlin yesterday evening, hosted by Amazon in their Computer Vision division in Berlin. Two talks were scheduled, one with images and a second with words. I explain.
Before pizza time
Here we learn about soft shadow removing. I liked this talk because it combined computer vision (CV) and machine learning (ML) and it's a problem I'm aware of as I'm regularly facing it when I'm post-processing my spherical panorama pictures taken under the sun - you can see my shadow in the picture.
What I remember from the hard shadow problem description is that a big part of the solution is to be able to isolate in the picture the shadow areas. What is a shadow area you may wonder? It's a part - or parts - of an image where the brightness has been drastically reduced such that they appear almost grey but there is still some color information available. Saying that we almost solve our problem: we need to find the color information in the shadow area and adjust its brightness to match the non shadowed neighbor area. You may have to operate in a different color space than RGB to keep the chromatic information undamaged and to change only the pixel brightness/luminance. A good image segmentation is an inevitable step.
For hard shadow the segmentation is an "easy" task as the transition between shadow/not-shadow areas is fast/brutal, in another word not soft. The problem with the soft transition is that is required a lot of human inputs to mask the image - in the sens of creating a mask that isolate the shadow areas from the others - and we want to automate this task.
A solution proposed yesterday was to use machine learning in order to make your system learning about the difference image with and without shadow. The speaker talked about the problem of getting data - which is a recurrent part of machine learning problem modelisation and any other scientific problems - and how he did create his data-set: computer generated images with Maya where he could get two sets, one with shadow and another without for the same scene.
After that I got a bit lost of on what the author does when he found out where the shadow areas were. But assuming the areas have been well discriminated you still need to adjust the brightness level. From that two solutions at least: if the area is homogeneous then a simple scaling factor/function should do something, treating the background - or the area - as a texture can be helpful too especially if you plan is to use in-painting techniques. But the chosen solution is of course linked to what you want to do: preserving information in the image - then I will say no in-painting - or tricking the eye/human brain such that the image appears nice without shadows - then go for in-painting.
After pizza time
A complete different topic to follow but not less interesting. It was about text and word analysis. For an introduction you can check WordNet to have a glimpse of what that field is. But back to the second speaker, his problem was to see if we can predict an affiliation to a political party based on text analysis.
As the speaker did mention it this is/was a work in progress where the first task was to establish a usable data-set for building the classifier. The text of each party manifesto was employed for that purpose.
Once you have your classifier what you want is to evaluate it. All the interventions, talks given by the government members, parliament members are the perfect data sources to be used for that as well article from different newspapers could be feed to the system.
This work goes as well into the direction of sentiment analysis and a temporal parameter is something you want to have in such problem. Depending of who is running the country, who has the majority at the parliament the roles, the words play/use by the people representatives evolve. It might be obvious but this kind of tool can tell us how much we perceive the words, talks given by our politicians and how much they or we interpret/dream/hallucinate about different situations.
Building such system wasn't too complicated - if I got it right from the speaker(s) - and the main challenges were/are to get clean data. As for all machine learning you need clean data, in every basic or applied research actually.
Before pizza time
Here we learn about soft shadow removing. I liked this talk because it combined computer vision (CV) and machine learning (ML) and it's a problem I'm aware of as I'm regularly facing it when I'm post-processing my spherical panorama pictures taken under the sun - you can see my shadow in the picture.
What I remember from the hard shadow problem description is that a big part of the solution is to be able to isolate in the picture the shadow areas. What is a shadow area you may wonder? It's a part - or parts - of an image where the brightness has been drastically reduced such that they appear almost grey but there is still some color information available. Saying that we almost solve our problem: we need to find the color information in the shadow area and adjust its brightness to match the non shadowed neighbor area. You may have to operate in a different color space than RGB to keep the chromatic information undamaged and to change only the pixel brightness/luminance. A good image segmentation is an inevitable step.
For hard shadow the segmentation is an "easy" task as the transition between shadow/not-shadow areas is fast/brutal, in another word not soft. The problem with the soft transition is that is required a lot of human inputs to mask the image - in the sens of creating a mask that isolate the shadow areas from the others - and we want to automate this task.
A solution proposed yesterday was to use machine learning in order to make your system learning about the difference image with and without shadow. The speaker talked about the problem of getting data - which is a recurrent part of machine learning problem modelisation and any other scientific problems - and how he did create his data-set: computer generated images with Maya where he could get two sets, one with shadow and another without for the same scene.
After that I got a bit lost of on what the author does when he found out where the shadow areas were. But assuming the areas have been well discriminated you still need to adjust the brightness level. From that two solutions at least: if the area is homogeneous then a simple scaling factor/function should do something, treating the background - or the area - as a texture can be helpful too especially if you plan is to use in-painting techniques. But the chosen solution is of course linked to what you want to do: preserving information in the image - then I will say no in-painting - or tricking the eye/human brain such that the image appears nice without shadows - then go for in-painting.
After pizza time
A complete different topic to follow but not less interesting. It was about text and word analysis. For an introduction you can check WordNet to have a glimpse of what that field is. But back to the second speaker, his problem was to see if we can predict an affiliation to a political party based on text analysis.
As the speaker did mention it this is/was a work in progress where the first task was to establish a usable data-set for building the classifier. The text of each party manifesto was employed for that purpose.
Once you have your classifier what you want is to evaluate it. All the interventions, talks given by the government members, parliament members are the perfect data sources to be used for that as well article from different newspapers could be feed to the system.
This work goes as well into the direction of sentiment analysis and a temporal parameter is something you want to have in such problem. Depending of who is running the country, who has the majority at the parliament the roles, the words play/use by the people representatives evolve. It might be obvious but this kind of tool can tell us how much we perceive the words, talks given by our politicians and how much they or we interpret/dream/hallucinate about different situations.
Building such system wasn't too complicated - if I got it right from the speaker(s) - and the main challenges were/are to get clean data. As for all machine learning you need clean data, in every basic or applied research actually.
09/04/2015
Deep learning (ou deep learning in French)
What was your question already?
How to explain deep learning to your friends, family members, neighbors, random stranger, dog? A very good question indeed. Rather than going deeply into neural networks and other festivities let's start with describing the problem(s) we want to solve. Or least let's give an example of what we are trying to do here.
Over the years I had to come with strategies if I wanted to explain what I do for living. Giving keywords such "color science", "computer vision", "image processing", "digital photography" is usually not enough or saying "I do work with images" neither. I always found interesting to answer the question "why to you want to do that?" or "which problem do you want to solve?". So to explain what I can do I try to give an idea of the tasks I have to solve.
What is the problem you are trying to solve already?
In some way asking these questions is already machine learning/deep learning-ish approach of solving a problem. In theory if someone asks you to solve a problem he knows the kind of results he want to obtain for a given input or starting point. What he doesn't know is what is happening between these two stages. Applied mathematics and optimization are a reasonable standard solution: you develop of model that recreate more-less accurately what is happening between these two stages, then for a new entree point your model will predict what an output will be.
I'm sure "big data" is an expression you have heard in the past years or months. It has of course different meaning depending who to is giving a definition. But, coming back to images and the incredible amount of images we are producing daily there is a need to develop solutions, tools to be able to interact with these images. You have in your hand an extremely large image database and using keyword as a search query isn't enough anymore. So here is the problem: how to navigate, how to browse into large image database in a more natural way? There is a bit of database here but that is not the main point of my article, check my past post on graph and database if you are interested.
Face recognition to recognition of everything
Working with images is fascinating, you see one image and automatically you extract some of its information. Of course there is a long learning curve, when you see a tree, a car, a known object in a picture you don't even realize it, you know, you have learned over the years you spent on earth to recognize, categorize, organize the continuous stream of visual information that come to your eyes and is later processed in your brain.
If you think of face recognition, the mathematical tools are now pretty standard. We can with high probability find out faces in images, classification comes after the recognition. And if you train your model you will be able to recognize semi automatically in a database faces of different persons as the tools/filters can be tuned for a given target. It can be scary of course if the threshold that decide for a true recognition/classification isn't verified by a real human and that action lead to a rocket launch. Actually any automatic action issued from an algorithm decision having impact on a human being is pretty bad (hello mass surveillance and hello Terminator). You want help from robots not to help robots or it's too late anyway.
An idea behind deep learning is to be able to learn what are into images - in a similar way as we human do - to extract features and to perform tasks on other images based on a trained neural network. I'm making shortcuts but that's the idea. To understand and to later mimic how information is circulating into the brain has been a dream of many researchers. Neural networks go into that direction. If a few years ago the algorithms were limited because of computer power the global picture is different now.
What is also interesting is that new strategies had to be developed to overcome the overload of data. In a way the system were "over learning" and people talked about over-fitting the data. And it makes sens. If I'm not too mistaken our brain is not indefinitely expandable, meaning we are sorting information continuously. One big part of these tools is to perform drop-out which can be explained as "now that our system can learn we have to teach him to forget part of what he knows in real time".
Cross disciplines
A chance I see - for me - is the need in some industries for expert being not only expert in one field. Specially for this kind of large scale problems involving images, computer vision, real time and fancy applied research projects. To know only about machine learning or statistic is not enough, to know both about computer and machine learning tools is better.
[We talk later about existing and possible applications.]
How to explain deep learning to your friends, family members, neighbors, random stranger, dog? A very good question indeed. Rather than going deeply into neural networks and other festivities let's start with describing the problem(s) we want to solve. Or least let's give an example of what we are trying to do here.
Over the years I had to come with strategies if I wanted to explain what I do for living. Giving keywords such "color science", "computer vision", "image processing", "digital photography" is usually not enough or saying "I do work with images" neither. I always found interesting to answer the question "why to you want to do that?" or "which problem do you want to solve?". So to explain what I can do I try to give an idea of the tasks I have to solve.
What is the problem you are trying to solve already?
In some way asking these questions is already machine learning/deep learning-ish approach of solving a problem. In theory if someone asks you to solve a problem he knows the kind of results he want to obtain for a given input or starting point. What he doesn't know is what is happening between these two stages. Applied mathematics and optimization are a reasonable standard solution: you develop of model that recreate more-less accurately what is happening between these two stages, then for a new entree point your model will predict what an output will be.
I'm sure "big data" is an expression you have heard in the past years or months. It has of course different meaning depending who to is giving a definition. But, coming back to images and the incredible amount of images we are producing daily there is a need to develop solutions, tools to be able to interact with these images. You have in your hand an extremely large image database and using keyword as a search query isn't enough anymore. So here is the problem: how to navigate, how to browse into large image database in a more natural way? There is a bit of database here but that is not the main point of my article, check my past post on graph and database if you are interested.
Face recognition to recognition of everything
Working with images is fascinating, you see one image and automatically you extract some of its information. Of course there is a long learning curve, when you see a tree, a car, a known object in a picture you don't even realize it, you know, you have learned over the years you spent on earth to recognize, categorize, organize the continuous stream of visual information that come to your eyes and is later processed in your brain.
If you think of face recognition, the mathematical tools are now pretty standard. We can with high probability find out faces in images, classification comes after the recognition. And if you train your model you will be able to recognize semi automatically in a database faces of different persons as the tools/filters can be tuned for a given target. It can be scary of course if the threshold that decide for a true recognition/classification isn't verified by a real human and that action lead to a rocket launch. Actually any automatic action issued from an algorithm decision having impact on a human being is pretty bad (hello mass surveillance and hello Terminator). You want help from robots not to help robots or it's too late anyway.
An idea behind deep learning is to be able to learn what are into images - in a similar way as we human do - to extract features and to perform tasks on other images based on a trained neural network. I'm making shortcuts but that's the idea. To understand and to later mimic how information is circulating into the brain has been a dream of many researchers. Neural networks go into that direction. If a few years ago the algorithms were limited because of computer power the global picture is different now.
What is also interesting is that new strategies had to be developed to overcome the overload of data. In a way the system were "over learning" and people talked about over-fitting the data. And it makes sens. If I'm not too mistaken our brain is not indefinitely expandable, meaning we are sorting information continuously. One big part of these tools is to perform drop-out which can be explained as "now that our system can learn we have to teach him to forget part of what he knows in real time".
Cross disciplines
A chance I see - for me - is the need in some industries for expert being not only expert in one field. Specially for this kind of large scale problems involving images, computer vision, real time and fancy applied research projects. To know only about machine learning or statistic is not enough, to know both about computer and machine learning tools is better.
[We talk later about existing and possible applications.]
Inscription à :
Articles (Atom)