27/05/2015

Struggle for social graph and datavizzz


The holly Grail of the day
Build an interactive data visualization of my own networks where I could jump from one network to the other and navigate in time.  On the paper it sounds easy: use your own network data (facebook (FB), linkedin (LI), twitter, instagram, EyeEm...) to exercise yourself on social graph. In other words use tools from your beloved statistic toolbox (Matlab, Python, R...).

The why
Why, why and why using your own data? First reason and obvious to me, you know the data - or at least part of it - and it should be bit easier to navigate through them. About the first why bother to do that? Once again it's simple and the answer is curiosity. The more people use a buzz word in all conversations the less they understand what it means and I don't like to not understand.

Social graphs are interesting because they illustrate part of our multiple identities - this of course if you decided to look at your own network instead of looking at the interaction between people forming a group which is also interesting (data journalism loves to dissect political social network to find out who are the leaders). We don't know the same people/don't play the same character depending of the network as they describe different interactions (e.g. FB vs LI).

The reverse engineer path
The path I did follow wasn't probably the most efficient but I'm getting better every day. Plotting a social graph isn't the most difficult task. Using gephi you can relatively fast generate beautiful graphs. In parallel I took in statistic and social network analysis to refresh parts of my brain on the topic.

The prototype
As inmaps isn't available any more I ended up on another automatic solution called socilab.con that requires you to log with your linkedin account. It's nicely made, you get a graph and several score values that describe your network and which role you play in it. Sadly it is limited to 500 contacts, so if your contact list is much bigger the analysis is incomplete. But this website allows you to download this version of your contact list. And actually what you are downloading is the formatted data from your LI account under the form of an adjacency matrix. I had to clean a bit the data using Python and Pandas which make any manipulation of csv file a real pleasure.

The adjacency matrix
This matrix - if I understood correctly - should be square where both columns and rows have the same names: your contact name list. Depending of the cell value 0 or 1 you know if your contact know each other or not, the matrix isn't symmetric. It's a particular case of data, because if you look at a FB group of people liking peanut butter toast for diner they may not know each other but they are all connected by their irrational attraction to fatty cream and low safe consideration.

Where the trouble starts
It starts right when you want to access your data... Building by hand this matrix is doable but is a really silly task. And both LI and FB do make the task easy neither. You will need to play with their API (I haven't checked for twitter, instagram and more yet) to access your account and download/build your matrix.



24/05/2015

Tour de Berlin 2015 - First stage Mauerpark to Kladow

Four people signed up, but one decided to sleep all day long and another did prefer to drink Caipirinha at the Karnaval der Kulturen happening the same weekend... Only two braves and their bike on the departure line!

1:33pm and Steffen is joining me. We have bikes, water, yoghurt cake, smart phone and offline map. Go go go.

A few km later we are in the pampa. The time to leave the city, to be still between East and West and to reach a border to the nature. We go over Tegel, Hohen Neundorf, Henningsdorf, aiming for Spandau but always staying away from it.

Relatively fast it's 50km on our counter. Down Spandau - I know the part because of previous summer expedition in this area looking for lakes to swim - the way is going in the forest again and is passing Sacrower See just after the Gross Glienicker See which happen to have been crossed by the wall delimitation.

From that point is becoming  a bit tricky. If you do want to follow the real Mauer path you have to take a boat to reach the Glienicker bridge... So instead the indicated way is to go to Kladow which is facing S-Wannsee. There is one of the public transport ferry lines. It's a big boat, there is place of at least 50bikes. But it's a nice day, no way we can get in without waiting hours.

6:03pm there is only one thing we can do while brainstorming how to go to back to Berlin: drinking beers and eating sausages in one of the beer gardens. Either we go to Potsdam and from there jump into the S-Bahn, but it means 20km more or we go back to Spandau by another way and following the water. We choose the second option which only 12 km to the daily ride.

Going back North we made another stop in Gatow where a fair is taking place. This time no sausages but delicious spätzle with cheese and beer of course. Our stomach feeling satisfied we continue and somehow ends up in the S-Bahn station Streesow.

9:33pm the sun is going down. A lot of km in our legs. It's nice to be transported by the outdoor S-Bahn that slowly brings us back to civilisation. I go down at S-Friedrich Strasse and cycle a few hundred meters to meet another friend for a last beer in this lovely summer evening.

Can't wait to complete this tour!

22/05/2015

Deep learning talk @Zalendo Tech Event

First Zalendo Tech Event at their Tech HQ nearby Alexanderplatz yesterday evening. To open their series of Meetup event Zalendo invited Professor Sepp Hochreiter of Johannes Kepler University in Linz to talk about deep learning. 

attentive crowd

About the talk
The talk was good but not adapted to an academic audience. If you are familiar with the topic you probably wouldn't have learned something new. But the talk did lead to interesting - and often expected - questions around and about deep learning. Sadly - to me - it was more where does it work?, what are the best parameters? than how does it work actually? 

As the speaker did remind to us, neural networks (NNs) aren't new on the market. They were discoveries years ago, it was promising and then nothing, other techniques were used, leaving specialists in their niche. I do remember courses during my master in image processing about 15 years ago [in Pierre et Marie Curie Paris VI] where the person teaching and introducing KNN and NNs sounds both excited and disenchanted. This until computers got faster (thanks to cpu, gpu, many-core, cluster, graphic card programming "et j'en passe") and suddenly it was possible to use NNs, to get results, to reproduce them and to beat classification challenges by far comparing to the expert of the field.

For every new promising technique there is the temptation to use if for everything in a brute force manner. But it doesn't work all the time. One remark given by the speaker is these solutions work when you are overloaded with data, when you immersed into data. It's not a surprise that big players such as Google, Facebook, Amazon and more are heavy on growing their deep learning team.

About automation, AI and drugs
You hear and see more and more presentations about deep learning, artificial intelligence (AI) where people are dreaming of AI being able to put words on a given image in a similar way a human will do. It's kind of working but there is no magic. It made me remember about an experiment where the researchers claimed to be able to produce images/video corresponding to the images we see in our dreams. Often people fear - and they can - about computer taking control over us, making decisions for us until we start working for them.

It is interesting to understand why pharmacy companies - those making drugs - are so big into deep learning. Bio-Informatics offer the perfect environment for developing big data solution. Here I'm not talking about the phase where drug need to be tested and evaluated on human but what happen before. Biology and chemistry (or computer chemistry) can be simulated using pretty accurate models, meaning you don't need to run an actual biological or chemical experiment. You can simulate the experiment, generate a huge amount of data and let your algorithm do the analysis. And guess what, computer vision, machine learning, deep learning - not to mention optimization - are part of the solution. And the faster you get your results, the faster you have a new drug to potentially introduce on the market hopefully before your competitor. I'm not sure "normal" people got a glimpse on that side of research, in that field it's actually the biological/chemical experiment that will validate a virtual experiment (remember to watch Terminator 4 or 5 at leas the last on screen...).

About the big brain project and graphic cards and evolution
Research is cool. It's very interesting to see how connections/links between highly specialized fields are happening to build a new framework for research. The big brain project (not sure about the name but there is the US and the EU version) is the perfect example, different fields from neuroscientists to computer graphics and hardware manufacturers need to collaborate to build this virtual brain model.

One of the last comment from the speaker yesterday had a pertinent echo in my head. This comment illustrates perfectly how technology is evolving and frameworks are crossing their paths. He told us that graphic card manufacturer (such as nvidia to not name them) are now developing hardware dedicated to run deep learning process, once again the hardware architecture helping to fasten a programmed algorithm. But until when and is it a good approach? 

Years ago and not so long time ago when computers were already getting faster, people were designing hardware to run image processing/computer vision algorithms. This because the computers in their at-this-time state weren't fast enough. Like the brain was too small and needed to grow or modify its physical body to evolve. But then computer became faster and those special design weren't adaptable enough, too specialized. I feel that we are living a similar state with deep learning. The question will be is hyper-specialization of computer hardware the solution - momentarily for sure - for deep learning or not?

About the future
We are all doomed. Soon computer will be smart enough to redesign their body when they will reach their limits to overpass them. I haven't any spoiler about how and when, out Mayan friends had a big fail about it three years ago, we have to be patient.

  

16/05/2015

Dans l'enfer du Märkisches Schweiz

We did it, after 12hours we did it! We went through all normal stages of an impossible bike mission. Small teaser we did follow a kind of a non logical order to do these stages, meaning we first took a lot of breaks and detours and then we started the tour. Also we weren't completely lucky. The tale step by step.

11:37 Lichtenberg train station
The five fantastics are almost all in time for the regional bahn. First detail thousand people with bikes were also in time and the train only have 5 bike places... It's a no go. We change platform and take the S-Bahn until the end to S-Strausberg and decide to do the planned bike tour in reverse mode. The goal once in Strausberg will be to cycle 35km to the Bahn station Obersdorf.

Coffee and bockwurst break in Strausberg
Already 5km (!!!) and first stop in the lovely old Strausberg village tiny burg. Train in Obersdorf is scheduled for 18:39, we have time. We completely deserved this coffee break at Milchbar.

After getting lost in the big city Strausberg while trying to start pursue our soon to be called odyssey we arrive at the lost Pyramide Grazau. Nice little village around and about 15km on the counter. It's also our first detour as leaving the pyramid part of the team decided - unintentional of course - to go toward Ekner which absolutely on the opposite direction of our planned trip.

15:03 picnic at the lac
Back on track we make our first real stop for the picnic. The public beach of Lange See is just for us. Incredible birds are flying around - mostly ducks - and it's a lovely Saturday with friends. Only thing is we started the trip 2hours ago and have done only 20km on the 35km, also some drops are coming.

In the heart of Märkisches Schweiz
The next 15km allows to experience the beautiful region despite the clouds. It's going up and down but completely doable even with shitty bikes. Thanks to our guide tour leader - we call it the PR girl or Anne who is screaming about the beauty of the region while cycling in the middle of the roads with the wind we barely understand what she says but she looks happy so we don't disturb her - we discover more lovely villages.

In Buckow - also called the pearl of the Märkishes Schweiz - the rain shower is a kind of warning: do we continue to Obersdorf under the rain or we go back now to where we start??? We decide to now listen to the sky messages and pursue again our trip.

18:15 Seeing Obersdorf and leaving
The landscape is still lovely and hilly.  Three hamlets further full of white and black sheeps we finally reach - in time we thought - our final destination and train station of Obersdorf.

The story is repeating itself, the train is full of bikes and we can't get in...

After strong negotiation between the five fantastics two main ideas arise on the top of the consensus being we go back to Strausberg where we know there will be S-Bahn every 10min, this instead of waiting one hour for the next train still with the chance to not have places, again.

The Obersdorf agreement
First idea is we follow the same way, we know it and it's all along a bike way which is good when we don't have lights on our bikes. Second idea is we take short-cuts by going on roads we don't know with cars on it and we don't have lights... First option is chosen, the reason win, Cartesian spirit for life, on the road again, only 35km remains.

21:12 Strausberg again
A last rush on the platform to change train with bags on our shoulder, open beers in hand and the door closing... But we made it and we are mainly dead.







01/05/2015

Voir des oeuvres d'art ou voir des gens regardant des oeuvres d'art?

La bonne question me direz vous. Ayant fini ma journée tôt - car elle avait elle même débuté très tôt - je décidais d'occuper mes trois heures de libre avant d'aller diner chez des amis en début de soirée par un mini opening-marathon-challenge. Challenge qui s'est aussi transformé en une course contre la montre et un jeu de piste avec l'orage qui tourbillonnait et se rapprochait de Berlin en fin d'après-midi.

18:40 direction la Loock Galerie sur la Postdamer Straße. A mi-chemin je fais une mini-pause Côtes du Rhône aux autres Galeries, par mini-pause j'entends "achat d'un bouteille et non dégustation". Je poursuis, traverse Potsdamer Platz, slalome entre les touristes, passe la Philharmonie et arrive dans le quartier des galeries. Trois voir quatre galeries sont ouvertes, du monde dedans et du monde dehors. J'aime bien cet endroit, les bâtiments sont propices à héberger des lieux d'exposition, suffisamment neutres et modulables et parfois des belles images.

Des belles images donc, en grand, en noir&blanc, de lieux et gens aux Etats-Unis. Rien de très nouveau mais je me répète, des belles images, bien composées, propres, pour des vues presque figées, on ne sent pas vraiment l'instantané, mais au moins ca me parle. En bonus la bière du vernissage est une Flensburger, un gage de gout de de qualité. La population est assez anglophone, américaine dirais-je et je glane quelques bouts de conversation où chacun y va de son anecdote géographique pour montrer qu'on voyage.

Environ douze minutes après avoir averti FourSquare de ma présence je file. J'ai encore au programme trois lieux et un diner dans moins de trois heures. Dehors le vent souffle et le gros nuage gris foncé tonne et des éclaires sont visible.

19.46 je pénètre dans la WeGalery. Le temps de souffler je crée le lieu sur FourSquare et commence à regarder autour de moi. Nous sommes dans le bas de Friedrichstrasse vers la Mehringplatz. La rue est un mélange de vieux bâtiment assez beaux et d'autre reconstruits assez laids. Il n'y a pas grand monde et le vernissage a débuté il y a déjà une heure. Dehors c'est l'apocalypse, les gens arriveront plus tard ou mouillé ou les deux.

Une exposition groupée, des artistes italiens, d'ailleurs la population est majoritairement italienne, habillée classe sobre avec gout à l'italienne. Le stagiaire ou l'assistant de la galerie est en panique et commence à courir partout et en prenant des photos avec son téléphone pommé. On doit être une vingtaine, probablement le record du jour et il veut en garder des preuves! Dans son rush il shoote du pied un bout d'une des oeuvres d'art exposée, au moins ils se passent des choses. Les oeuvres sont à l'image des visiteurs, propres, ca brille d'argent mais pas d'or, les formes sont plaisantes à l'oeil, déjà vu mais harmonieux, penchant parfois vers le design. Une connaissance arrive mais je dois filer, de plus la tempête fait une pause et je me dois d'en profiter! De plus mauvais choix de bière.

20:14 changement de lieu et de décor pour le Kunstquartier Bethanien et une autre exposition vernissage appelée hate10 - Kongress der Moglichkeiten ou le congrès des possibles? Ca se passe dans une partie uniquement du bâtiment où ils  passent parfois des choses intéressantes en particulier dans les ateliers et autres workshops. Mais pas cette fois. C'est hipster à souhait, tout parait bricolé, pas fini, bancal, cheap. Ca contraste pas mal avec ce que je viens de quitter. Population assez jeune avec au milieu un ou deux vieux artistes habillés jeunes essayent de communiquer. Tout ca pour dire que je ne reste pas longtemps. Un gros cliché que cet évènement.

Les jours de vernissages sont comme du pain béni pour nos amis outre-quiévrains sociologues ou anthropologistes. Le moment parfait pour observer des tribus, pas forcément pour voir des oeuvres d'art... Les vêtements changent mais les rapports de force restent les mêmes. Il faudra sans doute revenir une autre fois pour revoir quelque unes des images et installations aperçues ce soir.

20:58 je me rapproche de la Zion Kirche, en traversant Berlin j'ai pu voir barricades prêtes pour aujourd'hui, les colonnes de voiture Polizei prêtes, le gros de l'orage est passé mais comme chaque 30 avril au soir la ville s'attend au pire pour le lendemain. Mal sehen comme on dit.

Puis le dîner eu lieu, la dégustation de vin aussi et la soirée fût très agréable. Pour nos amis statisticiens, quatre personnes, trois nationalités - française, espagnole et anglaise - avec au menu un plat italien, tout ca dans une ville allemande.