Senior Data Engineer & Manager
My role is to identify real business needs in order to provide simple solutions.
In addition to robust working methods in Data processing, I have developed a particular expertise around geographic data. My related experiences as a manager, tech lead and scrum master strengthen my efficency in a team in the service of projects.
The BI/Big Data team provides key indicators to employees at Mappy's. Mainly in the form of dashboards. The figures are processed from the daily 200 GB of logs generated by the 700 servers that feed Mappy's services (map server, search engine, routing engine, etc.). The main business domains are:
- analysis of server performance: response time, availability, usage by our customers (pagesjaunes.fr, pap.fr, etc.)
- study of user interest in each mode of transport offered on Mappy
- ROI for the 5 million local buisnesses displayed on the map (number of clicks, phone numbers seen, etc.)
- small data (test results, application crashes, audience, etc.)
- processing of users location in order to study how they commute and travel
Methodology : Scrum, Continuous Integration (several deployments per day)
Technologies : Scala, Spark, Hadoop, ElasticSearch, Hive, Indexima, Tableau
The Map team develop servers that produces maps consumed over the HTTP protocol. The load from the websites and mobile applications of Mappy, Pagesjaunes (french yellow pages) and other customers reaches of 8,000 calls per second in picks.
The map team is also in charge of displaying traffic information.
Mission : migration to vector maps. Unlike raster maps (or bitmaps) where the pixels looks bigger when zoomed, vector maps give a great impression of fluidity, and allow a personalized display on the user's terminal.
Methodology : Scrum, Continuous Integration (deployment every Tuesday)
Technologies : Python, Postgresql, PostGIS, Varnish, zmq, Jenkins, Mapbox-Vector-Tiles
Mappy's Backend Services ("Back") department is made up of four teams in charge of cartographic services:
- route calculation
- geocoding and search engine
- immersive views and points of interest (POI)
AHelp team leaders to cut big evolutions into small functional increments
Organize packaging and delivery processes (every Tuesdays)
Training of Scrum Masters
Helped product teams to express their need. Removed unecessary constrains
Made the 3 frontend teams and the 4 backend teams agree on REST interfaces.
Helped the technical teams to ask the right questions and find their own answers.
Introduced asynchronous flow in the backends.
Migration of Mappy's mapping lecacy services from Windows / Visual C++ / SQLServer to Linux and open source components, which provide a standard REST API
- Map (8,000 requests / sec): Python / Tornado / Mapnik / PostGIS / Varnish
- Suggest: Python / Tornado / PostGIS / SolR
- Routing: Python / Tornado / C++ / RabbitMQ / Contraction hierarchies
Management of the CoPS team (Backoffice for collecting immersive photos from merchants around an iPad application) - 9 months
Management of the route computation team and training of the new manager - 6 months
The Geodata team (4 people) organizes the collection by panoramic views in France.
LThe images from the drivers' hard drives are processed and then blurred automatically (90%) as well as by 40 operators in Romania. Finally, they are published via a REST service, in compliance with french privacy policies.
Backoffice : Flash / PHP / MySQL / Python / Java / AWS.
Service Backend : Java / MongoDB - 25 To d'images
Méthodologie : Scrum, Intégration continue
Maintenance and evolutions of the CORTO software.
Integration of maps of client communities, whatever the format.
OOrganization of data collection between city services and partners (firefighters, municipal and national police, social landlords, car insurers) while respecting anonymity and privacy.
Technologies : Flash / ActionScript, PostGIS, MS access, ASP, GDAL / OGR, Delphi
Mapping software for urban areas. Local authorities could study data at the exact adress
Navideo : video rental inventory on the internet
On their way back from work, users could pick up a DVD they had booked. The software was would help rental stores to shifting its customer relationship and advertising campaigns to digital in order to reduce the cost.