My First Six Months As A Data Engineer
How it started…
May 2nd was my first working day as a data engineer at amsterdam&partners. A day that I won’t easily forget. Of course, on this very day, the Dutch Railways had a malfunction on the route between Hilversum and Amsterdam. It took some time to get to work. Nice entry…
Anyway, this new adventure already started on April 26th with the department visiting STRAAT Museum in Amsterdam. I got to know a few colleagues and they met me. And they still allowed me to start on May 2, which is quite remarkable…
In the beginning, I met with new colleagues to find out what the organization does and what they do within the organization. It soon becomes apparent that it is not easy to understand what the organization is about. Fortunately, for me, the organization itself is also busy reassessing who they are, what they are there for, etc. Concerning getting to know the organization, I hit the jackpot. Besides that, my position — data engineer — is also new in the organization. During the various conversations and sessions, the importance of data for the organization becomes clear (Note to self: managing expectations will be part of my job description).
I would like to say that I had a very warm welcome, something which I hadn’t had in a long time.
Project and other adventures
Soon I was drawn into a project where we look at which data infrastructure is needed based on a specific case. I also get some swimming rings and then I am thrown in at the deep end regarding the database of the I amsterdam City Card. The I amsterdam City Card gives you access to public transport, canal cruises, bicycle rental, museums and discounts at restaurants, concerts and attractions. A product that is important for our organization.
To understand the database, I want to build a dashboard in Tableau. We use Tableau to visualize data. I’ve worked with Tableau before, but it’s been a while. I grabbed a pen and paper — yes, I’m old school in that regard — and sketched the dashboard I want to build.
City Card Scorecard
Below is the result of my sketch. The numbers are fictitious.
After the necessary research (data and Tableau) I managed to create the dashboard below — with fictitious data. Some changes have been made to the sketch, mainly because I only wanted to work with the city card database. I managed to keep the use of other data to a minimum. I do know that if I had to do this again with what I know now, I would do it differently. The bottom row is more of the same, so that gives room for a different visualization.
I liked this exercise. I’ve been busy with Tableau Prep and Tableau Desktop, and the database feels familiar too. I also benefited greatly from the book: Storytelling with Data: A Data Visualization Guide for Business by Cole Nussbaumer Knaflic.
Data Engineer and next steps
Through my participation in the aforementioned project, it is becoming increasingly clear that a good description of where we want to go and what is needed to get there is very useful. A brain dump. I thought of a few A4 sheets. But in my case that usually gets out of hand, resulting in a firm document about data, data warehouses, data models, data quality, business intelligence (tools) and the data maturity model.
Based on the document choices can be made and I will be working on that.
About me
I’m Henk van der Duim, Data Engineer at Stichting amsterdam&partners.
Follow me on Medium for regular updates on Data, AI and other topics:
Also, I am open to connecting all data enthusiasts across the globe on Linkedin:
