We chatted to Andrea Louw about what her role entails, the things that inspire her, and how she grew from a fresh-out-of-university hire, into her current role.

How did you end up in a data engineering role at Yoco?

I studied BSc. Hons Applied Mathematics, and then started at Yoco as a Junior Data Scientist in April 2017. Besides doing internships during my studies, this was my first official job. After about a year and a half, I was promoted to Data Scientist. After doing increasingly more work on data cleaning, transformation, pipelines, and warehousing, I officially transitioned to a Data Engineering role in April 2020.

For the uninitiated, what does a Data Engineer do exactly? I know there are a lot of misconceptions about your role.

There are. There’s sometimes the misconception that I am involved with the creation or generation of data.

I'm involved in setting up and maintaining processes that collect the data from various production databases or integrated services, and transforming, cleaning, and loading it into our analytical database.

I don’t have much control over the quality of the data that is created at each of the data sources, but I suggest improvement, or transform it so that it’s more usable for our analysts..

I would describe my role as this: “I’m not the data creator. I package, transform, and deliver the data to our data warehouse, of which I am a primary caretaker...

The analysts and other data consumers would build their reports and analysis on the data that is available in the warehouse”.

Tell us a little bit about what you get up to on a typical day?

I work very closely with the Analytics team at Yoco. I try to stay informed about what data they need for their reports, what form they need it in, what cleaning steps would be helpful, and how fresh they would need the data to be.

This means that a lot of my day entails making small changes to existing data pipelines and frequently executed data queries to fix bugs or make the data sets more useful. I’m responsible for setting up new integrations with an ETL tool, or building an integration in Airflow if necessary. I also need to monitor the health of our cloud resources and make sure our analytical database stays performant. This is not part of my job per se, but from time to time I like to help other teams with data-informed alerting.

Data Engineering is, in essence, about helping other teams.

It’s therefore important to take the time to really understand what those dependent on your work need, and be willing to learn new tricks and adjust according to what would make their jobs the easiest.

What are the instances that make you go “Yes! This is why I love data engineering!”?

I love problem solving and exploring solutions, so when someone experiences a problem or is trying to answer a question, I’m inspired when I can offer them a way of approaching it. I enjoy the gratification when a new tool or service eventually works smoothly after a long time of struggling with it and figuring it out.

I get a kick out of building things that make my teammates’ work easier for them.

How have you been supported in growing as a Data Engineer during your time at Yoco?

Yoco has helped me to explore new avenues where I felt I could excel and grow more, even when these did not always match my role description.

I was given the opportunity to change roles that play better to my strengths and the direction I want to grow in, and Yoco paid for online courses and certifications in Data Engineering.

It’s also a great learning opportunity to be able to try out various online tools that are not affordable or easy to access as a non-business user. Because my work adds value, I’m rewarded by being able to work on projects that I personally find interesting.

Lastly, how can aspiring Data Engineers follow in your footsteps?

This would depend a bit on where you are coming from. As an analyst, I got started out of necessity, by building repeatable data transformation to simplify building reports as well as adding new data integrations to supplement our existing data.

My responsibility and control I had over the incoming data and data warehouse grew with time. That said, after covering the basics of databases and data structures, I would recommend doing an online course on big data processing tools (like Spark) as these aren’t typically covered in programming courses. Also, try to expose yourself as much as possible to the popular cloud-based big data tools (like Amazon Redshift).

—

Ready to grow your career in data at Yoco? Have a look at our open roles here.