For students and early-career professionals

Data Engineer

Builds pipelines that collect, clean, and deliver data for analytics and products.

12 chapters · About 3–5 months · 15–30 min a day

Start this plan free →

Chapter 1 is free.

Skills

14 skills, and what you'll learn in each

Each skill is a few short lessons, then practice. You rate yourself at the start and again at the end.

Core to the job

Chapter 2

SQL

Write queries that filter and summarise data and join tables

What you'll learn

  • Query output shape
  • Join logic
  • Grouping level
  • Query plan

Practice

  • Find the bug in a SQL query15 min

Chapter 2

Python

Summarise a data file and write a small reusable function

What you'll learn

  • Data frame basics
  • Function inputs and outputs
  • Read a traceback
  • Clear code names

Practice

  • Load a CSV and Summarise It in a Jupyter Notebook20 min
  • Build an Automated Data Quality Report in Python45 min

Chapter 2

Git

Save clear changes to a project's history and request review

What you'll learn

  • Change states
  • Short-lived branches
  • Merge conflict
  • Useful commit message
  • Pull request review

Practice

  • Make a Commit and Undo a Mistake15 min

Chapter 3

Data cleaning

Clean a messy data set and note each fix

What you'll learn

  • Tidy table shape
  • First data profile
  • Missing value choice
  • Repeatable cleaning steps

Practice

  • Clean a Messy Dataset and Document Your Fixes10 min

Chapter 3

Data transformation

Write and test one step that reshapes raw tables

What you'll learn

  • Transformation layers
  • Model grain
  • dbt materializations
  • Schema tests
  • Data lineage
  • Model references
  • The model graph
  • Templated SQL
  • Kinds of test

Practice

  • Choose a view, a table or an incremental model for five transformations25 min

Chapter 4

Data pipelines

Describe a data source, then automate copying its data daily

What you'll learn

  • ETL and ELT
  • Pipeline dependencies
  • Idempotent loads
  • Data quality checks
  • Safe backfills
  • Batch and streaming
  • Data contracts
  • Work across machines

Practice

  • Make four loads safe to run twice25 min
  • Pick Batch, Micro-Batch or Streaming for Three Feeds20 min

8 more in the plan

Data qualityWrite checks for a table and run them before sharingCh 4

What you'll learn

  • Data checks
  • Plausible values
  • Root cause fix

Practice

  • Write 5 data quality checks for a table20 min
Data modelingOrganise data into linked tables and describe each columnCh 6

What you'll learn

  • Table grain
  • Primary key
  • Table relationships
  • Normalised design
  • Star schema
  • Model levels
  • Transactional and analytical

Practice

  • Sketch a data model for a small app30 min
Cloud data platformsQuery a cloud database and check costs before adding dataCh 6

What you'll learn

  • Cloud service choice
  • Object storage
  • Managed warehouse
  • Cloud costs
  • Permission basics
  • What a query costs
  • Reading less data

Practice

  • Run a SQL Query in a Free Cloud Sandbox20 min
CommunicationRewrite an update for two different readers, cutting extra linesCh 7

What you'll learn

  • Lead with the point
  • Match detail to the reader
  • Said versus heard
  • Specific feedback

Practice

  • Write a Six-Line Delay Message10 min
Working with othersContribute to group work, ask for help and credit teammatesCh 7

What you'll learn

  • Early team friction
  • Clear ownership
  • Challenge ideas, not people
  • Lead or follow
Documenting data workWrite data field definitions that others can use unaidedCh 7

What you'll learn

  • Metric definition
  • Data lineage
  • Assumptions and trade-offs
  • Documentation types

Practice

  • Design a simple data-issue tag taxonomy20 min
Problem solvingTest each possible cause of a problem instead of guessingCh 9

What you'll learn

  • Problem vs symptom
  • Clean problem splits
  • Root cause
  • Deciding under uncertainty

Practice

  • Separate the Problem from the Symptom10 min
Critical thinkingSpot a weak assumption and ask where numbers come fromCh 9

What you'll learn

  • The hidden assumption
  • Argue the other side
  • The base rate
  • Fact vs inference
  • Wanting it to be true

Your plan

Chapter by chapter.

12 chapters, in this order. Chapter 1 is free.

1~1 wk

See data engineer work up close

StartFree
21–2 wks

Shape data with SQL and Python

Skills
31–2 wks

Turn messy data into clean models

Skills
41–2 wks

Keep data pipelines running and correct

Skills
51–2 wks

Build a daily pipeline into a dashboard

Project
61–2 wks

Organise data well on cloud platforms

Skills
71–2 wks

Document your data work for reuse

Skills
8~1 wk

Talk to two working data engineers

People
91–2 wks

Find the cause before you fix it

Skills
101–2 wks

Decide how you will get in

Decide
111–2 wks

Answer the questions interviewers ask

Interview
12on your timeline

Send your first applications

Apply

The job

What a Data Engineer does

Builds reliable data pipelines so teams can trust reports and data-powered features in apps and systems.

The work

  • Design data pipelines to move data from apps, files, and APIs into databases
  • Build ETL and ELT jobs that clean, join, and standardize messy data
  • Set up data models so analysts and products can query data quickly and safely
  • Monitor pipelines and fix failures, delays, and data quality issues
  • Work with analysts and engineers to define data needs and success checks
  • Improve performance and cost by tuning queries, storage, and compute
  • Document datasets, definitions, and pipeline behavior for other teams

A typical day

  1. Check pipeline alerts, rerun failed jobs, and investigate root causes
  2. Build or update a pipeline, then test with sample and backfill data
  3. Review pull requests and align on data definitions with stakeholders
  4. Tune a slow query or job, then add monitoring and quality checks

Tools

Databases: PostgreSQL, MySQL, SnowflakeData processing: Spark, PandasOrchestration: Airflow, PrefectCloud storage: Amazon S3, Google Cloud StorageVersion control: GitContainers: DockerMonitoring: Grafana, Datadog

Ways in

There is more than one route.

Ways to study

  • Bachelor degree in computer science, software engineering, or information systems
  • Bachelor degree in math, statistics, or engineering plus strong programming practice
  • Data engineering bootcamp or structured online certificate with projects
  • Workplace training through IT or software apprenticeships

How people get their first job

  • Build a portfolio with 2 to 3 end-to-end pipelines and clear documentation
  • Apply for internships in data, analytics engineering, or software engineering
  • Start in data analyst or BI roles and take ownership of data pipelines
  • Earn a practical SQL certificate and a cloud fundamentals certificate
  • Contribute to an open-source data tool by fixing bugs or improving docs

How the work is changing

What AI is doing to this role

How AI is changing this role · one of 6 tasks we track

Design and build data pipelines end to end

Sped up a lot

What AI does

Text-to-SQL tools like Databricks AI draft pipeline code from plain English.

Still yours

You still design the schema and decide how data flows.

Reviewed September 2026

In the app · Premium

The rest is in the app

  • How AI affects the other 5 tasks
  • Whether this job is growing or shrinking
  • How hard the first job is to get
  • Similar roles that are easier to get into
  • Updated every month, with sources
See it in the app →

How we rate a job →

Already working

Already a Data Engineer?

Plan your move to Senior Data Engineer: 10 chapters that end with a strong case for your next review.

Grow in the role →

Other roles in Data & Analytics

Questions

Questions about this plan

How long does it take to become a data engineer with Welica?

The plan is 12 chapters, 3–5 months at 15 to 30 minutes a day. You can go faster or slower; the plan moves with you.

Do I need a degree?

Not always. Common routes are bachelor degree in computer science, software engineering, or information systems, bachelor degree in math, statistics, or engineering plus strong programming practice, data engineering bootcamp or structured online certificate with projects, or workplace training through IT or software apprenticeships.

What is free?

Chapter 1, See data engineer work up close, is free for good. Premium unlocks the rest of the plan.

Start the Data Engineer plan.

Chapter 1 free. About 15 to 30 minutes a day.

Start on web Continue your session Download for MacMac Get the iPhone appiPhone Get it on Google PlayAndroid