Lead Data Engineer

Permanent | Lead Data Engineer | London

Salary: £70,000 - £120,000 depending on experience.
Key Responsibilities:

Architect and build scalable data pipelines and infrastructure to support AI and product systems.
Design and maintain data ingestion, transformation, and storage architectures for operational and AI workloads.
Develop and manage batch and real-time data pipelines.
Build and optimize systems for vector search, retrieval, and ML data pipelines.
Ensure data reliability, security, and governance across the platform.
Collaborate with AI and backend engineering teams to support training, inference, and product features.
Implement monitoring, observability, and data quality frameworks.
Optimize performance of large-scale datasets and query systems.
Contribute to technical architecture decisions and long-term data strategy.
Help build and mentor the future data engineering team as it scales.

Requirements:

Core Experience:
7+ years of experience in data engineering or backend engineering roles.
Strong experience designing and building data pipelines and distributed data
systems.
Experience working with relational databases (PostgreSQL preferred, but MySQL or similar is acceptable).
Experience with NoSQL databases.
Experience with vector databases used in modern AI systems.
Strong programming experience in Python.

Data & Infrastructure:
Experience with some of the following technologies is highly desirable:

Frameworks / Infrastructure:

Apache Spark
Apache Airflow
Kafka
Elasticsearch / OpenSearch

Databases

PostgreSQL
MongoDB
Vector databases such as Qdrant, Milvus, or pgvector
Python Libraries
Pandas
Polars

Engineering Skills:

Experience building scalable backend systems.
Experience designing data models and storage architectures.
Strong understanding of data processing performance and optimization.

What We’re Looking For:

Experience in a startup is advantageous!
A high-ownership engineer who enjoys building systems from scratch.
Someone excited to design the data backbone of an AI company.
Strong problem-solving ability and architectural thinking.
Comfortable working in a fast-moving startup environment.
Motivated by the opportunity to build technology that transforms an entire industry.