BI Data Engineer

• Redshift Optimisation: Optimising cluster and table configurations, including distribution, sort keys, and compression, to improve query performance. Analysing long-running queries and applying query-level optimisations, while leveraging data aggregation and retention strategies to improve performance, reduce storage, and manage data growth effectively.
• Data Aggregation & Retention: Designing and building roll-up layers that preserve analytical value while enabling safe deletion of raw-level data. This includes schema design, backfills, data reconciliation, and ensuring backward compatibility for existing reports and consumers.
• Data provision: Enabling teams to access data by creating curated, fit-for-purpose datasets and APIs for product, operations, and integrator-facing systems. These externally consumed solutions require high availability and strong data reliability.
• Infrastructure and Performance. Strengthening pipeline monitoring through automated setup and built-in notifications, while proactively identifying query optimisation opportunities and cost-efficiency improvements. Defining solutions to manage data limitations and retention, with data volume remaining a key constraint shaping the overall roadmap.

What will you be doing at Yoti?

• Own Redshift performance and capacity, balancing reliability, query performance, scalability, and cost.
• Design and deliver data aggregation and retention solutions, including roll-ups, reconciliation, backfills, and safe removal of historical source data.
• Build data-provisioning solutions for product, operational, and integrator-facing consumers, ensuring reliable and accurate access to curated data.
• Build and maintain reliable ingestion and ETL/ELT pipelines, including third-party API integrations and failure handling.
• Establish data quality, reconciliation, and monitoring practices for critical data flows and reporting.
• Apply AI and automation to improve engineering workflows, monitoring, optimisation, and delivery.
• Make informed trade-offs between infrastructure improvements, feature delivery, and compliance priorities.