Data Platform Engineer

职位要求 / 描述

ABOUT BLOCK LABS Block Labs is a premier technology studio operating at the bleeding edge of Web3, Artificial Intelligence, and iGaming. We don't just ship features; we engineer high-scale, production-grade platforms that power the next generation of digital products. We are a collective of senior engineers, product strategists, and builders who refuse to compromise on architecture. Whether we are designing autonomous multi-agent AI systems, building decentralized financial infrastructure, or architecting high-frequency iGaming platforms, our standard is excellence. We move fast, but we build for the long term. If you are looking to work alongside a team that values deep technical expertise, thoughtful system design, and product ownership, Block Labs is where you belong. THE ROLE Data & Intelligence now sits at the centre of several products we are developing, and we need a platform that is both dependable and capable of supporting more advanced intelligence over time. This role reflects that shift. We are designing a new data platform that will act as the backbone for everything from real time decisioning to predictive modelling. As a Data Platform Engineer in the Data Team, you will own the end-to-end real-time pipeline, serving data across a unified analytical warehouse and feature-serving layer. You are not building dashboards. You are engineering the commercial nervous system of a multi-tenant platform designed to scale from one operator to 10x with marginal infrastructure cost. KEY RESPONSIBILITIES: - Design, build, and maintain scalable data pipelines using AWS Glue (PySpark), or equivalent orchestration and transformation tools. - Engineer and optimise the ClickHouse warehouse for sub-second query performance across all back-offices. - Implement data contracts between back-office and the platform. Onboarding a new operator is a config change, not new tables, topics, or feature views. - Build the feature-serving layer providing pre-computed features to AI agents at millisecond latency. - Integrate with third-party databases, back-office APIs, and external systems (CRM, affiliates, acquisition platforms). - Establish monitoring, alerting, and maintenance procedures including pipeline health checks, freshness monitoring, anomaly detection, and data contract SLA enforcement. - Own CI/CD and infrastructure-as-code for data workloads. - Collaborate with data scientists, agent engineers, BI developers, and infrastructure teams to translate data requirements into reliable, production-grade pipelines. ABOUT YOU: - 3+ years building and operating production data pipelines at scale, with hands-on experience across both streaming and batch paradigms. - Expertise in Apache Kafka (or Amazon MSK): topic design, consumer group management, offset handling, schema registry operations, and production troubleshooting of lag, rebalancing, and throughput issues. - Strong SQL and warehouse engineering skills: experience with columnar analytical databases (ClickHouse strongly preferred, or similar: Druid, BigQuery, Redshift). - PySpark / Spark Streaming proficiency: writing transformation jobs that normalise, enrich, and enforce business rules on event streams. Experience with AWS Glue, Apache Airflow, or Apache NiFi is a strong plus. - Data modelling discipline: ability to design normalised, multi-tenant schemas where tenant isolation is a filter, not a fork. Experience with data contracts and schema governance. - CI/CD and infrastructure-as-code experience: automated testing of data pipelines, version-controlled deployments (CloudFormation, Terraform, or CDK), and familiarity with containerised workloads (ECS Fargate or Kubernetes). - Data quality and observability mindset: experience implementing pipeline health monitoring, automated data validation (Great Expectations or equivalent), freshness checks, and anomaly detection. NICE TO HAVE - Experience in iGaming, online ca

技能关键字

职责方向