Data Engineer job description WHAT YOU’LL DO: Design, develop, and maintain ETL/ELT pipelines to transform raw, multi-source data into clean, analytics-ready tables in Google BigQuery, using tools such as dbt for modular SQL transformations, testing, and documentation. Integrate and automate affiliate data workflows, replacing manual processes in collaboration with the related stakeholders. Proactively monitor and manage data pipelines using tools such as Airflow, with proper alerting and retry mechanisms in place. Emphasize data quality, consistency, and reliability by implementing robust validation checks. Build a Data Consistency Dashboard (in Looker Studio, Power BI, Tableau or Grafana) to track schema mismatches, partner anomalies, and source freshness, with built-in alerts and escalation logic. Ensure timely availability and freshness of all critical datasets, resolving latency and reliability issues quickly and sustainably. Control access to cloud resources, implement data governance policies, and ensure secure, structured access across internal teams. Monitor and optimize data infrastructure costs, particularly related to BigQuery usage, storage, and API-based ingestion. Document all pipelines, dataset structures, transformation logic, and data contracts clearly to support internal alignment and knowledge sharing. Build and maintain postback-based ingestion pipelines to support event-level tracking and attribution across the affiliate ecosystem. Collaborate closely with Data Scientists and Product Analysts to deliver high-quality, structured datasets for modeling, experimentation, and KPI reporting.
