Senior Data Engineer

Fireart Studio

КомпанияFireart Studio
УровеньSenior

Описание

We are looking for a Data Engineer to design, develop, and optimize our data infrastructure on Databricks. You will architect scalable pipelines using BigQuery, Google Cloud Storage, Apache Airflow, dbt, Dataflow, and Pub/Sub, ensuring high availability and performance across our ETL/ELT processes. You will leverage Great Expectations to enforce data quality standards. The role also involves building our Data Mart (Data Mach) environment and implementing CI/CD best practices. A successful candidate has extensive knowledge of cloud-native data solutions, strong proficiency with ETL/ELT frameworks (including dbt), and a passion for building robust, cost-effective pipelines.

Обязанности

Data Architecture & Strategy – Define and implement the overall data architecture on GCP, including data warehousing in BigQuery/Databricks, data lake patterns in Google Cloud Storage, and Data Mart (Data Mach) solutions. Integrate Terraform for Infrastructure as Code to provision and manage cloud resources efficiently. Establish both batch and real-time data processing frameworks to ensure reliability, scalability, and cost efficiency.
Pipeline Development & Orchestration – Design, build, and optimize ETL/ELT pipelines using Apache Airflow for workflow orchestration. Implement dbt (Data Build Tool) transformations to maintain version-controlled data models in BigQuery, ensuring consistency and reliability across the data pipeline. Use Google Dataflow (based on Apache Beam) and Pub/Sub…

Вакансия на сайте компании

Откликнуться в JobToo

Senior Data Engineer

Описание

Обязанности

Вакансии этой компании

Похожие вакансии