DevOps Engineer (Big data)

Location: London, UK Working Mode: 3 days on site per week Description We're looking for a DevOps Engineer (Azure & Big Data) to join our team in London, UK You will play a key role in designing and implementing the global marketing data platform that consolidates client profiles and marketing preferences into a centralized master repository. This system will manage data pipelines from multiple sources, handle master data management elements , and distribute data via APIs and bulk feeds to marketing platforms. The role requires expertise in Infrastructure as Code (IaC) , CI/CD automation , and experience in cloud-native and big data ecosystems. Responsibilities Design, implement, and maintain CI/CD pipelines leveraging tools like GitLab, Jenkins, or TeamCity Enable Infrastructure as Code (IaC) practices using Terraform for scalable deployments in Azure Collaborate closely with architects on end-to-end solution designs , ensuring alignment with enterprise architecture standards Manage Azure PaaS components , including Azure DevOps, Vault, and other security and configuration services (AKS experience is beneficial) Integrate and optimize data processing frameworks with Spark, Databricks, or Cloudera platform (Hue, Hive, HDFS) Support ETL pipelines built for master data management, customer profiles, and consent management workflows Ensure environments are stable, secure, and optimized for high-volume data flows across multiple downstream systems Troubleshoot environment, deployment, or automation issues quickly and effectively Partner with engineering and data teams to implement best practices for DevOps, governance, and cloud operations Requirements (Must Have Skills) Proven DevOps expertise in cloud environments , including IaC and CI/CD pipeline management Strong hands-on skills in Terraform and scripting (Shell or PowerShell) Experience working with GitLab and integrating automation into build and release cycles Exposure to Azure Cloud and PaaS services, including Azure DevOps and Vault Familiarity with data engineering and big data ecosystems , including Spark and Cloudera stacks (hue, hive, HDFS) Excellent problem-solving skills and ability to communicate technical solutions effectively Previous experience working on data-driven projects and master data management solutions Nice to Have Skills Hands-on experience with Azure Data Factory and Databricks for advanced data processing Knowledge of AKS (Azure Kubernetes Services) for containerized workloads Background in development using Java, C#, or JavaScript for automation or service integration Knowledge of customer data platforms, personalization strategies, and data governance requirements