Data Engineer

Technical Skills Expertise in DBT (preferred) for data modeling. Strong SQL skills with hands-on experience in: o Snowflake o Azure SQL o Stored procedures Proficiency in Python for data engineering workflows. Hands-on experience with: o Databricks & PySpark o Azure Data Services (ADF, ADLS, Synapse) Strong knowledge of Snowflake architecture and schema design. Experience with data validation, quality frameworks, and analysis tools. Familiarity with GitHub and CI/CD practices. Role demands a highly skilled Data Engineer to design, build, and optimize scalable data pipelines and data platforms. The ideal candidate will have strong expertise in data modeling, cloud-based data architectures, and modern data engineering tools across Azure, Snowflake, and Databricks environments. Key Responsibilities Data Engineering & Pipeline Development Design, develop, and maintain robust ETL/ELT pipelines using Databricks, PySpark, and Azure Data Factory (ADF). Build scalable and efficient data ingestion frameworks for structured and unstructured data. Optimize pipeline performance through performance tuning and orchestration best practices. Data Modeling & Management Develop and maintain data models using modern tools (DBT preferred). Implement Master Data Management (MDM) solutions to ensure data consistency and integrity. Design scalable and efficient Snowflake schemas (star/snowflake schema, dimensional modeling). Database & Query Optimization Write and optimize advanced SQL queries across Snowflake, Azure SQL, and Synapse. Develop and manage stored procedures and database objects. Ensure efficient data retrieval through indexing, partitioning, and query optimization. Cloud & Platform Integration Work with Azure data services including: o Azure Data Factory (ADF) o Azure Data Lake Storage (ADLS) o Azure Synapse Analytics o Azure SQL Database Integrate and maintain Snowflake with Azure ecosystem. Python Development Develop data transformation and automation scripts using Python libraries: o pandas o pyodbc o SQLAlchemy Build reusable components for data processing and validation. Data Quality, Validation & Monitoring Implement data validation rules, quality checks, and anomaly detection frameworks. Perform root cause analysis for data inconsistencies. Develop dashboards or tools for data quality monitoring. Collaboration & DevOps Use GitHub for version control, branching strategies, and code reviews. Manage workload scheduling and dependency management for pipelines. Collaborate with cross-functional teams including data analysts, data scientists, and business stakeholders. Required Skills & Qualifications Bachelor’s or Master’s degree in Computer Science, Information Systems, or related field. Strong experience in data engineering and data platform development. Preferred Qualifications Experience implementing data governance and MDM solutions. Knowledge of performance tuning in distributed processing systems. Familiarity with workflow orchestration tools. Experience in agile environments (SCRUM/Kanban).