Senior Data Engineer - Contango
Posted on :
08-01-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Our Team
Our team comprises highly skilled professionals who bring a wealth of international experience in technology entrepreneurship. We thrive on ambition and recognize that fun and performance go hand in hand. Together we create a dynamic environment where excellence is the currency of success.
Position Overview
As a Data Engineer you will play a crucial role in designing building and maintaining our data infrastructure. You will be responsible for ensuring the availability integrity and efficiency of our data pipelines enabling our organization to make informed datadriven decisions. The value you bring will be instrumental in delivering measurable value to Contango and beyond using data and AI.
Responsibilities
- Leadership and ownership: own overall data lake and data warehouse architecture with the responsibilities to design architect and maintain the data warehouse infrastructure to support data management data governance analytics ML & AI needs and address future challenges.
- Data pipeline development: Design build and maintain scalable data pipelines for ingesting processing and transforming data from various sources ensuring data quality scalability and reliability.
- Data modelling: Develop and maintain data models architecture patterns schemas and structures that support the needs of data analysts data scientists and other stakeholders.
- ETL (Extract Transform Load): Create and optimize ETL processes to extract data from diverse sources transform it into usable formats and load it into data warehouses or other storage solutions.
- Data governance quality and validation: Establish and enforce data governance policies standards procedures and best practices and implement data quality checks validation processes and error handling to ensure the accuracy and consistency of data.
- Performance tuning: Continuously monitor and optimize the performance of data pipelines to meet business requirements and scalability needs.
- Data security: Implement and maintain data security measures to protect sensitive information and ensure compliance with data privacy regulations.
- Collaboration: Work closely with data analysts data scientists and other stakeholders to understand their data requirements and provide support in data access and availability.
- Documentation: Maintain thorough documentation of data engineering processes data models and data dictionaries.
- Stay informed: Keep up to date with emerging trends and technologies in data engineering to ensure our data infrastructure remains cuttingedge.
Qualifications
- Bachelors degree in computer science information technology or a related field (masters degree is a plus)
- Demonstrable experience of minimum 8 years with at least 5 years in a leadership position as a Senior Data Engineer or similar with a portfolio of projects 10 years of experience in data engineering with a focus on designing and implementing data lakes data warehouses data modelling ETL processes and data transformation pipelines
- Experience in cloudbased data platforms (e.g. Spark Azure Synapse Analytics etc.)
- Proficiency in programming languages such as Python Java or Scala
- Strong knowledge of database systems (SQL and NoSQL) data warehousing and big data technologies
- Knowledge of data governance principles data quality management and regulatory compliance
- Experience with orchestration services (i.e. Azure Data Factory (ADF) Azure Logic Apps Azure Functions Databricks Jobs Airflow etc.)
- Familiarity with data engineering integration tools and frameworks and ETL processes
- Experience with cloud platforms like AWS GCP or Azure
- Experience with API development and integration with data services
- Excellent communications leadership and collaboration skills
- Ability to work in a fastpaced dynamic environment and lead multiple projects and teams to deliver highquality results
- Technical Expertise:
- Programming languages: Python Java Scala and R.
- Data management & databases: Oracle SAP SQL NoSQL Data Warehousing
- Big data technologies: Apache Hadoop Spark Kafka etc.
- Cloud platforms: Experience with Microsoft Fabric Azure (Synapse Analytics Databricks Machine Learning AI Search Functions etc.) Databricks on Azure AWS (S3 Redshift SageMaker etc.) Google Cloud Platform (BigQuery Dataflow etc.).
- Data governance: Experience with Ab Initio Informatica Collibra Purview IBM InfoSphere Great Expectations Deepchecks Databricks Unity Catalog Databricks Delta Sharing Databricks Catalog Explorer Databricks Audit Logging and Databricks Identify Management.
- Data visualization: Familiarity with Matplotlib Seaborn Tableau Power BI
- DevOps & MLOps: CI/CD principles Docker MLFlow Kubernetes
Benefits
- Competitive salary and benefits package.
- Opportunity to work on cuttingedge technology projects.
- Collaborative and innovative work environment.
- Professional growth and training opportunities.
If you possess the requisite experience qualifications and leadership acumen to drive our success and shape the future of Data and AI innovation within the UAE we invite you to submit your application.
This job has been sourced from an external job board.
More jobs on
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobs.ae is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
