Senior Data Engineer

Our customer is a leading organization undergoing digital transformation, specializing in implementing advanced AI solutions across their business operations. They're committed to leveraging cutting-edge technology to drive innovation and efficiency.

The customer is currently undergoing a significant cloud transformation journey, migrating from on-premises infrastructure to cloud-based solutions. This project involves building  scalable infrastructure to support their AI/ML initiatives, ensuring seamless deployment and operation of machine learning models in production environments.

You'll be joining a collaborative team of DevOps engineers, data scientists, and ML engineers focused on creating and maintaining the infrastructure that powers the organization's AI solutions. The team values innovation, technical excellence, and continuous improvement in delivering secure, efficient, and scalable AI systems.

• Interview process that respects people and their time
• Professional and open IT community
• Internal meet-ups and resources for knowledge sharing
• Time for recovery and relaxation
• Bright online and offline events
• Opportunity to become part of our internal volunteer community

Infrastructure Development & Standards

• Lead the design, development, and implementation of scalable, high-performance data infrastructure to support diverse data sets and downstream applications in fintech and audit domains.
• Establish and enforce best practices, protocols, and policies for data management, processing, and infrastructure maintenance.
• Create standards for system architecture to ensure traceability, usability, integrity, and scalability of all data systems.
• Proactively identify opportunities to optimize and future-proof infrastructure for evolving data needs.

Data Architecture, Governance & Pipelines

• Design, implement, and maintain efficient ETL pipelines to process and harmonize multiple data sources.
• Build infrastructure to support advanced applications, such as knowledge graphs and AI-driven tools, ensuring seamless interoperability with analytical workflows.
• Implement data governance frameworks to ensure data security, compliance, and integrity.

AI/ML Enablement & Data Readiness:

• Design and implement processes that ensure data is cleansed, normalized, and transformed to meet the requirements of machine learning models.
• Collaborate with data scientists to build and maintain end-to-end ML pipelines that efficiently deliver data for model development, testing, and production.
• Ensure that data platforms are optimized for high-speed data access and transformation necessary for real-time or batch ML processing.
• Establish monitoring systems for ML pipelines to detect data drift, quality issues, or performance bottlenecks.
• Integrate data versioning and lineage tracking to enhance reproducibility and auditability of ML experiments.

Collaboration & Leadership

• Partner with data scientists and engineers to ensure infrastructure aligns with analytical and research objectives.
• Mentor and guide other engineers, instilling best practices in infrastructure design, data management, and software development.
• Foster a culture of collaboration, continuous learning, and high standards within the team.

Education

• BS in Computer Science, Data Engineering, or a related field

Experience

• 5+ years of experience in Data Engineering, preferrably in financial domain
• Demonstrated success in designing and deploying scalable ETL pipelines and infrastructure for large-scale datasets.
• Proven track record of establishing standards and best practices for data architecture and processing systems.
• Experience working in an agile development environment with test-driven development methodologies.

Technical Expertise

• Proficient in data engineering programming languages such as Python, R, SQL/no-SQL, and languages for big data and cloud platforms.
• Proficiency with Azure (or AWS as alternative) and experience in managing cloud-based systems.
• Proficiency in data preparation for AI/ML enablement, including data quality, infrastructure setup, data validation and data versioning aspects

Soft Skills

• Strong leadership and mentoring capabilities, with the ability to guide teams in adopting best practices.
• Excellent organizational skills, with a focus on creating maintainable and scalable systems.
• Exceptional communication skills, enabling effective collaboration across cross-functional teams.