Senior Manager Data Engineering

Job Summary:

We are seeking a highly motivated and experienced Data Engineer Lead to oversee the design, development, and management of our data infrastructure. This role involves leading a team of data engineers to build scalable data pipelines, ensure data quality, and optimize data processes to support business intelligence, analytics, and AI/ML initiatives.

The ideal candidate will possess a blend of strong technical skills, leadership abilities, and a keen eye for detail. You will work closely with stakeholders across departments to ensure that data systems meet business requirements.

Key Responsibilities:

1. Data Architecture & Strategy:
   • Design and implement scalable, reliable, and secure data architectures.
   • Define best practices for data management, governance, and security.
   • Establish and maintain a cloud-based data platform (e.g., AWS, Azure, GCP).
2. Data Pipeline Development:
   • Build and maintain ETL/ELT pipelines for ingesting structured and unstructured data from various sources.
   • Ensure data pipelines are optimized for performance and reliability.
   • Develop batch and real-time data processing solutions.
3. Data Quality & Governance:
   • Implement data quality frameworks to ensure accuracy, consistency, and reliability.
   • Establish data governance policies, including data security, privacy, and compliance.
4. Team Leadership & Mentorship:
   • Lead a team of data engineers, providing technical guidance and mentorship.
   • Manage project timelines, delegate tasks, and ensure delivery of high-quality data solutions.
   • Foster a culture of continuous learning and improvement within the team.
5. Collaboration with Stakeholders:
   • Work closely with data analysts, data scientists, and business stakeholders to understand data requirements.
   • Collaborate with DevOps, Software Engineering, and Product teams to integrate data solutions.
6. Technology & Tooling:
   • Stay updated with the latest trends and best practices in data engineering.
   • Recommend and implement appropriate data tools and technologies to improve data processes.
   • Ensure that the data infrastructure aligns with business goals and scales with growth.

Required Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field. A Master's degree is a plus.
• 6+ years of experience in data engineering, with at least 2 years in a leadership role.
• Proficiency in programming languages such as Python, Java, or Scala.
• Expertise in SQL and experience with data modeling.
• Hands-on experience with cloud platforms (AWS, Azure, or GCP).
• Experience with data orchestration tools (Apache Airflow, Luigi, etc.).
• Knowledge of distributed data processing frameworks (Spark, Hadoop).
• Familiarity with data warehousing solutions (Snowflake, Redshift, BigQuery).
• Experience with real-time data streaming (Kafka, Kinesis).
• Understanding of data security, GDPR, and data governance best practices.

Preferred Qualifications:

• Experience with containerization (Docker, Kubernetes) and CI/CD pipelines.
• Knowledge of AI/ML model deployment pipelines.
• Experience with NoSQL databases and graph databases.
• Familiarity with tools like dbt, Power BI.

Key Competencies:

• Strong problem-solving and critical-thinking skills.
• Excellent communication and stakeholder management skills.
• Ability to balance technical depth with business acumen.
• Leadership and team management capabilities.
• Strong organizational and time-management skills.