Architect - Data Engineer

Senior Data Architect

Job Summary:

The Senior Data Architect will play a pivotal role in leading the design, development, and optimization of scalable data architectures and pipelines that support data-driven decision-making within the organization. This role is essential for ensuring the efficient processing, transformation, and storage of large volumes of data, enabling the business to harness actionable insights and drive innovation.

Key Responsibilities:

• Lead the design, implementation, and maintenance of scalable data pipelines for ingesting, processing, and transforming large data volumes using tools such as Databricks, Python, and PySpark.
• Architect scalable and efficient data solutions by selecting appropriate architectures, including modern frameworks whenever possible.
• Design and optimize data models and schemas to ensure efficient storage, retrieval, and analysis of both structured and unstructured data.
• Develop, optimize, and automate ETL workflows to extract, transform, and load data into data warehouses, data lakes, or lakehouses.
• Utilize big data technologies such as Spark, Kafka, and Flink to facilitate distributed data processing and advanced analytics.
• Deploy and manage data solutions on cloud platforms like AWS, Azure, or Google Cloud Platform (GCP) while leveraging cloud-native services for improved data architecture.
• Implement data governance, quality, and security measures, ensuring adherence to best practices and compliance.

Requirements:

• Proven experience as a Senior Data Engineer or Data Architect, with hands-on expertise in building and optimizing data pipelines and designing data architectures.
• Extensive experience with big data tools and technologies, showcasing strong problem-solving and analytical abilities to address complex data challenges.
• Excellent knowledge of data engineering principles, practices, and an ability to communicate technical concepts clearly to non-technical stakeholders.
• Proficiency in programming languages such as Python, Java, Scala, or SQL for data manipulation and ETL processes.
• Familiarity with data governance frameworks and best practices in managing data quality and security.
• Understanding of modern data architectures, including lakehouses, and experience with ETL tools like Apache Airflow or Azure Data Factory.
• Ability to write clean, scalable code and utilize version control systems such as Git for collaborative development.

Preferred Qualifications:

• Experience with real-time data streaming technologies such as Apache Spark Streaming or Kafka for processing data as it arrives.
• Background in software engineering is a plus, enhancing your ability to integrate analytical solutions and software development.
• Strong understanding of CI/CD pipelines and containerization technologies like Docker to facilitate rapid deployment and testing.
• Experience mentoring junior data engineers, contributing to their professional growth and the overall success of the team.
• Ability to evaluate and recommend new technologies, frameworks, and tools to improve processes and solutions.

Benefits:

• Competitive salary and performance-based bonuses reflecting your contribution to the organization.
• Comprehensive health, dental, and vision insurance plans for you and your family.
• Generous paid time off, including vacation days, sick leave, and paid holidays, ensuring a healthy work-life balance.
• Opportunities for professional development and continued education, including certifications and training programs.
• Flexible work arrangements, including remote work options to accommodate personal and professional needs.
• Collaborative and innovative work culture that encourages creativity and values diversity.
• Access to the latest tools and technologies to support your work and foster your growth within the organization.