AVP Data Engineer

·       Data Pipeline Development :

o    Design, develop, and maintain data pipelines to ingest, process, and transform data from various sources into usable formats.

o    Implement data integration solutions that connect disparate data systems, including databases, APIs, and third-party data sources.

·      Data Storage and Warehousing :

o    Create and manage data storage solutions, such as data lakes, data warehouses, and NoSQL databases.

o    Optimize data storage for performance, scalability, and cost-efficiency.

·        Data Quality and Governance :

o    Establish data quality standards and implement data validation and cleansing processes.

o    Collaborate with data analysts and data scientists to ensure data consistency and accuracy.

·        ETL (Extract, Transform, Load) :

o    Develop ETL processes to transform raw data into a structured and usable format.

o    Monitor and troubleshoot ETL jobs to ensure data flows smoothly.

·        Data Security and Compliance :

o    Implement data security measures and access controls to protect sensitive data.

o    Ensure compliance with data privacy regulations and industry standards (e.g., GDPR, HIPAA).

·        Performance Tuning :

o    Optimize data pipelines and queries for improved performance and efficiency.

o    Identify and resolve bottlenecks in data processing.

·        Data Documentation :

o    Maintain comprehensive documentation for data pipelines, schemas, and data dictionaries.

o    Create and update data lineage and metadata documentation.

·        Scalability and Reliability :

o    Design data infrastructure to scale with growing data volumes and business requirements.

o    Implement data recovery and backup strategies to ensure data availability and resilience.

·        Collaboration :

o    Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver data solutions.

o    Provide technical support and guidance to data users.

·        Continuous Learning :

o    Stay updated on emerging technologies, tools, and best practices in data engineering.

o    Implement new technologies to enhance data processing capabilities.