Loading open roles
GirnarSoft
Analytics Engineer
GirnarSoft · posted 4 months ago
Kolkata, Chennairecruitment + vendor
About the role
Analytics Engineer – Expectations for Working in Data Democ
1. AE Community Engagement
Skill Description: Proactively shares knowledge, mentors others, and contributes to the wider analytics community.
Key Responsibilities & Expectations:
- Actively support the wider AE community within Data Democ, including providing support through pairing sessions and constructive feedback during code reviews.
- Contribute and ideally lead within knowledge-sharing initiatives to upscale the entire team.
- Act as an onboarding buddy for subsequent hires, guiding them through our data stack, project structures, and team conventions.
- Be a consistent, helpful presence in team communication channels (e.g., Slack), assisting others with technical challenges and answering questions that fall outside your immediate project work.
2. Coding Maintenance & Improvements
Skill Description: Identifies and implements opportunities to refactor and improve the existing codebase for better performance, readability, and efficiency.
Key Responsibilities & Expectations:
- Proactively identify and refactor complex, inefficient, or outdated data models to improve performance and maintainability.
- Actively seek better ways of doing something, feeding back into the central DD pillars to help assist in the creation of best practice content
- Continuously assess the dbt project for opportunities to improve structure, readability, and efficiency, and take the initiative to implement these changes.
3. Coding Standards
Skill Description: Champions and enforces coding standards, ensuring the entire team produces consistent, high-quality, and well-documented code.
Key Responsibilities & Expectations:
- Fully understand the VMO2 standards and the reasons they exist
- Contribute to, advocate for, and enforce our SQL and dbt style guides to ensure consistency across the entire codebase.
- Perform thorough code reviews that assess not only the correctness of the logic but also its adherence to established standards for formatting, naming, and documentation.
- Lead by example by producing exceptionally clean, well-documented, and conformant code in all of your own work.
4. Communication
Skill Description: Effectively communicates complex technical concepts to both technical and non-technical audiences.
Key Responsibilities & Expectations:
- Translate ambiguous business requirements into clear, actionable technical plans and data models.
- Present technical concepts, such as data model designs or architectural changes, to business stakeholders in a way that clearly articulates the business value and impact.
- Tailor communication to the audience, presenting detailed technical specifications to engineering peers and simplified, outcome-focused explanations to business users.
- Clearly communicate the downstream impact of changes to data models to all affected teams and dashboard users.
5. Critical Thinking
Skill Description: Goes beyond the immediate request to understand the underlying business problem and challenges assumptions.
Key Responsibilities & Expectations:
- Challenge initial requests to uncover the true underlying business question or goal, rather than simply delivering a literal, short-term solution.
- Anticipate future business needs and design data models that are versatile and scalable enough to answer not just the immediate question, but a whole class of future questions.
- Demonstrate "big picture" thinking by designing solutions that benefit the entire data platform, not just a single, isolated request.
6. Data Exploration
Skill Description: Independently explores new datasets to validate quality, understand their structure, and identify potential uses.
Key Responsibilities & Expectations:
- Conduct thorough profiling and validation of new data sources before integration into DBT pipelines to assess quality, identify inconsistencies (e.g., NULLs, duplicates), and confirm suitability.
- Proactively analyse data to uncover insights, trends, or data quality issues, and share these findings with relevant business or product teams.
- Independently gain a deep understanding of the structure, grain, and relationships within raw source data to inform effective data modelling.
7. Data Modelling Techniques and Architecture
Skill Description: Designs and implements robust, scalable, and efficient data models and architectures.
Key Responsibilities & Expectations:
- Design, build, and maintain the “business layers” of our data warehouse, creating scalable and performant data models that serve as a "single source of truth" for key business domains.
- Architect and implement domain-specific data marts and "One Big Table" (OBT) models that simplify and accelerate analysis for business users in BI tools.
- Make deliberate, well-justified decisions about data modelling techniques (e.g., dimensional vs. OBT), clearly defining the grain and structure of tables to meet business needs.
8. Development Lifecycle
Skill Description: Understands and follows the full software development lifecycle, including requirements gathering, development, testing, and deployment.
Key Responsibilities & Expectations:
- Ability to manage the end-to-end development of data models, from translating a business need (e.g., a Jira ticket) into a technical design, to coding, testing, documenting, and deploying the solution.
- Work proficiently within our CI/CD framework (e.g., GitLab CI/CD) to automate testing and deployment processes, ensuring code is reliable before it reaches production.
- Treat analytics code with the same rigor as software engineering code, adhering to established development, testing, and release processes.
- Actively engage with the other roles and Pillars within Data Democ to enable a complete end-to-end understanding of where your role fits within the bigger picture of developing a product
9. Environment Configuration
Skill Description: Understands the development and production environments, and the configs within them, on the analytics platform.
Key Responsibilities & Expectations:
- Appreciation for local and cloud-based development environments, including dbt profiles and database connections.
- Suggest use of environment variables to manage configuration differences between development, staging, and production, avoiding hardcoded values.
- Troubleshoot environment-specific issues that may arise during development or deployment.
10. Impact Awareness
Skill Description: Understands the downstream impact of their work on other teams, systems, and business processes.
Key Responsibilities & Expectations:
- Before making any change to a core data model, perform a thorough impact analysis to identify all downstream dependencies, including other models, BI dashboards, and operational systems.
- Take full ownership of communicating changes to stakeholders, providing clear and timely notifications about what is changing, why it is changing, and what impact they should expect.
11. Independent Working
Skill Description: Can take a high-level objective and break it down into smaller tasks, working autonomously to deliver the outcome.
Key Responsibilities & Expectations:
- Take ownership of large, ambiguous projects and independently break them down into a clear project plan with actionable milestones.
- Autonomously investigate and resolve complex data discrepancies or bugs, tracing issues through the entire data pipeline from source to BI tool.
- Work with a high degree of autonomy, driving projects forward without the need for constant supervision or step-by-step instructions.
12. Industry Knowledge and Best Practice
Skill Description: Stays up-to-date with the latest trends, tools, and best practices in the data industry.
Key Responsibilities & Expectations:
- Actively stay informed about modern data tools, trends, and best practices by engaging with industry content (e.g., blogs, conferences, forums).
- Go beyond theory by applying new knowledge and proposing how modern techniques (e.g., dbt model contracts, data quality tools) could be implemented to improve our data platform.
- Champion modern analytics engineering principles within the team.
- Ensures compliance to current standards, but proposed new ideas to the central C4E team that could inform better ways of working based on new features or tools.
13. Optimisation
Skill Description: Identifies and resolves performance bottlenecks in queries and data models.
Key Responsibilities & Expectations:
- Analyse query performance and cost using tools like the BigQuery Query Explainer, and rewrite queries to optimize execution and reduce cost.
- Design and implement efficient data processing strategies, such as converting full-refresh models to incremental models to minimize data processing costs.
- Optimize the materialization strategy (table, view, incremental) of dbt models to balance performance, cost, and data freshness requirements.
14. Quality Control & PII
Skill Description: Implements robust data quality testing and ensures the security and privacy of sensitive data.
Key Responsibilities & Expectations:
- Implement comprehensive data quality testing in dbt, including both generic tests (e.g., not_null , unique ) and custom business logic tests to ensure data accuracy and integrity.
- Design and enforce robust procedures for handling Personally Identifiable Information (PII), ensuring data is appropriately masked, hashed, or access-controlled, especially in non-production environments.
- Take ownership of data quality for the models you build, treating it as a critical feature of your deliverables.
15. Resolving Issues
Skill Description: Independently and effectively troubleshoots and resolves complex technical issues.
Key Responsibilities & Expectations:
- Methodically debug failing dbt models and data pipeline issues, using compiled SQL and direct database queries to isolate and identify the root cause of errors.
- Independently investigate business-reported data discrepancies by performing data lineage analysis and comparing data at each stage of the transformation process.
- Demonstrate persistence and creativity in resolving complex and ambiguous technical problems.
16. SQL Skills
Skill Description: Demonstrates advanced SQL skills, including complex joins, window functions, and performance tuning.
Key Responsibilities & Expectations:
- Write clean, performant, and highly readable SQL.
- Confidently use advanced SQL functions (e.g., window functions, complex CTEs, analytical functions) to implement sophisticated business logic efficiently.
- Translate complex business requirements into elegant and effective SQL queries.
17. Scheduling of Transformations
Skill Description: Manages and optimizes the scheduling and orchestration of data transformation jobs.
Key Responsibilities & Expectations:
- Design and manage dbt Cloud job schedules to optimize for cost, data freshness, and efficiency, understanding the dependency graph ( DAG ) of the entire project.
- Split monolithic runs into logical, smaller jobs based on data sources and business requirements (e.g., hourly source data runs vs. daily mart builds).
- Clearly understand and manage the dependencies between different models and jobs to ensure they run in the correct order and maintain data integrity.
18. Source Control
Skill Description: Uses Git and source control effectively for collaboration, versioning, and code management.
Key Responsibilities & Expectations:
- Strictly adhere to a feature branch workflow using Git for all development work, creating clear and well-documented pull requests for peer review.
- Independently and confidently handle Git operations, including branching, merging, rebasing, and resolving merge conflicts.
- Maintain a clean and logical commit history that clearly communicates the evolution of the codebase.
19. Technical Debt
Skill Description: Identifies and prioritizes the reduction of technical debt in the analytics codebase.
Key Responsibilities & Expectations:
- Proactively identify sources of technical debt within the analytics codebase and articulate the business impact of not addressing them.
- Make a compelling case to stakeholders and product managers to prioritize refactoring and tech debt reduction work in development sprints.
- When tactical shortcuts are necessary to meet deadlines, take responsibility for documenting this debt in a backlog (e.g., Jira) with a clear plan for future resolution.