Computer Programmer III, Data Engineer - City of Vancouver

AI Summary

The City of Vancouver is hiring a Computer Programmer III, Data Engineer in Vancouver. This role focuses on advanced data engineering, including data ingestion, data lake and data warehouse design, and building robust data pipelines. Key duties involve complex programming and analysis, leading data development standards, and providing technical solutions. Top requirements include extensive experience with ETL/ELT tools, SQL environments, and DataOps methodologies.

About the Opportunity

Join the City of Vancouver, recognized among BC’s Top Employers, as a Computer Programmer III – Data Engineer. This pivotal role involves performing critical data engineering work to support capital and long-term technology project implementations. The incumbent will apply strong knowledge and skills relating to data ingestion, data storage, processing, modelling, data lake and data warehouse design, and implementation. A core responsibility is building efficient data pipelines, ensuring seamless data integration, rigorous data testing, and effective data orchestration. This position requires performing complex programming and analysis work, alongside configuring, adapting, and altering data structures, stores, and pipelines across multiple applications built in various development languages to meet intricate requirements. The Data Engineer will also take on a project leader capacity, supervising other data specialists and engineers, incorporating best technology solutions, and demonstrating leadership in the development of new data integration and ETL/ELT programming and development standards. Collaboration with Computer Programmer/Analyst staff and other team members is essential to achieve the overall objectives of the City and Technology Services. The Programmer shares valuable knowledge and experience of computer techniques to provide innovative solutions to technical and procedural problems, exercising considerable independence of judgement in all technical aspects of the work.

Key Responsibilities and Technical Expertise

As an expert technical resource, this role is crucial for data ingestion and pipelines, including ETL/ELT development, data modelling, and preparing data for analysis from diverse platforms. The Data Engineer will analyze highly complex business requirements to suggest robust solutions that meet both functional and non-functional requirements, considering cost, security, and existing investments. Generating technical specifications to design or redesign data warehouse, data lake, and other enterprise components associated with data delivery is a key duty. Leveraging industry best practices, the role involves designing, testing, implementing, and supporting these critical data solutions, while assuring quality, security, and compliance requirements are consistently met. The development of ETL/ELT within project contexts, ensuring alignment with current and evolving architecture, is paramount. This includes preparing comprehensive Technical Design and Specification documents, creating implementation instructions, and supporting deployments. The position also involves conducting design and peer reviews, individual and team unit testing, and overseeing technical issues and the work of other developers. Participation in the development of architectural framework and design, evaluating and introducing new tools and techniques to the development team, and proactively identifying areas for improvement are also expected. Consultation with managerial, engineering, business, and technical personnel to clarify program intent and requirements, identify problems, and suggest changes is a regular part of this role. The incumbent will perform or direct revision, repair, or expansion of existing IT systems to increase operating efficiency or adapt to new requirements, and assist in the training and development of other team members.