Data Engineer - Scotiabank, Toronto

AI Summary

Scotiabank is hiring a Data Engineer in Toronto, Ontario, Canada. This role focuses on building high-performance, scalable data pipelines and processing large datasets in a distributed environment.

Key duties include developing production-grade code using Apache Spark on Azure Databricks. Top requirements include deep expertise in Apache Spark, strong programming skills in Python, Scala, or Java, and at least three years of experience as a Data Engineer.

About the Role

Join a purpose-driven, winning team at Scotiabank, committed to results within an inclusive and high-performing culture. We are seeking a hands-on Data Engineer with deep expertise in Apache Spark and strong programming skills in Python, Scala, and Java. This critical role is centred on building high-performance, scalable data pipelines and efficiently processing large datasets within a distributed environment.

You will primarily work on Spark-based data processing running on Azure Databricks, developing production-grade code that supports enterprise analytics, reporting, and various data products. This is an engineering-heavy position for someone who enjoys writing clean, efficient code and optimizing distributed workloads. Responsibilities include designing, developing, and maintaining large-scale Spark applications, building and optimizing both batch and streaming data pipelines, and ensuring production-quality Spark code with a strong focus on performance, scalability, and reliability. Your experience will be crucial in optimizing Spark jobs through partitioning, caching, and memory tuning, as well as developing reusable Spark frameworks and utilities. Collaboration with platform, analytics, and data science teams will be essential to support downstream use cases, alongside debugging and troubleshooting Spark job failures in production while adhering to best practices for code quality, testing, and documentation.

Skills and Experience

To succeed in this Data Engineer role, candidates should possess a Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience. A minimum of three years of experience as a Data Engineer or in a similar capacity is required, demonstrating a proven track record.

We are looking for individuals with strong hands-on experience with Apache Spark and proficiency in one or more of the following languages: Python, Scala, or Java. A strong understanding of distributed systems and data processing concepts is fundamental, alongside solid experience with SQL for data transformation and analysis. Familiarity with data lakes and columnar formats such as Parquet and Delta Lake is also important. Candidates should have familiarity with Git and standard software engineering practices, coupled with strong problem-solving skills and meticulous attention to detail. Experience running Spark on Azure Databricks, migrating Spark jobs from on-prem Hadoop/Cloudera to cloud platforms, familiarity with orchestration tools like Airflow or Azure Data Factory, and exposure to cloud storage (ADLS Gen2) and cloud security concepts are considered valuable assets.

Why Join Scotiabank

Scotiabank is committed to fostering an inclusive culture where every employee is empowered to reach their fullest potential, respected for who they are, and embraced through bias-free practices and inclusive values. We champion diversity and provide opportunities for all employees to learn, grow, and participate through various Employee Resource Groups that span diverse gender identities, ethnicity, race, age, ability, and veterans.

We value the unique skills and experiences each individual brings to the Bank and are dedicated to creating and maintaining an inclusive and accessible environment for everyone. Scotiabank continuously works to remove and prevent barriers, building a diverse and inclusive workplace while meeting accessibility requirements. Employees benefit from upskilling opportunities through online courses, cross-functional development, and tuition assistance. Our competitive Rewards program includes a bonus, flexible vacation, personal and sick days, with benefits commencing on day one. Enjoy a dynamic ecosystem with amenities like free tea and coffee, universal washrooms, and ample space for team collaboration. We also offer opportunities for community engagement and belonging through various programs, no matter where you choose to work from.