Senior Data Engineer

2 days ago

Apply Now

Receive Emails with Similar Jobs

Catalyst

WebsiteLinkedInAll Job Openings

Catalyst (www.catalystinc.com) is a marketing agency that helps clients develop more profitable customer relationships$1. .$1

CRM • Database Development • Analytics • Omnichannel Marketing • Strategic Planning

51 - 200

Description

• Drive high impact, cross-functional data engineering projects built on top of a modern, best-in-class data stack, working with a variety of open source and Cloud technologies • Solve interesting and unique data problems at high volume and large scale • Build and optimize the performance of batch, stream, and queue-based solutions including Kafka and Apache Spark • Collaborate with stakeholders from different teams to drive forward the data roadmap • Implement data retention, security and governance standards • Work with all engineering teams to help drive best practices for ownership and self-serve data processing • Support and expand standards, guidelines, tooling and best practices for data engineering at Catalyst • Support other data engineers in delivering our critical pipelines • Focus on data quality, cost effective scalability, and distributed system reliability and establish automated mechanisms • Work cross functionally with application engineers, SRE, product, data analysts, data scientists, or ML engineers

Requirements

• 3+ years of experience successfully implementing modern data architectures • Strong Project Management skills • Demonstrated experience implementing ETL pipelines with Spark (we use Pyspark) • Proficiency in Python, SQL and/or other modern programming language • Deep understanding of SQL/New SQL with relational data stores such as Postgres/MySQL • A strong desire to show ownership of problems you identify • Experience with modern Data Warehouses and Lakes such as Redshift, Snowflake, and Databricks Delta Lake • Experience with distributed streaming tools like Kafka and Spark Structured Streaming • Familiarity with an orchestration tool such as Airflow, dbt, and Delta Live tables • Experience with automated testing for distributed systems (unit testing, E2E testing, QA, data expectation monitoring) • Experience working with application engineers, product, and data scientists • Experience with leveraging caching for performance using data stores such as Redis and ElasticSearch • Experience with maintaining and scaling heterogeneous and large volumes of data in production • Practical experience with DevOps best practices (CICD, IAC) is a plus • Familiarity with Change Data Capture systems is a nice to have

Benefits

• Highly competitive compensation package, including equity - everyone has a stake in our growth • Comprehensive benefits, including up to 100% paid medical, dental, & vision insurance coverage for you & your loved ones • Unlimited PTO policy encouraging you to take the time you need - we trust you to strike the right work/life balance • Monthly Mental Health Days and Mental Health Weeks twice per year