We are looking for a talented Data Engineer to design, build, and maintain ETL/ELT pipelines that consolidate data from more than 30 agency sources into a centralized Data Lake. The successful candidate will play a critical role in creating and managing data schemas, ingestion connectors, and streaming feed integrations to ensure smooth and reliable data flow. A strong emphasis will be placed on maintaining high data quality through validation, cleaning, and ensuring data availability for various analytics platforms. Additionally, the role involves monitoring pipeline performance and optimizing storage, processing, and data access to support efficient and scalable data operations.
Key Responsibilities:
- Develop, deploy, and maintain ETL/ELT pipelines that integrate multiple agency data sources into the Data Lake.
- Design and manage data schemas, ingestion connectors, and streaming feed integrations to facilitate seamless data ingestion.
- Ensure data quality by performing validation, cleaning, and maintaining consistent availability for analytics platforms.
- Monitor the performance of data pipelines and implement improvements to optimize storage, processing efficiency, and data accessibility.
Required Qualifications:
- Master’s degree in Computer Science, Data Engineering, Information Technology, or a related discipline, equivalent to 16 years of formal education.
- Proficiency with ETL tools and technologies essential for pipeline development and maintenance.
- Strong programming skills in SQL and Python or comparable languages to support data manipulation and automation tasks.
There are no additional preferred qualifications or benefits specified for this position.