Senior Data Engineer

Confidential

Multiple Cities, Pakistan

Apply Now

Posted 2 days ago 272 views Report Job

PKR. 400,000 - 600,000/Month

Responsibilities:

Responsible for the design, deployment, and maintenance of the business’s data platforms
Implement strategies directed at acquiring data and promoting the development of new insights across the business
Owns and extends the business’s data pipeline through the collection, storage, processing, and transformation of large datasets.
Work with senior data science management and departments beyond the Data and Analytics department in analyzing and understanding data sources
Participating in design and providing insights and guidance on database technology and data modeling best practices
Develops and manages scalable data processing platforms that he uses for exploratory data analysis and real-time analytics
Builds a metadata system where all available data is maintained and cataloged
Developing ETL processes that convert data into formats through a team of data analysts and dashboard charts
Process unstructured data, clean data, extract features from labeled/unlabeled data
Prepare data for training machine learning models
Retrieves and analyzes data using SQL, and Excel, among other data management systems
Builds data loading services for the purpose of importing data from numerous disparate data sources, inclusive of APIs, logs, relational, and non-relational databases
Continual improvement of the business’s data platforms through his observations and well- researched knowledge
Monitor the existing metrics, analyze data, and lead partnerships with other Data and Analytics teams to identify and implement system and process improvements
Development of reliable data pipelines that translate raw data into powerful features and signals
Designs, architects, implements, and supports key datasets that avail structured and timely access to actionable business insights
Overseeing activities of the junior data engineering teams, ensuring proper execution of their duties and alignment with business vision and objectives
Performs similar duties as he deems fit for the proper execution of his duties and duties as delegated by the Head of Data Science

Requirements:

Bachelor’s in Computer Science, Computer Engineering, or a related field
05 years of working experience working in a data engineering department, preferably as a Data Engineer in a fast-paced environment and complex business setting
Demonstrated experience in building and maintaining reliable and scalable ETL on big data platforms as well as experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop and Spark
Must also have had experience in data warehousing inclusive of dimensional modelling concepts and demonstrate proficiency in scripting languages, for example, Python, Perl, and so forth
Demonstrate machine learning experience and experience with big data infrastructure including MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc.,
Deep knowledge of data mining techniques, and relational, and non-relational databases
Oversees and manages junior data engineering teams and to ensure effective management, he must be capable of conveying information and instructions clearly down the line to the junior team
Sound understanding of building complex ETL pipelines using either open-source tools such as Luigi and airflow or cloud-based solutions
Highly proficient in the use of MS Office products (Word, Excel, PowerPoint).
Exceptional skills in SQL server reporting services, analysis services, Tableau, Salesforce, integration services, or any other data visualization tools
Ability to perform complex data analyses with large data volumes
Strong knowledge of Linux, OS tools, and file-system level troubleshooting
Substantial experience working with big data infrastructure tools such as Python, SQS, and Redshift
Experience with Kafka and Apache superset
Azure Cloud Applications and Services
A suitable candidate will also be proficient in Scala, Spark, Spark Streaming, Azure, AWS
Result-driven individual, passionate and a self-starter, proactive requiring minimal supervision

Working Hours:

Eastern Standard Time (EST)