Data Engineer – System Limited – Pakistan
System Limited invites applications for Data Engineer in Multiple cities of Pakistan
Job Title:
Data Engineer
The ideal candidate will have a strong background in data engineering, with expertise in building scalable and efficient data pipelines. The primary responsibilities will include designing, developing, and maintaining data processing systems using technologies such as NIFI, Airflow, Python, and Spark. Additionally, experience with AWS services (S3, Glue, Athena, EMR, EC2, Lambda), Hive, Redshift, Databricks, Jenkins, and version control systems like GitHub/Bitbucket is highly desirable.
Responsibilities:
- Design, develop, and maintain scalable data pipelines using NIFI, Airflow, Python, and Spark.
- Implement data processing solutions to extract, transform, and load (ETL) large volumes of data from various sources.
- Collaborate with cross-functional teams to understand data requirements and develop efficient solutions.
- Optimize and tune data pipelines for performance, scalability, and reliability.
- Implement best practices for data quality, data governance, and data security.
- Work closely with data scientists and analysts to support their data requirements and use cases.
- Monitor and troubleshoot data pipeline issues to ensure continuous data availability and integrity.
- Stay updated with the latest technologies and trends in the field of data engineering.
Requirements:
- Bachelor’s degree in Computer Science, Engineering, or related field.
- Proven experience as a Data Engineer or similar role.
- Strong proficiency in programming languages such as Python and Scala.
- Hands-on experience with data processing frameworks like Apache Spark.
- Experience with workflow management tools such as Apache NIFI and Apache Airflow.
- Solid understanding of cloud platforms, especially AWS services (S3, Glue, Athena, EMR, EC2, Lambda).
- Familiarity with data warehouse technologies like Redshift and Hive.
- Experience with Databricks for big data analytics and machine learning is a plus.
- Proficiency in continuous integration/continuous deployment (CI/CD) tools like Jenkins.
- Familiarity with version control systems such as GitHub or Bitbucket.
- Strong problem-solving skills and attention to detail.
- Excellent communication and collaboration skills.
