Data Engineer (Manager to Associate Director)
25-10-2023
Job Type
Permanent
Emp Type
Full Time
Salary Type
Annual
Salary
Negotiable
Job ID
32177
Job Description
As a Data Engineer, your role will involve constructing a high-quality data solution that aligns with clients' objectives.
You will evaluate and enhance the current processes of clients, developing and testing architectures like databases and large-scale processing solutions. Additionally, you will create ETLs/ELTs to handle data from different sources and formats, contribute to the selection of data solutions and development tools, and establish API connections using various languages and tools.
Responsibilities:
- Build enterprise-grade data solutions to execute clients' vision
- Understand and improve clients' current state processes
- Develop, construct, and test architectures like databases and large-scale processing solutions
- Create ETLs/ELTs to handle data from various sources and formats
- Advise on data solutions and development tools selection
- Develop APIs and use scripting languages to integrate systems
- Build transformation and validation code for complex data aggregation and calculation
- Document technical specifications for solution requirements
- Plan, design, and lead the implementation of a large-scale data platform
- Collaborate with technical architects and application designers on data conversion requirements and procedures
- Analyze and integrate new data sources into existing pipelines and models
- Develop go-to-market strategies and expand team capabilities
Requirements:
- Bachelor's degree (or higher) in mathematics, statistics, computer science, engineering, or related field
- At least 3 years of experience with data pipeline tools such as Azure Data Factory, Airflow, Informatica, or Databricks
- Hands-on programming skills in SQL, Java, Python, C++, Scala, SAS, or Kafka
- Practical experience with implementing large-scale enterprise data solutions like data lakes, data warehouses, or data mesh
- Deep understanding of cloud computing and data technologies (Azure, AWS, Google Cloud, Alibaba Cloud, Tencent Cloud)
- Familiarity with Agile and DevOps methodologies
- Knowledge of emerging technologies (Big Data, Blockchain, Internet of Things, Artificial Intelligence) is a plus
- Strong problem-solving skills and logical thinking
- Excellent multitasking and task management abilities
- Confident in decision-making and explaining processes
- Excellent communication and interpersonal skills
- Proficient in spoken and written English and Chinese.