Responsibilities
- Collaborate with Data Architects and Data Integration Engineers to enhance and maintain data pipeline architectures in compliance with established standards.
- Build and manage medium to complex data sets that satisfy functional and non-functional business needs.
- Design and execute internal process enhancements, including automating manual workflows, improving data delivery efficiency, and restructuring infrastructures for better scalability.
- Develop infrastructure for optimal extraction, transformation, and loading (ETL) of data from diverse sources using big data technologies.
- Create analytics tools that leverage data pipelines to deliver actionable insights related to customer acquisition, operational efficiency, and other key business performance indicators.
- Support stakeholders, including domain leads and teams, in addressing data-related technical challenges and fulfilling data infrastructure requirements.
- Ensure compliance with data security protocols, including encryption, obfuscation, and role-based access controls.
- Develop data tools to assist analytics and data science teams.
Functional Competencies
- Comprehensive knowledge of data and analytics frameworks, including data lakes, warehouses, marts, and reporting systems.
- Ability to define data retention policies, monitor performance, and recommend necessary infrastructure changes based on functional and non-functional requirements.
- Advanced expertise in the data engineering domain.
- Extensive experience utilizing big data tools and developing data solutions for advanced analytics.
- Over five years of hands-on experience with a solid foundation in data engineering.
- Proficient programming skills in Java, Python, and SQL.
- Demonstrated experience with database systems, including the Hadoop ecosystem, cloud platforms (AWS, Azure, Google Cloud), in-memory databases (e.g., HANA, Hazelcast), traditional RDBMS (e.g., Teradata, SQL Server, Oracle), and NoSQL databases (e.g., Cassandra, MongoDB, DynamoDB).
- Practical knowledge of data extraction and transformation tools, encompassing traditional ETL solutions (e.g., Informatica, Ab Initio, Alteryx) and modern big data tools.
Qualifications
- Background in programming, databases, and/or big data technologies, OR
- Bachelor's or Master's degree in software engineering, computer science, economics, or a related engineering field.
Desired Skills and Experience
Data Integration, Data Pipeline Development, ETL (Extract, Transform, Load), Big Data Technologies, Data Warehousing, Data Lakes, Cloud Data Solutions, Data Migration, SQL Programming, Data Modeling, Programming (Java, Python, Scala, R), Database Management (Hadoop, Teradata, Oracle, SQL Server), Cloud Platforms (AWS, Azure, Google Cloud), NoSQL Databases (MongoDB, Cassandra, DynamoDB), In-Memory Databases (SAP HANA, Hazelcast), Data Visualization Tools (Tableau, Power BI), API Development and Integration, Version Control (Git, SVN, Informatica Intelligent Cloud Services (IICS), Apache Kafka, Spark, Airflow, Alteryx, Snowflake, Databricks, AWS Redshift, Azure Data Factory, problem-Solving, Team Collaboration, Communication (Fluent English), Stakeholder Management, Analytical Thinking, Adaptability, Agile Methodologies (Scrum, Kanban), JIRA/Confluence, Requirement Gathering, Process Optimization, AWS Certified Data Analytics, Microsoft Azure Data Engineer Associate, Google Professional Data Engineer, Certified Informatica Developer
Argyll Scott Asia is acting as an Employment Agency in relation to this vacancy.
