Data Engineer 墨尔本悉尼布里斯班

You’ll spend time on the following:

• You will partner with teammates to create complex data processing pipelines in order to solve our clients’ most ambitious challenges
• You will collaborate with Data Scientists in order to design scalable implementations of their models
• You will pair to write clean and iterative code based on TDD
• Leverage various continuous delivery practices to deploy, support and operate data pipelines
• Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available
• Develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions
• Create data models and speak to the tradeoffs of different modelling approaches
• Seamlessly incorporate data quality into your day-to-day work as well as into the delivery process 

Here’s what we’re looking for:

• You have a good understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop
• You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting
• Hands on experience in MapR, Cloudera, Hortonworks and/or cloud (AWS EMR, Azure HDInsights, Qubole etc.) based Hadoop distributions
• You are comfortable taking data-driven approaches and applying data security strategy to solve business problems 
• You’re genuinely excited about data infrastructure and operations with a familiarity working in cloud environments
• Working with data excites you: you can build and operate data pipelines, and maintain data storage, all within distributed systems
• Assure effective collaboration between ThoughtWorks’ and the client’s teams, encouraging open communication and advocating for shared outcomes

工作内推 微信 18627123156

本地公司

Consulting

With our roots in custom systems and agile software delivery, we’re at the forefront of defining the tech principles used by some of the world’s most successful organizations. We have invested in organic growth, building on the strategy, design, data and engineering capabilities required to bring a truly integrated approach to solving our clients toughest challenges. Our collaborative, cross-functional teams deliver real results, fast.