Job Category: Information Technology

Job Source:

Job Summary

  • No. of Vacancy: N/A
  • Job Location: Melbourne, Australia
  • Employment Type: Full Time
  • Salary: N/A
  • Gender: Any
  • Age Limit: N/A
  • Experience: N/A
  • Career level: Mid-Senior level
  • Posted On: Nov 8, 2018
  • Application Deadline: Dec 8, 2018

Job Description/Responsibility

  • About the role 
    The Big Data Test Engineer will expand and optimise our clients’ data and data pipeline architecture, as well as optimise their data flow and collection for cross functional teams.
  • Responsibilities include: 

    • End-to-end testing of the Hadoop data lake solution including functional, system integration, regression, reconciliation and validation testing 
    • Write and execute simple and complex test cases in Big Data implementation 
    • Build a robust and efficient test framework for testing the ingestion, processing, transformation, storage and consumption layers in the Big Data architecture 
    • Assemble large, complex test data sets that meet functional / non-functional testing requirements 
    • Escalate key testing issues and provide feedback to the development team and other stakeholders 
    • Work with stakeholders including product owner, data engineers, architect and design teams to provide inputs to the testing processes in SIT and UAT in line with industry best practices. 

    About the Candidate 
    They will need the ability to optimise data systems and build them from the ground up. They will support software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. 

Experience Requirements

  • N/A

Education Requirements

  • N/A

Skills Requirements

  • • Python, Java, Spark, NiFi, Kafka, Hbase, Hive 
    • Proven working experience as Big Data engineer or tester for 2+ years. 
    • Experience in test driven Big Data development and test automation. 
    • Experience in testing the streaming technologies like NiFi, Kafka, etc. with industry standard Big data testing tools. 
    • Ability to write and execute simple and complex test cases in Big Data implementation. 
    • Experience in working on Big data technologies and concepts such as HDFS, Hive, NiFi, Spark, Spark streaming and NoSQL DB like HBase etc. 
    • Ability to interpret the development code and transformation logic to form valid test cases. 
    • Experience in one or more of Java, Scala, Python and Shell Scripting. 
    • Ability to work in team in diverse/ multiple stakeholder environment 
    • Experience in working in a fast-paced Agile environment 
    • BS in Computer Science, Statistics, Informatics, Information Systems or another quantitative field 
    Preferable skills and experience 
    • Experience in building testing framework for enterprise data lake is highly desirable 
    • Hands on development in the areas of building data pipelines in NiFi, Kafka, Spark Streaming, etc. 
    • Knowledge of CI/CD pipeline. 


  • • Formal training with industry recognized certifications 
    • The ability to interact with peers in a sharing and inclusive community 
    • Exciting and challenging projects 
    • A well-structured and tailored career framework 
    • A culture of collaboration and recognition 
    • Excellent remuneration 

Apply Instruction

This is a great role, where you will work alongside highly regarded consultants in a busy project environment.
If the above sounds like you and you are ready to commence your next challenge, please apply now or send your CV directly to or call at  +61 283107040 for a detailed discussion

About the Company

Company Name: IT Alliance Australia
Address: Level 57, 19 Martin Place Sydney , NSW 2000, , AUSTRALIA

Company Profile: IT Alliance Australia is an Australian company having operations in Australia and Asia. We provide IT Recruitment, Resourcing and Training services. Our clients are Medium and Large Private & Government organizations As today’s business environment is increasingly competitive, with huge pressure to provide agile, cost efficient and effective solutions, which is why the team here at IT Alliance Australia deliver the skills, technology and professional expertise that can help any business meet that challenge and succeed. Our team brings together vast experience and innovative thinking to provide IT Resourcing, IT Recruitment and IT Training services that deliver exceptional results.

Company Website:

Simillar Jobs

Personal Details

Documents for this application

Files can be up to 2MB for file types.
.pdf .doc .docx .rtf .txt

Cover Letter


All personal information submitted by you as part of an application will be used by us in accordance with our privacy statement

Upcomming Courses

More Courses


More Article