Description

Big Data Developer – Python/Pyspark, Hadoop/Hive

On behalf of our client in the Banking Sector, PROCOM is looking for a Big Data Developer – Python/Pyspark, Hadoop/Hive.

Big Data Developer – Python/Pyspark, Hadoop/Hive – Job Description

  • Responsible for overall design development coding testing and implementation of automated ETL program from scratch
  • Experience with querying and analyzing large enterprise data sets, drawing insights and inferences, and communicating output
  • Supports project through the documentation, monitoring and version control
  • Highly autonomous and productive in performing activities, requiring only minimal direction from or interaction with reporting manager
  • Keeps stakeholders informed through effective verbal and written communication
  • Develops code (writes software application programs)
  • Develops and codes programs to integrate multiple subsystems
  • Identifies and recommends system enhancements to improve or expand customer services by anticipating customer problems and recommending solutions
  • Plans, schedules, and coordinates project design and implementation teams
  • Assists and trains others on technical or industry-related issues. Evaluates hardware or software product alternatives and makes recommendations
  • A vacancy exists for a senior developer to join the team in the delivery and development of this project and to eventually contribute to future enhancement, support and maintenance post-delivery
  • Prime responsibility will be to develop detailed design specifications, create and review code and assess impact on other applications
  • Role will work closely with other team members to ensure consistency in coding standards and approaches and to optimize any synergy and re-use opportunities
  • Responsible for leading technical aspects of project from inception through to deployment
  • Participate in analysis and system design specification sessions
  • Documentation of design options and solutions according to System Development Life Cycle (SDLC) methodology
  • Ensure that solutions adhere to development best practices and architectural principles

Big Data Developer – Python/Pyspark, Hadoop/Hive – Mandatory Skills

  • Experience with coding, development and implementation of ETL data engineering solutions and processes utilizing Python and PySpark.
  • 3+ experience in automation and development using PySpark, Python and 5 years with Hadoop, Hive and Impala. Advanced SQL knowledge is a key requisite.
  • Advanced skills in querying, joining, manipulating and summarizing datasets of various types
  • Experience supporting project through the documentation, monitoring and version control
  • Experience in automation and development using PySpark, Python: 3 years
  • Hadoop, Hive (programming language) and Impala: 5 years
  • 5+ years of Advanced SQL knowledge is a key requisite

Big Data Developer – Python/Pyspark, Hadoop/Hive – Nice to Have Skills

  • Previous Banking or FI experience

Big Data Developer – Python/Pyspark, Hadoop/Hive – Assignment Start Date

ASAP – 6 months to start

Big Data Developer – Python/Pyspark, Hadoop/Hive – Assignment Location

Toronto, ON -Work Remotely

Tagged as: Data Engineering, Hardware

Source: