Databricks Data Engineer

Inizio Partners Corp
New York, NY
Category Data Analyst
Job Description
Our client is seeking a Databricks Data Engineer to lead and/or assist in designing and developing data systems, tailoring solutions to meet client-specific requirements. Responsibilities include designing and implementing databricks-based solutions, engaging with clients to evaluate their current and future needs, and developing comprehensive architecture solution roadmaps.

Requirements

  • 5+ years experience in architecture, design, and implementation using Databricks
  • Experience in designing and implementing scalable, fault-tolerant systems
  • Deep understanding of one or more of the big data computing technologies such as Databricks, snowflake
  • In-depth knowledge of spark internals, catalyst optimization, and Databricks runtime environment
  • Must have experience in implementing solutions using Databricks
  • Experience in Insurance (P&C) is good to have
  • Programming Languages SQL, Python
  • Technologies Databricks, Delta Lake storage, Spark (PySpark, Spark SQL)
  • Good to have - Airflow, Splunk, Kubernetes, Power BI, Git, Azure DevOps
  • Project Management using Agile, Scrum
  • B.S. Degree in a data-centric field (Mathematics, Economics, Computer Science, Engineering, or other science field), Information Systems, Information Processing, or engineering
  • Excellent communication & leadership skills, with the ability to lead and motivate team members
  • Ability to work independently with some level of ambiguity and juggle multiple demands

Benefits

  • Generous Paid Time Off
  • 401k Matching
  • Retirement Plan
]]>