Role OverviewDesign and implement distributed data processing pipelines using Spark, Hive, Sqoop, Python, and other tools and languages prevalent in the Hadoop ecosystem. Build utilities, user defined functions, and frameworks to better enable data flow patterns. Research, evaluate and utilize new technologies/tools/frameworks centered around Hadoop and other elements in the Big Data space.
What You Will Do
Work with teams to resolving operational & performance issues, and work with architecture/engineering leads and other teams to ensure quality solutions are implemented, and engineering best practices are defined and adhered to.
Why It Might Be a Fit
Strong development skills around Hadoop, Spark, MapReduce, Hive, and experience with performance/scalability tuning, algorithms and computational complexity.
Requirements
- MS/BS degree in a computer science field or related discipline
- 6+ years’ experience in large-scale software development
- 2+ year experience in Hadoop
- 2+ year experience in Data Science
- Strong development skills around Hadoop, Spark, MapReduce, Hive
- Strong Java programming, Python, shell scripting, and SQL
- Good understanding of file formats including Parquet, Avro, JSON and others
- Good understanding of R, TensorFlow, SAS or similar
- Experience with performance/scalability tuning, algorithms and computational complexity
- Experience (at least familiarity) with data warehousing, dimensional modeling and ETL development
- Proven ability to work cross functional teams to deliver appropriate resolution
]]>