Data Software Engineer

Natural Intelligence is a global leader in the online comparison market. Our goal is to simplify decision-making and to connect customers with valuable brands.

Our technology helps millions of consumers around the world make smarter, more confident buying decisions. We do so by creating trusted comparison websites that enable consumers to easily compare products and services online and form high-value relationships with the brands they choose. Our partner brands, some of which are among the most prominent websites worldwide, utilize our platform to maximize their online presence. 

We are looking for a strong and passionate big data developer that will help us build the data platform and data frameworks for the company, to support analytics & reports, machine learning algorithms and other data needs.


Responsibilities

  • Use your skills and experience to design and develop our innovative big data platform
  • Use big data frameworks such as Spark, Kafka, Redshift, Airflow, Flink, k8s, Hudi and so on to develop large scale data pipelines
  • Lead, deliver and own high scale projects end to end
  • Work closely with the data science team 
  • Lead technological initiatives and improvements while instilling development of “best practices”
  • Develop processes and tools to monitor, analyze, maintain and improve data operation, performance, and usability

Requirements

  • At least 2 years experience in Scala/Java or equivalent language 
  • Proven experience in building, optimizing, and maintaining large scale big data pipelines using popular open-source frameworks (Kafka, Spark, Hive, Presto, Airflow, etc)
  • Strong SQL abilities and hands on experience with SQL and NoSql DBs
  • Experience with AWS cloud services such as EMR, Redshift, S3, Athena, Glue - advantage
  • experience in Python 
  • Experience in stream processing frameworks such as Flink
  • Experience with scientific computing packages (numpy, scipy, pandas etc.) - advantage
Submit Application

Apply for this position

Data Software Engineer