Big Data Developer

24 May 2023
21 Jun 2023
Orlaith O'Mahony
Job Type

Data Ingestion Engineer

This company is the leading provider of predictive media intelligence, tracking and predicting engagement with the world's news stories each day. Their platform is used each day by journalists at hundreds of top tier publications, and communications professionals at all 10 of the top 10 global PR agencies, dozens of international brands, and NGOs, international organisations and not-for-profits.

As an Irish company, their product and engineering functions are central to their business. This presents a great opportunity to directly input into what they build and how they build it. There is a direct link between your work, their customers and the continued success of this company.

As a Backend Engineer on their Sources team you will play a pivotal role in how the company ingests, transforms and enriches substantial volumes of data describing how all media and content is consumed and interacted with throughout the Internet. If building and maintaining systems that deal with the realtime ingestion and processing of many billions of events per day excites you - this may be the role for you.


  • Maintain, extend and scale a distributed platform that can seamlessly scale to handle billions of ingested events per day
  • Build and populate data pipelines for collection, storage, processing, and analysis of data
  • Build integrations with external APIs, connectors and services for collection of social media/news content and the subsequent scoring and enrichment of this contentAdvocate for improvements to data quality, security, and performance
  • Solve technical problems of moderate scope and complexity.
  • Recognize impediments to our efficiency as a team ("technical debt"), propose and implement solutions


  • 3+ years in software development
  • Knowledge of software design principles and leading software development practices
  • Strong communication & collaboration skills
  • Willingness to get things done, learn new things, take initiative and challenge existing assumptions and conventions
  • Experience building and deploying to any cloud service (GCP, AWS, Azure, etc.)
  • Experience building and operating distributed systems at scale
  • Deep experience of at least one modern programming language such as Java, Scala, Elixer etc.

Nice to have:

  • Experience with high volume event streaming / queueing systems (Kafka, Kinesis, RabbitMQ etc.)
  • Experience working with schedulers in a distributed, service oriented environment (Airflow, Step Functions, Argo etc.)
  • Experience with observability principals (Instrumentation, Tracing, Telemetry)
  • Experience in building real-time integrations with a variety of external APIs, connectors and services (Restful/Streaming integrations, disparate authentication mechanisms, rate limit considerations, etc.)
  • Experience with IaC and DevOps methodologies
  • Knowledge of basic Linux administration, Kubernetes and Google Cloud Platform
  • Experience working in an agile environment with iterative development and fast feedback


  • Competitive salary
  • Health insurance
  • Bonus for individual and team performance
  • Pension
  • Great working environment - remote first or hybrid model
  • An opportunity to help define an entirely new industry category
  • Excellent opportunity to grow in one of Dublin's fastest growing home-grown companies

Their ethos:

This company believes in maintaining a friendly work environment, a healthy work-life balance, and compensating their employees fairly for their input. You'll be part of a team that believes in mutual support and education, and for a company where a work week isn't just the gap between weekends, but an opportunity to do work that is impactful and innovative. The team also love eating and socializing together, annual and seasonal company and team retreats, healthy and unhealthy snacks and other perks.

Apply for Big Data Developer

Already uploaded your CV? Sign in to apply instantly


Upload from your computer

Or import from cloud storage

Your CV must be a .doc, .pdf, .docx, .rtf, and no bigger than 1MB

4000 characters left

Marketing Communication

We would like to send you electronic communications regarding employment related opportunities, services and information that we consider suitable for you.

I would like to receive communications.

I would like you to share my personal data with other Companies within the ManpowerGroup.

You will receive an email containing the responses you have provided today with a link that will allow you to update your consent preferences at any time in the future. You can view a list of ManpowerGroup companies by clicking here.

When you apply for a job we will send your application to the named recruiter, who may contact you. By applying for a job listed on you agree to our terms and conditions and privacy policy. You should never be required to provide bank account details. If you are, please email us.

More searches like this