Big Data Developer

Data Ingestion Engineer

This company is the leading provider of predictive media intelligence, tracking and predicting engagement with the world's news stories each day. Their platform is used each day by journalists at hundreds of top tier publications, and communications professionals at all 10 of the top 10 global PR agencies, dozens of international brands, and NGOs, international organisations and not-for-profits.

As an Irish company, their product and engineering functions are central to their business. This presents a great opportunity to directly input into what they build and how they build it. There is a direct link between your work, their customers and the continued success of this company.

As a Backend Engineer on their Sources team you will play a pivotal role in how the company ingests, transforms and enriches substantial volumes of data describing how all media and content is consumed and interacted with throughout the Internet. If building and maintaining systems that deal with the realtime ingestion and processing of many billions of events per day excites you - this may be the role for you.

Responsibilities:

• Maintain, extend and scale a distributed platform that can seamlessly scale to handle billions of ingested events per day
• Build and populate data pipelines for collection, storage, processing, and analysis of data
• Build integrations with external APIs, connectors and services for collection of social media/news content and the subsequent scoring and enrichment of this contentAdvocate for improvements to data quality, security, and performance
• Solve technical problems of moderate scope and complexity.
• Recognize impediments to our efficiency as a team ("technical debt"), propose and implement solutions

Requirements:

• 3+ years in software development
• Knowledge of software design principles and leading software development practices
• Strong communication & collaboration skills
• Willingness to get things done, learn new things, take initiative and challenge existing assumptions and conventions
• Experience building and deploying to any cloud service (GCP, AWS, Azure, etc.)
• Experience building and operating distributed systems at scale
• Deep experience of at least one modern programming language such as Java, Scala, Elixer etc.

Nice to have:

• Experience with high volume event streaming / queueing systems (Kafka, Kinesis, RabbitMQ etc.)
• Experience working with schedulers in a distributed, service oriented environment (Airflow, Step Functions, Argo etc.)
• Experience with observability principals (Instrumentation, Tracing, Telemetry)
• Experience in building real-time integrations with a variety of external APIs, connectors and services (Restful/Streaming integrations, disparate authentication mechanisms, rate limit considerations, etc.)
• Experience with IaC and DevOps methodologies
• Knowledge of basic Linux administration, Kubernetes and Google Cloud Platform
• Experience working in an agile environment with iterative development and fast feedback

Benefits:

• Competitive salary
• Health insurance
• Bonus for individual and team performance
• Pension
• Great working environment - remote first or hybrid model
• An opportunity to help define an entirely new industry category
• Excellent opportunity to grow in one of Dublin's fastest growing home-grown companies

Their ethos:

This company believes in maintaining a friendly work environment, a healthy work-life balance, and compensating their employees fairly for their input. You'll be part of a team that believes in mutual support and education, and for a company where a work week isn't just the gap between weekends, but an opportunity to do work that is impactful and innovative. The team also love eating and socializing together, annual and seasonal company and team retreats, healthy and unhealthy snacks and other perks.