Senior Kafka Platform Engineer - Data Platforms

This job is brought to you by Jobs/Redefined, the UK's leading over-50s age inclusive jobs board.

Senior Kafka Platform Engineer - Data Platforms

Job no: 502558
Work type: Permanent Full Time
Location: London - Hybrid
Categories: Data Architecture & Data Engineering

Senior Kafka Platform Engineer - Data Platforms

Working at our iconic office in King's Cross, London, and virtually via Microsoft Teams

Your new role at NewDay

We are seeking a skilled Senior Kafka Engineer to join our Data Platform Engineering team in London.

As the Senior Kafka Engineer, you will lead the architecture, design, and implementation of Kafka in a multi-tenant environment. You will ensure that the platform meets the needs of multiple teams while maintaining high levels of performance, reliability, and security. The role holder will work with Platform Engineers, Technical Architects, Data Engineers, and Analysts at NewDay and with B2B customers of the Data Platform.

You'll deliver

• Configure, deploy, and manage Confluent Kafka platform.
• Automate processes and ensure observability using Infrastructure as Code (IaC) with Terraform.
• Troubleshoot Kafka issues across different environments and ensure problem resolution.
• Configure Kafka connectors for enterprise systems and fine-tune brokers for optimized performance.
• Handle version upgrades, disaster recovery (DR) solutions, and high availability (HA) strategies.
• Design Kafka topics, schemas (Avro), and ksqlDB objects.
• Collaborate with Data Engineers to develop and test event streaming applications and data pipelines.
• Work closely with AWS and Azure Security teams to integrate security into Kafka deployments.

What you'll bring

• Kafka & Cloud Expertise: Skilled in configuring and managing Confluent Kafka on AWS & Azure, integrating with diverse data sources, and optimizing performance.
• Automation & DevOps: Proficient in automating Kafka processes using Terraform and CI/CD tools like GitHub Actions and Kubernetes (EKS) for streamlined deployments.
• Distributed Systems & Troubleshooting: Strong background in managing distributed data systems, identifying issues, and implementing solutions across environments.
• Security & Collaboration: Experienced in working with security teams to ensure secure deployments and collaborating with cross-functional teams to design and implement scalable data pipelines.

We need knowledge, experience + expertise in:

• Skilled in building Kafka producer and consumer applications, and managing Confluent Kafka in AWS and Azure.
• Proficient in integrating Kafka connectors with various data sources and working with distributed data systems.
• Experienced Terraform developer with a strong understanding of Infrastructure-as-Code and cloud computing.
• Deep knowledge of computer networks, including cloud networks, VPCs, firewalls, proxies, and security configurations (AWS, IAM, AD integration).
• Expertise in virtualisation/containerisation, Continuous Integration/Deployment (CI/CD) methodologies, and Kubernetes (EKS).
• Familiar with ELK stack (Elastic, Fluentd, Kibana) and Python functional development, as well as Linux scripting.
• Proficient in both SQL and NoSQL technologies, and experienced working in Agile environments.
• Strong commitment to problem-solving, delivering high-quality results, and ensuring timely customer support.

And would love you to know or learn:

• Takes pride in ownership of work, e.g. effective automation pipelines
• Ability to debug, troubleshoot, and resolve complex technical issues
• Self-driven and motivated, with a strong work ethic and a passion for problem-solving
• Cares about improving the efficiency of the other team members
• Loves to work iteratively, start simple and add-in complexity
• Persistence and dedication to finish things out
• Strong interpersonal, planning and time leadership skills
• Ability to build strong and effective working relationships on an ongoing basis

Where next? Let's talk about this role - Apply now or contact talent@newday.co.uk with any queries.

About NewDay

We help people move forward with credit and help our colleagues to move their careers forward too.

We use our highly flexible, scalable, and multi-product digital credit engine to power over 120 million transactions every year. Our brands include Aqua, marbles, fluid and Bip. We partner with leading brands such as John Lewis, AO, Argos and DEKO.

Over 5 million UK customers are supported by our award-winning customer service.

At NewDay, we value all types of diversity. We're an equal opportunity employer and believe that our differences create a vibrant, authentic working culture. We want all our colleagues to feel able to bring their whole selves to work. We don't discriminate on the basis of age, physical or mental disability, gender reassignment, marriage and civil partnership, pregnancy and carer status, race (including colour, nationality, and ethnic or national origin), religion or belief, sex and sexual orientation. We make sure that every job is crafted to be inclusive and that people with disabilities or caring responsibilities can take part in the application and interview process. Tell us if you need accommodations: we'll put reasonable adjustments in place to support you.

Our dynamic NewDay culture

We're focused on what will drive impact in helping people move forward with credit. Our distinctive culture is geared to spark innovation and team working - with lots of open doors for development. Our customers can rely on us because we aim high, support each other, do the right thing and build for the future.

We invest in our colleagues. On top of a strong market competitive salary, you get a bonus opportunity that matches the impact (delivery + values) you drive in your role. We also help you retire better with market leading pensions.

At NewDay, #yourwellbeing matters: You get 26 days holiday and can buy up to 5 more after probation. Then you'll get extra days as you build your career with us.

NewWork, our flexible, hybrid working approach, helps you to manage your work/life balance - and even bolt on work time in other countries before or after your holiday. And when you're in the office, you get free healthy breakfast, fresh juices, lunch, barista coffee etc.

Our tax efficient green car and cycle to work schemes save you money (and help the planet).

Ask your Talent Acquisition Partner to tell you more about any of our perks.

We work with Textio to make our job design and hiring inclusive.

#LI-Hybrid

Advertised: 06 Nov 2024 GMT Standard Time
Applications close: 27 Nov 2024 GMT Standard Time