Senior/Lead Data Engineer

This job is brought to you by Jobs/Redefined, the UK's leading over-50s age inclusive jobs board.

Please note that this role requires security clearance at SC level. You must be SC cleared to be considered for the role.

Tasks and Responsibilities:

Engineering:

• Ingestion configuration.
• Wring python/pyspark and spark SQL code for validation/curation in notebook.
• Create data integration test cases.
• Implement or amend worker pipelines.
• Implement Data validation/curation rules.
• Convert data model data mapping the technical data mapping.
• Implement Integrations.
• Data migration implementation and execution.
• Data analysis (profiling etc).
• Azure devops/github configurations for ADF and data bricks code.

Our ideal candidate:

Strong experience in Designing and delivering Azure based data platform solutions, technologies including

1. 1. Azure ADF and Data bricks, Azure Functions,App Service ,Logic app,AKS,Azure app servie,Webapp.
   2. Good knowledge in real time streaming applications preferably with experience Kafka Real time messaging or Azure function ,Azure service bus.
   3. Spark processing and performance tuning.
   4. File formats partitioning for e.g. Parquet,JSON,XML,CSV.
   5. Azure Devops/git hub.

• Hands on experience in at least one of Python with knowledge of the others.
• Experience of synchronous and asynchronous interface approaches.
• Knowledge in data modeling (3NF/Dimensional modeling/Data Vault2).
• Work experience in agile delivery.
• Able to provide comprehensive documentation.
• Able to set and manage realistic expectations for timescales, costs, benefits and measures for success.

Nice to Have:

• Experience with integration and implementation of data cataloging tool like Azure Purview/Collabra.
• Experience in implementing and integrating visualizations tools like Power BI/Tableau etc.
• Experience C# application development (ASP.net MVP).

The Cognizant community:
We are a high caliber team who appreciate and support one another. Our people uphold an energetic, collaborative and inclusive workplace where everyone can thrive.

• Cognizant is a global community with more than 300,000 associates around the world.
• We don't just dream of a better way - we make it happen.
• We take care of our people, clients, company, communities and climate by doing what's right.
• We foster an innovative environment where you can build the career path that's right for you.

About us:
Cognizant is one of the world's leading professional services companies, transforming clients' business, operating, and technology models for the digital era. Our unique industry-based, consultative approach helps clients envision, build, and run more innovative and efficient businesses. Headquartered in the U.S., Cognizant (a member of the NASDAQ-100 and one of Forbes World's Best Employers 2024) is consistently listed among the most admired companies in the world. Learn how Cognizant helps clients lead with digital at www.cognizant.com

Our commitment to diversity and inclusion:
Cognizant is an equal opportunity employer that embraces diversity, champions equity and values inclusion. We are dedicated to nurturing a community where everyone feels heard, accepted and welcome. Your application and candidacy will not be considered based on race, color, sex, religion, creed, sexual orientation, gender identity, national origin, disability, genetic information, pregnancy, veteran status or any other protected characteristic as outlined by federal, state or local laws.

Disclaimer:
Compensation information is accurate as of the date of this posting. Cognizant reserves the right to modify this information at any time, subject to applicable law.

Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government issued ID during each interview.