Databricks Senior Data Engineer

Posted 2 Days Ago
Be an Early Applicant
Dublin
Senior level
Information Technology
The Role
The Senior Data Engineer will design and implement cloud-native data solutions, build MLOps pipelines, establish data governance standards, oversee data integration, and optimise performance. They will work with Databricks, Snowflake, and AWS to enhance data accessibility and scalability.
Summary Generated by Built In

About us:

Amach is an industry-leading technology driven company with headquarters located in Dublin and remote teams in UK and Europe.

Our blended teams of local and nearshore talent are optimised to deliver high quality and collaborative solutions.

Established in 2013, we specialise in cloud migration and development, digital transformation including agile software development, DevOps, automation, data and machine learning…

We are seeking an accomplished Senior Data Engineer to join our Dublin-based team. This role provides an exciting opportunity to influence our data architecture, working with innovative cloud technologies to drive impactful, data-centric projects. The ideal candidate will have in-depth experience with Databricks, Snowflake, AWS, and MLOps to support and enhance the deployment and scalability of machine learning models. You’ll play a pivotal role in ensuring data accessibility, optimising data sourcing pipelines, and enhancing the performance of large-scale data solutions. 

Key responsibilities & duties include:

  • Design and Implement Cloud-Native Data Solutions: Develop scalable, resilient data platforms using cloud-native technologies, data mesh frameworks, and integration across diverse data sources
  • Build and Maintain MLOps Pipelines: Use tools like MLflow to create reliable, efficient pipelines for deploying machine learning models in production environments
  • Establish Data Governance and Quality Standards: Develop and uphold data governance practices, ensuring robust data quality and compliance using tools like Databricks Unity Catalog
  • Oversee Data Integration and Migration: Lead migration projects from legacy data systems to modern cloud platforms, focusing on optimising operational costs and efficiencies
  • Performance Optimisation and Tuning: Use tools such as Snowflake and Delta Lake to enhance data accessibility, reliability, and performance, delivering robust, high-quality data products

Key Projects/Deliverables:

  • Data Mesh Architecture: Design and deploy data mesh frameworks to support seamless data integration and scalability across business functions
  • Operationalise MLOps Pipelines: Develop and manage MLOps pipelines to streamline machine learning model deployment and operational efficiency
  • Data Migration and Cost Optimisation: Lead large-scale data migrations to Azure and AWS, focusing on business-critical data sources and substantial cost reductions
  • Data Governance Applications: Create applications to reinforce data governance, quality, and enterprise standards, ensuring a secure and compliant production environment 

Required Experience:

  • Data Engineering Expertise: Proven experience in architecting and delivering large-scale, cloud-native data solutions
  • Advanced Knowledge in Databricks and Snowflake: Hands-on experience in Databricks, Spark, Pyspark, and Delta Lake, with strong skills in data warehousing and lakehouse solutions
  • MLOps Skills: Practical experience in MLOps, ideally with MLflow for model management and deployment
  • Cloud Proficiency: Strong knowledge of AWS, with additional experience in Azure advantageous for multi-cloud setups
  • Programming Proficiency: Advanced coding abilities in Python, SQL and Scala
  • Tooling Competence: Familiarity with version control (GitHub), CI/CD tools (Azure DevOps, GitHub Actions), orchestration tools (Airflow, Jenkins), and dashboarding tools (Tableau, Alteryx)

 Desirable Skills:

  • Experience with Synapse Analytics, Netezza, and legacy data systems
  • Knowledge of data governance best practices and tools
  • Excellent problem-solving skills, with the ability to work both autonomously and collaboratively in cross-functional teams

What’s in it for you: 

  • An opportunity to join a fast-growing company  
  • Options for career advancement 
  • Learning and development opportunities 
  • Flexible working environment
  • Competitive rates based on experience 

Not for you?

Check out all of our open positions in our careers page and follow us on LinkedIn for future opportunities.

P.S. Share this with friends and co-workers! Don't be afraid they'll steal it from you, if you're amazing and smart we'll find a role for you. We are growing fast and we are always looking for talented people.

At Amach, we strive to be an inclusive community of open-minded individuals with different backgrounds and we are committed to fostering, cultivating and preserving a culture of diversity, equity and inclusion. We strongly believe that a diversity of experience and background is essential to create a fulfilling environment and better solutions for our people and our customers. All Amach employees and contractors are expected to honour this policy and act to ensure that every individual is respected in the workplace. 

Your personal data

Amach will process your personal information in accordance with the EU's General Data Protection Regulation (GDPR). We will comply with data protection law and principles, which means that your data will be:

  • Used lawfully, fairly and in a transparent way
  • Collected only for valid purposes and not used in any way that is incompatible with those purposes
  • Relevant to the purposes we have told you about and limited only to those purposes
  • Accurate and kept up to date
  • Kept only as long as necessary for the purposes we have told you about
  • Kept securely

If you would like to contact us about your data, please use the following address: [email protected]

Top Skills

Python
Scala
SQL
The Company
HQ: Dublin, Dublin
139 Employees
On-site Workplace
Year Founded: 2013

What We Do

We help mature organisations evolve into modern digital businesses with faster time to market, increased operational stability and security.

We offer a suite of technical services delivered by an experienced team of subject matter experts. Our services can be provided as a fully managed service or as an embedded part of your team. We focus on both short-term and long-term goals that emphasise business outcomes for our customers.

Why choose Amach:
1. Business Agility​ - We focus on removing your IT debt, so that your company can focus on delivering business value to your customers at pace in a secure and reliable manner.
2. Cost Reduction ​- We help reduce your IT costs across all domains. This includes operational costs in both cloud and your data centres, licencing, evergreening, and reducing project delivery timelines.
3. Operational Stability ​ - We will modernise your IT systems ensuring operational stability and resilience, to ensure the end user experience for both customers and staff are enhanced.
4. Enable Innovation ​- By removing IT debt, we create space for your team to focus on innovation, ensuring business longevity and futureproofing. We bring both sector experience and lessons learned.
5. Security ​ - We address the overall architecture to ensure security by convention vs configuration. We promote DevSecOps practices, so security isn’t seen to prevent teams from delivery in an efficient manner.
6. Improve Employee Experience - We create and implement your EUX strategy that adapts to a changing world, while improving security and user efficiency - resulting in improved colleague satisfaction and retention.
7. Sustainability ​ - Achieve your sustainability goals through optimising your overall IT footprint and reducing costs.

We thrive on delivering customer value in every interaction.

If you are interested in hearing more or to see how we could help you, please get in touch [email protected]

Similar Jobs

DraftKings Logo DraftKings

Data Science Engineer

Digital Media • Gaming • Information Technology • Software • Sports • eSports • Big Data Analytics
Hybrid
Dublin, IRL
5300 Employees

DraftKings Logo DraftKings

Senior Data Science Engineer

Digital Media • Gaming • Information Technology • Software • Sports • eSports • Big Data Analytics
Hybrid
Dublin, IRL
5300 Employees

Block Logo Block

GTM Analytics Manager, EU

Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Hybrid
Dublin, IRL
12000 Employees
Hybrid
Dublin, IRL
289097 Employees

Similar Companies Hiring

Rapid7 Thumbnail
Software • Security • Sales • Information Technology • Cybersecurity • Cloud • Artificial Intelligence
Boston, MA
2400 Employees
Skillsoft Thumbnail
Software • Information Technology • HR Tech • Edtech • Conversational AI • Consumer Web • Artificial Intelligence
Nashua, NH
2900 Employees
Toast Thumbnail
Software • Information Technology • Hospitality • Food • Fintech • Cloud
Boston, MA
4500 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account