Design and develop a big-data processing engine and heavy ETL pipelines (including geo-location analysis) across cloud and on-prem. Build distributed, partitioned pipeline architectures using Spark/PySpark, Scala, Airflow, Kubernetes, and cloud platforms; follow Agile, clean-code, and software design best practices while supporting deployments and client-facing interactions.
Description
We are looking for a Big Data / Data Engineer to join our growing team. In this role, you will be responsible for the design and development of our big data processing engine a heavy ETL pipeline handling complex logic including geo-location analysis, running across both cloud and on-prem environments.
Requirements:
- 4+ years of experience in Data Engineering
- 2+ years of hands-on experience with Scala (must) or Java (optional)
- 2+ years of experience with SQL and NoSQL DBs
- Hands-on experience with Spark & PySpark (must)
- Experience with Airflow (must)
- Experience with AWS and/or GCP
- Experience with Kubernetes (k8s) and on-prem deployments
- Solid understanding of distributed systems design, pipeline architecture, and partitioning strategies
- Deep understanding of Agile principles, practices and values
- Solid understanding of design patterns, software development techniques, and clean code practices
Advantage:
- Experience with Apache Iceberg, Helm Charts, or data catalog tools
- Experience with Spring Framework
- Experience with microservice architecture
- Experience working with DevOps and CI/CD practices
- Customer-facing experience or comfort working directly with clients
- Bachelor's, BSc or Master's degree in Computer Science or Engineering
Similar Jobs
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Lead the operational PMO for change management on a large transformation: own the integrated change plan, governance, reporting/dashboards, risk & issues, standards and toolkits, stakeholder alignment, resource planning, onboarding, and continuous improvement to enable Functional and BU Change Leads to deliver.
Top Skills:
ApmgConfluenceExcelJIRAMs ProjectO9OraclePower BIProsci AdkarSalesforceSap S/4 HanaSharepointSmartsheetTeams
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
As a Finance FP&A Manager, you will lead financial planning processes, ensure compliance, analyze financial performance, and support decision-making within a Finance team, while driving continuous improvement initiatives.
Top Skills:
Data AnalysisFinancial ModelingFinancial PlanningReporting
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Lead development and execution of HTA, value, and evidence strategies for rare disease therapies. Manage HEOR study design and execution (RWE, registries, PRO/PCOA), develop economic models and global value dossiers, coordinate cross-functional and regional input, disseminate evidence via publications/conferences, and support access and reimbursement objectives to optimize patient access.
What you need to know about the Dublin Tech Scene
From Bono and Oscar Wilde to today's tech leaders, Dublin has always attracted trailblazers, with more than 70,000 people working in the city's expanding digital sector. Continuing its legacy of drawing pioneers, the city is advancing rapidly. Ireland is now ranked as one of the top tech clusters in the region and the number one destination for digital companies, with the highest hiring intention of any region across all sectors.
