ACD Direct Logo

ACD Direct

Senior Data Engineer

Posted Yesterday
Be an Early Applicant
Remote
Hiring Remotely in CAN
Senior level
Remote
Hiring Remotely in CAN
Senior level
Design, build, and maintain scalable Databricks-based ETL/ELT pipelines and data models optimized for AI/ML. Manage orchestration, data quality, governance, and CI/CD (ADO → GitHub) while collaborating with cross-functional teams in Agile delivery.
The summary above was generated by AI

About ACD

For more than a century, AIA Contract Documents (ACD) has supported architecture, engineering, and construction professionals by delivering a shared industry standard to align parties on a project.

What began in 1888 with the development of standardized construction contracts has evolved into a comprehensive suite of contract tools and foundational workflows that not only shape how the industry works today, but uniquely position ACD to help firms navigate construction’s growing complexity.

In a world driven by scale, fragmentation, and AI-generated decisions that prioritize speed over a clear understanding of risk, ACD serves as a trusted anchor, ensuring project participants can reduce disputes and negotiations, while achieving faster alignment and more predictable outcomes for the future.

AIA Contract Documents is seeking a Senior Data Engineer to support the buildout of data infrastructure powering upcoming AI initiatives. This role will play a critical part in designing and structuring data systems that enable scalable, high-quality inputs for AI and machine learning models.

The ideal candidate brings deep experience in modern data engineering practices, strong familiarity with Databricks, and the ability to operate independently in a fast-evolving environment with limited oversight.

Key Responsibilities

  • Design and implement scalable data models optimized for AI/ML use cases, ensuring data is structured for effective model training and inference

  • Architect and manage data pipelines using Databricks, including orchestration, job scheduling, and workflow optimization

  • Develop and maintain robust ETL/ELT processes to support data ingestion, transformation, and delivery across systems

  • Leverage Databricks Asset Bundles to manage deployment of data assets (pipelines, jobs, notebooks, and files) across environments

  • Collaborate with cross-functional teams to align data architecture with AI initiative requirements and business objectives

  • Ensure data quality, integrity, and governance standards are met across all pipelines and datasets

  • Contribute to CI/CD practices using Azure DevOps (ADO), with a future transition to GitHub-based workflows

  • Participate in Agile development processes, including sprint planning, stand-ups, and iterative delivery

  • Other duties as assigned

Qualifications

  • 5+ years of experience in Data Engineering or related field

  • Strong expertise in Databricks, including pipeline development, orchestration, and data architecture

  • Experience designing data models to support AI/ML applications

  • Proficiency in Python (PySpark) and SQL

  • Hands-on experience with Azure-based data environments

  • Experience with CI/CD tools such as Azure DevOps (ADO)

  • Ability to work independently with minimal oversight and ramp quickly in a fast-paced environment

  • Experience with Databricks Asset Bundles for deployment and environment management, preferred

  • Familiarity with transitioning CI/CD workflows to GitHub, preferred

  • Experience working in Agile development environments, preferred

  • Exposure to AI/ML workflows and data requirements for model development, preferred

What Success Looks Like

  • Efficient, scalable data pipelines supporting AI initiatives

  • Well-structured, high-quality datasets optimized for machine learning

  • Streamlined deployment and orchestration of Databricks assets across environments

  • Strong collaboration with stakeholders to deliver data solutions aligned with business needs

Similar Jobs

Yesterday
Easy Apply
Remote or Hybrid
Easy Apply
Senior level
Senior level
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Design, build, and maintain SparkSQL/PySpark data pipelines in the central data lake to ingest IoT, product, and unstructured data (video/audio). Produce reliable computed tables for analytics, model training, and dashboards; integrate external datasets; ensure high data quality and uptime; and collaborate with Data Science, ML, and cross-functional teams.
Top Skills: AirflowAWSAzureDagsterData LakeDatabricksDelta LakeETLGCPGitGitPrefectPysparkPythonRest ApisSparkSparksqlSQL
2 Days Ago
Easy Apply
Remote or Hybrid
Easy Apply
Senior level
Senior level
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Design, build, and operate scalable data ingestion, replication, and lakehouse infrastructure to move petabytes of data into a Delta Lake on S3. Improve reliability, observability, security, and developer experience for Spark/Databricks processing. Develop internal libraries and tooling (Go/Python), collaborate with cross-functional teams, and help shape long-term data platform and AI-ready infrastructure.
Top Skills: SparkAws DynamodbAws KinesisAws LambdaAws RdsAws S3Aws SqsGoJavaPythonScala
3 Days Ago
In-Office or Remote
Senior level
Senior level
Big Data • Cloud • Digital Media • Machine Learning • Mobile • Software • Industrial
Design, build, automate and maintain scalable ELT/ETL data pipelines and data architecture on cloud (AWS). Collaborate with analysts, data scientists and stakeholders to ensure data quality, governance, monitoring and performance. Apply ML-driven validation, anomaly detection, automated schema evolution and optimizations to support analytics and AI-native workloads.
Top Skills: AirflowAWSHadoopHiveJSONParquetPrestoPythonSnowflakeSparkSQL

What you need to know about the Dublin Tech Scene

From Bono and Oscar Wilde to today's tech leaders, Dublin has always attracted trailblazers, with more than 70,000 people working in the city's expanding digital sector. Continuing its legacy of drawing pioneers, the city is advancing rapidly. Ireland is now ranked as one of the top tech clusters in the region and the number one destination for digital companies, with the highest hiring intention of any region across all sectors.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account