CrowdStrike Logo

CrowdStrike

Manager, AI Benchmarking and Evaluation Research (Remote, ROU)

Posted 9 Hours Ago
Be an Early Applicant
Remote or Hybrid
Hiring Remotely in România
Mid level
Remote or Hybrid
Hiring Remotely in România
Mid level
Leads a team researching and evaluating AI, LLM, and agentic systems for cybersecurity workflows. Defines evaluation strategies, benchmark datasets, metrics, methodologies, and reproducible testing pipelines grounded in SOC operations. Assesses model accuracy, robustness, reliability, and operational effectiveness across incident response, threat hunting, alert triage, and investigations. Partners with engineering, product, and threat research teams to translate findings into improvements and communicates results to technical and executive stakeholders.
The summary above was generated by AI

As a global leader in cybersecurity, CrowdStrike protects the people, processes and technologies that drive modern organizations. Since 2011, our mission hasn’t changed — we’re here to stop breaches, and we’ve redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep their businesses running, their communities safe and their lives moving forward. We're proud to work for a mission-driven company leveraging AI to transform the way we work. CrowdStrikers drive their careers through flexibility and autonomy while also being expected to contribute to a culture of responsible AI adoption, experimentation, and innovation. We use an AI-first mindset as a force multiplier to proactively and continuously accelerate execution, build expertise, uncover insights, and solve complex problems. We’re always looking to add talented CrowdStrikers to the team who have limitless passion, a relentless focus on innovation and a fanatical commitment to our customers, our community and each other. Ready to join a mission that matters? The future of cybersecurity starts with you.


About the Role:

The CrowdStrike Data Science Team is looking for an experienced and driven leader to build and guide a team dedicated to designing and building evaluations for AI models that perform cybersecurity tasks. Our mission is to establish rigorous, reproducible standards for measuring how well AI and agentic systems support real-world security operations.

As the Evaluation Lead, you will bring deep, hands-on SOC expertise to the table, defining what "good" looks like for AI models operating in security workflows. You will build the datasets and methodologies that ground our models in the realities of frontline defense, and lead a team that turns findings into actionable insights for engineering and product stakeholders.

What You'll Do:

  • Lead, mentor and grow a team of researchers and engineers focused on evaluating AI models applied to cybersecurity tasks

  • Define the strategy, roadmap and success metrics for evaluating AI/LLM and agentic systems across security use cases (e.g. incident response, threat hunting, alert triage, investigation)

  • Design and standardize evaluation methodologies, benchmark datasets and reproducible testing pipelines rooted in real SOC workflows

  • Assess models for accuracy, robustness, reliability and operational effectiveness in security-analyst scenarios

  • Establish quantitative and qualitative metrics that reflect how AI models and agentic systems perform against genuine threat-detection and response challenges

  • Collaborate cross-functionally with engineering, product and threat-research teams to translate evaluation findings into scalable improvements

  • Communicate results, trade-offs and recommendations clearly to both technical and executive audiences

What You'll Need:

  • Hands-on SOC experience with a strong understanding of day-to-day security operations

  • At least 3 years in a management or team-leadership position, with a proven track record of mentoring and growing technical teams (formal manager experience is a plus)

  • Strong knowledge of incident response and threat hunting, including detection, investigation and remediation workflows

  • Broad knowledge of the cybersecurity landscape, including attack vectors, defense mechanisms, and the analyst workflows that AI aims to augment

  • Practical experience working with AI, including familiarity with AI/LLM capabilities and knowledge of agentic systems and how they operate

  • Ability to define, design and standardize evaluation methodologies and reproducible testing pipelines

  • Exceptional communication skills, including the ability to present complex technical findings clearly to both technical and non-technical audiences

  • Demonstrated track record of delivering results, supported by shareable projects or measurable outcomes

  • A genuine passion for working at the forefront of AI and cybersecurity, with a proactive mindset toward continuous learning and innovation in a rapidly evolving field

  • The following is required within every job description: "Proven experience utilizing AI technologies to enhance decision-making, streamline workflows and processes, improve efficiency and drive business outcomes.

Bonus Points:

  • Relevant security certifications (e.g., GCIA, GCIH, GCFA, OSCP, or equivalent)

  • Experience building or working with LLM/agentic evaluation frameworks and benchmarking pipelines

  • Programming experience in Python and familiarity with data-science tooling

  • Familiarity with SOC platforms, SIEM/SOAR tools, and detection engineering

  • Exposure to adversarial testing, red-teaming, or robustness evaluation of AI systems

  • Understanding of MITRE ATT&CK and how it maps to detection and response workflows

What We Offer:

  • Competitive salary

  • Stock options

  • Private Healthcare insurance

  • Life insurance

  • Training budget

  • Working with the latest technologies

  • Flexible time off

  • Team hangouts

#LI-JP1

#LI-GT1

#LI-Remote


Benefits of Working at CrowdStrike:

  • Market leader in compensation and equity awards
  • Comprehensive physical and mental wellness programs
  • Competitive vacation and holidays for recharge
  • Paid parental and adoption leaves
  • Professional development opportunities for all employees regardless of level or role
  • Employee Networks, geographic neighborhood groups, and volunteer opportunities to build connections
  • Vibrant office culture with world class amenities
  • Great Place to Work Certified™ across the globe

CrowdStrike is proud to be an equal opportunity employer. We are committed to fostering a culture of belonging where everyone is valued for who they are and empowered to succeed. We support veterans and individuals with disabilities through our affirmative action program.


CrowdStrike is committed to providing equal employment opportunity for all employees and applicants for employment. The Company does not discriminate in employment opportunities or practices on the basis of race, color, creed, ethnicity, religion, sex (including pregnancy or pregnancy-related medical conditions), sexual orientation, gender identity, marital or family status, veteran status, age, national origin, ancestry, physical disability (including HIV and AIDS), mental disability, medical condition, genetic information, membership or activity in a local human rights commission, status with regard to public assistance, or any other characteristic protected by law. We base all employment decisions--including recruitment, selection, training, compensation, benefits, discipline, promotions, transfers, lay-offs, return from lay-off, terminations and social/recreational programs--on valid job requirements.


If you need assistance accessing or reviewing the information on this website or need help submitting an application for employment or requesting an accommodation, please contact us at [email protected] for further assistance.


Similar Jobs at CrowdStrike

9 Hours Ago
Remote or Hybrid
Senior level
Senior level
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Leads research and production development of generative AI and large language model systems for cybersecurity. Responsibilities include defining AI research strategy, developing novel reasoning and summarization approaches, selecting model architectures and deployment strategies, building large-scale production systems, mentoring scientists, establishing evaluation standards, and guiding cross-functional technical initiatives. The role also represents the company through publications, speaking engagements, and technical thought leadership.
Top Skills: Cloud TechnologiesDeep LearningGpu TechnologiesLarge Language Models (Llms)Python
11 Hours Ago
Remote or Hybrid
Entry level
Entry level
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Design, implement, and maintain IAM and access-control systems for CrowdStrike’s global, multi-cloud Falcon platform. Manage IAM lifecycle operations, security governance, least-privilege controls, threat modeling, audits, and incident response. Develop automation and integrations, ensure access-system reliability at scale, and advise product and engineering stakeholders on security. The role also involves improving cybersecurity practices, supporting compliance, and applying AI technologies to enhance workflows and decision-making.
Top Skills: AnsibleAws IamAzure EntraBashDirectory ServicesFederationGcp IamGdprGoIso 27001KerberosLinuxMfaMicrosoft Active DirectoryNistOci IamOidcPythonSaltstackSAMLService MeshSoc 2SsoTerraformUnix
11 Hours Ago
Remote or Hybrid
Entry level
Entry level
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Designs, implements, and maintains IAM and access-control systems for CrowdStrike’s global, multi-cloud platform. Responsibilities include IAM lifecycle operations, incident response, least-privilege design, threat modeling, governance, auditing, scripting, automation, and advising product and engineering stakeholders. The role focuses on secure, reliable access across thousands of services and users while improving cybersecurity capabilities and supporting compliance and risk management.
Top Skills: AnsibleAws IamAzure EntraBashDirectory ServicesFederationGcp IamGoKerberosLinuxMfaMicrosoft Active DirectoryOci IamOidcPythonSaltstackSAMLService MeshSsoTerraformUnix

What you need to know about the Dublin Tech Scene

From Bono and Oscar Wilde to today's tech leaders, Dublin has always attracted trailblazers, with more than 70,000 people working in the city's expanding digital sector. Continuing its legacy of drawing pioneers, the city is advancing rapidly. Ireland is now ranked as one of the top tech clusters in the region and the number one destination for digital companies, with the highest hiring intention of any region across all sectors.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account