Research Scientist

Job description

## About Us [FAR.AI](http://FAR.AI) is a non-profit AI research institute working to ensure advanced AI is safe and beneficial for everyone. Our mission is to facilitate breakthrough AI safety research, advance global understanding of AI risks and solutions, and foster a coordinated global response. Since our founding in July 2022, we've grown to [40+ staff](https://www.far.ai/about/team), published [40+ academic papers](https://scholar.google.com/citations?user=FVJ24k8AAAAJ), and convened leading [AI safety events](https://far.ai/events/). Our work is recognized globally, with publications at premier venues such as NeurIPS, ICML, and ICLR, and features in the [Financial Times](https://www.ft.com/content/175e5314-a7f7-4741-a786-273219f433a1), [Nature News](https://www.nature.com/articles/d41586-024-02218-7) and [MIT Technology Review](https://www.technologyreview.com/2020/02/28/905615/reinforcement-learning-adversarial-attack-gaming-ai-deepmind-alphazero-selfdriving-cars/). We conduct pre-deployment testing on behalf of frontier developers such as OpenAI and independent evaluations for governments [including the EU AI Office](https://www.far.ai/news/far-ai-selected-to-lead-eu-ai-act-cbrn-risk-consortium). We help steer and grow the AI safety field through [developing](https://arxiv.org/abs/2405.06624) [research](https://arxiv.org/abs/2506.20702) [roadmaps](https://www.researchgate.net/publication/396910034_Open_Technical_Problems_in_Open-Weight_AI_Model_Risk_Management) with renowned researchers such as Yoshua Bengio; running [FAR.Labs](https://www.far.ai/programs/far-labs), an AI safety-focused co-working space in Berkeley housing 40 members; and supporting the community through [targeted grants](https://www.far.ai/programs/grantmaking) to technical researchers. ## About FAR.Research We explore promising research directions in AI safety and scale up only those showing a high potential for impact. Once the core research problems are solved, we work to scale them to a minimum viable prototype, demonstrating their validity to AI companies and governments to drive adoption. We are aiming to rapidly grow our team in the following areas especially, at varying levels of seniority: - **Evals and red-teaming**: Conducting pre- and post-release adversarial evaluations of frontier models (e.g. [Claude 4 Opus](https://x.com/ARGleave/status/1926138376509440433), [ChatGPT Agent](https://cdn.openai.com/pdf/839e66fc-602c-48bf-81d3-b21eacc3459d/chatgpt_agent_system_card.pdf), [GPT-5](https://cdn.openai.com/gpt-5-system-card.pdf)); developing [novel attacks](https://www.far.ai/news/defense-in-depth) to support this work; and exploring new threat models (e.g. [persuasion](https://arxiv.org/abs/2506.02873), [tampering risks](https://arxiv.org/abs/2507.11630)). - **Infrastructure:** Maintaining GPU compute infrastructure to support experiments with open-weight models and developing new tooling to allow our research teams to scale their fine-tuning and post-training workflows to frontier open-weight models. We are also seeking more senior candidates in the following research areas: - **Mitigating AI deception**: Studying when [lie detectors induce honesty or evasion](https://www.far.ai/news/avoiding-ai-deception), and developing model organisms for deception and sandbagging - **Adversarial Robustness**: Working to rigorously solve these security problems through building a science of security and robustness for AI, from [demonstrating superhuman systems can be vulnerable](https://far.ai/post/2023-07-superhuman-go-ais/), to [scaling laws for robustness](https://www.far.ai/news/does-robustness-improve-with-scale) and [jailbreaking constitutional classifiers](https://arxiv.org/abs/2506.24068) - **Mechanistic Interpretability**: F[inding](https://arxiv.org/abs/2502.12892) [issues](https://arxiv.org/abs/2508.16560) [with](https://arxiv.org/abs/2505.11756) Sparse Autoencoders, probing deception using [AmongUs](https://arxiv.org/abs/2504.04072), understanding [learned planning](https://far.ai/post/2024-07-learned-planners/) in SokoBan and interpretable data attribution. [FAR.AI](http://FAR.AI) is one of the largest independent AI safety research institutes, and is rapidly growing with the goal of diversifying and deepening our research portfolio. We would welcome the opportunity to add new research directions if you are a senior researcher with a strong vision and would like to pitch us on it. ## **About the Role** We organize our team as Members of Technical Staff, with significant overlap between scientist and engineer roles. As a scientist, you will take ownership of and accelerate existing AI alignment research agendas. You can publish research findings broadly and engage with the AI alignment community. If you are an experienced research scientist, then we would be excited to incubate your agenda at FAR using our existing infrastructure and world-class team. You will receive engineering mentorship via code review, pair programming and regular 1-to-1s. Alongside the engineers, you will be involved in develop scalable implementations of machine learning algorithms and using them to run scientific experiments, You are encouraged to develop your research taste, proposing novel directions and joining a research pod which suits your interests. You are welcome to take time to study and to attend conferences free of charge. Our technical team is organized into research pods to enable continuity of organizational structure whilst each pod can pivot through varied research projects. Beyond [FAR.AI](http://FAR.AI), you can work with national AI safety institutes, frontier model developers and top academics. ## **About You** _We are excited by unconventional backgrounds_. You may have the following: - New and under-explored AI alignment idea(s). - Experience leading and/or playing a senior role in research projects related to machine learning. - Ability to effectively communicate novel methods and solutions to both techni