Research Engineer - Forecast Coherence & World Models

Metaculus is the premier public forecasting platform with over 2,800,000 forecasts on over 20,000 questions. Metaculus is "epistemic infrastructure" like Wikipedia and Our World In Data, providing the world with trusted, accurate information on a variety of questions, particularly in science and technology. Our mission is to enable everyone to model, understand, predict, and navigate the world’s most important and complex challenges.

As a Research Engineer focused on Forecast Coherence & World Models, you'll pioneer the development of technologies that make forecasts and their underlying world models transparent and accessible. You'll work at the intersection of software engineering and epistemics, building tools that help domain experts and forecasters create more coherent predictions, and support better decision-making on complex, cross-disciplinary problems.

This role emphasizes independent experimentation and rapid prototyping. You will run LLM experiments, design and test new approaches, and build MVPs to validate ideas. You will also design methods to elicit, aggregate, and display the relationships between forecasts (e.g. causal connections, necessary conditions) while leveraging AI to assist in world model construction and identify missing variables that could improve forecast quality.

Responsibilities

• Independently scope and run experiments with LLMs, prompt engineering, and novel approaches to eliciting and aggregating forecast relationships
• Build MVPs and prototypes to test and iterate on ideas quickly
• Design and implement methods for eliciting, aggregating and visualising coherence rules across forecasts, including causal/correlational relationships, martingale processes, necessary/sufficient conditions, etc.
• Develop intuitive interfaces that make complex forecast relationships and world models accessible to stakeholders
• Build AI-powered tools for world model elicitation and legibilization, including autonomous coherence rule detection and latent variable suggestion systems
• Integrate new capabilities into the main Metaculus platform and potentially build stand-alone tools for specialised use cases

Our Tech Stack

• Frontend: JavaScript, TypeScript, React, Next.js, D3, Tailwind CSS
• Backend: Python, Django, PostgreSQL, Redis, Sentry, Heroku, AWS
• Collaboration: GitHub, Slack, Notion, Figma

Requirements:
• Clean Code Practices: Commitment to writing maintainable, testable, and well-documented code
• Problem Solving: Strong analytical skills with the ability to tackle ambiguous problems in novel domains
• AI integration: prompting, RAG, fine tuning, etc.
• Communication: Excellent written and verbal communication skills, essential for remote collaboration
• Mission Alignment: Genuine interest in improving humanity's ability to predict and navigate complex challenges

Bonus Qualifications

• Full-Stack Engineering: experience with both frontend and backend development
• Experience with Django and Next.js frameworks
• User-Centric Development: Experience building intuitive interfaces for data visualization and interaction
• Background in statistics, probability theory, or causal inference
• Familiarity with directed acyclic graphs (DAGs) and graphical models
• Experience building data visualization tools for complex relationships
• Experience with forecasting platforms or prediction markets
• Background in epistemics, decision theory, or related fields

Benefits:
Starting annual salary in the range of $120k-$180k, plus equity, with the possibility of higher compensation for exceptional candidates.

In addition to the salary range listed above:

• We reimburse employees for health insurance costs, depending on country. In the US, we reimburse up to the legal maximum under a QSEHRA plan
• In the US, we offer a 401k plan for employees
• We provide generous parental leave
• As applicable, we provide relocation support and compensation adjustments when employees move to higher cost-of-living areas
• We offer travel opportunities for regular in-person working sessions and team retreats in the US and Europe
• We encourage our employees to meaningfully engage with and attend technical conferences and community events
• We are an equal opportunity employer that understands and appreciates the benefits of a diverse workforce