Beta FutureWork.fyi

What ML still hasn't solved.

priorwork.fyi finds what has been done. This finds what still needs doing: open problems that several papers independently say are holding their work back, each checked against the index to see whether anyone has solved it since. If you have wondered what you should work on in ML, start here.

Gathering open problems…

Open research problems in machine learning, by area

Each entry is a gap several independent papers ran into and named as a limitation of their own work. Enable JavaScript for the full version, with the evidence and the papers behind each one.

Open problems in Machine Unlearning

A Standardized Multi-Architecture Testbed of Memorized Concepts, Generation Fingerprints, and Editable Structures in Text-to-Image Models Beyond Stable Diffusion v1.x
Empirical Robustness and Sensitivity of Machine Unlearning Under Realistic Data and Checkpoint Constraints
Machine Unlearning and Verification Under Black-Box Model Access Constraints
Benchmarking the Adversarial Robustness and Reversibility of LLM Unlearning
Machine Unlearning Under Degraded Operational Preconditions
Scalability and Robustness of Machine Unlearning Under Varying Forget-Set Regimes
Machine Unlearning Without External Discriminators or Teacher Reasoning Scaffolds

Open problems in Model Safety, Alignment & Jailbreaks

A Framework for Deploying Activation- and Logit-Based Methods Against Closed-Source LLM APIs via Instrumented Proxy Models
Recovering Internal Attention and Activation Signals from API-Visible Model Behavior
Safety Monitoring and Alignment Interventions Under Black-Box and API-Only Constraints
Decision-Only and Logit-Free Safety Auditing and Alignment Evaluation for Black-Box Models
Robust Evaluation of Non-Binary and Phrasing-Agnostic Refusal in Safety-Aligned Language Models
Evaluating the Cross-Modal Robustness of Text-Centric LLM Safety and Jailbreak Defenses
Evaluating the Generalisation of Small-Model Safety and Jailbreak Techniques to 70B+ and Frontier Models

Open problems in Hand-Object Interaction & Affordance

Hand-Object Affordance Estimation Beyond Clean Pre-Scanned Scenes and Predefined 3D Grids
Robust 3D Hand-Object Pose and Interaction Estimation Under Upstream 2D Segmentation and Localization Noise
Fine-Grained 3D Hand Affordance Grounding and Generation for Small-Part Articulated Interactions
Robust Hand-Object Interaction Learning Beyond Cascaded Parametric Pose Estimators
Benchmarking Hand-Object Interaction Models Across Diverse Hand Morphologies and Embodiments
Dynamic Closed-Loop Execution and Task-Conditioned Hand-Object Manipulation

Open problems in Protein & Biomolecular Sequence Modeling

Benchmarking Surrogate-Trained Sequence Optimization Against Empirical Fitness Landscapes
A Systematic Robustness Study of Genomic Foundation Models Beyond the Human Reference Genome
Benchmarking and Mitigating Model Sensitivity to Predicted Structure Quality in Protein Sequence Modeling
A Benchmark of Experimentally Validated Functional Motifs for Training and Evaluating Motif-Conditioned Generative Models
Quantifying and Mitigating Surrogate Oracle Exploitation in Biomolecular Sequence Optimization
Benchmarking the Fidelity of In Silico Protein Design Proxies Against Experimental Biophysical Assays

Open problems in Watermarking & Deepfake Detection

Benchmarking Watermarking and Deepfake Detection Across Non-Latent and Flow-Matching Generative Architectures
Black-Box and Model-Agnostic Deepfake and Watermark Verification for Closed-Source Generative Models
Black-Box and Architecture-Agnostic Watermarking and Defense for Diffusion Models
Cross-Family Generalization and Robustness of Text Watermarking Attacks and Defenses
Black-Box Verification and Trigger Generation for Model Watermarking and Detection
Cross-Paradigm Robustness Benchmarking for Deepfake Detectors

Open problems in Language Model Interpretability & In-Context Learning

Cross-Linguistic Robustness of Interpretability and Cognitive Alignment in Language Models
Benchmarking Whether LLM-Based Methods Survive a Change of Model: A Transfer Study Across Access Levels and Model Families
Benchmarking Interpretability and In-Context Learning Mechanisms Beyond Toy Synthetic PCFGs
Testing the Transferability of Toy-Model Theoretical Guarantees to Deep Transformer Architectures
A benchmark of naturally occurring hallucinations with model-generated ground truth, replacing synthetic entity-swap surrogates for detection evaluation

Open problems in 3D Point Clouds & LiDAR Perception

An Outdoor-Scale Benchmark for 3D Scene Understanding Methods Currently Validated Only on Indoor ScanNet-Derived Data
Benchmarking and Adapting Indoor-Trained 3D Perception and Generation Models to Outdoor and LiDAR Point Cloud Scenes
Robust Multimodal 3D Perception Under Sensor Extrinsic and Intrinsic Miscalibration
Benchmark and Training Protocols for 3D Perception from Unpaired or Calibration-Free 2D-3D Data

Open problems in Continual Learning & Catastrophic Forgetting

A Standardized Transfer-Validation Benchmark for Methods Validated Only on Small-Scale Image Classification
Characterizing and Reducing Pre-Trained ViT Dependence in Transfer-Based Methods
Sub-Quadratic Covariance Sketching for Exemplar-Free Continual Learning
Task-Agnostic Inference for Modular and Masked Continual Learning Architectures

Open problems in Deep Learning Theory & Optimization Dynamics

Empirical Benchmarking of Generalization Failure in Alternative Optimization Dynamics
Bridging the Activation Divide in Neural Network Optimization Dynamics Theory
Extending Two-Layer Network Theory to Deep and Structured Architectures
Cross-Architecture Empirical Validation of Deep Learning Theory Predictors

Open problems in Digital Humans, Avatars & Virtual Try-On

Joint Pose Refinement and Error-Tolerant Conditioning for Downstream Human Avatar Synthesis
Overcoming Parametric 3DMM Expressiveness and Mouth Interior Bottlenecks in Monocular Facial Avatars
Reducing Downstream Dependency of Garment/Avatar Generation on Upstream 2D and 3D Human-Parsing Predictions
Benchmarking and Improving 3D Human Body Model Robustness Across Underrepresented Demographics and Non-Standard Morphologies

Open problems in Domain Adaptation & Generalization

Domain-Invariant Learning Without Domain Labels or Target-Domain Data at Training Time
Cross-Architecture Robustness and Evaluation of Domain Adaptation in Object Detection
Systematic Evaluation of Data Augmentation Techniques Across Disparate Distribution Shift Types
Cold-Start Streaming Domain Adaptation and Discovery Without Labeled Base Sessions or Offline Pre-Passes

Open problems in Knowledge & Dataset Distillation

Knowledge Distillation Under Strict Black-Box Teacher Preconditions
Distillation Under Zero-Data and Unlabeled-Stream Preconditions
Cross-Modal 3D Distillation Without Paired and Synchronized LiDAR Streams
Knowledge Distillation and Rollout Transfer from Black-Box Teachers Without Logit Access

Open problems in Language Model Evaluation & Benchmarking

Probability-Free LLM Evaluation and Benchmarking for Black-Box Models
Methods and Benchmarks for LLM Judging, Ranking, and Evaluation That Work Without Token-Level Probabilities
Label-Free and Transferable Calibration for Large Language Model Outputs
Multi-Model Evaluation and Ensembling Without Per-Step White-Box Logit Access

Open problems in Learning with Noisy & Partial Labels

Eliminating the Auxiliary Clean-Label Anchor Required to Bootstrap Training under Imperfect Supervision
Validation-Free Hyperparameter Selection for Robust Learning Pipelines
Robust Learning under Incomplete, Evolving, and Annotator-Heterogeneous Label Spaces
Hyperparameter-Free Generalization: Benchmarking and Predicting Sensitivity of New Hyperparameters Across Tasks

Open problems in Offline & Model-Based RL

Cross-Domain Evaluation and Robustness of Offline Model-Based RL Across State-Action Representations
Stabilizing Value Learning and Policy Extraction in Sparse-Reward Offline Reinforcement Learning
Cross-Modality and Sim-to-Real Robustness Evaluation of Offline and Model-Based Reinforcement Learning
Empirical Robustness of Model-Based RL Across Simulator Differentiability, Parallelism, and Task Horizons

Open problems in RL Theory & Markov Decision Processes

Sample-Efficient Reinforcement Learning Under Markovian Trajectory Sampling Without a Generative Model
Empirical Robustness and Scalability Evaluation of Provably Efficient RL Algorithms
Reinforcement Learning Algorithms Without Generative Model and Reset Oracles
Regret Bounds with Problem-Dependent Constants Removed: Minimax Characterization of the Gap Between Instance-Independent and Instance-Dependent Guarantees

Open problems in RLHF & Preference Optimization

Eliminating Matched-Pair Preconditions and Scale Sensitivity in Contrastive Preference Guidance
Quantifying Evaluation Circularity and Judge Bias in Preference Optimization
Preference Optimization Under Unlabeled and Heterogeneous Real-World Annotator Noise
Empirical Robustness of Preference Optimization Across Synthetic, Multi-Annotator, and Real-World Feedback Distributions

Open problems in Semantic & Instance Segmentation

Overcoming the Mask Proposal Recall Ceiling in Open-Vocabulary Segmentation
Benchmarking and Bridging Idealized Preconditions in Promptable Segmentation
Benchmarking and Developing Segmentation Without Dense Initial Annotation Preconditions
Hierarchical Layout and Vector Segmentation Under Incomplete Multi-Level Supervision

Open problems in 3D Gaussian Splatting & Novel View Synthesis

Dynamic 3D Gaussian Splatting Under Non-Smooth and Discontinuous Motion Dynamics
Cross-Domain Robustness and Generalization Benchmarking for 3D Gaussian Splatting
Robust Wide-Baseline 3D Gaussian Splatting Under Unreliable Foundation-Model Geometry Priors

Open problems in Aerial, Satellite & BEV Perception

Robust Cross-View Perception and Localization Without Paired Ground-Level Street Imagery
Benchmarking and Mitigating UAV Perception Failure Under Rapid Motion Blur, Scale Shifts, and Occlusion
Robustness and Degradation Benchmarking of Map-Conditioned Perception Under Imperfect OpenStreetMap Priors

Open problems in Algorithmic Fairness

A Unified Formulation for Multiclass and Regression Targets under Postprocessing-Based Guarantees
Benchmarking Algorithmic Fairness Methods Under Violations of Sensitive Attribute Preconditions
Robustness and Sensitivity of Fairness Interventions Under Misspecified Merit Distributions and Latent Parameters

Open problems in Information Extraction & Entity Recognition

Error-Aware Entity Linking Evaluation and Propagation-Bounded Pipelines for Downstream NLP Tasks
Quantifying and Mitigating Upstream Parser Brittleness in Syntactic Information Extraction
Semantic Typing and Grounding of Pronominal and Nominal Mentions Across Divergent Coreference Schemas

Open problems in Low-Resource Languages & Dialects

A Benchmark and Labeling Framework for Community-Contingent Hate Speech Annotations, Replacing Single Gold Labels with Distributions of Plausible Ones
Reliability-Calibrated Annotation Protocols for Subjective and Low-Resource labeling Tasks
Metric Inflation and Reliability Limits in IPA-Space Generation Evaluation

Open problems in Molecular Modeling & Drug Design

Benchmarks and Training Protocols for Size-and-Structure Transfer in Molecular ML
Lossless Generalization Across Molecular Scaffold Diversity: Closing the Accuracy Gap on Unseen Compounds
Benchmarking Cross-System and Cross-Ensemble Generalization in Molecular Generative and Transition Models

Open problems in Person Re-Identification & Face Recognition

Robust Cross-Modality and Cross-Platform Retrieval for Person and Vehicle Re-Identification
Cross-Distribution Robustness and Granularity Evaluation in Cloth-Changing Person Re-Identification
Cross-Encoder and Target-Free Robustness Benchmarking for Facial Representations

Open problems in Text-to-Image & Video Generation/Editing

Dense Motion Supervision Without Optical-Flow Ground Truth
Systematically Benchmarking and Calibrating VLM-as-a-Judge Failure Modes in Image and Video Generation
Benchmarking and Mitigating Upstream LLM Prompting Dependencies in Text-to-Visual Generation

Open problems in Vision-Language Understanding & Captioning

Cross-Cultural Knowledge Grounding for Vision-Language Understanding
Robust Vision-Language Pre-training Under Cascading Pseudo-Label Noise
Benchmarking the Cross-Model and Cross-Failure Generalization of Vision-Language Hallucination Mitigation

Open problems in 3D Shape Modeling & Surface Reconstruction

Reducing the Computational and Fidelity Cost of Distilling 2D Diffusion Priors into Explicit 3D Representations
Cross-Category and Vocabulary Generalization Benchmark for Text-to-3D Shape Models

Open problems in Audio, Speech & Music Generation

Black-Box Optimization and Guidance for Audio Generation Without Target ASR White-Box Access
Conditioning Audio and Speech Generation on Unstructured Holistic Text Descriptions

Open problems in Bandits & Online Learning

A Standard Benchmark Suite and Evaluation Protocol for Validating Bandit, Online Learning, and Sequential Decision Algorithms on Real-World Logged Data
Instance-Dependent Robustness Guarantees Beyond Worst-Case Regret Bounds for Constrained Online Learning

Open problems in Camera Pose, SLAM & Structure-from-Motion

Robust Pose and 3D Structure Estimation Under Imperfect and Open-World Instance Masks
Reducing Reliance on Pre-Built 3D Supervision for Training 3D-Aware Perception and Reconstruction Models

Open problems in Causal Inference & Discovery

Robustness and Adaptation of Causal Frameworks to Arbitrary DAGs and Non-Linear SCMs
Benchmarking Causal Effect Estimation Across Multi-Valued and Continuous Treatment Regimes

Open problems in Diffusion Models

Black-Box and Few-Step Control of Diffusion Models Without Intermediate State Access
Extending Class-Conditional ImageNet Generation Methods to Text-Conditioned, Open-Domain Synthesis

Open problems in Efficient Attention & State-Space Models

Evaluating Toy-Task Inductive Biases and Mechanistic Claims Across Standard Pre-training and Deep Architectures
A Matched-Scale Testbed and Extrapolation Protocol for Testing Whether Toy-Scale Evidence Survives at Realistic Scale

Open problems in Equivariance & Symmetry in Neural Networks

Auditing the Gap Between Theoretical Equivariance Guarantees and Empirical Performance in Finite, Symmetry-Broken Architectures
Benchmarking Equivariance Degradation Across Discretization, Domain Shift, and Hybrid Pipelines

Open problems in Federated Learning

Client-Scalable Federated Learning Under Severe Data Partitioning and Heterogeneity
Eliminating Server-Side Auxiliary Data Dependencies in Federated Model Alignment and Calibration

Open problems in Inference Efficiency & Long Context

Systematic Cross-Scale Robustness and Evaluation of Efficient Inference and Long-Context Methods
Calibration-Free and Gradient-Free Predictors for Efficient Inference

Open problems in Language Agents & Tool Use

Benchmarking and Diagnosing Agent Framework Failures Across Open-Weight LLM Backbones
Benchmark Verification for Language Agents Without Ground-Truth Test Suites or Reference Repositories

Open problems in Multi-Agent & Goal-Conditioned RL

Evaluating the Robustness and Limits of Tree-Search RL Under Stochastic Dynamics and Function Approximation
Policy Generalization Across Large Agent-Count Shifts in Multi-Agent Reinforcement Learning

Open problems in Multimodal Fusion & Multi-Task Learning

Empirical Robustness and Scalability Benchmarking for Multi-View Multimodal Fusion
Systematic Robustness Benchmarking of Incomplete Multimodal Learning Under Non-Random and Structured Missingness

Open problems in Neural Text Generation & Decoding

Zero-Shot Machine-Generated Text Detection Under Strict API and Log-Probability Constraints
Benchmarking and Extending Machine-Generated Text Detection Under Black-Box, Mixed-Author, and Multi-Source Settings

Open problems in Out-of-Distribution & Anomaly Detection

Representation Adaptation and End-to-End Learning for Anomaly Detection Beyond ImageNet Backbones
Cross-Modality Robustness and Evaluation of Anomaly Detection Methods Across Tabular and Unstructured Data

Open problems in Quantization & Low-Precision Training

Characterizing and Mitigating Low-Bit Quantization Breakdown in Small-Scale Language Models
Post-Training Realization of Training-Time Quantization Objectives on Fixed Pre-Trained Models

Open problems in Question Answering & Reading Comprehension

Principled Allocation of Low-Rank Adaptation Capacity When Full Fine-Tuning Is Unaffordable
Multilingual Robustness of Table Question Answering Models

Open problems in Scientific ML: PDEs & Neural Operators

Systematic Generalisation and Robustness Evaluation of Neural Operators in 3D PDE Systems
Semi-Supervised Neural Operator Learning for Parameter-Scarce Trajectories

Open problems in Sentiment & Emotion Analysis

Cross-Lingual Robustness and Generalizability of Empathetic Dialogue Systems
Benchmarking Emotion Analysis and Decoding Across Model Access Regimes and Representation Constraints

Open problems in Uncertainty Quantification & Conformal Prediction

Principled Multiclass Extensions of Binary-Only Theoretical Guarantees in Prediction Methodology
Evaluating Uncertainty Quantification and Conformal Prediction Methods Beyond Binary Outcomes

Open problems in Visual Reasoning & Embodied Navigation

Systematic Generalisation and Robustness Evaluation of Visual Reasoning Methods Across Distinct Perceptual Regimes
Visual Reasoning Without Dense Scene-Graph and Rationale Preconditions

Open problems in Word Embeddings & Lexical Semantics

Robustness Benchmarking of Lexical Semantics Methods Under Language Model Scarcity and Distribution Shift
Cross-Lingual and Natural-Distribution Robustness Benchmarking for Paraphrase-Invariant Semantic Models

Open problems in Adversarial Robustness & Backdoor Attacks

Backdoor and Poisoning Attacks Under Constrained Training-Time Access

Open problems in Classical Supervised Learning: Trees, Ranking & Aggregation

Empirical Robustness and Scaling of Jacobian- and Pairwise-Based Aggregation Beyond BERT-Base

Open problems in Computational Social Science & Content Moderation

Benchmarking Systematic Biases and Reproducibility in LLM Pragmatic Discourse Annotation

Open problems in Depth & Stereo Estimation

Systematic Cross-Domain Real-World Evaluation of Synthetic-Trained Depth and Stereo Models

Open problems in Dialogue & Conversational Systems

Benchmarking Dialogue Management and Planning Beyond Predefined Action Spaces and Static Plans

Open problems in Differential Privacy

Robustness and Sensitivity of Public-Data-Assisted Differential Privacy Under Distribution Shift and Proxy Misspecification

Open problems in GANs, Style Transfer & Image Translation

Evaluating and Extending Latent Image Manipulation Beyond Domain-Specific StyleGAN2 Backbones

Open problems in Graph Neural Networks & Node Classification

Architecture-Agnostic Black-Box Explanation for Graph Neural Networks

Open problems in Human Pose & Motion Estimation

Robust 3D Pose Lifting Under Empirical Upstream 2D Detector Errors

Open problems in Image Restoration & Super-Resolution

Cross-Domain Robustness Audit of Image Restoration Models Under Physical Sensor and Optical Degradations

Open problems in Medical Imaging

Open-Vocabulary Diagnostic Knowledge Retrieval for Medical Imaging Without Predefined Ontologies

Open problems in Meta-Learning

Empirical Robustness and Precondition Boundaries of Meta-Learning Under Task Distribution Shifts

Open problems in Multilingual & Cross-Lingual Transfer

A Resource-Floor Benchmark: Measuring the Minimum Data, Lexical, and Pretrained-Model Prerequisites Below Which Multilingual NLP Methods Fail

Open problems in Recommender Systems

Cross-Domain and Interaction-Density Robustness of Sequential Recommender Systems

Open problems in Retrieval-Augmented Generation

Cross-Task Calibration and Failure Modes in RAG Hallucination Detectors

Open problems in Robot Learning & Embodied Control

Closed-Loop Trajectory Generation for Robot Manipulation Without External Motion Planners

Open problems in Self-Supervised & Contrastive Representation Learning

A Scaling-Law Characterization of Regularization and Transfer Benefits That Vanish at Scale

Open problems in Submodular Optimization & Approximation Algorithms

Robustness and Generalization of Graph Submodular Optimization to Weighted, Directed, and Attributed Networks

Open problems in Syntactic & Semantic Parsing

Benchmarking the Robustness of Syntax-Guided NLP Across Parser Degradation, Languages, and Formalisms

Open problems in Time-Series Forecasting

A Standard Benchmark and Protocol for Genuine Zero-Shot Transfer to Truly Unseen Domains in Structured Perception Tasks

Open problems in Video Tracking & Action Recognition

Diagnosing and Overcoming the Short-Horizon Saturation Barrier in Dense Video Models

Open problems in Video-Language Understanding & Temporal Grounding

Benchmarking and Mitigating Spatio-Temporal Grounding Degradation in Continuous Video-LLMs