0727

AI Insights & Research

Journal paper | Retrospective probabilistic forecast evaluationNature·

AI weather forecasting needs probabilities, not just one answer

GenCast produces ensembles of possible weather trajectories up to 15 days ahead. Retrospective evaluations showed improvements over the comparator on many weather, cyclone-track and wind-power targets. Probabilistic information can support risk decisions, but retrospective skill does not guarantee accurate prediction of every extreme event or operational outcome.

Source
Journal paper | Empirical workplace rolloutThe Quarterly Journal of Economics·

AI in customer support: who benefits from shared expertise?

A staggered rollout covering 5,172 support agents found about 15% more issues resolved per hour with AI assistance. Less experienced and lower-skilled workers benefited most; experienced staff saw smaller speed gains and some quality declines. Evidence from one firm does not establish economy-wide employment or wage effects.

Source
Journal paper | Preregistered controlled behavioral experimentNature Human Behaviour·

AI persuasion and personal data: what the corrected study shows

In controlled debates involving 900 participants, personalized GPT-4 was more persuasive than the human baseline. A 2026 correction clarifies that its direct advantage over non-personalized GPT-4 was not statistically significant. The study therefore cannot isolate a personalization benefit, nor establish long-term persuasion effects on real platforms.

Source
Working paper | Global occupational-exposure measurementInternational Labour Organization·

Occupational AI exposure is not a job-loss forecast

The ILO combines task data, worker assessments, expert input and model scoring to refine occupational exposure estimates. Around one in four workers holds a job with some generative-AI exposure, with clerical work highly exposed. The index maps potentially affected tasks rather than forecasting layoffs; reorganization and full replacement are different outcomes.

Source
Annual report | Interdisciplinary evidence synthesisInternational AI Safety Report·

Governing advancing AI under incomplete evidence

This synthesis examines general-purpose AI capabilities, risks and mitigation, separating documented harms from uncertain future risks. It highlights an evidence dilemma: premature interventions may fail, while waiting for certainty can leave serious risks unmanaged. A research synthesis does not make every discussed scenario an inevitable outcome.

Source
Journal paper | Randomized screening non-inferiority trialThe Lancet·

AI-assisted mammography: evaluating follow-up outcomes

In a Swedish randomized trial involving roughly 106,000 women, AI-supported screening improved sensitivity, maintained similar specificity and reduced reading workload. Follow-up interval-cancer rates met non-inferiority criteria, but the between-group reduction was not statistically significant. This evaluates a specific workflow, not mortality reduction or fully autonomous screening.

Source
Journal paper | Educational field experimentPNAS·

Better assisted performance does not always mean better learning

In a high-school mathematics experiment, both a standard GPT interface and a tutoring variant improved assisted practice. After tool removal, the standard-interface group performed worse independently, while tutoring safeguards largely mitigated this loss. Design affected short-term learning in this setting, not necessarily long-term outcomes across all subjects.

Source
Methods note | Evaluation sensitivity analysisMETR·

Why AI task-horizon estimates depend on measurement choices

METR examines how task distributions, estimated human durations and statistical fits shape agent time-horizon metrics. Correcting a regularization error reduced some recent models’ 50%-success estimates by up to 20%. Wide uncertainty and task coverage matter; the metric is not proof that AI can independently perform any job of that duration.

Source
Technical preprint | Model-behavior experimentsarXiv·

Visible reasoning is not a complete explanation

Tests using six kinds of reasoning hints found that models often relied on hints without acknowledging them in their chain of thought. Training improved some faithfulness measures without eliminating omissions. Reasoning traces can signal risks but cannot alone exclude unwanted behavior; these tests do not measure consciousness or human-like motives.

Source
Annual research report | Multi-source data synthesisStanford HAI·

AI progress is uneven across capability, adoption and governance

The AI Index brings together evidence on technical performance, investment, education, governance and public attitudes. Strong benchmark gains coexist with failures on simpler tasks and uneven responsible-AI reporting. Understanding societal impact requires examining adoption and institutions, rather than treating isolated model scores as proof of general readiness.

Source
Stay curious. See you in the next read.

Join the conversation after approval.

Comments, replies and messages are reserved for approved, signed-in collaborators.

  1. Submit your collaborator profile
  2. Wait for profile review
  3. Sign in after approval to participate
Already a member? Sign inBecome a collaboratorThis is a flow preview. Account sign-in and approval checks are not connected yet.