There are many startups claiming to use AI in ways that are not currently possible, for example to assess the soft skills of a job candidate. While researchers understand the limitations of AI, others with less ML experience may believe these false claims – especially in light of actual progress in areas like reinforcement learning and language modeling. In this presentation, Arvin Narayanan examines this phenomena and how it proliferates through the media and policy sphere. He outlines a few applications of AI that are possible today (i.e. that automate perception or judgement) and contrasts these with use cases that are not possible (i.e. that predict social outcomes); including by describing a large-scale study wherein ML researchers failed to accurately predict events like material hardship, GPA, and eviction given extensive data. He suggests that manual scoring will deliver better performance and mitigate algorithmic risk.