School of Social and Political Science

Description

Abstract

This paper explores scaling as material practice, imperative, belief system, imaginary and mathematical law(s) in today’s AI, in particular in the development of large language models. After briefly reviewing the place of LLMs and of scaling in the history of ‘connectionist’ AI, I’ll turn to what for this talk I will take as the canonical statement: ‘The intelligence of an AI model roughly equals the log of the resources used to train and run it … you can spend arbitrary amounts of money and get continuous and predictable gains’ (Sam Altman).It's worth taking this ‘law’ seriously, rather than simply trying to debunk it, because ‘arbitrary amounts of money’ are indeed being spent on AI ($7.8 trillion from 2025 to 2030, according to a Citigroup estimate), with, e.g., significant environmental consequences. The pivot of the talk is simple: a logarithmic function, at least of the kind Altman is referring to, manifests diminishing returns. Hence the questions: how far out do you go on a diminishing-returns curve, at the cost of how many trillion dollars and gigatonnes of carbon, and why?

Location: G.01 HSY Teaching Centre

High School Yards

Edinburgh, EH1 1LZ

And online - STIS Seminar Zoom Link:

https://ed-ac-uk.zoom.us/j/82140544359?pwd=Eoxde3Vj_00WiBRxOt47aCdJ0Dy3CITi.T-3gLpgTSI0MkJAy
Passcode:RJxn7F5R