My first work computer was a water cooled IBM 370 which lived in an air conditioned, soft static floored purpose built room, with a dedicated clean power supply. A powerful thing, it could support up to a dozen users.
Today my watch has more processing power and memory than that IBM 370.
Listen to the architects of Chat GPT, like Sam Altman of Open AI, and they will talk through the enormous increases in scale they’re achieving from each generation of the software, as the software teaches itself to create smarter models thousands of times smarter. And more energy efficient.