Large-scale language models have revolutionized the field of artificial intelligence over the past few years. Two of the most important people in the field, whose research has played a pivotal role in the development of modern AI, first met under very interesting circumstances.
Nobel laureate Jeffrey Hinton spoke about his encounter with Ilya Satskeva, former principal scientist at OpenAI. In 2007, Jeffrey Hinton was teaching at the University of Toronto, and Ilya Satskeva was completing his master’s degree in computer science. Around that time, Jeffrey Hinton says he first came into contact with Elijah.
“I was in the office, I think it was Sunday, programming,” Hinton says. “I heard a knock on the door. It wasn’t just a knock, it was some kind of urgent knock. So I answered the door and this was a young student there. , and he said he’d been cooking French fries over the summer, but he’d rather work in my lab,” Hinton recalls.
It’s common now for professors to be approached in this way by students, but Ilya was different from other professors. “I said, well, let’s make an appointment and talk.” So (Ilya) said, how about now? That was Elijah’s personality,” Hinton says.
“So we talked a little bit and I gave him a paper to read. It was a Nature paper on backpropagation. Then we had another meeting a week later, and he came back, and he said I didn’t understand it,” Hinton recalls.
“And I was very disappointed. I thought he seemed like a bright guy, but it’s just the law of chaining. It’s not that hard to understand. And he said, ‘Oh, no, no, I understand that.’ I did. I don’t understand why you don’t give a gradient to a rational function optimizer. It took me years to think about it, but it stuck with me. “He was very talented and his raw instincts about things were always very good,” he recalls.
It turns out that Ilya Satskeva not only fully understood the paper, but also pointed out ways to improve the technology in ways that took scientists years to realize. Maybe that’s when Hinton realized Elijah was special. Elijah eventually began pursuing a PhD in computer science under Hinton and received his degree in 2013. Hinton and Sutskever, along with Alex Krizhevsky, developed AlexNet, a CNN network that became a kind of forerunner of modern AI models. In 2013, Geoffrey Hinton and Ilya Sutskever started working at Google Brain after the company they founded was acquired by Google. Hinton remained at Google, and Ilya Sutskeva joined Greg Brockman and Sam Altman to launch OpenAI in 2015.
Of course, OpenAI launched ChatGPT in late 2022, ushering in the current AI revolution. That research was supervised by Ilya Sutskever, who was the company’s chief scientist. Jeffrey Hinton won the Nobel Prize in Physics this year for his invention that made machine learning possible. Meanwhile, Ilya left OpenAI under difficult circumstances and is now working on SSI, which he says is a one-off attempt to discover superintelligence. And while we haven’t quite reached superintelligence yet, one wonders how different the world of AI might have been if Elijah hadn’t knocked on Professor Hinton’s door that Sunday all those years ago. will make you wonder.
