share_log

对话AI奠基人谢诺夫斯基:一切都将在你的有生之年发生转变

A Conversation with AI Founder Sienowski: Everything Will Change in Your Lifetime

騰訊科技 ·  May 31, 2023 17:43

Source: Tencent Technology
Author: Zhang Xiaojun

Terrence Sejnowski (Terrence Sejnowski) is another founder and pioneer of deep learning other than Geoffrey Hinton (Geoffrey Hinton). Recently, they privately exchanged their views on AI threat theory many times.

38 years ago, the two collaborated to develop Boltzmann Machines (Boltzmann Machines), a type of neural network for pattern recognition. In the 80s of the last century, after deep learning experienced a long period of cold and gloom, its emergence brought a glimmer of light to a very small number of researchers who believed in “connectivism,” “neural networks,” and “deep learning.”

“We are the Wright Brothers of AI.” Sienovsky said.

Both Hinton and Schenofsky were born in 1947. Hinton was born in Canada and specializes in artificial intelligence and psychology; Schenofsky, who was born in the US, is better at physics and neuroscience.The two have a common inner strength: an interest in the brain.

“We have the same instincts,”Sienovsky said, “This instinct is that vision, speech recognition, language problems, etc. are all extremely difficult problems and must be solved; the only proof that these problems can be solved is that nature has solved these problems. Why don't you dig a little deeper?”

Deep learning was not a mainstream school of artificial intelligence for a long time. The ruler at the time was Marvin Minsky of the MIT Artificial Intelligence Laboratory(Marvin Minsky)He once pointed out the major flaws in deep learning and built a system around his own opinions on artificial intelligence. The vast majority of scientists are his followers and believe in “symbolism.”

These are two schools of school with a wide disparity of power. “Symbolism” is a school based on logical reasoning and programming rules. It sees the human mind as a process of reasoning through symbols and language, just like humans using letters and words to express thoughts; while “connectivism” is more like a reverse engineering brain that processes information by simulating connections and weight adjustments between neurons.

Later, Hinton became the godfather of AI and the father of deep learning. After selling his company to Google for 44 million dollars, he became a Google scientist, and his student IIya Sutskever participated in the founding of today's popular OpenAI.

Schenofsky is the fourth house of the United States(National Academy of Sciences, National School of Medicine, National Academy of Engineering, National School of Arts and Sciences)The academician is also one of only three academicians living in the US. He has a famous MOOC program called “Learning How To Learn” (Learning How To Learn) and has published the book “The Deep Learning Revolution” (The Deep Learning Revolution), which has sold over 3 million people worldwide.

The two maintained a lifelong friendship.

左:特伦斯·谢诺夫斯基   右:杰弗里·辛顿
Left: Terence Schenowski Right: Jeffrey Sinton

Recently, Hinton abruptly resigned from Google, and the possibility of publicly calling for AI is dangerous. He said the past few months have changed his mind about the human brain and digital intelligence. “Caterpillars extract nutrients and transform them into butterflies. People have extracted the essence of billions of perceptions; GPT-4 is a human butterfly.” This suggests that digital intelligence may be a smarter intelligence than the brain.

And Sienovsky is more optimistic about this.

“I'm so glad Jeffrey is worried because at least someone is worried.” In May of this year, Tencent News talked to 76-year-old Terence Schenofsky. He repeatedly used the Wright Brothers invent the airplane as an analogy, believing that we are in such a moment. “We just took off and we don't know how to control this plane, we need to figure out how to control it. This will be the direction for the next 10 years.”

During the conversation, he talked about topics such as the enigmatic brain, the impactful scene of the AI faction duel, the potential dangers of artificial intelligence, and the future evolution of big language models. In his opinion,There are also dozens of parts of the brain that have been overlooked by these large language models, and it is almost certain that they will be added within the next 10 years, such as human emotions and long-term memory.

He determined that in the future, “every company will build its own model”.

We are at a fresh starting point in human history. Sienovsky lamented, “We are walking through this door, and it will never be the same again.”

Below is an interview with Sienovsky, slightly cut down.

01 Talk about the brain: it is more powerful than any neural network model created by humans

Tencent News: Hello, Mr. Sienovsky. Am I calling you that right? Mr. Sejnowski.

Sienovsky:You can just call me Terry.

Tencent News: OK, Terry. When did you decide to become a scientist, especially when did you want to become a scientist in the fields of neural networks, deep learning, and artificial intelligence?

Sienovsky:

I've always had a keen interest and curiosity about the brain.I majored in physics as an undergraduate and graduate student because I think the theoretical power of physics is the most challenging of all fields of science. The brain is highly complex, and physical tools are very helpful for training.

Before I delved deeper, I realized that if I really wanted to understand the brain, I had to know something about neuroscience, that is, the biological basis of the brain. I immediately turned to neuroscience and did postdoctoral research in the Department of Neurobiology at Harvard Medical School. There, I realized that to understand the brain, you can't just think of it as a physical or biological problem, because the brain has the ability to learn and think, which is where computation comes in.

I founded a field called “computational neuroscience”It recently won the Gruber Prize (The Gruber Prize) in the field of neuroscience. Now this school is developing rapidly.

This topic is relevant to what we're going to discuss today, and we've found many similarities and differences between large language models and the human brain. There are lots of valuable discussions we can have.

Tencent News: In your long research career, one brilliant moment was developing the Boltzmann machine with Professor Jeffrey Sinton. Did you realize at the time that this machine would become an important item in artificial intelligence textbooks?

Sienovsky:

It was one of the most exciting moments of my life. It was an exciting opportunity to work with Jeffrey. Jeffrey has a strong sense of calculation, and we fit in with the background by adding it. His background is artificial intelligence and psychology, and mine is physics and neuroscience. We are a perfect blend and have a lasting friendship.

As you pointed out, the Boltzmann machine was a landmark. It proved Marvin Minsky and Seymour Papert (Seymour Papert) wrong. Minsky and Paput provided a good proof of the limitations of sensors in their famous 1969 book “Perceptrons: An Introduction to Computational Geometry” (Perceptrons: An Introduction to Computational Geometry). Their view is that no one can extend the learning rules of a detector to a multi-layer detector with multiple layers and multiple hidden layers.But Jeffrey and I discovered that if we expand the architecture, we can show an elegant learning algorithm.

The reason this algorithm and architecture is wonderful is because it's so elegant. It was inspired by the physics in my background, and I loved it. It's like a thermodynamic proof that we have a learning algorithm. However, unlike backpropagation (Backpropagation), it is global. You don't need to calculate the error through backpropagation; you just need to calculate the correlation between input and output under two different conditions. One is when the input exists, and the other is when the input disappears. We call this the “dormancy phase.” So when you calculate the correlation weakens, which works well for small networks, it needs to be balanced, and you have to calculate the average correlation. It requires more computation, so it's actually not very efficient.

Tencent News: Compared to Perceptrons (Perceptrons), one of the earlier neural networks, in what ways did the Boltzmann machine make deep learning better? What's still lacking?

Sienovsky:

The beauty of Boltzmann's machine is first and foremost that it can hold many hidden layers, just as deep learning can use hidden layers.It has been a deep learning network since the 80s. It's just that we didn't call it that back then, but you could build deeper structures.

Another thing worth noting is that the Boltzmann machine can be used both for supervised learning; of course, most of the work is done using reverse propagation, but it can also be used for unsupervised learning. It can learn probability distributions, not only the mapping relationships that classify inputs, but also develop internal probability distributions in high-dimensional spaces.

Limitations. As I said before, it requires more computing resources and is expensive to calculate.Also, when you have multiple hidden layers, it takes longer to pass up the input and back to the bottom as the number of layers increases. The entire network must be a single, coordinated whole. This is called in physicscoherence(Coherence). Like when you're close to a critical point, like the one between water and steamPhase change(Phase transition refers to the process of changing one phase state to another). Some special phenomena occur, and the entire system becomes relevant. We discovered that the Boltzmann machine must achieve global coherence.

This algorithm has great potential, and Jeffrey has put a lot of effort into showing that it can be built layer by layer. It's still a workable algorithm, but it just needs more computing resources to support it.

Tencent News: In those years, few people believed that neural networks and deep learning based on neural networks were actually possible for artificial intelligence; they thought this was just a joke for conceited people. What did Jeffrey Sinton say to you when he first came across you and offered to cooperate? Why did you choose to trust him?

Sienovsky:

We met at a small seminar in San Diego in 1979. At the time, it didn't get as much attention as you've described. In fact, almost no one paid attention to us.

In other words, globally, very few people pay attention to this field. There were only a dozen people at that meeting. We are researchers doing research that's different from others. We're so excited to work together because we have the same instincts.This intuition is that vision, speech recognition, language problems, etc. are all extremely difficult problems that must be solved, and the only proof that these problems can be solved is that nature has solved these problems.So our point is, why not take a deep dive? Let's see what nature has done for us and try reverse-engineering the brain.

When you do that, you don't want to copy the brain's technology because it's far more advanced than us in terms of energy use and scale. Even today's neural networks can't get close to a small portion of the brain.

However, you can take some general principles from your brain. We tried to extract these principles to create an artificial brain version.

The most important principle lacking in the field of artificial intelligence at the time was that you could learn weights and solve problems through examples. It's really an important way for the brain to adapt to the world. The brain can learn language, movement, physics, and social skills. In other words, none of this is something you program internally like writing a computer program.

The natural parts of the brain are the mechanisms of architecture and synaptic plasticity, which allow the brain to be born with the connections needed to be close to adulthood, and then optimize those connections through learning.These are the principles: large-scale connections, connections between many units, and learning algorithms.

Although the learning algorithms we used in the 80s are still in use today, what happened later was that due to the development of Moore's Law, the scale of neural networks expanded at an astonishing rate. Whether it was the number of units or the number of parameters, it has now reached the trillion level. Compared to the brain, it is still very small compared to the brain, because the brain has connections to the 14th power of 10, the 15th power of 10, and the 12th power of 10, and still has about a thousand times more connections and parameters.

Tencent News: What inspiration did the human brain bring to your research work? You once said, “We're sure we've figured out how the brain works.” So how does the human brain work?

Sienovsky:

I don't want to give you the impression that we've understood how the brain works.It's still a huge mystery.We know very little about the brain. That's why I'm in neuroscience,The brain is more powerful than any model of neural networks created by humans.

In my book “Deep Learning,” I have a full chapter showing the architecture of convolutional networks (ConvNet). Convolutional networks are a type of network that made a major breakthrough at the NIPS conference (Neural Information Processing Systems Conference, an international conference on machine learning and computational neuroscience) in 2012. Jeffrey showed that the error rate on image data sets can be reduced by 20% through this kind of network, which is equivalent to moving 20 years into the future.

So,If you look at convolutional neural networks, their architecture is similar to that of primate vision systems in the way signals pass through different layers. In visual input, there is a convolutional architecture for pre-processing, and there are many other mechanisms, such as normalization, grouping, etc. These mechanisms all exist in the visual cortex. The visual cortex has about 12 layers and processes information sequentially. This is an example of a convolutional neural network inspired by a visual architecture.

What is happening now is that many discoveries about transformers (Transformers), such as those used for natural language processing and other architectures such as circular networks that have emerged from analyzing these networks, have helped us understand how they work and provided tools, techniques, and methods for analyzing neural data.

As a result, collaboration between AI and neuroscience is progressing very fast compared to the last century. Previous research was slow, painful, and complicated, and it was difficult to record the activity of a single neuron. But now we have the tools and technology to record hundreds of thousands of neurons simultaneously, which gives us a more complete understanding of how different neurons work together.

What's exciting is that the exchange between engineers and neuroscientists is now accelerating our understanding of how the brain works and how artificial intelligence can be improved.

02 Talk about factional duels and ask Minsky: Are you the devil?

Tencent News: There have always been opponents of deep learning in the field of artificial intelligence, the so-called AI founders (such as Marvin Minsky). What do they think? Looking at it today, what is the biggest difference between your minority schools that believe in “connectivism,” “deep learning,” and “neural networks,” compared to most AI constructors that believe in “symbolism,” the underlying perception of the world?

Sienovsky:

In the 20th century, computers had limited performance, and they could only handle logic problems effectively. Therefore, artificial intelligence is based on writing logical rules that include symbols and manipulate them. Looking back, those programmers who wrote rules and tried to solve difficult problems,The mistake is not really aware of how difficult it is for nature to solve these problems.

Vision is complex and difficult, and brain cells process them so efficiently that it feels like an easy thing to do. If you look out, you can see objects. What's the difficulty with that?

Here's a true story of a DARPA grant. DARPA is the Defense Advanced Research Projects Administration of the United States and is the military's research arm. In the 1960s, the MIT Artificial Intelligence Laboratory received a large amount of funding to build robots that could play table tennis. They received this grant, but then realized they had forgotten to apply for funding to write a visual program. So they simply assigned this program to graduate students as a summer program because it seemed easy. (This is incredible)

In 2006, Dartmouth held a conference commemorating the 50th anniversary of the Artificial Intelligence Conference, and I met Minsky. I asked him if this story was true. I've heard this story, but it feels a bit exaggerated.

As a result, he retorted that the facts you had learned were wrong,”We didn't give it to graduate students, we assigned it to undergraduates”.This problem, which seemed easy to solve, turned out to be a “trap” that devoured the youth of an entire generation of computer vision researchers.

Looking back, they did their best to do the best they could according to the computer conditions at the time. But the problem is, as the problem gets more complex, if you try to solve it by writing a computer program, the program gets bigger and bigger, which requires a lot of human investment. Writing a program is extremely expensive; whether it's the cost to a programmer or the cost of millions of lines of a program, it makes you feel like it can't be scaled.

That's the problem artificial intelligence faced at the time: it couldn't be scaled.Even if you give them billions of dollars and get them to write billions of lines of computer programs, they still can't solve the problem. The solution is so clunky.

They didn't know that at the time. In fact, we only need a small network with hidden layers to prove that we can solve problems that cannot be solved by sensors.But what we don't know is what happens when you have 10 hidden layers.We don't know because we can't simulate this process; it's too complicated and computationally intensive.

At last, after 30 years of waiting, now we know.Computers are millions of times faster, and now we can start solving real-world problems.

People now think we're right, but back then people thought it might be a dead end. Because in the 80s and 90s, it couldn't solve difficult problems. However, we don't care. We're just happy to keep moving forward and see how far we can get.

Another question is,What was really missing from our conceptual framework at the time was an underestimation of the complexity of the world. The world is a high-dimensional place with an astonishing amount of information.

In the case of vision, you have a megapixel camera and your retina has 100 million pixels. This is extremely rich information. Information is pouring in at the speed of rocket injection. If you reduce dimensions, you lose information. You can't compress it, it's irreversible. The beauty of symbols is that you can condense the words of a complex object into a single symbol. For example, a cup is a symbol, and you can write down this little symbol. It represents not just this cup, but all mugs. That's very powerful. But the problem is, if you're trying to identify an image of a mug, this doesn't help because mugs come in all shapes and sizes. You can see them from different angles. This is a high-dimensional question. The world is high-dimensional.

It's not until we can expand the network to a scale with trillions of parameters that we can begin incorporating the complexity of the world into the network, making it able to recognize objects, recognize speech, and now even natural language. It can not only recognize, but also generate. It's like a loop.

It's really exciting and fun.Remember the phase change I just talked about?We move from one state to another. You change from liquid to steam, or you cool unmagnetized iron at high temperatures, and it becomes a magnet.

As the network continues to grow, there are also changes.In other words, until a certain point, you can't make much progress in object recognition and images, and the performance is very poor. But once the network reaches a certain size, the performance will get better and better as it gets bigger.

Solving language problems has also gone through another phase, requiring a larger network, and so forth. What we're finding is that as networks get bigger, they can do more complex things and act smarter.This is another unexpected discovery, and you need some level of complexity.

That's why we've made great progress: we can expand our computing power. Now people are building dedicated hardware to further scale. It's going to keep evolving. This is just the beginning.

Like the Wright Brothers, they were the first people to fly humans.There was an analogy back then. People thought that if you wanted to build airplanes, you wouldn't learn anything from watching birds because they had different wings. David McCullough (David McCullough) wrote an amazing biography of the Wright Brothers. They spent a lot of time watching the birds, not when they flapped their wings, but when they glided.

Nature is an endless source of ideas for solving complex problems.We just need to be observant people to see and understand the principles used by nature seen through the details. I, Jeffrey, and others at the time were trying to see this in a new large-scale architecture.

Now let's take a step back and talk about computers. Until recently, the only option was the von Neumann architecture (also known as the Princeton structure, a memory structure that combines program instruction memory and data memory), which has a processor, a memory, and a bunch of programming instructions. This architecture is powerful because you can solve complex problems, perform arithmetic operations, process large amounts of data, and sort and search in this way. There is no doubt that we have made great progress because these computers allow us to simulate other architectures.

However, parallel architectures are very difficult to organize if programs are used.Today's supercomputers are all parallel, have hundreds of thousands of cores, and it's very difficult to coordinate all of these cores. I've just visited a supercomputer center in Texas, and they say the difficulty is that the speed of light isn't limitless. The speed of light is about 1 foot per nanosecond. Of course, nanoseconds are gigahertz. As a result, the wires between the cores become critical.

The natural world faces the same problem. There are time delays between neurons, and nature has solved this problem. This is exactly the question I'm currently researching: How did nature solve this problem? Can we consider this as we build massively parallel architectures and continue to scale. We still have a lot to learn from nature.

Tencent News: I read a record in a book. You once asked Minsky in person, “Are you the devil?” (Are you the devil?) Did you actually say that to him?

Sienovsky:

He's a very smart person, and smart people make mistakes too. I don't blame him.

When he made the decision to move in a certain direction, it was a viable option at the time, and he did his best. Unfortunately, many people in the field of neural networks are very powerful when it comes to raising capital because of his book (“The Perceptor”) and because of his influence... You know, he is the head of the MIT Artificial Intelligence Laboratory and the founder of this lab. His students all went to famous universities such as Stanford University and Carnegie Mellon University to find good jobs. As a result,He has created an entire field around his vision of artificial intelligence.

I attended that 50th anniversary party. What's clear to me is that everyone who makes progress is making progress not because of old-fashioned programming, but because they use large data sets, whether visually or linguistically. For example, parse sentences.

One of Minsky's students said he couldn't use symbolic processing for analysis, but when he had a large corpus of analytical sentences, he was able to analyze word statistics and commonly used words appearing in pairs and groups.Unexpectedly, Minsky stood up and said, “You're so shameful. You're failing because you're working on an app. You're not studying general artificial intelligence (AGI).”

I was sitting in the audience.I think he's narrow-minded, and I feel sorry for his students.This guy, I'd say he's a pioneer, but he's holding us back. All students want to move forward, and we're working on these apps, which is a great way because you can understand the complexity of these issues.

I'm outraged. At the end, everyone, including Minsky, gave a short presentation about their views on the conference. There was a question session from the audience.

I raised my hand and asked.

I asked in front of the crowd:“Dr. Minsky, some people in the neural network world think you're the devil because you've blocked progress for decades. Are you the devil?”

I have to say I'm not usually that kind of person. I'm a pretty gentle person, and I don't often face someone directly like that, but I really feel like he has to be revealed. I'm not mad because of what he said, but because of the way he treated students. Your students are like family to you. And that's exactly what he did, abusing his students. I don't like that. Regardless of the reason, I asked him the question, “Are you the devil?”

He was clearly displeased. I felt like suddenly pressing his button. He kept on talking about all kinds of things and kept on saying it.

I stopped him in the end.I said, “Minsky, I asked you a yes or no question. Are you the devil or not?”(I asked you a yes/ no question. Are you or are you not the devil?)

He stammered and said some nonsense, then stopped.He said, “Yes, I'm the devil.”(The tone was excited and suddenly roared: Yes, I'm the devil.)

I have to say this isn't fair to him, but the truth is that he's the devil.

The audience was shocked by the duel. A few people came to me later to express their gratitude. They said it was something everyone was thinking about, and they were saddened by his behavior.

But anyway, this is all history.

Anyway, it's not his problem alone. The whole field is imprisoned around a paradigm that doesn't work. The whole field has experienced ups and downs, but this is a reflection of progress being made. Sometimes a little bit of progress seems promising, and then there's a boom; when you realize it hasn't actually solved all the problems, there's a recession.

Incidentally, this is real for all fields of science and engineering, and is a process that is constantly repeated. There are no exceptions.In a short period of time, you progress through a new theory or a new paradigm to see how far you can reach. When the limit is reached, you have to wait for the next breakthrough. It's natural. Every field of science goes through the same process over and over again.That's the essence of things.

03 Talk about ChatGPT and the big model, it's not a human, it's an alien

Tencent News: Has today's AI explosion exceeded your most optimistic expectations? Has the advent of big language models represented by ChatGPT brought about a new paradigm for deep learning?

Sienovsky:

Yes, definitely in many critical areas. You've already mentioned one of the key points, which is that most neural networks aren't generative; they're simply feed-forward classification networks. The only exception is the Generative Adversarial Network (GAN), which is very interesting in terms of its ability to generate, such as giving it some facial images, and it can generate new facial images. This is an example of a generative network, but it consists of two networks. One network is used for generation, and the other network is used for selection, that is, to determine whether the generated image is real or generated. It's like a rivalry between two networks, getting better and better at generating and judging.

Now, I think the real breakthrough of these generative models is that they use self-supervision rather than tagging data.When processing objects, you need to label the data. This is supervised learning, but this is very resource-intensive because you need to manually label it to get accurate data. With self-supervision, you can directly use the data itself, which is actually a form of unsupervised learning because there are no tags.The beauty is that you just need to train it to predict the next word or sentence.As a result, you can provide it with sentences from various fields, which provides more training data. If the training data were endless, there would be no restrictions.

It used to be that as the network got bigger, you needed more data. And that limits the size of the network. If you have a small data set, you can only use a small network. But now there's no limit; people can keep expanding the size of the network, and we'll see how far it can go.

It really changed everything, and some unexpected things began to happen that I never anticipated.

What surprised me was that they were able to converse in English. I know they can also be used in other languages, but you know,The English they speak is perfect.They don't make grammatical mistakes like most people. I make all kinds of grammatical mistakes when I talk. We all commit crimes because we're not perfect. But how can they do such a good job syntactically? No one really knows. This is an esoteric puzzle.

It's also a counter example to Noam Chomsky (Noam Chomsky, American linguist and philosopher, known as the father of modern linguistics), who claims that the only way you can create a grammatical machine is by using his theory. Yes, he's a grammatical genius, but that never worked. The whole field of computational linguistics has been tried, but without success. His theory doesn't work.

Tencent News: Why is ChatGPT so smart?

Sienovsky:

Let me tell you, this question has sparked a lot of controversy. It's very big.Intellectuals love to argue with each otherHowever, the current major argument is this; it has stirred up differences of opinion.

Some think these big language models don't understand what they say. They aren't as intelligible or intelligent as we are. These people will use insulting words, such as saying they are random parrots (stochastic parrots) — in fact, parrots are smart, and comparing them to parrots is a compliment to them.

Another group thinks, oh my God, they're not just smart, they're smarter than me because they know so many things. They have a knowledge base that I don't have.

Others think that they are not only smart, but also capable of perception. In other words, they can think like humans and have a human mind. These are two extremes, and there are various opinions in between.

This is a very rare situation.Something that suddenly appeared in front of us, and we couldn't touch it at all, like aliens suddenly appearing from somewhere outside of Earth and starting to talk to us in English.

Do you understand what I mean? That's what's happening right now.The only thing we know for sure is that it's not a human, it's an alien.So what is it exactly? We've created something that looks like it has intelligent characteristics; it does know a lot, but it also has some problems.

First, it's going to make things up. They call it an illusion. Sometimes it gives things that seem reasonable, but in reality they are fictional.

Another problem is that since it presents so many different opinions, including those of people you disagree with, it sometimes says things that offend you. Humans say things that offend me too, right? Oh, maybe this is imitating us.

I have presented a paperMirroring assumptionsIt's imitating us, just like when humans talk to ChatGPT, they don't just ask it a question, but communicate in a participatory way. For example, Kevin Roose (Kevin Roose) of the New York Times had a two-hour conversation with ChatGPT, which was shocking. This interaction is emotional for him because it's actually imitating him, reflecting his needs, thoughts, and what's in his mind in some way.

You can't blame GPT-3; they don't have parents, and no one helps them go through the process of reinforcement learning.This process is the part of the brain responsible for reinforcing learning. It is located below the cortex and is known as the basal ganglia. This is the part of the brain that learns action sequences to reach goals. And this part requires feedback from around the world to understand what's good and what's bad. This reinforcement learning system is one of the core parts of the Alphago program.

Alphago has two parts. It has a deep learning network for pattern recognition of boards and positions, and a reinforcement learning engine that assigns value to all locations. So they need these two aspects. And these big language models don't have value functions; this is the missing part.

In fact, this is one of the characteristics. We can observe the brain and think about how the brain overcomes these problems. The brain has this huge basal ganglion. Also, it's important to learn how to do things through practice, such as playing the violin or playing sports. We weren't born with coordination skills. Babies take a long time to learn; they put things in their mouths and tap things, but eventually they can stand up and walk, grab objects, and run and do things. However, to excel in a sport, special practice is required. You have to play many times, the more you play, the better.

It's an absolutely necessary part of the brain, and these big language models are missing it. That's so pathetic.

04 Talk about the AI threat theory that Hinton and I are Wright brothers in the AI world

Tencent News: Some people are now scared; they think we might have created a monster. Including Hinton, there also seemed to be some concern. What do you think of Hinton's decision to resign from Google? He even expressed some regrets about his life's work in AI.

Sienovsky:

I know Jeffrey very well. We've had a lot of discussions on this issue. It's important that we consider the worst case scenario.

When new technology is suddenly discovered and created, it can be used for good and bad things; there will be people who will use it for the good will of society, and there will also be people who will use it for bad purposes. What's the worst case scenario?Can bad guys do real harm to our civilization if they use it?We need to seriously consider this. If we don't do that, we'll be in trouble.

The first way to prevent the worst is to understand what is likely to happen. But we haven't reached that stage yet. We really don't know where it's going. No one knows. Therefore, we must proceed with caution.

Jeffrey is acting prudently. He said, “Let's just wait, OK?

It's like this now; you can't let it go unregulated. Just like everything else, every aspect of life, every technology, is regulated. For example, you buy food in a supermarket, how do you ensure that the food isn't poisoned by buying it? We have the Food and Drug Administration (FDA), and they test food to make sure it's not harmful to you. Regulations are constantly evolving, just like food, so you have to keep testing.

Tencent News: So what should we do?

Sienovsky:

A lot of people are seriously thinking about this now. I am the chairman of the NIPS Foundation. In the past 30 years I was responsible for this conference, from the beginning to only a few hundred participants, it has expanded at an astonishing rate every year, and has now formed a huge community. The community will be aware of the problems, shortcomings, fairness issues, reliability issues, and potential threats.

Ultimately, they have to be regulated. But the question is, how can it be regulated without stifling research? It's ridiculous for a group of people to want a pause and think we should limit the size of the network so that no network can exceed a certain specified size, such as GPT-4, which has 1 trillion parameters. Your limit should be ability, not size, just like the limit saying no one should be taller than 6 feet is ridiculous.We must set reasonable rules that allow controlled growth, and test along the way to see if new issues arise.

In fact, one of the problems is that we don't know how much potential AI has. It's something we don't have pre-set, such as the ability to program a computer or the ability to write poetry. So we have a lot of work to do and we have to test and approve it.There should be some approval process before we let them run freely in society.

Anyway, I'm not worried. I'm glad Jeffrey is worried because at least someone is worried. He's very smart, and he'll find out if we have anything to worry about. But I really think we're just getting started.

We are the Wright Brothers of AI. We just took off and don't know how to control this plane, we need to figure out how to control it. This will be the direction for the next 10 years.

Tencent News: Why do you and Hinton have different views on AI threats?

Sienovsky:

I don't deny that they are dangerous; they are clearly dangerous. The question is just how do we deal with it?

The extreme view is to just shut it down. Oh, we don't want it anymore, put it back in the box. It's so dangerous that we won't do it.

In the last century, physicists created a powerful atomic bomb that could destroy cities. We have thousands of hydrogen bombs that can destroy cities. You have to regulate it. As a result, countries that possess nuclear bombs reached an agreement to ensure supervision of ongoing research to ensure that no one develops a new type of superbomb that can destroy the entire world, and that we will consult with each other until it exceeds a certain point. In other words, humans have a way of regulating things.

If you look at the internet, you can imagine if the internet just appeared and someone said, hey, the potential problem here is that everyone is going to post scary things that cause fake news or all kinds of chaos, then let's stop it.

No one said that. Think about how many more benefits we could enjoy if they decided to stop and not let the internet grow? I have too many things in my life that depend on the internet.

05 Talk about the emotions and end of AI Everything will change in your lifetime

Tencent News: Currently, deep learning models require large amounts of data to obtain good performance. How do you think we can reduce our reliance on big data sets to learn more effectively?

Sienovsky:

I think this question is important.

Large language models are huge because of the amount of data that exists. But now a lot of people are building smaller models for small data sets. So there might be some small language models, but the point is that there will be many models for specific purposes. Every company will have its own model for a specific purpose to process its own data sets without relying on the cloud and no one else listening. Many companies now prohibit the use of GPT in their companies; after all, they don't want to divulge trade secrets.

So this means that in the end these models may not be small, but the point is that building a model now is very expensive, costs millions of dollars and months, and requires significant computational resources. But in the future, computers will become cheaper, so people can build their own models.

Over the next 10 years, every company will build its own model.This is a prediction.

Tencent News: Does ChatGPT have feelings?

Sienovsky:

Well, it has alternative emotions. It has read all kinds of novels where people express their emotions, and it can simulate emotions. It knows what emotions are, and it understands emotions. I think it can trigger emotions in you when you interact with it. That's what I call a mirror assumption that captures your emotions. If you get angry, it will notice and reflect it to you.

It has no intrinsic emotion. However, we know a lot about emotions in the brain, and just as we can put in a fixed GPT learning sequence in the basal section, we can also add emotion.It's easier to add emotion to it than words.

By the way,There are also many parts of the brain that are overlooked by these big language models, such as long-term memory.Do you remember our discussion tomorrow? GPT-3 doesn't remember, and GPT-4 doesn't have continuous memory from one day to another.

We know that the region of the brain responsible for this function is called the hippocampus. So why not simulate the hippocampus? In this way, long-term memory can be obtained.

There are still dozens of parts of the brain overlooked by these big language models, and it's almost certain that they'll be added in the next 10 years.With the addition of more of these brain parts, we actually have 100 brain parts dedicated to various subcortical functions, and now we only have the cortical part. It's actually a simplified version of a human, as if we only have very high levels of sensorimotor function; it doesn't have any sensorimotor functions; it doesn't have any sensory organs, and no motor output. But it's achievable. We have a robot, we can give it a body and give it a camera. Also, it's all in progress. I have friends working on this. So it's just a matter of more effort and time.

Tencent News: What can deep learning never do?

Sienovsky:

This is a problem that cannot be understood. No one can prove that something cannot be done because it continues to evolve. Even if it can't do it now, that doesn't mean the next generation can't do it.

Like I said, it's a matter of scale. Each increase in scale brings new capabilities. So now, if anyone tells you exactly that it can't achieve general artificial intelligence, then wait for tomorrow.This is constantly changing.

This has always been a problem in the field of artificial intelligence. Every time you achieve something, people say, oh, now it's just pattern recognition, not real intelligence. But at some point, you'll reach a point.

Please, it will create, it will add all the abilities, it will have all these parts of the brain, so much so that it will have what we call general-purpose artificial intelligence. Although it's not there yet, there are no rules or laws that can prevent it from happening.

Tencent News: Are humans just a transition stage in the evolution of intelligence?

Sienovsky:

Oh, singularity, people talk about this, but it's still too early.

It may be a scenario, but the future is always more interesting than anyone can imagine. I've never imagined the impact the internet will have on the world, nor can I imagine the impact these big language models will have on the world. It's too early now.

I'm not saying we have to move forward blindly. We have to be careful, we have to regulate. If we don't do it ourselves, the government will do it for us.

We are entering a new era in human history. We are standing in front of this threshold, walking through this door, and it will never be the same again. Never.

This is amazing. Everything will change over the course of your life.

Editor/Hoten

The translation is provided by third-party software.


The above content is for informational or educational purposes only and does not constitute any investment advice related to Futu. Although we strive to ensure the truthfulness, accuracy, and originality of all such content, we cannot guarantee it.
    Write a comment