Source: Smart investors.

$NVIDIA (NVDA.US)$The latest stock price has soared to a new historical high, with a market cap reaching $3.53 trillion, nearing Apple's $3.6 trillion.

Recently, $Arm Holdings (ARM.US)$ In a new podcast hosted by CEO René Haas, Huang Renxun said, "We are working hard to accelerate the pace."

This podcast was just launched on October 9, with Nvidia founder Huang Renxun as the first guest.

Huang Renxun and Haas have a close relationship with each other's companies. Before joining ARM, Haas worked under Huang Renxun for seven years as the general manager of Nvidia's computing product business.

Just to mention it here.

In 2016, under the leadership of Masayoshi Son, SoftBank Group spent approximately $33 billion to acquire the UK chip design company ARM, and in 2020 agreed to sell it to Nvidia for a negotiated price of $40 billion.

However, due to regulatory challenges, both parties abandoned this deal in February 2022.

This outcome later turned out to be a blessing for SoftBank. In September 2023, ARM went public in a high-profile manner, with its stock price more than doubling, becoming an important part of SoftBank's global strategy in the field of artificial intelligence.

The conversation between the two chip executives now shows this familiar feeling.

They discuss their shared history and also have some inspiring dialogues interspersed.

Jensen Huang told Hass that Nvidia will surpass GPU, design the entire computing system, including networks, switches, software, and other chips, to improve performance without increasing energy and cost requirements.

"We hope to reduce costs so that we can achieve this new type of inference with the same cost and responsiveness as in the past."

He added that artificial intelligence chatbots will be able to conduct a more thorough study of ideas, reasoning, and reflection on answers through thousands of thought calculations, and then draw conclusions. "The quality of the answers will be much better."

But compared to before, Huang Renxun talks more about the power of software. This is another strength of Nvidia, but it has not received much applause under the shining hardware business.

In this conversation, Huang Renxun and Rene Haas delved into key strategies and challenges in driving innovation in the artificial intelligence and computing industry, covering core issues such as system design, chip advancements, energy efficiency, and how to maintain long-term compatibility of technology architectures.

Huang Renxun also mentioned that we are now witnessing an "industrial revolution" in the computer industry, where AI factories are no longer just tools like traditional computers, but have become a 24/7 operating "manufacturing plant" constantly generating intelligence.

He also admitted that making huge technological breakthroughs involves pain and struggle.

By the way, on October 7th, the latest Bloomberg Billionaires Index showed that Huang Renxun's personal wealth reached $109 billion, ranking 13th on the global rich list, exceeding Intel's total market cap of $96.5 billion.

Rene: Welcome to Tech Unheard, a podcast that takes you deep into the most exciting developments in the tech field. I am Rene Haas, CEO of ARM. At ARM, we are shaping the future of computing with the industry's most powerful and efficient computing platform, aiming to unleash the full potential of AI.

Our technology is at the innovation core of leading global companies. In this podcast, I will share insights, stories, and visions of the future with some of the smartest people in the industry.

Today, I am very honored to have a conversation with NVIDIA's CEO Jensen Huang, who is a true visionary, my former boss, and also my personal mentor.

We will delve into his career journey, the future of AI, and how NVIDIA's unique, innovative, and ambitious culture continuously pushes the boundaries of technology.

We sat down for this conversation at NVIDIA's headquarters in Santa Clara.

Jensen Huang: It's great to have you back.

René: Yeah, it feels really good to be back here. Happy to see you, and it's great to be back at NVIDIA.

Hmm, this building didn't exist before. When I worked here many years ago, these buildings didn't even exist. How many years has it been now?

Jensen Huang: It's been 20 years. I started working here in 2006, and left in 2013.

René: Yeah, 20 years ago. These buildings didn't exist. But coming back feels very familiar. Thank you for taking the time.

Thank you for your invitation, I'm glad to have you here.

About organizational culture

René, now NVIDIA has developed so much, what I have always been curious about is recruitment. NVIDIA's culture is very unique, the company has its unique way of doing things.

How do you identify talents that can be successful at NVIDIA?

We don't always succeed in this. Look, you are the best example (laughs).

It's a kind of adventure. I don't think the interview process is the best way to determine if someone is suitable. You know, everyone can show a very suitable appearance in the interview.

Through conversation, you can build a good impression. You can even learn how to interview by watching YouTube.

Of course, technical questions will also be shared, and everyone will try to make the interview process as strict and challenging as possible.

But this is really difficult. My own method usually involves retrospective verification, where I ask some questions to previous employers that I initially intended to ask the applicants.

Because you can deceive yourself by preparing a moment of brilliant performance, but you cannot escape the past. Therefore, I think this retrospective verification is useful.

I like to ask in-depth questions to see how they reason. However, I believe that ultimately, NVIDIA has been successful in many aspects.

You also know that our turnover rate is very low, the company environment is very diverse, with many interesting backgrounds and people. People in our company come from top companies around the world, and here, they become even more successful.

I believe that creating a great company is partly about finding excellent talents.

On the other hand, building a great company also requires creating conditions for these people to do better than they expect. Often, this requires transparently explaining NVIDIA's vision, strategy, and reasons for our success.

You know, I have spent a lot of time doing this.

René, yes, I have also noticed that your company has always been known for its transparency of information, especially in explaining the challenges, opportunities, and strategic aspects we are executing. Information flows quite smoothly within the company, making it clear to everyone what the company is doing.

Huang Renxun, yes, I always find it strange that when the company has too many 'information silos', many decisions can only be made based on the 'need to know'.

Of course, everyone doesn't need to know everything, but the more they know, the more empowered they are to make the right decisions for the company. So, I tend to make mistakes in terms of information transparency rather than restricting information.

I also tend to empower people. As a result, our company, despite its large size, operates like the smallest 'big company' in the world.

We have approximately 0.03 million employees, maybe slightly more.

They make hundreds of decisions every day. If all 0.03 million employees move in the direction that benefits the long-term development of the company when dealing with these often ambiguous decisions, the cumulative effect is very rapid.

Lei Nei, one thing has always surprised me, which is the point you mentioned. I don't know if it's because you have hired the right people or it's a self-selection process? The senior management team is very good at adapting to uncertainty, and you delve into different levels of the organization, such as confirming which projects are most important.

I am very curious about how this happened. Is it because as you expanded the company, your senior management team remained aligned with your vision, thus making the company develop like this? Because I think it's very impressive.

When I worked at NVIDIA, many executives were completely accepting that you would bring in the right people at any time to solve problems.

Hwang In-hyun, I have never asked them. The reason I did this is that you don't need to seek permission for obvious things.

NVIDIA was designed to be a full-stack computing company, our mission is to build GPUs, network chips, and switches. We will do chip architecture and design, develop system applications, and even create algorithms and solvers.

So the question is, how do you build all parts separately while ensuring that they work together? The way we solve this problem is by breaking the organization's "silos".

We see the organization as a place where leaders can nurture talent, create conditions for success, provide support, help them overcome obstacles, and so on.

However, the real "boss" is the mission, which spans the entire company. Therefore, it can cover systems, chips, network chips, software, and algorithms, covering various areas.

By organizing work in this way, we also create transparency, all silos become permeable. When the organization is permeable, the effect is usually better because you have more people to help you criticize and improve it.

So, I really like this "penetrability" of our company. I like everything to be transparent, everyone is helping me do things better. It's not like everything is hidden in the dark.

About acquisitions and mergers

René, you almost acquired us, that would have been interesting, but you acquired Mellanox. I know you still regret it.

Huang Renxun, I know you still feel sad about it.

René, I'm still sad. [Laughter] I cry a bit every day.

Huang Renxun, thank you. [Laughter] But you did a great job.

René, but you did acquire Mellanox, which was not only an excellent strategic acquisition, but also seamless integration into your company from an external perspective. As you mentioned, mission comes before everything else.

It looks like it was executed very smoothly. How was this process accomplished? M&A is not easy, and acquisition is very challenging.

Huang Renxun, indeed very challenging.

First, there were about 10 to 12 people in the Mellanox management team, who have now become part of NVIDIA Israel's management team and have joined our executive team (E-staff).

We have architects, researchers, software system developers, chip developers, we have network interface cards and switches, and MVlink switches. Previously, we only had the InfiniBand product line, but now we have a complete Ethernet product line. Since our merger, Mellanox's product portfolio has quadrupled.

They are integrated into every aspect of what we do.

Looking back at this acquisition, initially our vision was that the computing unit was no longer just the GPU. The GPU used to be an external device, in fact, ARM played an important role in helping us transition to a company that builds SoCs (System on Chip).

It is important to note that an SoC is basically a complete computer system, whereas an independent GPU only starts up after all parts of the computer have started. The CPU starts first, then the ROM, then the operating system, and finally the GPU. In the case of SoC, you have to boot up the entire system yourself.

Therefore, NVIDIA has evolved from an algorithm company (essentially a GPU company is an algorithm company) to a computing company gradually, which is our first transformation.

Initially, developing SoCs was not easy, but now we have produced some great products. Next, our next step is to build systems, and DGX-1 is our first system.

In fact, I still really like Shield, our Android TV computer. I have a sentiment for it because it is the first complete system built by NVIDIA. We created this product and learned a lot from it.

Looking back, we feel that Shield was indeed a remarkable product we launched. It is still the most popular Android TV box to date.

At that time, people were still using it to play PlayStation or Xbox, paired with a display screen. We were wondering, how should we proceed?

The Birth of DGX-1

René: Yes, Shield is still one of my favorite NVIDIA products.

Huang Renxun: Yes, I almost forgot about that era. It was a wonderful old time, that was a system from which I learned a lot.

Even today, we are still maintaining its software. We never expected this product to find its position in the market.

I remember the team suddenly needed to find a complete set of components and bill of materials. Like I said, we really didn't know what we were doing at that time.

René: Yes, DGX-1, this computer that changed everything, how was it born?

Huang Renxun: Actually, DGX-1 is a very large Shield. Yes, a very large Shield.

For me, Shield is made of plastics, while DGX-1 weighs 600 pounds, the transition between them is not a big problem. The real issue is that now we can build complete computer systems.

When we acquired Mellanox, the major concept was that computers are no longer just nodes, but the entire data center. The data center will become a computational unit.

If you cannot design GPUs, CPUs, network interfaces, switches, all transmission devices and connect them together, start up the entire system, connect and run all components from scratch, and allocate workloads, then you will not truly understand how to build an AI supercluster.

This transformation, this vision is so clear that it has largely inspired the cohesion of two teams.

To inspire a team, you need a very clear vision, and we did have a very clear vision at that time. Moreover, this vision was very specific because you could see it, sitting there, running before your eyes, which is the supercluster that includes all the equipment of the two companies.

Therefore, this vision is both clear and inspiring. As a CEO, you must be able to make abstract things concrete, and then we set out to achieve it.

About Vision and Intuition

René, you have done an outstanding job. Coming back to the topic of vision, I have always told others, Shield is a good example, early CUDA chasing oil & gas is also a good example, many times these are completely unexpected things.

People are not aware of this. In fact, that was our first attempt. Yes, indeed, it was the first time.

René totally did not anticipate what the ultimate killer application or final form would be, but you have amazing resilience to experiment and test these ideas early on when the market is not ready or the definition not clear.

What do you think this is attributed to? Is it excellent intuition? Or foresight?

We've had a lot of good instincts, you know, about ten such experiences in the time since the company was founded. And NVIDIA's advantage is that we are surrounded by extraordinary talent.

These are the world's best computer scientists, the most outstanding strategists, and business talents. They are not arrogant, they just want to do great things.

So I think, first, we have excellent talent; second, I think we are also outstanding in intuition.

We have good intuition about what problems need to be addressed and know how to move from today to what we want to be as a company. Therefore, I think our intuition is excellent in determining every step forward.

You know, many of the things we do, people will ask, why are we doing Shield? Isn't that a waste of time?

And I said, one day we will become a system applications company, all these systems will be connected to cloud computing service. Why not try smaller systems first? If we can't even do this well, we definitely won't be able to do bigger systems.

So we created a condition for the company to learn new skills, even if it fails, it will not harm itself, you know.

About Pain and Agony

Rene, is it only companies led by founders that can achieve things like this? Because very few companies have both a clear vision and the resilience to keep moving forward as you described.

There has been a lot of discussion recently about founder mode versus manager mode, clearly you are the founder, still leading the company after 30 years, achieving great success. Do you think the achievements you described can only be achieved by companies led by founders?

Huang Renxun, I don't think so. I think you have done a great job at ARM. When I see the work you have done, I feel very proud.

Rene, yes I have learned a lot from you, not just speaking the truth.

Huang Renxun, Yes, I really love seeing the work you do, it makes me happy, brings me great joy and pride. I don't think only founders can do this. I do think you need great resilience and perseverance. I describe it as "pain and agony," you know (laughter).

Reine, lesson learned.

Huang Renxun, yes, that's the feeling of pain and suffering.

Reine, I've experienced it too.

Huang Renxun, in many ways, you must adapt to it. You have to get used to the presence of pain and suffering. You know, the journey to success is not one achievement after another. It's not like that.

There will be huge setbacks, sometimes embarrassing moments, especially when you're a CEO and you haven't experienced them. But...

Reine, it will happen.

Huang Renxun, I hope it happens because it will be beneficial for you. But you know, in all these moments, I don't know what I've learned, but it has made me stronger. I know I can survive.

I didn't like that feeling at the time, but looking back at those moments, that's when you grow.

It is these moments that make you grow.

Yes. These are the moments you are most proud of. You are proud of yourself, proud of the company, because you have overcome those difficulties. So, I think our company is strong because we have many such stories. The company is filled with stories of setbacks, one after another.

Most people would think, 'Oh, this difficulty is nothing compared to the last one,' every time there is a problem, someone always says, 'It's nothing much.' This enables the company to continue moving forward during challenging times.

About the future of change

René, you and I roughly started in this industry at the same time, and some things related to AI make me feel like these transformations are something I never thought of.

I used to think that it would take several generations in the future to experience the speed of this kind of change, and everything happening now is simply unbelievable. It's like entering the 'ultimate frontier,' not knowing what else the future holds beyond the changes we see in the field of artificial intelligence.

What do you think about this? Are we going through an unprecedented industry transformation? Is there anything more profound than this? What we are witnessing now is simply incredible.

Yeong-hoon, I have always hoped that computers would exhibit intelligent behavior. We can write such excellent software, I thought we would solve problems by writing algorithms, making computers look very intelligent.

But I never thought this would lead to an industrial revolution.

What I'm saying is, what I've said many times before - this is the first time the computer industry has surpassed traditional computer industry. We are no longer just tools, no longer just instruments. We are now a manufacturing company.

What I mean is, for example, as we are talking now, our phones are in our pockets, not being used. When I'm not using it, it's not doing anything. And most computers are the same. My laptop is also sitting idle in the office, like most people's computers. You only use it when you need a tool.

However, this new AI factory industry is running 24/7, whether you are using it or not, they are producing data, ingesting data, generating data, creating intelligence.

Intelligence is being mass-produced. Whereas previously computers were tools, instruments, now they have become factories, a production equipment, capable of producing very valuable things on a very large scale.

This is a new era that our industry has never seen before. Computers have now become manufacturing tools, they have become the mechanical equipment behind these machines called intelligent tokens, this is an extraordinary idea.

We are at the beginning of a new industrial revolution.

René, Is its development speed even faster than you expected? You have been involved in all the work related to AlexNet and DGX-1, witnessing the speed of innovation.

From my perspective at ARM, since I took over, we have been diving deep into the development of ai. Its growth rate is much faster than I imagined two and a half years ago, even faster than I imagined a year ago.

You have been involved in all of this, has its growth rate also exceeded your imagination?

Ren Xuan Huang: No, we are trying to make it faster.

We have entered a cycle of one year. Because we are not just manufacturing chips now, we know that the pace of chip progress is not as fast as before. If you are lucky, the new process node may improve performance by a few percentage points. This is already amazing.

So how do we achieve multiple times the performance improvement through each generation of products?

Our approach is to design 6 to 7 new chips for each system, then reinvent the entire system through collaborative design, inventing new things, such as NVLink switches, and enabling us to transmit copper wires through the backplane of the entire system, connecting all GPUs in very large packages and 3D packaging.

We are using various technologies to achieve this goal.

The result is that we can achieve a 2 to 3 times performance improvement annually at the same energy consumption and cost. This is also another way to reduce the cost of ai by 2 to 3 times every year.

Its progress speed far exceeds Moore's Law. So by compounding this progress, in five or six years, ten years later, we can achieve a huge reduction in the cost of intelligence.

The reason we are doing this is because we believe now is the time when everyone recognizes the value of this technology. If we can greatly reduce costs, then we can do some new things in the reasoning phase, such as the way of reasoning.

ChatGPT is an amazing service that I use every day, I even used it this morning. When you hit enter, your prompt gets loaded, and then it generates the output.

But in the future, it will iterate over this answer repeatedly, possibly generating a tree search, or it will do some form of iteration and reflect on its own answer, eventually generating an output. It may have gone through hundreds or even thousands of iterations, but the quality of the answer will be much better.

We want to reduce costs so that we can offer this new way of reasoning at the same cost and speed of response as before.

The importance of software

René, I watched a demonstration of an OpenAI model, the way it goes through the reasoning process is truly astonishing. It goes through a logical tree, you can see it making decisions that are very similar to humans, but the speed is completely different from humans.

As you said, you are introducing a complete dataset and infrastructure into an industry at an unprecedented speed. CPUs are typically purchased every two to three years and lose value. But now, you are building new systems every year. People want to pay and deploy these systems as soon as possible.

When we talk about it now, it sounds easy, but you know, every year we are delivering new computers the size of this room. All the cables, networks, switches, software, it's really crazy.

Rene, what do you think? I'm not asking you to make future predictions, but this is more of a question about absorbing technology. Can it continue at the current pace?

Yes, I think it can. But it must be done in a systematic way, meaning everything we do is done in an architectural manner.

This means that the software you developed for yesterday's clusters, such as Hoppers, can run on Blackwell, and this software will also run on Rubin. The software you developed for Rubin can also run on Hoppers.

This kind of architectural compatibility is very important because the industry's investment in software is far greater than hardware by a factor of thousands, not to mention that software is never obsolete.

So if you develop software or release software, you have to maintain it forever.

Therefore, the concept of CUDA is not because there are millions of people programming for it, but because there are billions of GPUs compatible with it.

Rene, software will not die.

Huang Renxun Yes. Therefore, no matter what investments you make on one GPU, it will continue on other GPUs. And all the software you write today will get better tomorrow. All the software we will write in the future will run on the existing installation base.

So first, we must be very disciplined in architecture; second, even at the system level, we are now very architectural.

We will change some of the technology to drive progress in system design without needing to abandon everything you did yesterday.

For example, when we first entered the datacenter business, the power distribution of large-scale data centers was about 12 kilowatts per rack, while Blackwell's power distribution was 120 kilowatts per rack. Its density is ten times that of the former. Of course, its density has increased tenfold, compressing millions of dollars worth of servers into one rack.

Therefore, the energy-saving and space-saving effects are simply incredible.

René This is very similar to our story. You know, ARM architecture has been around for 30 years, we have software with decades of history, and sometimes people don't always understand that.

Huang Renxun We care about all the work on every ARM chip. A few days ago, someone conducted some benchmark tests, and the results showed that Grace's performance per watt is four times better than the world's best CPU.

Energy efficiency is crucial, that's everything.

About More Mission

Lei Nei, when you switch from a 500-megawatt datacenter to a 5000-megawatt datacenter, do you start to see any architectural issues arise? Just looking at aspects like network latency, without involving proprietary content, do you start to see some issues from a high-level physical perspective?

Huang Renxun, everything will run into issues, physical laws cannot be violated, that's where the problem lies, but everything will run into issues first.

Of course, we are moving rapidly on the power density curve, experiencing exponential growth. From 12 kilowatts to 40 kilowatts, then to 120 kilowatts, 200 kilowatts, this number will go even higher.

We are compressing and intensifying computation as much as possible. As we do this, liquid cooling becomes more efficient, and we can use copper for longer durations.

It's good to use copper conductors for as long as possible, so you don't have to switch from electrical signals to optical signals.

We will ultimately have to switch to optical signals, but we will use electrical signals as much as possible.

Therefore, with as many data centers as possible, our solution is more cost-effective, more energy efficient, and more reliable. This leads us to the need for densification.

Another benefit of densification is that all GPUs in a specific rack or adjacent racks can function as one giant GPU. It's really amazing.

René, that's incredible!

One thing I've always been curious about is your keynote speech at Computex. I remember watching the content of your speech on a Sunday night, and the amount of content you covered was not only incredible but also, as someone who gives keynote speeches, I really admire how you manage it. Did you do a lot of rehearsals?

I remember when we worked together, sometimes you were still making changes late into the night before, but you still succeeded. However, the depth you are dealing with now, especially when it comes to data center architecture, you have expanded a lot. How did you prepare for all of this?

Huang Renxun, actually, we are preparing every day. You know, that's the benefit of our work; we are not actors. [laughs]

So, this is considered our work. We prepare every day.

But to be frank, many of the things you and I do are actually educational. To shape an industry, market, and introduce new ideas we are experimenting with, a lot of it is teaching, you know, it's not advertising.

We are a platform company, which means we cannot do our work alone; we need others to work with us. Therefore, we focus on teaching, inspiring, showcasing, maybe demonstrating, and hope to gradually make more and more people believe in CUDA, initially, now with NVIDIA accelerating computing, and together join us on our AI journey.

The next major project we are working hard on is Physics AI, how to teach AI to follow the laws of physics while also understanding these laws.

So I think this journey is quite long, so GTC and Computex are our opportunities to celebrate our ecosystem and the work they have done, teach them, and inspire them to take the next step.

René, they're actually very similar. We will have a QBU demonstration, and I will do the demo. The staff in charge will say, wow, these slides are too simple. That's probably what you talk about all day. I wonder, what makes it different?

Huang Renxun, it's still difficult, to be honest, because we haven't rehearsed. So it's not because we choose not to rehearse. There's simply no time for rehearsals until we have all the materials ready. So we just seize the time and do our best.

Thank you, Jenson.

Nice to see you. Well done. I'm proud of everything you've done.

Editor/rice