Inference market the new AI gold rush

Building on my post from yesterday....

in this post I would like to write more about the transition from AI infrastructure expansion to the inference market, taking into account my personal assessment and using some sources. The stocks mentioned do not constitute investment advice. As always, the lines are intended to provide impulses to understand the potential winners of tomorrow.

In recent years and currently, a large proportion of investment has gone into building the infrastructure for artificial intelligence.

Tech giants and cloud providers have built huge data centers to enable the training phase of AI models such as GPT-4 or Gemini [1].

However, a paradigm shift is now emerging: the focus is shifting from expensive training to the inference market, i.e. the practical use and monetization of AI models, as general AI models will eventually become a "commodity".

When we say that AI is becoming a commodity, this means that artificial intelligence is increasingly becoming a standard technology that is readily available and accessible to many companies and industries, similar to electricity, the internet or cloud computing today.

Standardization and availability

AI solutions are being standardized to such an extent that companies can use them without in-depth specialist knowledge.
Instead of creating individually developed AI systems, companies are turning to ready-made solutions or platforms (e.g. AI models, APIs or services from companies such as OpenAI, Google or Microsoft).

Lower costs

Economies of scale and technological advances are making AI systems cheaper and therefore affordable for more companies.
Cloud providers such as AWS, Azure or Google Cloud offer AI tools as pay-as-you-go services, which lowers the barriers to entry for smaller companies.

Wide range of applications

AI is used in all kinds of industries, from logistics and finance to healthcare and agriculture.
Technologies such as image and speech processing, automated decision-making and data analysis are becoming the basis for innovation.

Loss of competitive advantages

When AI is equally available to all companies, those that used to be leaders will lose their unique competitive advantage.
The difference will no longer lie in the use of AI itself, but in the efficient and creative application of the technology.

Long-term significance

AI as a commodity will lead to a democratization of technology where every company has access to the same basic tools.
Competition will shift to who uses AI better, rather than who owns or develops it.
At the same time, new differentiators could emerge, such as industry-specific customization or unique data.

In practice, this means that AI is no longer the exclusive domain of tech giants, but a tool that anyone can use to drive innovation.

📈 Why the shift to the inference market is inevitable

From training to inference: What's the difference?

Training:

AI models are trained with enormous amounts of data, a computationally intensive process that requires specialized hardware such as GPUs and TPUs. Companies such as Nvidia, AMD and Broadcom are benefiting greatly from the boom in data centers at this stage.

Inference:

Once training is complete, the models must be used in the real world, whether through chatbots, voice assistants or image processing. Efficiency is key here, as AI has to respond to millions of requests in real time.

Capital shift: the money follows the monetization

Infrastructure development completed:

Building training infrastructure is costly, but after a certain phase, the focus shifts to optimization and more efficient use over frugal training or specialized chips.

Energy efficiency and cost optimization:

Companies are looking for solutions that are not only powerful but also energy-efficient and cost-effective, a crucial factor in maximizing margins.

🔮 The growth market of the future

The inference market is expanding rapidly as AI applications penetrate more and more areas of life and work:

Automated customer service:

Chatbots and virtual assistants are replacing traditional call centers and offer round-the-clock support (e.g. IBM watsonx Assistant) [2].

Medical diagnostics:

AI-supported image analyses improve diagnostics and enable personalized therapies [3].

Language models & generative AI:

Applications such as ChatGPT or Google Bard are revolutionizing the field of generative AI.

Industry 4.0 & automation:

Smart factories and predictive maintenance increase efficiency in production [4].

🏆 The profiteers of the inference market - how are the Big Techs positioned?

📌 AMD
$AMD (+0,11 %) Nvidia's challenger in the inference sector

Technology:

AMD's MI300X is considered a direct competitor to Nvidia's H100 and offers advantages in terms of energy efficiency and costs [5].

Market outlook:

With a strong positioning in the cloud and data center sector, AMD could gain market share in the growing inferencing market

Nvidia $NVDA (-1,24 %) From training king to inference king?

Dominance in the training market:

Nvidia has dominated the training sector to date with its A100 and H100 GPUs [6].

Switch to inference mode:

With new software optimizations such as TensorRT and specialized hardware solutions such as the Grace Hopper superchips, Nvidia is preparing to be a leader in the inference sector as well.

Google
$GOOGL (-0,96 %) TPUs as the inference solution of the future

Own AI chips:

Google relies on its own Tensor Processing Units (TPUs) to reduce inference costs in the cloud [7].

Range of applications:

Optimized inference chips play an essential role in Google Cloud as well as YouTube AI recommendations.

Amazon
$AMZN (-0,61 %) The cloud giant with its own AI hardware

Inferentia chips:

Amazon's specially developed Inferentia chips are designed for the cost-effective operation of AI inference solutions [8].

Market penetration:

With a steadily growing number of AWS customers using customized AI inference solutions, Amazon remains a key player in the AI sector.

Microsoft
$MSFT (-0,71 %) Profiteer through OpenAI & Azure AI

Partnership with OpenAI:

The close cooperation with OpenAI positions Microsoft as one of the largest providers of inference solutions.

Azure AI:

With Azure AI, Microsoft is building a powerful cloud infrastructure for generative AI.

Future developments:

In-house AI chips could reduce dependence on external providers such as Nvidia in the long term.

Broadcom
$AVGO (-0,04 %) The "hidden champion" for inference networks

Network technology:

Broadcom's high-speed networking technology enables the fast data processing that is critical for AI inference.

Market Leadership:

As a leading provider of networking chips in hyperscale data centers, Broadcom benefits from the growing demand for AI-optimized solutions.

Qualcomm
$QCOM (-0,12 %) Inference on the edge market

Edge Computing:

In addition to the cloud, inference is also needed on mobile devices and IoT systems. Qualcomm offers powerful, energy-efficient AI chips for this.

Smartphone applications:

In particular, the use of Snapdragon AI processors in smartphones makes Qualcomm an important player in the edge sector.

🎯 Conclusion: the inference market could be the new AI gold rush

The AI boom is far from over, it's just shifting. While the expansion of infrastructure laid the foundation, the focus is now on continuous revenue from inference solutions.

Companies that specialize in energy-efficient and high-performance inference hardware have long-term growth potential.

Providers with their own AI solutions can benefit massively by reducing costs and improving margins.

Companies like Broadcom provide the technological foundation for fast and reliable AI inference.

The inference market has long been a reality and is not a short-term phenomenon.

❓Which companies do you see benefiting alongside the big players?

In yesterday's post, I briefly outlined my personal portfolio breakdown on the topic of the AI revolution.

Thanks for reading! 🤝

__________

Sources:

[1]

https://datacentremagazine.com/it/gartner-why-global-it-spending-will-hit-us-5-61tn-in-2025

[2]

https://www.ibm.com/de-de/products/watsonx-assistant?utm_content=SRCWW&p1=Search&p4=43700081249081072&p5=e&p9=58700008827051052&gclsrc=aw.ds&gad_source=1&gbraid=0AAAAA-oK3j5oGHzEFRXtK5errGcJ_JurZ&gclid=CjwKCAiAnpy9BhAkEiwA-P8N4u6i5OsTL36ctZDn93JgG58kV90ckf7U6o2QKeKPSUqqwHSw3MxoaRoC2IwQAvD_BwE

[3]

https://www.mckinsey.com/featured-insights/themes/whats-next-in-ai-and-healthcare

[4] https://www.deloitte.com/de/de/issues/innovation-ai/industrie-40.html

[5] https://www.amd.com/de/products/accelerators/instinct/mi300/mi300x.html

[6]

https://www.nvidia.com/en-us/data-center/

[7]

https://cloud.google.com/tpu

[8]

https://aws.amazon.com/de/ai/machine-learning/inferentia/

Tenbagger2024@Tenbagger2024

Thanks for the great contribution.
It remains and will remain exciting. As AI is new territory for many companies, the need for advice will increase. Consultants such as $ACN or $GLOB should benefit from this.
The available data must be analyzed and processed. Here, less is often more. One profiteer should be $EXLS.
In the end, however, it is probably also about which software companies build the best models here. Models that can make companies more profitable and increase margins.
I imagine CEO Mark Leonard will be on the lookout for this and $CSU will benefit from the AI boom.
At $APP, you can already see the impact that well-applied AI can have.
But AI will also play a huge role in future technologies such as robotics, autonomous driving, defense and aerospace. I see $NVDA, $TSLA and our German car manufacturers at the forefront here.
However, the danger with AI will also be that companies will shrink or even disappear

•

Kevin@VPT

@Tenbagger2024 With pleasure. Thank you for your additions.

It will certainly take some time to establish the broad corporate value and intensive use. The consulting sector will certainly benefit here... that's right.

I'll have to take a look at the stocks you mentioned. In a few years, every company will probably be an AI company that converts and uses various applications

••

@VPT
My personal opinion would be that many companies are more likely to develop their own AI models with their own high-quality data. And will rely less on other models. Data protection also plays a major role, especially in Europe. Edge computing should therefore play a greater role.
As you also say, consulting will play a role, which is why I'm surprised that this is not yet reflected in companies' share prices.
Do you see an opportunity here?

@Tenbagger2024 I also think that the future prospects, especially in the area of edge computing, are not yet priced into the European players, as the music is currently playing in the USA. Europe feels left behind in this area... but your point about data protection is of course the point 😂 As soon as the first real breakthroughs come with European models, things can get exciting and opportunities can open up.

It is questionable whether European players will be able to come up with sensible AI models in the race with the USA & China.
Otherwise, it could be that we will be "later" due to regulation and then develop niche models or applications for specific sectors and requirements. I think SAP and start-ups already have some approaches to this.

In terms of global competitiveness in terms of scaling, we will probably not become a pioneer, so the current big players will continue to benefit the most here.

On the subject of edge computing: it will certainly be most important in the areas you mentioned: Autonomous driving, machine control, medical diagnostics, etc. You probably need real-time decisions that are processed directly in the device

@Tenbagger2024 European stocks in the edge sector, I'm thinking of
- Siemens for Industry 4.0, i.e. automation in manufacturing,
- Bosch, as a supplier for cars
- Clearly. SAP for business software
- Schneider Electric also in the smart buildings and industry sector

@VPT
$HMS Take a look at HMS. They could now become strong through AI from the niche due to their expertise. Repairing and analyzing machines via remote diagnostics. Recognizing errors remotely in order to be able to initiate measures directly. Here, HMS is asking the hardware and software to create connectivity.
In Europe and Germany in particular, it is these companies that work in the niche market and supply specialized software. Such as $NEM or $MUM.
But I also see Siemens with its digital twin at the forefront here

Inference market: the new AI gold rush