“Every company will be an AI company, and every company will have its own AI factory. AI Infrastructure will be one of the most important capital investments of the future.” – Jensen Huang, CEO of NVIDIA

What Is An AI Factory?

A factory turns raw material into a product.

AI factories turn data and electricity into tokens -- responses, train of thought, or other AI generated content.

The name is an improvement that more accurately describes how modern compute facilities are used: to create AI, rather than just to store data.

What Does An AI Factory Look Like?

An AI factory is essentially a warehouse full of computers. These computers are GPUs (graphics processing units, likely made by Nvidia).

If you wanted a more powerful computer than the one that sits under your desk, you would add more GPUs. To do that, you'd need a bigger container. Keep adding GPUs and making the container bigger, until its the size of a warehouse, and you've got an AI factory.

What An AI Factory Does

AI factories are doing one of two things:

1. Training Mode: Training or fine-tuning new models.
2. Inference Mode: Serving tokens from already trained models to users.

Training Mode

During training, a model is asked to make predictions (generate tokens, images, etc). The model output is compared to a target, and the difference gives an idea of how it should improve. This process is repeated perhaps billions of times, making small improvements at each step. Until the model is trained, the predictions won't be great, so during training they are thrown away.

Each prediction is the result of a mathematical calculation run on a computer, the electricity used to run that calculation is the power demanded by the AI factory.

Fine-Tuning

Training mode also covers fine-tuning, which is where you take an existing model and show it more specific examples of the niche problem you wish it could solves, until it learns the pattern.

Example fine-tuning use-cases:

• Making a text model less biased.
• Teaching an LLM medical or legal jargon.
• Specializing a generic image classifier to recognize different types of specialized machinery on your construction site that you want to monitor.

Inference Mode

During inference (also referred to as serving), requests from real customers or end-users are sent to the AI factory so that an AI can generate a response.

Conclusion

Conclusion: AI factories are real, not hype, but definitely squeezing juice from the fresh marketing label.