What is Inference

 Any model has two phases. First, there's the training phase, where the model is trained on a dataset. It produces output, which is validated, and the updates are fed back for improvement. There can be multiple passes over the training data.

 

Once the model is ready, it is deployed into production, where it encounters real-world data. This process of putting the model into production and getting the actual output is called Inference.

 

Think of it in terms of food. Given some ingredients, a chef experiments with them. She creates a recipe, tries the food, and if she doesn't like it, she tweaks the recipe, makes another dish, and iterates multiple times before finalizing her recipe (or model). This was all training.

 

Now, she takes this recipe to her restaurant and puts it into production in the actual kitchen so that customers can enjoy it. This process is inference.

 

To summarize, machine learning inference is the process of running real-world or production data into a model to get the actual predictions. In other words, an inference model is a production-ready model.