In many industrial processes, a very specific need arises: to identify and classify products, parts or components quickly and reliably. Traditionally, this required developing specific models, collecting large volumes of images and undergoing long training processes, as explained in 10 things I wish I had known in my first factory AI project.
Today, thanks to VLM models (Visual Language Models), this process can be simplified enormously.
What is a VLM?
A Visual Language Model is a type of artificial intelligence capable of understanding both images and natural language instructions. This means it can analyse what it sees on a camera and compare it against examples and instructions provided by a person.
Instead of training a model from scratch with hundreds of labelled images, it is enough to show visual examples and describe what needs to be identified.
Traditional approach | VLM approach |
|---|---|
Requires hundreds of labelled images per category | Works with a handful of reference examples |
Training and validation across multiple cycles | Direct setup, with no prior training cycle |
Changing category requires retraining | Changing category requires updating the reference |
Requires a specialised technical profile | Configurable by an operator or process engineer |
How visual classification works in Rely
The process begins by creating a visual reference showing the different objects to be classified. This reference can include example images, visual instructions, text descriptions and the distinguishing features of each object.
This reference is known as Golden Reference.
When the camera captures a new image, the system automatically compares what it is seeing against the Golden Reference to determine which category the object belongs to.
A practical example: pen classification
To demonstrate how the system works, we set up a Golden Reference with different types of pens. Each model included specific features allowing it to be distinguished from the rest: some were distinguished by their shape, while others required identifying printed information on the object itself, such as the writing width.
When the camera captured an image, the system automatically analysed the object and returned the corresponding classification.
The most interesting aspect is that the artificial intelligence was never told that it was looking at pens. It simply learned to differentiate them from the provided examples, without anyone programming a single specific rule about what a pen is.
When visual (and textual) information matters
One of the advantages of VLMs is that they interpret not only shapes, but also text and context.
During testing we observed a particularly interesting behaviour. Some pen models differed solely by a printed inscription, such as "0.2" or "0.4". When this information was hidden, the system was unable to correctly identify the object and returned the result "none". When the visible information was shown again, classification was performed correctly.
This demonstrates that artificial intelligence uses both visual elements and textual information to make decisions, just as a person reading a product label would to distinguish it from an almost identical one.
Real-time decision automation
Once classification is obtained, additional logic can be incorporated using nodes, with no coding required. In the demonstration we set up a simple rule:
If the result contained the word "none", the system generated a KO.
If the object was correctly identified, it generated an OK.
In this way, visual classification ceases to be merely an analysis and becomes an automatic decision-making tool, capable of triggering alerts, rejecting products or initiating actions within the process.
Traceability and historical analysis
All inspections performed are recorded in the Rely history. This allows subsequent consultation of:
Results obtained
Inspection dates
Cameras used
Classes detected
Defects identified
Thanks to this traceability, companies can analyse trends, investigate incidents and continuously improve their quality processes, using the same logic we explained in How to avoid quality claims with machine vision.
Frequently asked questions
Do I need to train a model for every new object I want to classify?
No. With a VLM, it is sufficient to update the Golden Reference with new examples and instructions, without going through a traditional training cycle.
How many examples do I need in a Golden Reference?
Far fewer than in traditional training. In the case of the pens, a few examples per model were enough for the system to learn to differentiate them, including the printed information on each one.
Can a VLM be combined with traditional defect detection?
Yes. VLM classification and specifically trained defect detection can coexist within the same inspection workflow, depending on the requirements of each reference.
What happens if two objects are almost visually identical?
If the difference lies in a visible detail (an inscription, a colour, a subtle shape), the system can learn to distinguish them provided that detail is included and legible in the Golden Reference, as we saw with the writing width of the pens.
A new way to implement machine vision
VLM models are transforming how companies implement machine vision systems. Instead of relying on lengthy training and development processes, it is now possible to create visual classifiers using examples and instructions that are easily understood by any operator.
The result is a faster, more flexible and accessible deployment, capable of adapting to new products and requirements with far less complexity than traditional systems.
Do you have a classification case that currently forces you to program complex rules?
We can show you how it would be configured using a Golden Reference in a demo on your own product.