Skip to content

Dear customers! Please send orders and inquiries to [email protected] or to Telegram @servix_uz instead of calling by phone: due to the high workload, the operator cannot answer all incoming calls.

Servix

HPE ProLiant DL145 Gen11 with NVIDIA: an edge node for AI inference

Servix technical team · Updated

In brief. DL145 Gen11 paired with an NVIDIA accelerator is a node for inference on site, not for training models. The difference is fundamental: training requires tens of kilowatts and a cluster, inference — one card and a couple of hundred watts. A typical scenario in Uzbekistan is on-site video analytics: license plate recognition, people counting, workwear compliance monitoring, when sending the video stream to the head office costs more than processing it locally. The model is not in the Servix catalog; we calculate a node for a specific task on request.

Jump to: inference versus training · which card is needed · how much video memory · power and cooling · when it pays off · questions and answers

Inference and training are different machines

Confusion here is costly. Training a model means weeks of work by a cluster of cards at 300–⁠700 W each, with fast interconnect between nodes and a separate cooling loop. Inference is applying an already trained model to a data stream: one card, tens of watts or a couple of hundred, an ordinary server.

Ninety percent of business tasks are inference. The model is taken ready-made or fine-tuned once in the cloud, and then runs on site for years. Edge nodes with a single accelerator are designed exactly for this scenario.

Which card is needed

At the edge, what matters is not peak figures but the thermal envelope and form factor.

  • 70–⁠75 W cards with passive cooling and half length are the main choice for the edge. They are powered from the slot, require no additional cables and fit in a short chassis.
  • 150–⁠300 W cards provide more streams per node but require a full-size chassis, enhanced fans and higher-wattage power supplies. In a shallow cabinet, this is most often not feasible.
  • Cards of 400 W and above belong in a data center, not on a site.

Before choosing, calculate not “how many cards will fit” but how many camera streams need to be processed and at what frame rate. For tasks such as people counting, 2–⁠5 frames per second is enough, and one card handles dozens of streams; license plate recognition at speed needs more.

How much video memory

Video memory capacity is the first limit hit in practice. The model must fit into memory entirely together with processing buffers; if it does not fit, swapping with system memory begins, and performance drops several times over.

As a guideline: small detection and classification models fit into a few gigabytes, recognition and segmentation models need noticeably more, and language models on site require tens of gigabytes and usually do not run on an edge card. The card is chosen based on the specific model, not on general considerations.

Power and cooling

An accelerator changes the thermal balance of the cabinet. The practice is as follows:

  • Calculate the budget for the whole cabinet: server, accelerator, switch, UPS. A 75 W card adds about the same to the server at peak; a 300 W card changes the class of the entire installation.
  • Passive cards are cooled by airflow from the server fans. This means the fan kit must be rated for the card, not for an empty slot — a separate item in the configuration.
  • The UPS is sized by peak consumption with the accelerator under load, not by the rated power of the power supplies.

When it pays off

What needs to be calculated is not performance but the cost of the alternative. Local inference wins when:

  • the link to the head office is narrow or expensive, and the video stream is constant;
  • the decision must be made in fractions of a second — a barrier gate, a turnstile, an alarm;
  • data must not leave the site perimeter due to customer requirements.

If none of these conditions is met, it is cheaper to send frames to the head office and keep one powerful node for the whole organization, leaving an ordinary server without an accelerator on site — for example, DL320 Gen11 for 41,000,000 UZS or DL380 Gen11 for 49,000,000 UZS if you need spare slots.

We will calculate a node for your model and number of streams — with the card, power, cooling and storage. Write to us.

Questions and answers

How does inference differ from model training?

Training creates a model and requires a cluster of accelerators, weeks of time and tens of kilowatts. Inference applies a ready-made model to a data stream and needs one card and a couple of hundred watts. Inference is exactly what is done at an edge site.

Which GPU is needed for on-site video analytics?

For the edge, a passive half-length 70–⁠75 W card is usually chosen: it is powered from the slot, requires no additional cables and fits in a short chassis. More powerful cards require a full-size server, enhanced cooling and a different UPS.

How much video memory is needed?

The model must fit entirely into video memory together with buffers. Small detection models fit into a few gigabytes; recognition and segmentation require noticeably more. Calculate based on the specific model you plan to run.

Can models be trained on such a server?

Technically yes, practically there is no point: training on a single edge card will take weeks where a cluster would manage in hours. Training is done in the cloud or in a data center, and the ready-made model is deployed to the site.

How many cameras can one node handle?

It depends on the model and the required frame rate. For people counting and event detection at 2–⁠5 frames per second, one card processes dozens of streams; for license plate recognition at speed — only a few. The calculation is done for the specific task.

Related reading