Geinium
Home › Performance

Performance

How fast simulations, analyses and exports run, and what to expect on other hardware.

All numbers on this page were measured on the development PC: one NVIDIA GeForce RTX 5070 Ti (compute capability 12.0, 16 GB, driver 610.88) with the CUDA 12.2 build of Geinium, an Intel Core i7-14700K processor (8 performance and 12 efficiency cores) and Windows 11. It is the only graphics card the GPU engine has been run on. On another card the speed will differ; how to measure it is at the end of this page.

Simulation speed

Conditions: mdsim bench -nb gpu, 20,000 timed steps of the complete NPT protocol (thermostat, barostat, constraints, PME, energies every 100 steps), median of three runs, GPU otherwise idle. The time for 100 ns follows from the speed.

System and protocolns/day100 ns takeWith multiple time stepping (option)
Lysozyme in water, 33,752 atoms, 2 fs (the default)1164.42 h 4 min1326.9 ns/day
The same, 4 fs with repartitioned hydrogen masses, two LINCS iterations (option)2134.91 h 7 min2274.0 ns/day
A protein-ligand system in water, 67,156 atoms, 2 fs (the default)674.93 h 33 min748.2 ns/day
The same, 4 fs with repartitioned hydrogen masses, one LINCS iteration in the input1246.41 h 56 min1342.0 ns/day

These are the speeds of one replica. A job with three replicas takes three times as long when the replicas run one after the other; see below for running them side by side. Equilibration (200 ps) and minimisation add minutes, not hours. The pipeline writes two LINCS iterations at 4 fs, so a 4 fs job of the larger system runs somewhat slower than the last row. The output files of this release are byte-identical to those of release 1.0.0, which ran the default protocol 1.37 to 1.41 times slower: the gain came from scheduling, the pair search and the kernels, not from accuracy.

Why 100 ns cannot take minutes: at 2 fs, 100 ns are 50 million steps, and at 1164 ns/day each complete step of 33,752 atoms (all forces, PME, constraints, coupling) takes about 0.15 ms. The time step is limited by the fastest motions in the system, the vibrations of bonds to hydrogen; 4 fs with repartitioned masses is the largest step that keeps the dynamics correct, and it about halves the time.

What affects the speed

FactorEffectWhat you can do
Number of atomsRoughly in proportion: twice the atoms, a little more than half the speed (674.9 against 1164.4 ns/day).Use the smallest box that is safe: the default solute-box distance of 1.0 nm with a rhombic dodecahedron needs less water than a cube.
Time step4 fs with repartitioned hydrogen masses is about 1.8 times faster than 2 fs.Choose 4 fs on the Run MD page when the faster sampling matters more than staying with standard masses.
Multiple time steppingLong-range forces every second step: 1.11 to 1.14 times faster in the benchmark, 1.04 to 1.07 in 200 ps runs with all output files written.An option for the production run (protocol.mts_factor: 2); it changes the integration scheme, so it is not the default.
OutputWriting frames and energies costs time; the benchmark writes energies every 100 steps.The defaults (frames, energies and log every 10 ps) cost little. Avoid full-precision trajectories (trr_ps) unless needed.
Start-upAbout 1.4 s outside the step loop at every start of the GPU engine; the first start on a new card or driver also compiles the GPU code once (6 to 8 s).Matters only for very short runs.
Other programs on the GPU or processorThe engine raises the priority of the thread that drives the GPU, but a busy processor or another GPU program still slows it.Leave the PC to the simulation for timing measurements.
The graphics cardNewer and larger cards are faster; the memory needed is small.Run mdsim bench to see what your card does.

Several runs on one GPU

A single run of a mid-sized protein does not fill a modern GPU completely, so two to four runs side by side give more total throughput, while each single run becomes slower. Total throughput relative to one run, measured with mdsim bench -batch 4:

System2 runs3 runs4 runs
Lysozyme, 33,752 atoms1.101.151.18
67,156-atom protein-ligand system1.02
Water box, 6,495 atoms (another session)1.551.862.06

Small systems gain much; large systems gain almost nothing. In a job, "Concurrent replicas" on the Run MD page (project key replicas.concurrent) runs replicas side by side; "Automatic" measures the best number once per job (about a second per candidate, at most 4 lanes) and writes the result to pipeline.log. The results do not depend on it: replicas run side by side give the same files as replicas run one after the other. The Batch queue runs one job at a time.

The CPU engine

The CPU engine is the double-precision reference implementation; it gives the same results but is not a production engine. For 33,752 atoms on 8 threads it runs at about 6 to 9 ns/day, so 100 ns would take 11 to 17 days. It is used automatically when no usable GPU exists, and always for energy minimisation, which takes seconds to a minute (18 s in a 47,630-atom test job). On processors with performance and efficiency cores, its force kernel is about twice as fast when the threads are pinned one per performance core (engine.pin, mdrun -pin on).

Preparation, analyses, MM-GBSA and export

StepTimeConditions
Structure, ligand, topology and system stages of a receptor complex16 sA 61,705-atom system, with gap modelling
Modelling one missing loop1.5 s, 3 s, 20 sGaps of 2, 4 and 8 residues
Ligand parameters (AM1-BCC)Seconds for most drug-like ligands; 69 s for the largest of the test set44 drug-like molecules took 451 s in total, on 8 threads of a shared PC
Interaction analysis of a 100 ns replica6 to 14 s10,001 frames, 47,630 to 89,031 atoms, 8 threads
Detail analyses of the results export41 to 63 s per 100 ns job; about 10 s for a thinned trajectory of 1001 framesRun once per job; also from "Compute" on the Results page
MM-GBSA over 1,001 frames66 to 67 s (44 s with -fastexp)47,653 atoms per frame, 8 threads; the cost grows with the square of the solute size
A complete short job481 s for 23 stages3 replicas of 1 ns, 47,630 atoms, an earlier build

GPU memory

The engine estimates the memory a run needs before it allocates anything: about 0.4 GB, plus about 2 kB per atom, plus 48 bytes per PME grid point. Measured: 207 to 307 MiB for systems of 6,495 to 88,884 atoms. If the free memory of the GPU is smaller, the run uses the CPU engine and says why.

Which GPUs are supported

RequirementValueNotes
Graphics cardNVIDIA, compute capability 5.0 or newer (Maxwell, for example GeForce GTX 750 Ti and GTX 900 series, and every later generation)The program contains ready machine code for compute capability 6.1, 7.5, 8.6 and 8.9 and portable code for everything from 5.0 on, which the driver compiles for newer cards.
Driver527.41 or newerCards that run the portable code may need a driver at least as new as the toolkit (536.25); not tested.
DetectionAutomatic, at every startDriver, cards, memory and the FFT library are checked; nothing to configure.
No usable GPUThe CPU engine is usedWith a message that names the reason. The results are the same; the run takes much longer.
The GPU engine has been executed on one card, the RTX 5070 Ti of the development PC, through the portable code path. The machine code for the other generations is compiled but was not run on such cards. On any other card, run mdsim gpuinfo -selftest before the first job.

Measuring on your PC

rem which GPUs are there, and can the GPU engine use them?
mdsim gpuinfo -probe

rem does this GPU compute the same forces and energies as the CPU engine? (about two seconds)
mdsim gpuinfo -selftest

rem speed of a real run input, for example the production input of a finished job
mdsim bench -s P:\Geinium\Jobs\m3_dicyclomine\run\06_run\rep1\md.mdtpr -nb gpu

rem total throughput of 1 to 4 runs side by side, with a recommendation
mdsim bench -s P:\Geinium\Jobs\m3_dicyclomine\run\06_run\rep1\md.mdtpr -nb gpu -batch 4

bench runs 5000 timed steps by default (-nsteps) after a warm-up. For a fair number, close other programs that use the GPU and run it two or three times.