Skip to content

Benchmarks and verification ​

MILEVA's 2-D science path is the spherical annulus (radial measure r2dr, Hernlund & Tackley 2008) and its 3-D path is the Yin-Yang spherical shell. Evidence that the code solves the equations it claims to solve comes in three kinds, and this page keeps them apart:

KindWhat it showsExamples
Verification against published resultsThe same physical problem, solved by an independent code in the same geometry, gives the same numbers as the published tables.Hernlund & Tackley (2008) spherical-annulus tables; the Annulus column of Fleury et al. (2024); the Kageyama & Sato (2004) Yin-Yang rotation formulas.
Verification against exact identitiesA property every correct discretisation must satisfy, independent of any reference value: conservation, an exact null mode, an energy identity.Nu = 1 on conduction, closed energy budget, work = dissipation, rigid rotation annihilated, G=−BT.
RegressionThe code agrees with itself: two solvers on the same matrix, the serial and distributed assemblies, a stored earlier result. It protects against change, not against a wrong operator.MUMPS against UMFPACK, multigrid against MUMPS, serial against distributed 3-D assembly.

Same geometry

The 2-D path is compared only with spherical-annulus references (Hernlund & Tackley 2008; the Annulus column of Fleury et al. 2024). Cartesian box benchmarks, cylindrical annulus cases and Fleury et al.'s scaled cylinder column describe other geometries and are not used as targets.

Summary ​

Every spherical-annulus case with a published reference, at the finest grid run, against the reference value. Nu is Nutop, Vrms is in units of κ/D and ⟨T⟩ is the nondimensional volume mean over the r2 measure, all time averaged; Δ is MILEVA relative to the reference. Details, resolution series and notes for each row are in the sections below.

CaseReferenceGridNuNu ref.ΔNuVrmsVrms ref.ΔVrms⟨T⟩⟨T⟩ ref.Δ⟨T⟩
Basal heating, Ra 104, isoviscousH&T 2008, Table 1128 × 10244.054.18−3.1 %35.037.7−7.2 %
Basal heating, Ra 105, isoviscousH&T 2008, Table 164 × 5127.707.39+4.2 %161.8160+1.1 %
Basal heating, Ra 105, contrast 106H&T 2008, Table 264 × 5125.185.71−9.3 %535.8463+15.7 %
Internal heating, Ra 104H&T 2008, Table 364 × 51223.623.5+0.4 %0.3100.308+0.6 %
Internal heating, Ra 105H&T 2008, Table 364 × 51275.878.5−3.4 %0.3510.349+0.6 %
Internal heating, Ra 106H&T 2008, Table 364 × 512251.5265−5.1 %0.3410.350−2.6 %
Basal heating, Ra 104, f=0.6Fleury et al. 2024, Annulus64 × 5124.394.63−5.2 %39.546.6−15.2 %0.2950.283+4.2 %
Basal heating, Ra 105, f=0.6Fleury et al. 2024, Annulus64 × 5129.368.99+4.1 %176.7189.2−6.6 %

Blank cells: an internally heated shell with an insulating core has no basal Nu, and the mean temperature has not yet been extracted from the basally heated runs. The two f=0.4 runs of Fleury et al. are left out because they have not converged. The references give their time-dependent cases as means with a peak-to-peak spread (H&T's Vrms spread is ±90 at contrast 106, 36 and 160 for internal heating at Ra 105 and 106), and the seeded planform moves Nu at Ra 105 by up to 8 %, so differences of that size are within what the references themselves resolve. The internally heated rows at Ra 104 and 105 and the basally heated row at Ra 105 are from runs that settled in the planform of Hernlund & Tackley's Fig. 3; in the other planforms the same cases differ by up to 20 %, as discussed below.

Running the benchmark driver ​

benchmarks/hernlund_tackley_annulus.jl sets up the reference problem from scratch: it clears every optional physics flag, then switches on only what the case needs.

bash
# case, Nr, seeded wavenumber m (default 6); Nt = 8 Nr
julia --project=. benchmarks/hernlund_tackley_annulus.jl HT4 64
julia --project=. benchmarks/hernlund_tackley_annulus.jl HT5 128 6
julia --project=. benchmarks/hernlund_tackley_annulus.jl HTV5 64

# Fleury et al. (2024) annulus, Ra = 10^k, aspect ratio from MILEVA_F
MILEVA_F=0.6 julia --project=. benchmarks/hernlund_tackley_annulus.jl FL5 64
CaseRaViscosityReference
HT4, HT5, HT6104, 105, 106isoviscousH&T 2008 Table 1
HTV5, HTV6105, 106η=exp⁡[13.8156(0.5−T′)], contrast 106H&T 2008 Table 2
FL4 … FL8104 … 108isoviscousFleury et al. 2024, benchmarks/reference/fleury2024/

The setup is a Boussinesq, basally heated shell with free-slip walls, isothermal boundaries, no net rotation and a uniform grid. The layer is 1000 km thick with f=rin/rout=1.22/2.22=0.5495 for the H&T cases, and the viscosity is chosen to give the requested

Ra=ρ0gαΔTD3κη.

The variable-viscosity cases use the Frank-Kamenetskii "simple" form with θFK=ln⁡106 and η=ηRa at T′=0.5. The run goes to 0.6 thermal diffusion times (D2/κ is 31 688 Myr here), averages Nutop and Vrms (in units of κ/D) over the last third, and fits a straight line through that window. If the trend exceeds the larger of 2 % (Nu) / 5 % (Vrms) and the reference's own published fluctuation, it prints a NOT CONVERGED flag. Output goes to output/benchmarks/ht2008_<case>_N<Nr>…/, including diagnostics.csv with the energy budget.

These runs take hours to days at Nr = 64 and above. Use MILEVA_MAXSTEPS=<n> and MILEVA_OUTTAG=<tag> for short probes.

The "T on grid (TVD)" row below is reproduced with the env knob MILEVA_TGRID=1, which sets cfg.temperatureOnGrid = true (TVD advection on the grid) and adds the suffix _gridT to the output directory:

bash
MILEVA_TGRID=1 julia --project=. benchmarks/hernlund_tackley_annulus.jl HT5 64

Hernlund & Tackley (2008), spherical annulus ​

Reference values (Hernlund & Tackley 2008, 32 × 256). The 3-D column is the same study's spherical shell, shown for comparison with the annulus geometry.

CaseNu annulusVrms annulusNu 3-D shellVrms 3-D shell
Ra 104, isoviscous4.1837.73.8542.3
Ra 105, isoviscous7.391607.27160
Ra 106, isoviscous14.464015.9625
Ra 105, contrast 1065.71 ± 0.1463 ± 906.30405
Ra 106, contrast 10610.1 ± 0.11390 ± 7809.71804

The ± values are the paper's peak-to-peak spread for the time-dependent cases.

Measured ​

Seeded wavenumber m=6 unless stated. Nu is the time-averaged Nutop; budget_resid is the energy-budget residual from diagnostics.csv (see below). Nr = 32 is the reference's own resolution and is listed as the low end of the resolution series.

CaseNr × NtNu MILEVANu H&TVrms MILEVAVrms H&TNotes
Ra 10432 × 2564.244.1835.437.7steady, converged
Ra 10464 × 5124.144.1835.137.7steady, converged
Ra 104128 × 10244.054.1835.037.7Nu_top/Nu_bot 4.052/4.011, budget_resid −0.0004
Ra 10532 × 2568.767.39156.9160time dependent, budget_resid 0.0000
Ra 10564 × 5128.887.39158.2160converged flag
Ra 105128 × 10248.907.39159.6160time dependent, Nu_top/Nu_bot 8.97/8.96, budget_resid −0.001
Ra 105, T on grid (TVD)64 × 5128.977.39161.2160temperatureOnGrid = true, budget_resid +0.0108
Ra 105, m=464 × 5129.057.39176.2160Nu_top/Nu_bot 9.05/9.09
Ra 105, m=864 × 5128.407.39156.4160time dependent, budget_resid −0.0010
Ra 105, m=364 × 5127.707.39161.8160three plumes and three downwellings from 15 Gyr, as in H&T Fig. 3; Nu_top/Nu_bot 7.68/7.70 over that state, budget_resid −0.0005
Ra 105, contrast 10632 × 2564.665.71 ± 0.1440.6463 ± 90time dependent
Ra 105, contrast 10664 × 5125.185.71 ± 0.1535.8463 ± 90time dependent, Nu_top/Nu_bot 5.183/5.161, budget_resid +0.0015

Time series from each run's diagnostics.csv over 0.6 thermal diffusion times, seeded m=6 except the m=3 run in the Ra 105 row, whose Nu falls step by step as its plumes merge into the three of the reference run: Nutop solid, Nubot dotted, Vrms in units of κ/D, published reference values dashed. The Nr = 64 runs of the first two rows are read from the run log, which prints Vrms to two decimals in cm/yr, hence the steps in those curves. The bottom two rows are the Fleury et al. (2024) cases of the next section, with each aspect ratio's Fleury et al. (2024) value in the matching colour.

How to read it:

  • Ra = 104. Nu converges from above through the reference value (4.24, 4.14, 4.05 at Nr = 32, 64, 128, reference 4.18). The reference itself was computed at 32 × 256. Vrms is 35.4, 35.1, 35.0 against 37.7, steady with resolution.

  • Ra = 105. Nu depends on the planform, not on the resolution. Runs seeded with m=6 keep many narrow cells and give Nu 8.76, 8.88, 8.90 from Nr = 32 to 128 (the grid TVD path gives the same, 8.97), 20 % above the reference. Hernlund & Tackley's Fig. 3 shows three plumes and three downwellings for this case. A run seeded with m=3 starts with about ten plumes, which merge over 15 Gyr into exactly that pattern, and Nu falls with each merger, from 9.2 to 7.70 (+4.2 %), with Vrms 161.8 (+1.1 %). Wider cells carry heat less efficiently, so the cell count sets Nu. Fleury et al. (2024, their Table S3) obtain 7.16 and 160.0 for this case with GAIA. The seeded m=4 and m=8 runs (9.05 and 8.40) were averaged without checking which planform they had reached.

  • Ra = 105 at contrast 106 (the case that exercises the variable-viscosity operator): Vrms at Nr = 64 is 535.8 against the paper's 463 ± 90, and Nu rises toward the reference with resolution (4.66 at Nr = 32, 5.18 at Nr = 64, reference 5.71 ± 0.1).

  • Ra = 106 (HT6, HTV6): the driver supports these cases; no measurement is listed yet.

Internal heating (Table 3) ​

Purely internally heated, isoviscous, insulating core, the same shell, run with examples/annulus_internal_heating_2d.jl at 64 × 512, seeded m=3 at Ra 104 and 105 and m=6 at Ra 106, averaged over the last third of 0.3 diffusion times (Internally heated convection). ⟨T⟩′ is the volume mean over the r2 measure, (⟨T⟩−Ts)/ΔT.

RaH′⟨T⟩′ MILEVA⟨T⟩′ H&TVrms MILEVAVrms H&TNotes
1043.40.3100.30823.623.5seeded m=3; three downwellings, steady from 3 Gyr
1043.40.2780.30819.823.5seeded m=6; six downwellings, steady
1056.60.3510.34975.878.5 (peak to peak 36)seeded m=3; five to six downwellings, reorganises near 8 Gyr, budget_resid +0.0004
1056.60.3510.34969.478.5 (peak to peak 36)seeded m=6; budget_resid −0.0006
10614.00.3410.350251.5265 (peak to peak 160)time dependent (190 to 360), budget_resid 0.0000

In every case the surface heat flow equals the heat produced to within 0.4 %.

When quoting these numbers, state the seeded wavenumber cfg.m_wavenumber: more than one planform is stable at these Rayleigh numbers and the seed decides which one the run lands in. At Ra 104 this is the whole difference: the m=6 run holds six cells and is 16 % slow and 10 % cold, while the m=3 run lands in the three downwellings of Hernlund & Tackley's Fig. 3 and agrees to within 1 % in both. At Ra 105 the seed does not persist; both runs end in five to six cells, as in the reference figure, and differ only in Vrms. The reference has no error bars or convergence study, so the comparison is verification against published results.

Fleury et al. (2024), Annulus column ​

Fleury, Plesa, Hüttig & Breuer (2024, G3 25, doi 10.1029/2023GC011114) solved basally heated isoviscous convection in the 3-D sphere, the spherical annulus and a scaled cylinder, for f=0.2 to 0.8 and Ra = 104 to 108. Their tables (Zenodo doi 10.5281/zenodo.8047757) are in benchmarks/reference/fleury2024/, copied unmodified; the driver reads only the Annulus column. Their definitions match H&T (measure 2r2drdϕ, same Ra, Nutop normalised by f). Two differences matter when comparing: Fleury et al. start from a 5 % random perturbation with an unreported planform (MILEVA seeds m=6), and their top gradient is a one-sided difference between the last two shells.

Measured at Nr × Nt = 64 × 512:

CasefMILEVA Nu (top/bot)Fleury et al. NuMILEVA VrmsFleury et al. Vrms⟨T⟩ MILEVA / Fleury et al.Notes
Ra 1040.43.76 (3.74/3.60)3.8634.037.60.184 / 0.167not converged (Nu_top and Nu_bot still apart)
Ra 1040.64.39 (4.39/4.37)4.6339.546.60.295 / 0.283
Ra 1050.69.36 (9.36/9.33)8.99176.7189.2n/aconverged, budget_resid −0.001
Ra 1050.48.03 (7.98/7.91)5.59148.2133.7n/anot converged (Nu still trending)

The f=0.4 and 0.6 rows of Fleury et al. should not be interpolated to f=0.55: they state that their results reproduce H&T at 0.55, so its Nu(f) is not smooth there, most likely because the random start lands in different planforms.

MILEVA's time-averaged Nu (left) and Vrms (right) against the published value for every row of the two tables above except the two f=0.4 runs, which have not converged. Blue: Hernlund & Tackley (2008); orange: the Annulus column of Fleury et al. (2024). Marker shape gives the run: circle, square and diamond for Nr = 32, 64 and 128, the star for the grid TVD run, up and down triangles for the m=4 and m=8 planforms. Points on the dashed line equal the reference; the error bars on the reference axis are the published peak-to-peak spread of the time-dependent contrast-106 case.

Exact identities ​

These hold for any correct discretisation, whatever any reference says. Each is asserted by a test in tests/, run by tests/run_tests.jl.

Two of these identities, recomputed by docs/scripts/plot_conservation.jl with MILEVA's own diffuse_temperature, compute_diagnostics and assemble_stokes in the spherical annulus. Left: the conducting Nusselt number converges to 1 at second order, with the top and bottom values identical. Right: |W/D−1| on an exponential radial viscosity ramp shrinks under refinement at every viscosity contrast from 1 to 106.

Conduction: Nu = 1 and Nu_top = Nu_bot ​

tests/test_nusselt_conduction.jl produces the conducting steady state with the diffusion solver itself (backward Euler, Δt≫D2/κ) and evaluates compute_diagnostics on it. The wall flux is the one the diffusion operator uses, k(Tbc−Twall cell)/(Δr/2), with the Dirichlet value held at the face.

GeometryNr = 16Nr = 32Nr = 64
spherical annulus, |Nu−1|8.1e-42.0e-45.1e-5
cylindrical annulus, |Nu−1|2.5e-46.4e-51.6e-5

Second-order convergence, and Nutop=Nubot to round-off at every resolution (asserted to a relative 1e-9).

Energy budget closure ​

The same test takes one backward-Euler step from an arbitrary field and checks that the change of thermal energy equals the net wall power computed from the reported wall gradients:

ρcpddt∑cTc|c|=Pbot−Ptop.

Measured relative residual: 2.1e-15 (spherical), 5.9e-15 (cylindrical). With uniform internal heating the applied power equals the prescribed power to 1e-12 and the residual stays at 6e-15.

tests/test_marker_energy.jl checks that one marker temperature step (projection, grid diffusion, sub-grid relaxation, hand-back) moves exactly the energy the grid solver computed, ∑mwmΔTm=∑cΔTc|c|:

PathRelative residual
2-D spherical annulus, exact delivery5.3e-15
2-D cylindrical annulus, exact delivery2.6e-15
3-D Yin patch, exact delivery5.1e-11
3-D Yin patch, production cap deliveryScaleMax = 27.2e-2

In the last row the cap applies at Yin-patch edge nodes, which carry half-cell marker support.

In a running model the same bookkeeping is logged every diagEvery steps as budget_resid in diagnostics.csv: the change of thermal energy against the boundary and internal power accumulated every step with the diffusion solver's own theta weighting (boundary_power, utils/diagnostics_csv.jl), normalised by the larger of the surface and internal power. A conservative run gives about zero. Measured values on the H&T runs are in the tables above (|budget_resid| ≤ 0.002 on the marker path). The grid TVD path is slightly non-conservative (+0.011), and marker reseeding is the one remaining non-conservative operation on the marker path.

Work equals dissipation (2-D) ​

For any Stokes solution with no-normal-flow walls the buoyancy work equals the viscous dissipation, at any viscosity field:

W=∫f⋅vdV=∫2ηε˙:ε˙dV=∫4ηε˙II2dV=D,W/D=1.

Unlike rigid rotation, on which ε˙≡0, this identity exercises the viscosity-gradient part of the operator. Measured on a fixed analytic load, Nr = 24 / 48 / 96:

ContrastW/D
10.995 / 0.999 / 1.000
100.996 / 0.999 / 1.000
1031.002 / 1.001 / 1.001
1061.019 / 1.006 / 1.003

tests/test_annulus_operator.jl asserts W>0, D>0 and |W/D−1|<0.05 in the spherical annulus on an exponential radial viscosity ramp. Measured at Nr = 32:

Contrast1102104106108
W/D0.99710.99961.00401.01131.0211

At 108, |W/D−1| falls from 0.021 (Nr = 32) to 0.0063 (Nr = 64).

Rigid rotation is annihilated (2-D) ​

Rigid rotation has ε˙≡0, so it is an exact null mode of ∇⋅(2ηε˙) in every geometry. The operator returns a relative residual of 1.5e-16, 1.4e-16 and 2.3e-16 at Nr = 12, 24, 48 in the spherical annulus. tests/test_annulus_operator.jl asserts a residual below 1e-12.

Other exact properties in the suite ​

TestProperty
tests/test_conduction_conservation.jlRadial conduction is a flux difference that telescopes over a column; ALA diffusivity profile and conductive baselines in both geometries.
tests/test_quiescence.jlA model with no lateral structure, no boundary contrast and no sources does nothing: temperature stays uniform, velocity at round-off, energy unchanged.
tests/test_advection_consistency.jl3-D temperature advection: a uniform field is stationary, a divergence-free radial flow preserves a constant, no overshoot of global bounds, TVD less diffusive than upwind.
tests/test_energy_consistency.jlDissipation equals buoyancy work, converges under refinement and survives production-like viscosity fields; the two O(Di) terms of the ALA energy equation balance on the flow they describe.
tests/test_solution_convergence.jlA stationary Stokes solve settles under refinement.
tests/test_melt_conservation.jlMelt mass, composition and energy bookkeeping close.

3-D operator checks ​

Gradient and divergence are adjoint (G = −Bᵀ) ​

The finite-volume 3-D assembly (assemble_stokes_3d) makes the discrete gradient the negative transpose of the discrete divergence. Measured on an 8 × 8 × 24 Yin patch, restricting to interior velocity rows (excluding the Dirichlet wall rows and the pinned pressure cell):

Block‖G+BT‖
all velocity rows0 exactly
Vr0
Vθ0
Vϕ0

verify_g_minus_bt(L, meta) in solvers/stokes_assembly_3d.jl compares the full blocks, Dirichlet and pin rows included, and therefore returns about 0.40 on the same patch; that number does not measure the interior operator.

Work equals dissipation (3-D) ​

Same identity, Yin-Yang patch with a radial viscosity ramp, Nr = 8 / 12 / 16:

ContrastNr = 8Nr = 12Nr = 16
10.99611.00261.0029
1020.98800.99520.9985
1040.99090.99210.9953
1060.98120.98130.9841

Every contrast improves under refinement. tests/test_stokes_3d_variable_eta.jl asserts |W/D−1|<0.05 at Nr = 8 and 12 for contrasts 1 and 104, and that refinement does not make it worse; it measures 0.9961 / 1.0026 and 0.9910 / 0.9922, matching the table.

Rigid rotation (3-D) ​

With cfg.useCurvatureTerms = true (the default, the full spherical deviatoric stress) the rigid-rotation residual is 2.0e-16 / 2.5e-16 / 2.5e-16 at Nr = 8 / 12 / 16, independent of the viscosity field. tests/test_spherical_operator_nullmode.jl pins both branches.

Yin-Yang geometry ​

tests/test_yy_rotation_vs_ks2004.jl checks the vector rotation between the patches against Kageyama & Sato (2004), Eqs. 22-23; agreement is to machine precision. This is verification against a published analytic formula.

The 3-D checks on this page are exact identities of the operator and the Yin-Yang rotation; a comparison of Nu and Vrms with a published 3-D shell benchmark is not listed yet. How the patches are coupled across their seam is described on the Yin-Yang grid page.

Regression tests and smoke tests ​

TestWhat it comparesKind
tests/test_stokes.jlMUMPS against UMFPACK on the same assembled system.regression (both solve the same matrix, so it cannot detect a wrong operator)
tests/test_2d_vcycle_scgs.jl2-D multigrid against the MUMPS solution.regression
tests/test_assembly_3d_mirror.jlSerial and distributed 3-D assemblies give the same matrix and right-hand side to 1e-13.regression
tests/test_config_matrix.jl, tests/test_config_timeloop.jlEvery geometry, mode and advertised flag constructs and survives the time loop.smoke
tests/test_full_feature_smoke.jlEvery 3-D physics flag survives two steps.smoke
tests/smoke_stokes_3d_mg.jlOne 3-D multigrid Stokes solve (default target of bin/mileva test).smoke
tests/test_mpi_*.jlMPI tiers; need mpiexec, not run by tests/run_tests.jl.regression

Run the suite with

bash
JULIA_NUM_THREADS=4 julia --project=. tests/run_tests.jl

It reports 1404 passes, 0 failures, 0 errors in about five minutes.

Scripts outside the runner: tests/test_amr_full.jl (its baseline iteration count of 40 predates the current 3-D solver, which converges in 31), and tests/test_mdpi_eta_contrast.jl, tests/test_vcycle_visc_contrast_sweep.jl, tests/test_arrhenius_viscosity.jl, tests/test_fk_viscosity.jl and the smoke_* chunk scripts, which print their results rather than asserting them.

Related pages: Parameter reference, Function reference.