The world of weather forecasting has taken a significant leap forward with the introduction of NVIDIA’s FourCastNet3 (FCN3), the latest advancement in artificial intelligence-driven global weather prediction systems. This state-of-the-art system is part of NVIDIA’s Earth-2 initiative and offers unparalleled probabilistic skill, computational efficiency, spectral fidelity, ensemble calibration, and stability, particularly at subseasonal timescales. FCN3 sets a new benchmark in medium-range forecasting accuracy, competing head-to-head with established machine learning models like GenCast and outperforming traditional numerical weather prediction systems such as IFS-ENS.

One of the standout features of FCN3 is its remarkable speed and efficiency. A single 60-day forecast rollout at 0.25° and 6-hourly resolution is completed in less than four minutes using just one NVIDIA H100 Tensor Core GPU. This performance marks an eightfold increase in speed over GenCast and is sixty times faster than IFS-ENS. The rapid processing capabilities of FCN3 not only demonstrate its technological prowess but also highlight the potential for more timely and accurate weather predictions, which are crucial for planning and disaster management.

The system’s ensemble calibration and spectral fidelity are particularly noteworthy. Even at extended lead times of up to 60 days, FCN3’s ensemble members maintain realistic spectral properties. This achievement marks a significant advancement in data-driven weather prediction, allowing for large ensemble forecasts that extend from medium-range to subseasonal timescales. This capability is crucial for understanding and preparing for potential weather patterns and anomalies weeks in advance.

FCN3’s Advanced Architecture

The architecture of FCN3 is built on a fully convolutional, spherical neural operator framework. This setup leverages spherical signal processing primitives, distinguishing it from its predecessor, FourCastNet2, which relied on the Spherical Fourier Neural Operator. FCN3 integrates local spherical convolutions with spectral convolutions, utilizing Morlet wavelets in a discrete-continuous group convolution framework. This innovative approach allows for anisotropic, localized filters that are adept at capturing localized atmospheric phenomena while ensuring computational efficiency. A custom implementation in NVIDIA CUDA further enhances this efficiency.

FCN3 introduces stochasticity at each predictive stage through a latent noise variable, which evolves based on a diffusion process on a spherical surface. This hidden Markov model approach allows for the efficient generation of ensemble members in a single step. Unlike diffusion model-based methods, this technique ensures that FCN3 learns the correct spatial correlations inherent in stochastic atmospheric processes. The model is trained as an ensemble, minimizing a composite loss function that combines the continuously ranked probability score (CRPS) across both spatial and spectral domains.

Scaling and Performance

Scaling machine learning models is a critical aspect of achieving high performance, and this is especially true in data-driven weather models. With FCN3, NVIDIA has introduced a novel paradigm for model parallelism, inspired by traditional numerical weather modeling’s domain decomposition. This strategy allows larger models to fit into VRAM during training by distributing the model across multiple devices, thus reducing the disk I/O per device. Spatial operations, such as convolutions, are executed in a distributed manner using the NVIDIA Collective Communications Library (NCCL). FCN3 is trained on up to 1,024 GPUs, utilizing domain, batch, and ensemble parallelism. For those interested in exploring FCN3 further, the training code is available online.

In terms of performance, FCN3 outshines the best physics-based ensemble model, IFS-ENS, and matches the predictive skill of GenCast. On a single NVIDIA H100 GPU, FCN3 can generate a 15-day forecast with 6-hour intervals and 0.25° spatial resolution in just one minute. This represents a significant improvement in both speed and efficiency over existing models. The system’s probabilistic ensembles consistently exhibit spread-skill ratios near one, indicating that the predicted uncertainties align closely with observed atmospheric variability. This reliability is further confirmed by rank histograms and additional diagnostics, which show that the ensemble members are interchangeable with real-world observations.

Spectral Fidelity and Forecasting Accuracy

One of FCN3’s most impressive features is its ability to preserve atmospheric spectral signatures across all scales. The system faithfully reproduces the energy cascade and sharpness of real-world weather patterns, even at extended lead times of up to 60 days. Unlike many machine learning models, which may blur high-frequency features or produce noisy artifacts over time, FCN3 maintains stable, physically realistic spectra. This capability enables the creation of accurate, sharp, and physically consistent forecasts that extend well into the subseasonal range.

This accuracy is vividly illustrated in a case study involving FCN3’s predictions of 500 hPa wind intensities, initialized on February 11, 2020, shortly before Storm Dennis hit Europe. FCN3 accurately captured the magnitude of wind intensities and their variability across different scales, as demonstrated by the angular power spectral density of its predictions. This accuracy remains consistent even with extended rollouts of 30 days or more.

Getting Started with FourCastNet3

For those interested in utilizing FCN3, the fully trained model checkpoint is available on NVIDIA’s NGC platform. An easy way to run FCN3 inference is through Earth2Studio. To execute a single 4-member ensemble inference, the following code can be used:

“`python
from earth2studio.models.px import FCN3
from earth2studio.data import NCAR_ERA5
from earth2studio.io import NetCDF4Backend
from earth2studio.perturbation import Zero
from earth2studio.run import ensemble as run
import numpy as np

load default package

model = FCN3.load_model(FCN3.load_default_package())

determine output variables

out_vars = ["u10m", "v10m", "t2m", "msl", "tcwv"]

data source initial condition

ds = NCAR_ERA5()
io = NetCDF4Backend("fcn3_ensemble.nc", backend_kwargs={"mode": "w"})

no perturbation required due to hidden Markov formulation of FCN3

perturbation = Zero()

invoke inference with 4 ensemble members

run(time=["2024-09-24"],
nsteps=16,
nensemble=4,
prognostic=model,
data=ds,
io=io,
perturbation=perturbation,
batch_size=1,
output_coords={"variable": np.array(out_vars)},
)
“`

The results of this inference, depicted in the accompanying visualizations, showcase FCN3’s capability to predict total column water vapor and 10-meter zonal wind velocity, along with their respective ensemble standard deviations. For optimal performance, it is recommended to install torch-harmonics with custom CUDA extensions enabled and use automatic mixed precision in bf16 format during inference, which is the default setting in Earth2Studio.

Conclusion

FourCastNet3 represents a remarkable advancement in the field of weather forecasting, offering unprecedented speed, accuracy, and reliability. As the system continues to evolve and scale, it holds the promise of transforming how we predict and understand weather patterns on a global scale. For those interested in delving deeper into FCN3, a wealth of resources and training materials are available. The future of weather forecasting is here, and it is powered by AI. For more details on FourCastNet3, visit the NVIDIA blog.

For more Information, Refer to this article.