Papers
Machine learning for physics, as it is published.
New work from arXiv on CFD surrogates, neural operators and physics-informed learning, refreshed every hour. Search it, filter by topic, or start with the papers we keep coming back to.
Papers
New work from arXiv on CFD surrogates, neural operators and physics-informed learning, refreshed every hour. Search it, filter by topic, or start with the papers we keep coming back to.
564 papers in Turbulence · newest first
physics.flu-dyn
This work aims to incorporate basic calibrations of Reynolds-averaged Navier-Stokes (RANS) models as part of machine learning (ML) frameworks. The ML frameworks considered are tensor-basis neural network (TBNN), physics-informed machine learning (PIML), and field inversion & machine learning (FIML) in J. Fluid Mech., 2016, 807, 155-166, Phys. Rev. Fluids, 2017, 2(3), 034603 and J. Comp. Phys., 2016, 305, 758-774, and the baseline RANS models are the one-equation Spalart-Allmaras model, the two-equation - model, and the seven-equation Reynolds stress transport models. ML frameworks are trained against plane channel flow and shear-layer flow data. We compare the ML frameworks and study whether the machine-learned augmentations are detrimental outside the training set. The findings are summarized as follows. The augmentations due to TBNN are detrimental. PIML leads to augmentations that are beneficial inside the training dataset but detrimental outside it. These results are not affected by the baseline RANS model. FIML's augmentations to the two eddy viscosity models, where an inner-layer treatment already exists, are largely neutral. Its augmentation to the seven-equation model, where an inner-layer treatment does not exist, improves the mean flow prediction in a channel. Furthermore, these FIML augmentations are mostly non-detrimental outside the training dataset. In addition to reporting these results, the paper offers physical explanations of the results. Last, we note that the conclusions drawn here are confined to the ML frameworks and the flows considered in this study. More detailed comparative studies and validation & verification studies are needed to account for developments in recent years.
physics.ao-ph
Accurate prediction of atmospheric optical turbulence in localized environments is essential for estimating the performance of free-space optical systems. Macro-meteorological models developed to predict turbulent effects in one environment may fail when applied in new environments. However, existing macro-meteorological models are expected to offer some predictive power. Building a new model from locally-measured macro-meteorology and scintillometer readings can require significant time and resources, as well as a large number of observations. These challenges motivate the development of a machine-learning informed hybrid model framework. By combining some baseline macro-meteorological model with local observations, hybrid models were trained to improve upon the predictive power of each baseline model. Comparisons between the performance of the hybrid models, the selected baseline macro-meteorological models, and machine-learning models trained only on local observations highlight potential use cases for the hybrid model framework when local data is expensive to collect. Both the hybrid and data-only models were trained using the Gradient Boosted Decision Tree (GBDT) architecture with a variable number of in-situ meteorological observations. The hybrid and data-only models were found to outperform three baseline macro-meteorological models, even for low numbers of observations, in some cases as little as one day. For the first baseline macro-meteorological model investigated, the hybrid model achieves an estimated 29% reduction in mean absolute error (MAE) using only one days-equivalent of observation, growing to 41% after only two days, and 68% after 180 days-equivalent training data. The number of days-equivalent training data required is potentially indicative of the seasonal variation in the local microclimate and its propagation environment.
physics.flu-dyn
Computational Fluid Dynamics (CFD) simulations using turbulence models are commonly used in engineering design. Of the different turbulence modeling approaches that are available, eddy viscosity based models are the most common for their computational economy. Eddy viscosity based models utilize many simplifications for this economy such as the gradient diffusion and the isotropic eddy viscosity hypotheses. These simplifications limit the degree to which eddy viscosity models can replicate turbulence physics and lead to model form uncertainty. The Eigenspace Perturbation Method (EPM) has been developed for purely physics based estimates of this model form uncertainty in turbulence model predictions. Due to its physics based nature, the EPM weighs all physically possible outcomes equally leading to overly conservative uncertainty estimates in many cases. In this investigation we use data driven Machine Learning (ML) approaches to address this limitation. Using ML models, we can weigh the physically possible outcomes by their likelihood leading to better calibration of the uncertainty estimates. Specifically, we use ML models to predict the degree of perturbations in the EPM over the flow domain. This work focuses on a Convolutional Neural Network (CNN) based model to learn the discrepancy between Reynolds Averaged Navier Stokes (RANS) and Direct Numerical Simulation (DNS) predictions. This model acts as a marker function, modulating the degree of perturbations in the EPM. We show that this physics constrained machine learning framework performs better than the purely physics or purely ML alternatives, and leads to less conservative uncertainty bounds with improved calibration.
physics.flu-dyn
Turbulent flows are of central importance across applications in science and engineering problems. For design and analysis, scientists and engineers use Computational Fluid Dynamics (CFD) simulations using turbulence models. Turbulent models are limited approximations, introducing epistemic uncertainty in CFD results. For reliable design and analysis, we require quantification of these uncertainties. The Eigenspace Perturbation Method (EPM) is the preeminent physics based approach for turbulence model UQ, but often leads to overly conservative uncertainty bounds. In this study, we use Machine Learning (ML) models to moderate the EPM perturbations and introduce our physics constrained machine learning framework for turbulence model UQ. We test this framework in multiple problems to show that it leads to improved calibration of the uncertainty estimates.
physics.flu-dyn
The constants and functions in Reynolds-averaged Navier Stokes (RANS) turbulence models are coupled. Consequently, modifications of a RANS model often negatively impact its basic calibrations, which is why machine-learned augmentations are often detrimental outside the training dataset. A solution to this is to identify the degrees of freedom that do not affect the basic calibrations and only modify these identified degrees of freedom when re-calibrating the baseline model to accommodate a specific application. This approach is colloquially known as the "rubber-band" approach, which we formally call "constrained model re-calibration" in this article. To illustrate the efficacy of the approach, we identify the degrees of freedom in the Spalart-Allmaras (SA) model that do not affect the log law calibration. By subsequently interfacing data-based methods with these degrees of freedom, we train models to solve historically challenging flow scenarios, including the round-jet/plane-jet anomaly, airfoil stall, secondary flow separation, and recovery after separation. In addition to good performance inside the training dataset, the trained models yield similar performance as the baseline model outside the training dataset.
physics.flu-dyn
Assessing the compliance of a white-box turbulence model with known turbulent knowledge is straightforward. It enables users to screen conventional turbulence models and identify apparent inadequacies, thereby allowing for a more focused and fruitful validation and verification. However, comparing a black-box machine-learning model to known empirical scalings is not straightforward. Unless one implements and tests the model, it would not be clear if a machine-learning model, trained at finite Reynolds numbers preserves the known high Reynolds number limit. This is inconvenient, particularly because model implementation involves retraining and re-interfacing. This work attempts to address this issue, allowing fast a priori screening of machine-learning models that are based on feed-forward neural networks (FNN). The method leverages the mathematical theorems we present in the paper. These theorems offer estimates of a network's limits even when the exact weights and biases are unknown. For demonstration purposes, we screen existing machine-learning wall models and RANS models for their compliance with the log layer physics and the viscous layer physics in a priori manner. In addition, the theorems serve as essential guidelines for future machine-learning models.
physics.flu-dyn
We present a novel machine learning approach to reduce the dimensionality of state variables in stratified turbulent flows governed by the Navier-Stokes equations in the Boussinesq approximation. The aim of the new method is to perform an accurate reconstruction of the temperature and the three-dimensional velocity of geophysical turbulent flows developing non-homogeneities, starting from a low-dimensional representation in latent space, yet conserving important information about non-Gaussian structures captured by high-order moments of distributions. To achieve this goal we modify the standard Convolutional Autoencoder (CAE) by implementing a customized loss function that enforces the accuracy of the reconstructed high order statistical moments. We present results for compression coefficients up to 16 demonstrating how the proposed method is more efficient than a standard CAE in performing dimensionality reduction of simulations of stratified geophysical flows characterized by intermittent phenomena, as observed in the atmosphere and the oceans.
physics.flu-dyn
The current revolution in the field of machine learning (ML) is leading to many interesting developments in a wide range of areas, including fluid mechanics. Here we review recent and emerging possibilities in the context of predictions, simulations and control of fluid flows, focusing on wall-bounded turbulence. A number of important areas are benefiting from ML, and it is important to identify the synergies with the existing pillars of scientific discovery, i.e. theory, experiments and simulations. It is essential to adopt a balanced approach as a community in order to harness all the positive potential of these novel methods.
physics.comp-ph
Turbulence in fluids, gases, and plasmas remains an open problem of both practical and fundamental importance. Its irreducible complexity usually cannot be tackled computationally in a brute-force style. Here, we combine Large Eddy Simulation (LES) techniques with Machine Learning (ML) to retain only the largest dynamics explicitly, while small-scale dynamics are described by an ML-based sub-grid-scale model. Applying this novel approach to self-driven plasma turbulence allows us to remove large parts of the inertial range, reducing the computational effort by about three orders of magnitude, while retaining the statistical physical properties of the turbulent system.
cs.LG
Analysis of compressible turbulent flows is essential for applications related to propulsion, energy generation, and the environment. Here, we present BLASTNet 2.0, a 2.2 TB network-of-datasets containing 744 full-domain samples from 34 high-fidelity direct numerical simulations, which addresses the current limited availability of 3D high-fidelity reacting and non-reacting compressible turbulent flow simulation data. With this data, we benchmark a total of 49 variations of five deep learning approaches for 3D super-resolution - which can be applied for improving scientific imaging, simulations, turbulence models, as well as in computer vision applications. We perform neural scaling analysis on these models to examine the performance of different machine learning (ML) approaches, including two scientific ML techniques. We demonstrate that (i) predictive performance can scale with model size and cost, (ii) architecture matters significantly, especially for smaller models, and (iii) the benefits of physics-based losses can persist with increasing model size. The outcomes of this benchmark study are anticipated to offer insights that can aid the design of 3D super-resolution models, especially for turbulence models, while this data is expected to foster ML methods for a broad range of flow physics applications. This data is publicly available with download links and browsing tools consolidated at https://blastnet.github.io.
physics.comp-ph
Earth system models suffer from various structural and parametric errors in their representation of nonlinear, multi-scale processes, leading to uncertainties in their long-term projections. The effects of many of these errors (particularly those due to fast physics) can be quantified in short-term simulations, e.g., as differences between the predicted and observed states (analysis increments). With the increase in the availability of high-quality observations and simulations, learning nudging from these increments to correct model errors has become an active research area. However, most studies focus on using neural networks, which while powerful, are hard to interpret, are data-hungry, and poorly generalize out-of-distribution. Here, we show the capabilities of Model Error Discovery with Interpretability and Data Assimilation (MEDIDA), a general, data-efficient framework that uses sparsity-promoting equation-discovery techniques to learn model errors from analysis increments. Using two-layer quasi-geostrophic turbulence as the test case, MEDIDA is shown to successfully discover various linear and nonlinear structural/parametric errors when full observations are available. Discovery from spatially sparse observations is found to require highly accurate interpolation schemes. While NNs have shown success as interpolators in recent studies, here, they are found inadequate due to their inability to accurately represent small scales, a phenomenon known as spectral bias. We show that a general remedy, adding a random Fourier feature layer to the NN, resolves this issue enabling MEDIDA to successfully discover model errors from sparse observations. These promising results suggest that with further development, MEDIDA could be scaled up to models of the Earth system and real observations.
physics.flu-dyn
This study proposes a novel deep-learning-based method for generating reduced representations of turbulent flows that ensures efficient storage and transfer while maintaining high accuracy during decompression. A Swin-Transformer network combined with a physical constraints-based loss function is utilized to compress the turbulent flows with high compression ratios and then restore the data with the underlying physical properties. The forced isotropic turbulent flow is used to demonstrate the ability of the Swin-Transformer-based (ST) model, where the instantaneous and statistical results show the excellent ability of the model to recover the flow data with remarkable accuracy. Furthermore, the capability of the ST model is compared with a typical Convolutional Neural Network-based auto-encoder (CNN-AE) by using the turbulent channel flow at two friction Reynolds numbers = 180 and 550. The results generated by the ST model are significantly more consistent with the DNS data than those recovered by the CNN-AE, indicating the superior ability of the ST model to compress and restore the turbulent flow. This study also compares the compression performance of the ST model at different compression ratios (CR) and finds that the model has low enough error even at very high CR. Additionally, the effect of transfer learning (TL) is investigated, showing that TL reduces the training time by 64% while maintaining high accuracy. The results illustrate for the first time that the Swin-Transformer-based model incorporating a physically constrained loss function can compress and restore turbulent flows with the correct physics.
cs.LG
Simulating turbulent flows is crucial for a wide range of applications, and machine learning-based solvers are gaining increasing relevance. However, achieving temporal stability when generalizing to longer rollout horizons remains a persistent challenge for learned PDE solvers. In this work, we analyze if fully data-driven fluid solvers that utilize an autoregressive rollout based on conditional diffusion models are a viable option to address this challenge. We investigate accuracy, posterior sampling, spectral behavior, and temporal stability, while requiring that methods generalize to flow parameters beyond the training regime. To quantitatively and qualitatively benchmark the performance of various flow prediction approaches, three challenging 2D scenarios including incompressible and transonic flows, as well as isotropic turbulence are employed. We find that even simple diffusion-based approaches can outperform multiple established flow prediction methods in terms of accuracy and temporal stability, while being on par with state-of-the-art stabilization techniques like unrolling at training time. Such traditional architectures are superior in terms of inference speed, however, the probabilistic nature of diffusion approaches allows for inferring multiple predictions that align with the statistics of the underlying physics. Overall, our benchmark contains three carefully chosen data sets that are suitable for probabilistic evaluation alongside various established flow prediction architectures.
physics.flu-dyn
Supervised super-resolution deep convolutional neural networks (CNNs) have gained significant attention for their potential in reconstructing velocity and scalar fields in turbulent flows. Despite their popularity, CNNs currently lack the ability to accurately produce high-frequency and small-scale features, and tests of their generalizability to out-of-sample flows are not widespread. Generative adversarial networks (GANs), which consist of two distinct neural networks (NNs), a generator and discriminator, are a promising alternative, allowing for both semi-supervised and unsupervised training. The difference in the flow fields produced by these two NN architectures has not been thoroughly investigated, and a comprehensive understanding of the discriminator's role has yet to be developed. This study assesses the effectiveness of the unsupervised adversarial training in GANs for turbulence reconstruction in forced homogeneous isotropic turbulence. GAN-based architectures are found to outperform supervised CNNs for turbulent flow reconstruction for in-sample cases. The reconstruction accuracy of both architectures diminishes for out-of-sample cases, though the GAN's discriminator network significantly improves the generator's out-of-sample robustness using either an additional unsupervised training step with large eddy simulation input fields and a dynamic selection of the most suitable upsampling factor. These enhance the generator's ability to reconstruct small-scale gradients, turbulence intermittency, and velocity-gradient probability density functions. The extrapolation capability of the GAN-based model is demonstrated for out-of-sample flows at higher Reynolds numbers. Based on these findings, incorporating discriminator-based training is recommended to enhance the reconstruction capability of super-resolution CNNs.
physics.flu-dyn
To model complex turbulent flow and heat transfer phenomena, this study aims to analyze and develop a reduced modeling approach based on artificial neural network (ANN) and wrapper methods. This approach has an advantage over other methods such as the correlation-based filter method in terms of removing redundant or irrelevant parameters even under non-linearity among them. As a downside, the overfitting and randomness of ANN training may produce inconsistent subsets over selection trials especially in a higher physical dimension. This study analyzes a few existing ANN-based wrapper methods and develops a revised one based on the gradient-based subset selection indices to minimize the loss in the total derivative or the directional consistency at each elimination step. To examine parameter reduction performance and consistency-over-trials, we apply these methods to a manufactured subset selection problem, modeling of the bubble size in a turbulent bubbly flow, and modeling of the spatially varying turbulent Prandtl number in a duct flow. It is found that the gradient-based subset selection to minimize the total derivative loss results in improved consistency-over-trials compared to the other ANN-based wrapper methods, while removing unnecessary parameters successfully. For the reduced turbulent Prandtl number model, the gradient-based subset selection improves the prediction in the validation case over the other methods. Also, the reduced parameter subsets show a slight increase in the training speed compared to the others.
physics.flu-dyn
Developed turbulent motion of fluid still lacks an analytical description despite more than a century of active research. Nowadays phenomenological ideas are widely used in practical applications, such as small-scale closures for numerical simulations of turbulent flows. In the present work, we use a shell model of turbulence to construct a closure intended to have a solid theoretical background and to capture intrinsic probabilistic features of turbulence. Shell models of turbulence are dynamical deterministic systems used to model energy cascade and other key aspects of the Navier-Stokes such as intermittency. We rescale the variables of the Sabra model in a way which leads to hidden symmetries and universal distributions. We then use such fine distributions to write closures, i.e., missing expressions for some of the Sabra variables. Our closures rely on approximating probability density functions using a Gaussian Mixture Model, which makes them probabilistic by nature and allows us to write time-correlated closures. We also provide a framework where other Machine Learning tools can be employed with reduced black-box aspects.
physics.flu-dyn
The objective is to provide clear and well-motivated guidance to Machine Learning (ML) teams, founded on our experience in empirical turbulence modeling. Guidance is also needed for modeling outside ML. ML is not yet successful in turbulence modeling, and many papers have produced unusable proposals either due to errors in math or physics, or to severe overfitting. We believe that "Turbulence Culture" (TC) takes years to learn and is difficult to convey especially considering the modern lack of time for careful study; important facts which are self-evident after a career in turbulence research and modeling and extensive reading are easy to miss. In addition, many of them are not absolute facts, a consequence of the gaps in our understanding of turbulence and the weak connection of models to first principles. Some of the mathematical facts are rigorous, but the physical aspects often are not. Turbulence models are surprisingly arbitrary. Disagreement between experts confuses the new entrants. In addition, several key properties of the models are ascertained through non-trivial analytical properties of the differential equations, which puts them out of reach of purely data-driven ML-type approaches. The best example is the crucial behavior of the model at the edge of the turbulent region (ETR). The knowledge we wish to put out here may be divided into "Mission" and "Requirements," each combining physics and mathematics. Clear lists of "Hard" and "Soft" constraints are presented. A concrete example of how DNS data could be used, possibly allied with ML, is first carried through and illustrates the large number of decisions needed. Our focus is on creating effective products which will empower CFD, rather than on publications.
cs.LG
This article introduces a new Neural Network stochastic model to generate a 1-dimensional stochastic field with turbulent velocity statistics. Both the model architecture and training procedure ground on the Kolmogorov and Obukhov statistical theories of fully developed turbulence, so guaranteeing descriptions of 1) energy distribution, 2) energy cascade and 3) intermittency across scales in agreement with experimental observations. The model is a Generative Adversarial Network with multiple multiscale optimization criteria. First, we use three physics-based criteria: the variance, skewness and flatness of the increments of the generated field that retrieve respectively the turbulent energy distribution, energy cascade and intermittency across scales. Second, the Generative Adversarial Network criterion, based on reproducing statistical distributions, is used on segments of different length of the generated field. Furthermore, to mimic multiscale decompositions frequently used in turbulence's studies, the model architecture is fully convolutional with kernel sizes varying along the multiple layers of the model. To train our model we use turbulent velocity signals from grid turbulence at Modane wind tunnel.
physics.flu-dyn
In this study, we explore the application of an artificial recurrent neural network (RNN) called Long Short-Term Memory (LSTM) as an alternative to a turbulent Reynolds-Averaged Navier-Stokes (RANS) model. The LSTM models are utilized to predict the shear Reynolds stress in developed and developing turbulent channel flows. We conduct comparative analyses, comparing the LSTM results propagated through computational fluid dynamics (CFD) simulations with the outcomes from the model and data acquired from direct numerical simulation (DNS). These analyses demonstrate a good performance of the LSTM approach.
physics.flu-dyn
Differentiable physical simulators are proving to be valuable tools for developing data-driven models for computational fluid dynamics (CFD). In particular, these simulators enable end-to-end training of machine learning (ML) models embedded within CFD solvers. This paradigm enables novel algorithms which combine the generalization power and low cost of physics-based simulations with the flexibility and automation of deep learning methods. In this study, we introduce a framework for embedding deep learning models within a finite element solver for incompressible Navier-Stokes equations, specifically applying this approach to learn a subgrid-scale (SGS) closure with a graph neural network (GNN). We first demonstrate the feasibility of the approach on flow over a two-dimensional backward-facing step, using it as a proof of concept to show that solver-consistent training produces stable and physically meaningful closures. Then, we extend this to a turbulent flow over a three-dimensional backward-facing step. In this setting, the GNN-based closure not only attains low prediction errors, but also recovers key turbulence statistics and preserves multiscale turbulent structures. We further demonstrate that the closure can be identified in data-limited learning scenarios as well. Overall, the proposed end-to-end learning paradigm offers a viable pathway toward physically consistent and generalizable data-driven SGS modeling on complex and unstructured domains.