Large scale simulation of fluid structure interaction using Lattice Boltzmann methods and the `physics engine' (2008)

We study the methodology behind the simulation of fluid flow with up to 150,000~fully resolved rigid bodies incorporated in the flow. The simulation is performed using a 3D~Lattice Boltzmann solver for the fluid flow and a so-called rigid body physics engine for the treatment of the objects. The numerical methods, the necessary extensions and the coupling between both methods are presented in detail. Furthermore, the parallelisation is discussed and performance results are given for different test cases with up to 150,000~rigid bodies on up to 1025~processor cores. The approach enables a detailed simulation of large scale particulate flows, which are relevant for many industrial applications. References C. K. Aidun, Y. Lu, and E.-J. Ding. Direct analysis of particulate suspensions with inertia using the discrete Boltzmann equation. J. Fluid Mech., 373:287--311, October 1998. http://adsabs.harvard.edu/abs/1998JFM...373..287A. P. L. Bhatnagar, E. P. Gross, and M. Krook. A Model for Collision Processes in Gases. I. Small Amplitude Processes in Charged and Neutral One-Component Systems. Phys. Rev., 94(3):511--525, 1954. doi:10.1103/PhysRev.94.511. C. Binder, C. Feichtinger, H.J. Schmid, N. Thurey, W. Peukert, and U. Rude. Simulation of the Hydrodynamic Drag of Aggregated Particles. Journal of Colloid and Interface Science, 301:155--167, Jan 2006. doi:10.1016/j.jcis.2006.04.045. J. F. Brady and G. Bossis. Stokesian Dynamics. Annu. Rev. Fluid Mech., 20(1):111--157, 1988. doi:10.1146/annurev.fl.20.010188.000551. S. Chen and G. D. Doolen. Lattice Boltzmann method for fluid flows. Annu. Rev. Fluid Mech., 30:329--364, 1998. doi:10.1146/annurev.fluid.30.1.329. R. W. Cottle, J. S. Pang, and R. E. Stone. The Linear Complementarity Problem. Academic Press, 1992. S. Donath, K. Iglberger, G. Wellein, T. Zeiser, and A. Nitsure. Performance Comparison of Different Parallel Lattice Boltzmann Implementations on Multi-core Multi-socket Systems. International Journal of Computational Science and Engineering (IJCSE), accepted for publication 2008. K. Erleben. {Stable, Robust, and Versatile Multibody Dynamics Animation}. PhD thesis, University of Copenhagen (DIKU), 2005. ftp://ftp.diku.dk/diku/image/publications/erleben.050401.pdf. M. Griebel, S. Knapek, and G. Zumbusch. {Numerical Simulation in Molecular Dynamics. Numerics, Algorithms, Parallelization, Applications}, volume 5 of Texts in Computational Science and Engineering. Springer Verlag, 2007. doi:10.1007/978-3-540-68095-6. X. He and L.-S. Luo. Lattice Boltzmann model for the incompressible Navier-Stokes equation. J. Stat. Phys., 88:927--944, 1997. doi:10.1023/B:JOSS.0000015179.12689.e4. Information on the HLRB 2. http://www.lrz-muenchen.de/services/compute/hlrb/, Aug. 2008. K. Hofler, M. Muller, S. Schwarzer, and B. Wachmann. Interacting Particle-Liquid Systems. In E. Krause and W. Jager, editors, High Performance Computing in Science and Engineering '98. Springer Verlag, 1998. J. Horbach and D. Frenkel. Lattice-Boltzmann method for the simulation of transport phenomena in charged colloids. Phys. Rev. E, 64(6):061507, 2001. doi:10.1103/PhysRevE.64.061507. K. Iglberger, N. Thurey, and U. Rude. Simulation of moving particles in 3D with the Lattice Boltzmann method. Comp. Math. Appl., 55(7):1461--1468, 2008. doi:10.1016/j.camwa.2007.08.022. C. Korner, T. Pohl, U. Rude, N. Thurey, and T. Zeiser. Parallel Lattice Boltzmann Methods for CFD Applications. In A.M. Bruaset and A. Tveito, editors, Numerical Solution of Partial Differential Equations on Parallel Computers, volume 51 of Lecture Notes for Computational Science and Engineering, chapter 5, pages 439--465. Springer Verlag, 2005. doi:10.1007/3-540-31619-1_13. C. Korner, M. Thies, T. Hofmann, N. Thurey, and U. Rude. Lattice Boltzmann Model for Free Surface Flow for Modeling Foaming. Journal of Statistical Physics, 121:179--196, 2005. doi:10.1007/s10955-005-8879-8. A. J. C. Ladd. Numerical simulations of particulate suspensions via a discretized Boltzmann equation. Part 1. Theoretical foundation. J. Fluid Mech., 271:285--309, 1994. doi:10.1017/S0022112094001771. A. J. C. Ladd. Numerical simulations of particulate suspensions via a discretized Boltzmann equation. Part 2. Numerical results. J. Fluid Mech., 271:311--339, 1994. doi:10.1017/S0022112094001783. R. Mei, W. Shyy, D. Yu, and L.-S. Luo. Lattice Boltzmann Method for 3-D Flows with Curved Boundary. J. Comp. Phys., 161:680--699, 2002. doi:10.1006/jcph.2000.6522. I. Millington. Game Physics Engine Development. Series in Interactive 3D Technology. Morgan Kaufmann, 2007. CD included. P. Neumann. Numerical simulation of nanoparticles in Brownian motion using the lattice Boltzmann method. Master's thesis, University of Erlangen-Nuremberg, Computer Science 10 -- Systemsimulation, 2008. T.-W. Pan, D. D. Joseph, R. Bai, R. Glowinski, and V. Sarin. Fluidization of 1204 spheres: simulation and experiment. J. Fluid Mech., 451:169--191, 2002. doi:10.1017/S0022112001006474. T. Pohl. High Performance Simulation of Free Surface Flows Using the Lattice Boltzmann Method. PhD thesis, University of Erlangen-Nuremberg, Computer Science 10 --- Systemsimulation, July 2008. http://www10.informatik.uni-erlangen.de/Publications/Dissertations/Pohl_080713.pdf. T. Pohl, M. Kowarschik, J. Wilke, K. Iglberger, and U. Rude. Optimization and Profiling of the Cache Performance of Parallel Lattice Boltzmann Codes. Parallel Processing Letters, 13(4):549--560, 2003. doi:10.1142/S0129626403001501. T. Pohl, N. Thurey, F. Deserno, U. Rude, P. Lammers, G. Wellein, and T. Zeiser. Performance Evaluation of Parallel Large-Scale Lattice Boltzmann Applications on Three Supercomputing Architectures. Nov 2004. Supercomputing Conference 04. doi:10.1109/SC.2004.37. T. Preclik. Frictional Rigid Body Dynamics. Master's thesis, University of Erlangen-Nuremberg, Computer Science 10 --- Systemsimulation, 2007. Computer Science Department 10 (System Simulation), University of Erlangen-Nuermberg. D. Qi. Lattice-Boltzmann simulations of particles in non-zero-Reynolds-number flows. J. Fluid Mech., 385:41--62, April 1999. http://adsabs.harvard.edu/abs/1999JFM...385...41Q. Y. H. Qian, D. D'Humieres, and P. Lallemand. Lattice BGK Models for Navier--Stokes Equation. Europhysics Letters (EPL), 17(6):479--484, 1992. doi:10.1209/0295-5075/17/6/001. D. E. Stewart. Impact and Friction of Solids, Structures and Machines, chapter Time-stepping methods and the mathematics of rigid body dynamics. Birkhauser, 2000. M. Sturmer, J. Gotz, G. Richter, A. Dorfler, and U. Rude. Fluid flow simulation on the Cell Broadband Engine using the lattice Boltzmann method. Comp. Math. App., submitted 2007, accepted 2008, to be published. S. Succi. The Lattice Boltzmann Equation---For Fluid Dynamics and Beyond. Clarendon Press, 2001. G. Wellein, T. Zeiser, S. Donath, and G. Hager. On the single processor performance of simple lattice Boltzmann kernels. Computer and Fluids, 35(8--9):910--919, 2005. doi:10.1016/j.compfluid.2005.02.008. D. Yu, R. Mei, L.-S. Luo, and W. Shyy. Viscous flow computations with the method of lattice Boltzmann equation. Prog. Aero. Sci., 39(5):329--367, 2003. doi:10.1016/S0376-0421(03)00003-4. T. Zeiser, G. Wellein, A. Nitsure, K. Iglberger, U. Rude, and G. Hager. Introducing a parallel cache oblivious blocking approach for the lattice Boltzmann method. Progress in Computational Fluid Dynamics, 8(1-4):179--188, 2008. doi:10.1504/PCFD.2008.018088.