Reinforcement Learning for Swimming in a Concentration Gradient

Abstract

The coupled problem of hydrodynamics and solute transport for a microscopic swimmer is studied, with the Reynolds number set to zero and the Péclet number (Pe) ranging from 0 to 100. A suitable second order finite element method is proposed based on the FEniCS library. It is shown that errors in the temporal treatment of the geometry can destroy the dynamical properties of the system. The method is applied to compute the Sherwood number as a function of Pe in homogeneous fluids. The results agree with previously published asymptotic approximations and confirm that little gain in solute flux is achieved by swimming. A reinforcement learning framework is also introduced that considers the swimmer as a learning agent moving inside a fluid with a concentration gradient. The outcomes of the learning process show that learning locomotion is significantly easier than learning to move towards concentration maxima. Furthermore, the experiments show that the learning difficulty increases severely with the Péclet number. The results demonstrate the challenges that natural and artificial swimmers need to overcome to migrate efficiently in chemically inhomogeneous environments.