Posts | MAMOC

The Spatial Ecology of Predator-Prey Systems.

Sat, 18 Nov 2023 00:00:00 +0000

Abstract

This study focuses on two methods of computationally modelling systems of many interacting particles of two species. The first method is the general idea of reaction-diffusion systems, which are mathematical differential constructs that describe how spatial and temporal patterns can emerge from the interactions between mathematical species and their environment. Also considered are agent based models, which instead only use hyper-local information and do not rely on macroscopic differential equations at all. In particular, the predator prey system is considered from both perspectives. By coupling local interaction rules with movement, these models elucidate the dynamics of population spread, predator-prey relationships, and the emergence of patterns in ecosystems.

Introduction to reaction-diffusion systems

Theoretical ecology is a mathematical framework for understanding how the interactions of individual organisms with each other and with the environment determine the distribution of populations and the structure of communities. Many different models are needed, each based on some set of hypotheses about the scale and structure of the spatial environment and the way that organisms disperse through it. When many agents interact in simple ways, sometimes emergent properties can be observed in their collective, such as the propagation of wavefronts or the formation of patterns. These patterns are glimpses of structure from chaos, and have prompted countless minds to become fascinated by the connections between maths and biology.

Reaction-diffusion models are a way of translating the simple assumptions about ways agents can move and interact on the local level into global conclusions about the persistence or extinction of populations and the coexistence of interacting species. Born from the macroscopic observations of molecular diffusion, reaction-diffusion models now more widely refer to any event-driven system of interacting moving agents, and occur and many different scales.

In biology, the great success of reaction-diffusion mathematics is the Turing model, which describes how homogeneous groups of cells in an embryo can spontaneously differentiate into patterns, like the spots and stripes on animal skins [1].

On the left, a computationally generated Turing pattern. On the right, an example of Turing patterns found organically in nature, on the beautiful patterns on the skin of fish. Photo credit: Dennis Jacobsen/Shutterstock

In chemistry, the Belousov-Zhabotinsky chemical reaction is an incredible display of dispersive concentric patterns. Further examples are as distributed as econometric information diffusion and as grievous as modelling the spread of forest fires.

These systems are linked, and the outcomes we see from their patterns are directly caused by similarities in the form of the differential equations which govern their spatio-temporal evolution. They share the two species reaction diffusion equations, which are given by

$$ \begin{pmatrix} \dot q_{1} \\ \dot q_{2} \end{pmatrix} = \begin{pmatrix} D_{1} \\ D_{2} \end{pmatrix} \cdot \nabla^{2} \begin{pmatrix}q_{1} \\ q_{2} \end{pmatrix} + \begin{pmatrix} R_{1}(q_{1}, q_{2}) \\ R_{2}(q_{1}, q_{2}) \end{pmatrix},$$

where $q_{1,2}$ describes the concentration of either species, $D_{1,2}$ their diffusion coefficients, and $R_{1,2}(q_{1}, q_{2})$ are functions representing the agent’s interactive and local behaviour. Decomposing the equation, the first term on the right hand side can be recognised as Fick’s second law of diffusion, $ \dot q = D \nabla^{2} q$ for a concentration $q$ [2].

The dynamics of our model is specified by the rates at which individuals move and die or reproduce. As such, a good place to start is to consider the local mechanics of our agent movement and reactions, supposing that agents may only move to a randomly chosen nearest neighbour of their location, and reproduce or die at rates which depend on the number of individuals at the same location. Using some simplifying assumptions along the way, there are creative ways to derive the reaction term, which is usually, hopefully, linear in $q_{1,2}$. Often, however, we are not so lucky. Spatial models often unavoidably invite non-linear terms, resulting in chaotic long-term behaviour.

Lotka-Volterra equations.

Without spatial dependence.

This example, the Lotka-Volterra equations, are a model for the population dynamics of a predator species and a prey species, perhaps foxes and rabbits, and is a convinient way to explore how reaction terms work. Choosing to completely ignore spatial dimensions, the mean field assumption can be adopted: that all agents interact with the average effect of all others. Letting $ x_1(t) $ denote the prey population and $x_2(t) $ denote the predator population at time $t $,

$$ \dot x_1 = \alpha x_1 - \beta x_1 x_2,$$ $$\dot x_2 = \delta x_1 x_2 - \gamma x_2. $$

$ \alpha $ is the natural birth rate of the prey in the absence of predators.
$ \beta $ is the death rate of the prey due to predation.
$ \gamma $ is the natural death rate of the predators in the absence of prey.
$\delta $ is the rate at which predators increase by consuming prey.

Reading off from the general form of reaction-diffusion equations, $D_1$ and $D_2$ have been set to zero, and the remaining terms are the reactive terms $R_{1,2}(q_{1}, q_{2})$. Below, linear stability analysis is used with the method outlined by Strogatz [3 pp.151-152]. Firstly, the steady states of this system of differential equations are found by setting the time derivatives to zero. The trivial solution $x_1 = x_2 = 0$ indicates mutual extinction, but otherwise, solving for constants, we get ${x_1, x_2} = {\gamma / \delta, \alpha / \beta}$. Secondarily, to analyse the stability of these steady state solutions, our set of differential equations are linearised so that we have

$$J = \begin{pmatrix} \frac{\partial f_1}{\partial x_1} & \frac{\partial f_1}{\partial x_2} \\ \frac{\partial f_2}{\partial x_1} & \frac{\partial f_2}{\partial x_2} \end{pmatrix} = \begin{pmatrix} \alpha - \beta x_2 & -\beta x_1 \\ \delta x_2 & \delta x_1 - \gamma \end{pmatrix} = \begin{pmatrix} 0 & -\beta \frac{\gamma}{\delta} \\ \delta \frac{\alpha}{\beta} & 0 \end{pmatrix}.$$

The trivial solution has eigenvalues $\alpha$ and $-\gamma$, indicating a saddle point [3, p.130]. This is good, because saddle points are unstable, meaning the model does not predict spiralling uncontrollably towards extinction. The other matrix has eigenvalues $+i \sqrt{\alpha \gamma}$ and $- i \sqrt{\alpha \gamma}$, which indicates periodic trigonometric-esque solutions.

These curves represent a typical periodic steady state solution to the Lotka-Volterra equations

The model elegantly captures the core ideas: prey populations grow naturally but are eaten by predators, and predator populations decline without food but grow when they eat prey. That said, the model is limited, as it never predicts extinction for nonzero initial populations. A better model would include a notion of self competition in the individual populations, perhaps stemming from a fixed supply of food. Such a model would be self-limiting. A better model,

$$ \dot x_1 = \alpha x_1 (1 - x_1 / \eta_{1}) - \beta x_1 x_2,$$ $$\dot x_2 = \gamma x_2 (1 - x_2 / \eta_{2}) - \delta x_1 x_2 $$

introduces the notion of independent growth limitation on the individual populations $x_1$ and $x_2$ to the system through the Logistic model. The advantages is this allows one species to “win”, in addition to cases where both populations settle on fixed values. But there are limitations. The periodic/cyclic nature of the original Lotka-Volterra model is lost.

Adding spatial dependence to the Lotka-Volterra model

The main ingredient missing from these models is the absence of spatial dependence in the model. Further, there is reason to be concerned with the mean field assumption that we adopted before, which states that all agents interact with the average effect of all the others agents. In reality, agents exist in 2D space and can only interact with their immediate spatial neighbours, and the model should reflect this.

One option is to add in our our Fick’s law inspired diffusion terms, and so we define the spatial Lotka-Volterra equations

$$\frac{\partial x_1}{\partial t} = \alpha x_1 - \beta x_1 x_2 + D_1 \nabla^2 x_1,$$

$$\frac{\partial x_2}{\partial t} = \delta x_1 x_2 - \gamma x_2 + D_2 \nabla^2 x_2.$$

This set of differential equations can be modelled computationally on a randomised initial population to show the progression of the reaction-diffusion system. We can use an Euler-esque method for executing this progression, by multiplying our differential equations by a timestep $dt$ to get a $dX$, and adding that to $X$ each iterative timestep $t$.

steps = 2000
for t in range(steps):
 # Lotka-Volterra equations with added diffusion Laplacians
 laplacian_X1 = lapalcian(X1)
 laplacian_X2 = lapalcian(X2)

 dX1 = (alpha * X1 - beta * X1 * X2 + Dx * laplacian_X1) * dt
 dX2 = (-gamma * X2 + delta * X1 * X2 + Dy * laplacian_X2) * dt

 X1 += dX1
 X2 += dX2

Animated, we get our first look at the emergent patterns of spatial ecology. The background is black, and the areas that are more green have a higher density of prey, and the areas that are more red have a higher density of predators. Both populations are initialised with a random uniform distribution. The population densities are continuous variables, and do not represent discrete population sizes, as much the general spatial flux of species densities.

The patterns of dots and stripes that form and then disappear are reminiscent of Turing patterns, and in the appendix, we will consider whether these are Turing patterns, how they form, and why they also seem to disperse over time.

Agent based models for two species population dynamics.

Spatial ecology can also be modelled computationally using agent-based models, and the benefit of this is that it did not assume the truth of a set of universal differential equations. As such, agent based models offer a completely different method for simulating systems of interacting particles by operating exclusively at the local level, with no macroscopic information shared to individual agents. Diffusion occurs via steps in random directions. Reactions occur based on only local information, such as if two interacting agents of the same species at the same place have enough energy to procreate. While these models are much closer approximations to real life, they still lack the macroscopic dynamics of flocking, herding, fleeing, or hunting. As such, it may make sense to imagine these agents as small groups of microorganisms, diffusing due to the random flow of Brownian motion, rather than intelligent autonomous animals.

Program overview

The iterative time steps of my program is as simple as

TheGrid = ReactionDiffusionLibrary.Grid(gridxsize, gridysize)
Prey = ReactionDiffusionLibrary.Species("Prey", Prey_E0, Prey_EP, Prey_N, TheGrid)
Pred = ReactionDiffusionLibrary.Species("Predator", Pred_E0, Pred_EP, Pred_N, TheGrid)

steps = 2000
for step in range(steps):
 Prey.Move()
 Pred.Move()
 TheGrid.Interact(Prey, Pred)

where the Prey, Pred, and TheGrid instances are C# classes incorporated into the python script using pythonnet clr. The two main mechanisms of reaction-diffusion are contained in the Move method and the Interact method. Here is Move,

public void Move()
 {
 List<double[]> SpeciesCoords = new List<double[]>(AgentsList.Count);
 Random random = new Random();

 for (int i = 0; i < AgentsList.Count; i++)
 {
 // Move this agent to a random nearest neighbour square
 var agent = AgentsList[i];
 int dirIndex = random.Next(0, 5);
 agent.Move(Grid.Directions[dirIndex]);
 SpeciesCoords.Add(new double[] {agent.X, agent.Y});

 // Keep track of what agents are in what squares
 char yOrDCondition = PredOrPrey == "Prey" ? 'y' : 'd';
 agent.AgentId = $"{yOrDCondition}{i}";
 agent.AgentIndex = i;

 Grid.AgentsInGrid[(int)agent.X][(int)agent.Y].Add(agent.AgentId);
 }
 this.Grid.WriteAgentsInGridToFile();
 }

Each iteration, the coordinates of all the agents in the species are rewritten based on their movement in a random direction, and this also helped with dealing with the non constant number of agents due to births and deaths.

The one alteration I made to the program to make it not a strictly agent based model is the Dying boolean, which turns on when a species total number of agents dips below 1000 gridwide. This allowed agents to procreate on their own whenever the species has sufficiently low numbers rather than needing to randomly stumble upon another of their species on the grid. To justify this, we can imagine each dot as a small collection of microorganisms rather than an individual.

The Interact method iterates through each square in the grid, collecting pairs of agents. If two agents of the same species are in the same grid square, and they have above the minimum procreation energy, they will breed and produce another agent at that square. If a predator is in the same square as a prey, it will feed on the prey, killing it and absorbing its energy.

 public void Interact(Species PreySpeciesObj, Species PredSpeciesObj)
 {
 int ySize = GridYSize;
 int xSize = GridXSize;
 // Check each Grid cell
 for (int y_i = 0; y_i < ySize; y_i++)
 {
 for (int x_i = 0; x_i < xSize; x_i++)
 {
 List<string> agents = AgentsInGrid[y_i][x_i];
 if (agents.Count == 0) continue;

 List<int> preys = new List<int>();
 List<int> preds = new List<int>();

 // Check each agent in a Grid cell, and sort into species
 foreach (string agentStr in agents)
 {
 if (agentStr.StartsWith('y')) preys.Add(int.Parse(agentStr.Substring(1)));
 if (agentStr.StartsWith('d')) preds.Add(int.Parse(agentStr.Substring(1)));
 }

 // Predators breed and feed on prey
 Procreate(preds, PredSpeciesObj);
 for (int i = 0; i < preds.Count; i++)
 {
 if (i < preys.Count)
 {
 Agent predator = PredSpeciesObj.AgentsList[preds[i]];
 Agent food = PreySpeciesObj.AgentsList[preys[i]];
 predator.Energy += food.Energy;
 food.AddToDeathList();
 }
 }

 // Preys breed
 Procreate(preys, PreySpeciesObj);
 }
 }
 // At the end of the turn, make babies into adults and handle deaths
 PreySpeciesObj.NewDay();
 PredSpeciesObj.NewDay();
 ClearAgentsInGrid();
 }

Design considerations

Energy

Prey generate energy stochastically but passively, and will do so as long as they are not eaten. Predators can only generate energy by feeding on prey. Energy on average is gained rather than lost throughout the simulation. However, existing in the same gridsquare as another agent of your own species incurs an energy cost directly proportional to the number of agents there. For example, if $40$ agents occupy the same gridspace, they will each suffer an energy cost of $40$ energy that iteration. This is meant to replicate an energy cost due to limited food supply at any one area, and makes the populations display self-limiting behaviour.

Procreation

To procreate, agents must bump into each other on the same gridspace, and if both parents have above a minium energetic threshold, they will procreate and suffer an energy cost. There is no limitation on procreating with your own offspring. To avoid extinction in my simulations

Death

Agents can die in two ways, running out of energy or being eaten. In this model, only prey may be eaten. While prey passively generate energy, they may still die from running out of energy if they live too long in areas with a high density of prey. Predators more usually die from running out of energy, by not eating enough prey.

Learnings from animating the agent based model

The results are different from the Lotka-Volterra animation. Instead of smoothly fading between areas of high prey density and high predator density, we see vast areas of blank space. But still, some similar emergent properties can be seen, in particular the collections of agents evolving in wavefronts.

Explosive pockets of mass generations of new agents can be seen, but why do they occur? The spatial and energetic restrictions on when procreation can occur impede local procreation rates, which can lead to huge gains in average agent energy. If enough energy is stored up and a species begins to bump into itself, a chain reaction of procreation is set-off. The limiting factor of this is usually the rate of diffusion, which leads to most agents in the chain reaction dying due to the energy cost of occupying the same gridsquare as many others before they can move away to safer areas.

Expanding the size of the grid, the emergence of wavefronts becomes more obvious. We can begin to see expanding florets of prey, chased down by a secondary wave of predators. These expanding wavefronts sometimes stochastically form similar concentric strucures to the patterns formed by the Belousov-Zhabotinsky chemical reaction.

Each time the model is run, different patterns are generated. A glance at the population curves below for a selection of runs shows that in the early stages the curves appear self-similar. The long term behaviour is more unpredictable, and includes large spikes when the species has amassed a high energy and begins to collide into itself, provoking an uncontrolled chain reaction of procreation.

An overview of the population curves for nine runs of the agent based model. As throughout, the green line represents the prey population and red represents the predator population

Conclusion

Two models were examined in this article. Feel free to check out my github..

The agent based model was the most enjoyable, the most challenging, and took the most time. For performance reasons, the backend had to be rewritten in C#, but this was a fun challenge. The main interface and graphing remained in python. This model could easily be used for further research or study. For instance, agent based models have recently become hugely important in predicting the mature structure of bacterial biofilms, particularly to mirror how biofilms react to stimuli in lab scenarios.

Appendix

Derivation of how Turing patterns form and why they disperse in the spatial Lotka Volterra equations

Patterns seem to form in the spatial Lotka reaction-diffusion simulation for predators and prey in a 2D grid. These patterns seem similar to Turing patterns commonly seen in nature. The equations I have called the spatial Lotka-Volterra equations were formed by adding in diffusive Laplacian terms to the classic Lotka-Volterra equations.

$$\frac{\partial x_1}{\partial t} = \alpha x_1 - \beta x_1 x_2 + D_1 \nabla^2 x_1 $$

$$\frac{\partial x_2}{\partial t} = \delta x_1 x_2 - \gamma x_2 + D_2 \nabla^2 x_2 $$

To prove that the given system of equations produces Turing patterns [4], we need to perform a linear stability analysis around a homogeneous steady state solution, and look for conditions under which perturbations grow in a spatially inhomogeneous way. We will follow the method here [5]. First, let’s find the homogeneous steady state $(x_1^{*}, x_2^{*})$ for a spatially uniform state by setting

$$ \frac{\partial x_1}{\partial t} = \frac{\partial x_2}{\partial t} = 0, \nabla^2 x_1 = \nabla^2 x_2 = 0.$$

Subbing this into the above spatial Lotka Volterras, we get $$ 0 = \alpha x_1^* - \beta x_1^* x_2^*,$$ $$0 = \delta x_1^* x_2^* - \gamma x_2^*. $$

Solving for $ x_1^{*} $ and $ x_2^{*} $, $ x_2^{*} = \frac{\alpha}{\beta}$ and $ x_1^* = \frac{\gamma}{\delta x_2^{*}} = \frac{\gamma \beta}{\alpha \delta}$

Next, we linearize the system around the steady state by letting $ x_1 = x_1^* + u $ and $ x_2 = x_2^* + v $, where $ u $ and $ v $ are small perturbations. We then substitute these into our system and keep only linear terms in $ u $ and $ v $. After linearization, the system becomes: $$ \partial_t u = \alpha u - \beta (x_1^* v + x_2^* u) + D_1 \nabla^2 u$$ $$ \partial_t v = \delta (x_1^* v + x_2^* u) - \gamma v + D_2 \nabla^2 v $$

Substituting the steady-state values and simplifying we get $$ \partial_t u = (\alpha - \beta x_2^*) u - \beta x_1^* v + D_1 \nabla^2 uu = - \frac{\beta \gamma}{\delta} v + D_1 \nabla^2 u $$ $$\partial_t v = \delta x_2^* u + (\delta x_1^* - \gamma) v + D_2 \nabla^2 v = \delta x_2^* u + (\delta x_1^* - \gamma) v + D_2 \nabla^2 .$$

So, we have our fully linearized system which we can write in matrix form as:

$$ \begin{pmatrix} \partial_t u, \\ \partial_t v \end{pmatrix} =\begin{pmatrix} D_1 \nabla^2 & -\frac{\beta \gamma}{\delta} \\ \frac{\alpha \delta}{\beta} & D_2 \nabla^2 - \gamma \end{pmatrix} \begin{pmatrix} u \\ v \end{pmatrix} $$

Now we look for solutions in the form of $ e^{\lambda t + ik\cdot r} $, where $ \lambda $ is the growth rate of the perturbation, $ k $ is the wave number, and $ r $ is the position vector.

We substitute this form into the linearized system and solve for $ \lambda $. The dispersion relation is obtained by setting the determinant of the matrix minus $ \lambda I $ to zero:

$$ \det \begin{pmatrix} D_1 k^2 - \lambda & -\frac{\beta \gamma}{\delta} \\ \frac{\alpha \delta}{\beta} & D_2 k^2 - \gamma - \lambda \end{pmatrix} = 0 $$

Solving the characteristic equation, we get: $$ (D_1 k^2 - \lambda)(D_2 k^2 - \gamma - \lambda) - \left(-\frac{\beta \gamma}{\delta}\right)\left(\frac{\alpha \delta}{\beta}\right) = 0 $$

Expanding and simplifying gives us a quadratic equation in $\lambda $, $$ \lambda^2 + \lambda(D_1 k^2 + D_2 k^2 - \gamma) + h(k^2) = 0. $$ where $ h(k^2) = D_1 D_2 k^4 - \gamma D_1 k^2 - \alpha \gamma$.

For Turing instability, we need [6]:

the homogeneous system to be stable without diffusion, which we proved in the linear stability analysis in the previous section,
and the diffusion terms to destabilize the homogeneous steady state, meaning the real part of $ \lambda $ must be positive for some range of $k^2$.

And so, one way to find evidence of Turing instability would be to look for any sign change in the real part of $\lambda (k^2)$, which would indicate there exists a range of wavenumbers for which patterns will emerge.

Instead, we will reason that if we have $h(k^2) < 0$ for some $k^2$, it must be true that $ \lambda(k^2)$ is positive for that same interval of $k^2$, because of the positive $\lambda$ coefficients in $\lambda^2 - \lambda(D_1 k^2 + D_2 k^2 - \gamma) + h(k^2) = 0.$

Next, performing a quick change of variables $z = k^2$,

$$\frac{dh(z=k^2)}{dz}=2 D_1 D_2 z - \gamma D_1,$$

we aim to find the stationary point so set $\frac{h(z^{*})}{dz}$ to zero, and prove its minimum is negative. We get $z^{*}=\frac{\gamma}{2 D_2}$, and plugging this into $h(z)$ gives a value for $h_{min}$ (since $h(z)$ is clearly concave),

$$h_{min}(z^{*}=\frac{\gamma}{2 D_2}) = \frac{D_1 D_2 \gamma^2}{4 D_2} - \frac{D_1 \gamma^2}{2 D_2} - \alpha \gamma = - \frac{D_1 \gamma^2}{D_2} - \alpha \gamma$$

Since $ \alpha, \gamma, D_1, D_2 $ are necessarily $>0$, we have proved that for $h(k^2)$ is necessarily negative for some range of $k^2$, and that $\lambda $ is positive for that same range of $k^2$. Why this leads to Turing instabilities is explained in more detail here [6].

References

Learning to walk with Novelty Search

Mon, 30 Oct 2023 00:00:00 +0000

NeuroEvolution of Augmenting Topologies

NeuroEvolution of Augmenting Topologies (NEAT) is an evolutionary algorithm for generating intelligent behaviour in one layer neural networks. NEAT works by evaluating neural networks against a fitness function (performance metric) and breeding those networks that perform best. NEAT does not require the backpropogation algorithm or labelled examples to learn, therefore allowing it to be used in much broader settings than deep learning methods. Invented in 2002 by Stanley and Miikkulainen¹ the state of the art adaptations to NEAT have applications in game design², reinforcement learning and control problems³. This article will explore experiments regarding bipedal walking and the NEAT algorithm including its Novelty Search adaptation.

Skip to the experiments

The Algorithm

As NEAT is an evolutionary algorithm which obeys the high level process for optimisation shown in the diagram below.

High Level Evolutionary Process source Baeldung

A more detailed description of each step in the evolution process:

Initialisation: The first step is to generate a pool of candidate solutions. This can be done by randomly generating neural networks or by loading a set of pre-existing networks.
Selection: Next, each candidate solution is assessed on the problem. The best performing solutions are then selected to be parents for the next generation.
Crossover: Once the parents have been selected, they are combined to create new solutions. This is done by randomly selecting genes from each parent and combining them to create a new offspring.
Mutation: After crossover, the offspring are randomly mutated. This is done by making small changes to the genes of the offspring. Mutation helps to introduce new genetic diversity into the population, which can help to improve the performance of the algorithm.
Solution: The NEAT process continues to generate new solutions and assess them on the problem until a solution of sufficient quality is found. Once a satisfactory solution is found, the evolutionary process is stopped and the solution is returned⁴.

Population diagram source Paul Pauls

Genetic Encoding

As stated as a key problem in Stanley’s paper, a major challenge of NEAT is the genetic representation of the neural networks. It is important to ensure that dissimilar network toplogies can crossover in a meaningful way; new networks are given time to innovate (stopping them being removed from the population prematurely) and finally, how to incorporate Occam’s Razor into the problem without a contrived fitness function. These problems are solved in NEAT with the following solutions:

Utilising historical markings in the genetic encoding to identify topologies with the same origin,
Separate newly innovated groups into new species,
Instantiate the network with a minimal structure and only grow when necessary.

As such, the genetic encoding(genome) for an individual in the NEAT algorithm is as follows:

List of node genes defining its activation function and whether it is an input,output or hidden node,
List of connection genes where each connection gene specifies the in-node, the out-node, the weight of the connection, an enabled bit and an innovation number (used for crossover).

Example Genome/Phenotype encoding source K Stanley

Mutation

Mutation in NEAT is conducted with the following rules (for each rule it is also possible for nothing to happen to a connection, weight or node):

Connection weights are perturbed by a random amount,
A connection is added between two previously unconnected nodes,
A node is added where a connection was. The old connection is disabled and two new connections are added.

Crossover

Crossover in NEAT is performed with historical markings, this means whenever a new gene appears (node/connection) a global ‘innovation’ number is incremented and assigned to that gene. This means that genes with matching innovation numbers can be lined up. Genes that do not match are inherited from the more fit parent or, if they are equally fit, chosen randomly. This method is an incredibly low complexity way to align structure in the parents that is suitable for crossover. An example of crossover is provided below:

Matching up genomes for different network topologies using innovation numbers source K Stanley

Protecting Innovation

Finally, in order to combat the aforementioned problem of the innovations being wiped out before they have time to be optimised we utilise speciation. A distance measure must therefore be created in order to allocated species to individuals in the population. The distance measure ($\delta$) is then defined as a linear combination of excess (E, copies of the same gene) and disjoint(D, genes with not match in the corresponding parent) genes plus the average weight difference of matching genes ($\bar{W}$).

$$ \delta = \frac{c_1 E}{N} + \frac{c_2 D}{N} + c_3\bar{W} $$

Where $c_i$ is the importance of each factor and N is the number of genes. A compatibility threshold $\delta_t$ is then defined and each individual is placed into the first species for which its distance is less than the threshold to avoid overlap. Finally, in order to reduce one species dominating the population, NEAT utilises explicit fitness sharing mitigate this. Explicit fitness sharing works by taking every individuals fitness and dividing it by the number of individuals in the species, so we are saying a large species has a better chance to do better and this should be averaged out. Species then grow or shrink depending on their performance relative to the population average. $$ N\\_{j}^{'} = \frac{\sum\\_{i=1}^{N_j}f\\_{ij}}{f} $$ Where:

$N_{j}$ and $N^{’}_{j}$ are the old updated species size for species $j$,
$f_{ij}$ is the species size divided individual fitness of individual $i$,
$f$ is the mean adjusted fitness of the whole population.

Strengths and Weaknesses

NEAT based methods are particularly suited to problems where:

The problem is dynamic and therefore the neural network needs to be able to fundamentally change throughout its lifetime,
There is little or no knowledge of the optimal network structure for the problem,
The gradient of the objective function is degenerate or there are multiple objectives to be simultaneously optimised.

However NEAT performs poorly however in situations where:

The problem is too expensive to evaluate. NEAT requires evaluating each candidate solution on the problem. If the problem is too expensive to evaluate, then NEAT may not be practical,
The problem is inexpressible through a one layer neural network, however the adapation to NEAT, HYPER-NEAT⁵ addresses this issue,
The default encoding scheme in NEAT is not suitable for the network topology that is desired to be developed,
Limited computational resources because Neuroevolution is incredibly computationally expensive, requiring significant processing power and time, especially for complex tasks.

In particular this article will discuss the fitness and novelty based approaches to NEAT and their accompanying performances on the OPENAI gym bipedal walker environment⁶.

Learning to Walk

The first experiment I attempted was the OPENAI Gym Bipedal Walker environment, the aim was to utilise the NEAT algorithm and learn walking behaviour within a 24 hour training window or by completing the environment (whichever came first). The walker environment is deemed ‘completed’ if the model can complete 300 units of distance in 1600 time steps and the reward function gives an agent a reward for every unit of distance it travels and penalises an agent for the following:

The use of any motor joints incurs a small energy cost,
The walker falling over incurs a -100 score penalty.

Implicitly, this means the environment is telling us a ‘good’ model is one that travels lots of distance, with minimal energy expenditure and does not fall over. This is a good starting heuristic for ‘walking’ behaviour, but as will be discussed later, embedding more knowledge into the fitness function improves performance. This leads to my earliest implementation of the function that evaluates a genomes performance as follows:

def eval_genome(genome, config):
 # setup the model and the environment
 env = gym.make("BipedalWalker-v3", hardcore=False,)
 net = neat.nn.FeedForwardNetwork.create(genome,config)
 fitnesses = []

 # run the model in the environment 'runs_per_net' times
 for runs in range(runs_per_net):
 observation, info = env.reset()

 # Run the simulation until environment termination.
 fitness = 0.0
 while True:
 action = net.activate(observation)
 observation, reward, terminated, truncated, info = env.step(action)
 fitness += reward

 if terminated or truncated:
 break

 fitnesses.append(fitness)

 # The genome's fitness is its mean performance across all runs.
 return statistics.mean(fitnesses)

However, after running NEAT in this setup for various hyperparameter settings and a total of 40+ hours of training time, the model would get consistently stuck in a lunging behaviour that gained 6-8 units of distance and then stood still, as demonstrated in the clip below. This behaviour meant that because the walker did not fall over another, more promising, model that travels less than 106 units but falls over is considered worse (once the -100 penalty for falling over is applied) despite have more promising emergent walking behaviour. This problem meant the fitness function is not incentivising walking behaviour well and had to be adapted to avoid this local minima.

Rolling Window Approach

In order to fix the aforementioned problem with the fitness function and a long ignored problem with slow training times I decided to implement a further constraint on the model to address both these issues. These issues were solved by what I called the ‘rolling window’ approach. This approach gives the model a distance it must attain in a certain amount of time, penalising and ending the simulation run if the model does not attain these criterion. The process is outlined in Figure 5 below.

Flow Chart of Rolling Window Approach

The rolling window approach improves the training process in the following ways:

The adaptation to the loss function terminates any model that doesnt make it through the next window thereby improving performance (this caused time per generation to go from 20s to 1.2s in my training).
The adapation penalises any model that doesnt make it through a window(see ‘fitness = fitness - 50 + c’) thereby incentivising models to move $df$ in $dt$ time and hence alleviating the lunging problem.
‘-50’ is selected as the penalty for failing the first window, this is half the penalty for falling over but still sizeable enough to incentivise the walking behaviour desired.
Finally, in order to ensure that the penalty for failing at window $x$ is less than window $x+1$, $c\,$ is incremented after every window to make sure the penalty reduces the more windows the model makes it through. After picking 320 and 60 as good candidates for $dt$ and $df$ (when the fifth window is complete the model should have traversed 300 units) the model should complete the environment after making it through five windows.

The code for this approach is as follows:

def eval_genome(genome, config):
 # setup the model and the environment
 env = gym.make("BipedalWalker-v3", hardcore=False,)
 net = neat.nn.FeedForwardNetwork.create(genome,config)
 fitnesses = []

 # run the model in the environment 'runs_per_net' times
 for runs in range(runs_per_net):
 observation, info = env.reset()

 # Run the simulation until environment termination or window failure.
 fitness = 0.0
 time,dt,df,c = 0,0,0,0
 while True:
 time += 1
 dt += 1 % 320

 action = net.activate(observation)
 observation, reward, terminated, truncated, info = env.step(action)
 df = (df + reward)
 fitness += reward

 if terminated or truncated:
 break
 # if the model does not make it through a window
 if dt == 319 and df <= 59:
 fitness =fitness - 50 + c
 df=0
 break
 # if the model makes it through a window
 elif dt == 319 and df > 59:
 c += 6
 df=0

 fitnesses.append(fitness)

 # The genome's fitness is its mean performance across all runs.
 return statistics.mean(fitnesses)

Results

After running the NEAT algorithm with the rolling window setup, convergent walking or ‘shuffling’ behaviour was achieved after 35000 generations and stagnated until 40000 generations. The training process and final model are displayed in the videos below. Although the model is now moving, albeit inefficiently, the greedy nature of the fitness function means the environment was not completed (the maximum distance attained was 154 units of distance). In order to motivate NEAT to generate more complex walking behaviour, that will not necessarily show dividends straight away, a new fitness function should be considered.

Novelty Search

After listening to the TWiML podcast ⁷ with Kenneth Stanley (the co-author of the original NEAT paper) I discovered novelty search, a supposedly better parameterisation of the fitness function to generate complex behaviour. This new search algorithm is the basis of the second walker experiment. The idea of Novelty Search is to define a fit model as a ’novel’ model, this means rewarding solutions that are sparse in the environment space. For example, in a maze (the scenario the paper⁸ uses) a ’novel’ solution is one which reaches a new area of the maze rather than minimising something like the distance to the end (which dramatically increases the chance of getting stuck in a local minima).

Example of Novelty Search outperforming euclidean distance for maze solving source: Lehman and Stanley

In the example above from Lehman and Stanley’s paper on Novelty Search⁸ we see that novelty search dramatically outperforms a fitness based approach to maze solving. This setup is applied to the walker environment to see the performance of novelty search. The setup for the algorithm is as follows:

Novelty is defined as reaching a novel forward distance in the environment i.e the model has walked forwards to a spot no model has reached before,
A model is not deemed novel if it is within 1 unit of distance to the nearest three models,
An archive of previous solutions is initially filled with 200 models and then added to whenever a new model is deemed novel,
The environment is deemed completed if a model completes the 300 units in 1600 time steps.

Therefore, the code for implementing the novelty search is as follows:

def eval_genome(genome, config):
 global archive
 env = gym.make("BipedalWalker-v3", hardcore=False,)
 net = neat.nn.FeedForwardNetwork.create(genome,config)
 fitnesses = []

 for runs in range(runs_per_net):
 novelty = archive.copy()
 observation, info = env.reset()

 fitness = 0.0
 dt,df = 0,0
 while True:
 dt += 1 % 400
 # act on environment
 action = net.activate(observation)
 observation, reward, terminated, truncated, info = env.step(action)
 df = (df + reward)
 fitness += reward
 # early termination heuristic
 if dt == 399 and df < 34:
 df = 0
 terminated = True

 if terminated or truncated:
 if len(archive) < 200:
 archive.append(fitness)
 novel = 0
 else:
 # determine the models novelty
 novelty = [abs(fitness - x) for x in novelty]
 novelty.sort()
 novel = statistics.mean(novelty[:3])
 # if the model is novel add it to the archive
 if novel > 1 and fitness > 0:
 archive.append(fitness)
 # if the environment is completed end simultation
 if fitness >=300:
 break
 fitness = novel
 # heuristic to state a non-novel positive model is better
 # than one that falls over
 elif fitness > 0:
 fitness = -99
 else:
 fitness = -100
 break

 fitnesses.append(fitness)

 # The genome's fitness is its mean performance across all runs.
 return statistics.mean(fitnesses)

Results

After running NEAT on the new Novelty based fitness function a solution is found to the problem after just 4284 generations (shown in the video below). Although the computational cost is slightly higher than the previous rolling windows approach ($\approx 5s\,$ vs $\approx 1s\,$), Novelty search took almost 10 times less generations to converge to a solution that travels double the distance of the rolling window approach. Thefore, Novelty Search dramatically outperformed the basic NEAT implementation Below, it can be seen that the final model’s behaviour is considerably more complex, utilising more motor joints meaning the algorithm has incentivised learning more complex but meaningful behaviour as desired. Furthermore, in comparison to the distance based approach previously we can see a much more diverse set of optimal architectures being explored by the algorithm.

Conclusion

In conclusion I have learnd a lot about NeuroEvolution in the process of writing this article. In particular, I am glad that my motivation to learn about gradient free learning has led to me discovering these algorithms and I am pleased at the results I have gained. I have learnt about the basic NEAT algorithm and how it can be used in reinforcement learning situations alongside its Novelty Search adaptation. If i was to continue learning more in the area I would look into HYPERNEAT and the more recent developments in the NeuroEvolution space.

Kenneth O. Stanley and Risto Miikkulainen , Efficient Evolution of Neural Network Topologies ↩︎
Real-Time Neuroevolution in the NERO Video Game ↩︎
Kenneth O. Stanley and Risto Miikkulainen ,Efficient Reinforcement Learning through Evolving Neural Network Topologies ↩︎
Paul Pauls, A Primer on the Fundamental Concepts of Neuroevolution ↩︎
Kenneth O. Stanley, David D’ambrosio, Jason Gauci A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks ↩︎
OPEN AI gym Bipedal Walker Environment ↩︎
NeuroEvolution: Evolving Novel Neural Netowrk Architectures with Kenneth Stanley - TWiML Talk 94 ↩︎
Joel Lehman and Kenneth O. Stanley, Abandoning Objectives: Evolution through the Search for Novelty Alone ↩︎ ↩︎

PINNs, Not the Spiky Kind.

Fri, 28 Jul 2023 00:00:00 +0000

Physics Informed Neural Networks (PINNs)

Physics-informed neural networks (PINNs) are a deep learning framework for solving partial differential equations (PDEs). PINNs work by adding a specified PDE to the loss function of a neural network and utilising automatic differentiation to train the network. This allows PINNs to solve PDEs that are difficult or impossible to solve with traditional numerical methods, such as finite element methods (FEM). Invented in 2017 by M. Raisse et al. in two papers ¹ for the forward/inverse problems, which are defined as follows:

Foward Problem: The underlying equation and the boundary conditions are known, and the goal is to find the solution to the equation.
Inverse Problem: The underlying equation and the boundary conditions are known, but the values of some of the parameters in the equation are unknown. The goal is to find the values of these parameters and the solution, given experimental data.

In both forward and inverse problems, PINNs are trained on a dataset of known solutions to the underlying equation. This dataset can be generated using numerical methods, such as the finite element method, finite difference method or from physical experiments. PINNs are particularly well-suited for solving problems where:

The local consistency of FEM is broken like in integro-differential equations,
High-dimensional input domain settings, in which classical techniques (such as FEM) are prohibitively expensive²,
Parameter inference is occuring simultaneously (inverse problems).

Furthermore, they can be trained to generate solutions for multiple physical parameter values, by adding them as outputs from the model, e.g., multiple kinematic viscosity values in the Navier-Stokes equations. The PINN can also be probed for any continuous point in time/space unlike FEMs. In these restricted use cases, PINNs achieve performance on par or better than FEMs³. However PINNs are currently perform poorly at solving stiff equations, where stiffness is the difficulty to solve numerically, as ‘Neural Networks are biased in their frequency domain to drop off lower frequency terms which is what causes the issues with stiffness’³ and in the vast majority of cases PDEs do not satisfy the conditions for which FEMs are considerably faster or more accurate. This means that in industry the most popular use case of PINNs is an out the box model that has been trained with standard boundary conditions, which allows the customer to get useful solutions out.

This article explores PINNs by generating a solution for the Sprott attractor (forward problem) and the solution to an inverse problem involving the Navier-Stokes equations. These problems are chosen to benefit a PINN as the Sprott attractor exhbits sensitivity to initial conditions which make it hard for the FEM to match. Furthermore inverse problems, as mentioned before, are faster for PINNs as they can learn the parameters while training unlike FEMs.

Deep Neural Networks

In PINNS, the neural network (NN) is approximating the solution to the PDE. The PDE is defined on the domain $\Omega \subset \mathbb{R}^d$ with a solution $u(\mathbf{x})$ approximated by the neural network $f$ parameterised by $\lambda$. The input vector is $\mathbf{x}=(x_{1},…,x_{d})^T$. $$ f \left( \mathbf{x};\frac{\partial u}{\partial x\\_{1}},...,\frac{\partial u}{\partial x\\_{d}};\frac{\partial^2 u}{\partial x\\_1^2},...,\frac{\partial^2 u}{\partial x\\_1\partial x\\_d},...;\lambda \right). $$

Alongside suitable boundary conditions $\mathcal{B}(u,\mathbf{x})=0 \text{ on } \delta \Omega$. Although many NN architectures exist, the multi-layer perceptron is suitable for solving most PDEs and as such for the remainder of the article the architecture will be assumed to be a fully connected feed-forward network.

Adapted Loss Function

It was stated in the first paragraph that PINNs operate by $\textcolor{red}{\text{incorporating the PDE into the loss function}}$. So, in the usual manner, the model, loss and training dataset are denoted as $f$, $V$ and $\mathcal{T}$ respectively. Then $\mathbf{\theta}$ is defined as the weights and biases in the NN and $\mathcal{T}=\mathcal{T}_f \cup \mathcal{T}_b$ are two training sets where $\mathcal{T}_f \subset \Omega$ are the points in the domain and $\mathcal{T}_b \subset \Omega$ are the problem boundary conditions. Then the loss function can be setup as $$ \ell(\mathcal{T},f(x;\mathbf{\theta}),R) = \omega\\_f \textcolor{red}{R(f(\mathbf{x};\mathbf{\theta}),\mathcal{T}\\_f)} + \omega\\_b V(f(\mathbf{x};\mathbf{\theta}),\mathcal{T}\\_b) $$ where, for PINNs, $$ \textcolor{red}{R(f(\mathbf{x};\mathbf{\theta}),\mathcal{T}\\_f)}=\frac{1}{|\mathcal{T}\\_f|} \underset{x \in \mathcal{T}_f}{\sum} \left\lVert f(\mathbf{x};\frac{\partial u}{\partial x\\_{1}},...,\frac{\partial u}{\partial x\\_{d}};\frac{\partial^2 u}{\partial x\\_1^2},...,\frac{\partial^2 u}{\partial x\\_1\partial x\\_d},...;\lambda) \right\rVert\\_2^2 $$ $$ V(f(\mathbf{x};\mathbf{\theta}),\mathcal{T}\\_b)=\frac{1}{|\mathcal{T}\\_b|} \underset{x\\_i \in \mathcal{T}_b}{\sum} \lVert f(x\\_i;\mathbf{\theta}) - \mathcal{B}(x\\_i) \rVert\\_2^2 $$ There are two terms for the loss, induced on the boundary conditions and on the domain, that are simultaneously backpropogated to train the model. Observed or experimental data can be added into the boundary condition loss set as $(x_i,\mathcal{B}(x_i))$ pairs. Building upon the classic misty hill analogy for gradient descent⁴, the $\textcolor{red}{R(f(\mathbf{x};\mathbf{\theta}),\mathcal{T}_f)}$ (the loss induced by the governing PDE) provides a set of steps down the hill that act as a good heuristic to improve the chances of getting to the bottom (global minima and solution to the PDE). For example if the aim is to approximate a solution to the diffusion equation $\frac{\partial^2 f}{\partial x^2}= \frac{1}{\kappa} \frac{\partial f}{\partial t}$ then the corresponding PDE loss would be $\underset{f}{\text{argmin}}\,\left( \frac{\partial^2 f}{\partial x^2}- \frac{1}{\kappa} \frac{\partial f}{\partial t}\right)$ as the second expression is the minimisation task that corresponds to the NN obeying the PDE.

Gradient descent as steps on a misty hill

Automatic Differentiation

During the backpropogation process the traditional aim is to minimise the derivatives $\frac{\partial \ell(f)}{\partial x_1},…,\frac{\partial \ell(f) }{\partial x_d}$ ,i.e., the loss of the model with respect to the inputs. In the process of computing these derivatives we also end up computing the loss with respect to each model parameter and can then utilise a weight update algorithm such as ADAM⁵, L-BFGS⁶ or the traditional $w_{t+1}=w_t + \eta \frac{\partial \ell(f)}{\partial w}$. In PINNS we now have the boundary conditions of the PDE to enforce on the NN through the addition of the PDE error to the loss function. Furthermore, NNs only require one pass of the data through the approximating function however FEM require the computation of both $ \, f(x_1,…x_i,…,x_d)$ and $f(x_1,…x_i+ \Delta x_i,…,x_d) $ for all $i$. For large $\text{dim}(\mathbf{x})$, automatic differentiation is much more efficient.

The PINN

We now formalise the PINN algorithm and NN structure, copying from Lu Lu and DEEPXDE⁷.

Construct a Neural Network with accompanying parameters,
Create the two training datasets $\mathcal{T}_f,\mathcal{T}_b$,
Specify the loss function and weights associated to $\omega_f, \omega_b$,
Train the Neural Network to find find the best parameters that minimise the loss function.

Schematic of a PINN for solving the diffusion equation credit to Lu Lu et al. and DEEPXDE

Forward Problem: Sprott Attractor

The Sprott Attractor⁸ is a chaotic dynamical system proposed in a 2014 paper that exhibits sensitivity to initial conditions and exhibits complex structure. It will be treated as a forward problem and therefore that we know in advance the parameters $\alpha, \beta$ and select $t \in [0,20]$ as the domain to learn on. $$ f(t)=\begin{cases} \frac{dx}{dt} = y +\alpha xy + xz \\ \frac{dy}{dt} = 1 - \beta x^2 + yz \\ \frac{dz}{dt} = x - x^2 - y^2 \end{cases},\quad \alpha=2.07, \beta = 1.79 $$

Data/PDE Setup

The code in this section defines the geometric domain(time domain $(0,20)$), solver settings, and ODE system. The ODE system is a three-dimensional system of ordinary differential equations (ODEs) where $\texttt{x, y and z}$ are the three state variables of the system and $\texttt{o}$ is now the output of the model.

import deepxde as dde
import numpy as np

geom = dde.geometry.TimeDomain(0, 20)
dde.config.set_default_float("float64")
a = 2.07
b = 1.79


def ode_system(t, o):
 # https://www.dynamicmath.xyz/strange-attractors/ Sprott attractor

 x = o[:, 0:1]
 y = o[:, 1:2]
 z = o[:, 2:3]
 dx_t = dde.grad.jacobian(o, t, i=0)
 dy_t = dde.grad.jacobian(o, t, i=1)
 dz_t = dde.grad.jacobian(o, t, i=2)
 return [
 dx_t - (y + (x * a * y) + (x * z)),
 dy_t - 1 + b * x * x - y * z,
 dz_t - X + x * x + y * y,
 ]

Neural Network Setup

The code in this section sets up the neural network for solving the ODE system. The network architecture is 4 hidden layers, each with 20 neurons. The activation function for all layers is $tanh$ (notable for PINNS due its non-zero derivatives mitigating the “dying ReLU” problem⁹). The weights of the neural network are initialized using the Glorot normal initializer.

The code also defines a function called input_transform. This function takes a time $\texttt{t}$ as input and returns a vector of sinusoidal features that will be used as input to the neural network. As we know in advance that the solution contains sinusoidal like behaviour this feature layer will boost learning performance. The code then calls the $\texttt{apply_feature_transform}$ method on the neural network.

The code in this section is used to set up the neural network for solving the ODE system.

n = 10
x_true, y_true, z_true = gen_truedata(n)

t = np.linspace(0, 20, n).reshape(n, 1)
observe_x = dde.icbc.PointSetBC(t, x_true, component=0)
observe_y = dde.icbc.PointSetBC(t, y_true, component=1)
observe_z = dde.icbc.PointSetBC(t, z_true, component=2)
data = dde.data.PDE(geom,ode_system,[observe_x,observe_y,observe_z],40,10,anchors=t,num_test=100)
layer_size = [1] + [20] *4 + [3]
activation = "tanh"
initializer = "Glorot normal"
net = dde.nn.FNN(layer_size, activation, initializer)

def input_transform(t):
 return tf.concat(
 (
 t,
 tf.sin(t),
 tf.sin(2 * t),
 tf.sin(3 * t),
 tf.sin(4 * t),
 tf.sin(5 * t),
 tf.sin(6 * t),
 ),
 axis=1,
 )


net.apply_feature_transform(input_transform)

Train

The code in this section trains the neural network to solve the ODE system. The training process uses the Adam optimizer with a learning rate of 1e-3. The training is run for 200 iterations, and the loss history is monitored every iteration. The model is saved to a file after training is complete for later inference.

model = dde.Model(data, net)
model.compile("adam", lr=1e-3,)
losshistory, train_state =model.train(iterations=200,
 display_every=1,
 disregard_previous_best=False, )
model.save(save_path="./strange_attract-10t/model")

Results

The results show that the Sprott Attractor model can be accurately solved using a neural network with only 50 pieces of data. The model converges after 80 epochs, and the final plot of the three-dimensional attractor matches the true solution with a testing error of approximately 1e-4. This is a significant result, as it shows that PINNS can be used to solve complex ODE systems with relatively little data. The attractor is a three-dimensional object that has a complex and unpredictable structure. The neural network is able to reproduce this structure accurately.

Training animation for Sprott Attractor

Solution Plot

Inverse Problem: Inlet/Outlet Cavity

For the inverse problem, we consider a cavity filled with liquid that has an inlet port in the bottom left and outlet port in the top right, we aim to model the flow of liquid in this cavity.

The problem considered is one created by John Burkardt in ‘Centroidal voronoi tesselation-based reduced-order modelling of complex systems’¹⁰ as the problem considered has an accompanying dataset¹¹ of $(u,v)$ pairs for the domain that can be used for $\mathcal{T}_b$. The spatial domain is defined as $\Omega = (0,1)\times(0,1)$, time domain $(0,T)$ and use the 2d Navier-Stokes Equations with the following boundary conditions: $$ \begin{align} \frac{\partial \mathbf{u}}{\partial t} - \nu \Delta \mathbf{u} + \mathbf{u} \cdot \nabla \mathbf{u} + \nabla p = 0 \quad \text{in} \quad (0,T)\times\Omega \\ \nabla \cdot \mathbf{u} = 0 \quad \text{in} \quad (0,T)\times\Omega \\ \mathbf{u}(0,x)=\mathbf{u}_0(x) \quad \text{in} \quad \Omega \\ \\ \mathbf{u}=5(1-(y-0.1)^2/0.01) \quad \text{on} \quad (0,T)\times\Gamma_i \\ (\mathbf{n}\cdot\nabla)\mathbf{u}=0 \quad \text{on} \quad (0,T)\times\Gamma_o \\ \mathbf{u}=0\quad \text{on} \quad (0,T)\times\Gamma_d \end{align} $$

Spatial Domain $\Omega$

Data/PDE Setup

The code in this section defines the Navier-Stokes equations that will be solved. The equations are defined in terms of the velocity field $(u,v)$, the pressure $p$, and the streamfunction $\psi$.We make the assumption that $u = \psi_y \text{ and } v = −\psi_ x$ for some latent function $\psi(t, x, y)$. Under this assumption, the continuity equation $u_x +v_y=0$ will be automatically satisfied.

The free parameters $[C_1,C_2]$ to be learned, that make the problem inverse, are also defined. The code then defines the spatial and temporal domains for the problem. The spatial domain is a rectangle with dimensions $(0,0) \text{ to } (1,1)$. The temporal domain is the interval [0,6]. The geometry is a combination of the spatial and temporal domains. In the code below we manufacture adherance to the continuity equation $\nabla \cdot \mathbf{u}=0$ by using the stream function¹² $\psi(x,y)$ which only needs continuous first and second partial derivatives (implying the order of differentiation does not matter) and obeys $u(x,y)=\frac{\partial \psi}{\partial y}\, \text{ and }\,v(x,y)=-\frac{\partial \psi}{\partial x}$. Then, the continuity condition holds as follows: $$ \nabla (u,v) \equiv \frac{\partial u}{\partial x} + \frac{\partial v}{\partial y} = \frac{\partial \psi}{\partial y \partial x} - \frac{\partial \psi}{\partial x \partial y} = 0. $$ So the model has four outputs $(\psi,p,u,v)$ where $u,v$ are linked to $\psi$ with the PDE equations u_diff = u_real - u and v_diff = v_real - v. This allows the point set boundary condition to be applied in the next section.

import deepxde as dde
import numpy as np
from create_datasets import create_dataset
# Parameters to be identified
C1 = dde.Variable(0.8)
C2 = dde.Variable(1 / 300)

def Navier_Stokes_Equation(x, y):
 """
 System of PDEs to be minimized: incompressible Navier-Stokes equation in the
 continuum-mechanics based setup.

 """
 psi, p, u_real, v_real = y[:, 0:1], y[:, 1:2], y[:, 2:3], y[:, 3:4]

 p_x = dde.grad.jacobian(p, x, i=0,j=0)
 p_y = dde.grad.jacobian(p, x, i=0,j=1)

 u = dde.grad.jacobian(psi, x, i=0, j=1)
 v = - dde.grad.jacobian(psi, x, i=0, j=0)

 u_x = dde.grad.jacobian(u, x, i=0, j=0)
 u_y = dde.grad.jacobian(u, x, i=0, j=1)
 u_t = dde.grad.jacobian(u, x, i=0, j=2)

 v_x = dde.grad.jacobian(v, x, i=0, j=0)
 v_y = dde.grad.jacobian(v, x, i=0, j=1)
 v_t = dde.grad.jacobian(v, x, i=0, j=2)

 du_xx = dde.grad.hessian(u, x, i=0, j=0)
 du_yy = dde.grad.hessian(u, x, i=1, j=1)
 dv_xx = dde.grad.hessian(v, x, i=0, j=0)
 dv_yy = dde.grad.hessian(v, x, i=1, j=1)

 continuity = u_x + v_y
 x_momentum = u_t + C1 * (u * u_x + v * u_y) + p_x - C2 * (du_xx + du_yy)
 y_momentum = v_t + C1 * (u * v_x + v * v_y) + p_y - C2 * (dv_xx + dv_yy)
 u_diff = u_real - u
 v_diff = v_real - v
 return continuity, x_momentum, y_momentum, u_diff, v_diff

space_domain = dde.geometry.Rectangle([0, 0], [1, 1])
time_domain = dde.geometry.TimeDomain(0, 6)
geomtime = dde.geometry.GeometryXTime(space_domain, time_domain)

Neural Network Setup

The code in this section sets up the neural network for solving the Navier-Stokes equations. The neural architecture has 6 hidden layers, each with 50 neurons. As before, the activation function for all layers is $tanh$ and the weights of the neural network are initialized using the Glorot uniform initializer.

The code also creates a dataset of training data. The dataset is created by sampling the Navier-Stokes equations on a fine grid. The dataset includes 700 domain points, 200 boundary points, 100 initial points for $\mathcal{T}_f$ and 7000 for $\mathcal{T}_b$, and 3000 test points.

[ob_x, ob_y, ob_t, ob_u, ob_v] = create_dataset('data_full.csv',5000)
ob_xyt = np.hstack((ob_x, ob_y, ob_t))
observe_u = dde.icbc.PointSetBC(ob_xyt, ob_u, component=2)
observe_v = dde.icbc.PointSetBC(ob_xyt, ob_v, component=3)

data = dde.data.TimePDE(
 geomtime,
 Navier_Stokes_Equation,
 [observe_u, observe_v],
 num_domain=700,
 num_boundary=200,
 num_initial=100,
 anchors=ob_xyt,
 num_test=3000
)

layer_size = [3] + [50] * 6 + [4]
activation = "tanh"
initializer = "Glorot uniform"
net = dde.nn.FNN(layer_size, activation, initializer)
model = dde.Model(data, net)

Train

The code in this section trains the neural network to solve the ODE system. The training process uses the Adam optimizer with a learning rate of 1e-3. The training is run for 10000 iterations and the loss history is monitored every 10 iterations. The model is saved to a file after training is complete for later inference.

This model required hours of time to train with a GPU accelerated tensorflow backend, so we highly recommend doing the same if you try to replicate the experiment.

model.compile("adam", lr=1e-3, external_trainable_variables=[C1, C2])
loss_history, train_state = model.train(
 iterations=10000, callbacks=[variable], display_every=10,
 disregard_previous_best=False,)
model.save(save_path="./psi_pinn_nt/model")

Results

The results show that the PINN is able to accurately predict the flow of fluid in the domain. The predicted flow matches the true flow over the domain the NN was trained on, with non-degenerative performance on out of domain data. The testing error was on the order of 1e-3 across the six objective functions, which is a good indication that the neural network is accurately predicting the flow.

Solution Plot

Conclusion

I learned a lot about PINNs in the process of writing this article, specifically, testing my patience after spending one week debugging the PINN to discover that my visualization function was malformed. I learned about the basic structure of a PINN, how they are trained, and how they can be used to solve PDEs. I also learned about the advantages of PINNs over traditional numerical methods, such as FEM. I am excited to learn more about PINNs and their potential applications in the future.

Here are some specific things I learned:

PINNs are a relatively new research area, but they have already shown great promise for solving PDEs.
PINNs can be used to solve PDEs that are difficult or impossible to solve with traditional numerical methods.
PINNs can be trained on relatively little data, how little data was required to train the Sprott attractor was very impressive.
PINNs can often be more efficient than traditional numerical methods for high dimensional problems.

Thank you to Cam, Nathan, Rick and Ed for their support and putting up with my whining at spending one week debugging a single line of code and to Ben and James for their exhaustive grammar checking.