5 Day Mini-Course on: Physics Informed Machine Learning with Pytorch and Julia

Instructors: Arvind Mohan and Nicholas Lubbers, Computational, Computer, and Statistical Division, Los Alamos National Laboratory

Day 1: The killer feature: Automatic Differentiation: ENR2, S395

           0 - Theory of automatic differentiation 

           1 - Example Notebooks: Introduction to Pytorch

           2 - Example Notebooks:  Tape-based Automatic Differentiation

Zoom: https://arizona.zoom.us/j/83738249833  passcode: applied

A successful statistical description of most nonlinear systems is stymied by the lack of closure; the rates of change of lower order moments depend on moments of higher order leading to an infinite hierarchy. Most attempts to enforce an artificial closure fail. However, there is a natural asymptotic closure for fields of weakly nonlinear waves (think of a wind driven ocean surface). I will explain to you what (mild) premises are required to achieve this natural closure and the reasons it occurs. The corresponding kinetic equation for the energy density turns out to have very interesting stationary solutions corresponding to equipartition states and to finite flux solutions of Kolmogorov type, discovered originally by our good colleague Volodja Zakharov. But, despite many successes, the story is far from over as there are still open challenges, some of which I will tell you about.

Abstract:         We analyze an epidemic model on a network consisting of susceptible-infected-recovered equations at the nodes coupled by diffusion using a graph Laplacian.  The model has few parameters enabling fitting to real data. It also has the essential ingredient of importation of infected people. These two features were particularly important for the COVID-19 epidemic.  Using a complete graph describing the main airports on the planet, the model allowed us to predict the arrival of the COVID epidemic in Mexico in March 2020. It was also useful to evaluate deconfinement scenarios and prevent a so-called second wave.

The size of the outbreak is governed by the initial growth rate of the disease given by the maximal eigenvalue of the epidemic matrix formed by the susceptibles and the graph Laplacian representing the mobility. We use matrix perturbation theory to analyze the epidemic matrix and define a vaccination strategy, assuming vaccination reduces the susceptibles. When mobility and local disease dynamics have similar time scales, it is most efficient to vaccinate the whole network because the disease grows uniformly. If only a few vertices can be vaccinated, then which ones do we choose? We answer this question, and show that it is most efficient to vaccinate along an eigenvector corresponding to the largest eigenvalue of the Laplacian.

When mobility is slower than local disease dynamics the epidemic grows on the vertex with largest susceptibles. The epidemic growth rate is more reduced when vaccinating a larger degree vertex; it also depends on the neighboring vertices. These results are illustrated by numerical calculations on several examples.

Place: Math Building, Room 402  https://map.arizona.edu/89

Day 2: Automatic Differentiation in Julia. Math, 402

            1 - The many algorithmic approaches to AD

            2 - Example Notebooks: The Automatic Differentiation Ecosystem in Julia

Zoom:            https://arizona.zoom.us/j/83738249833   Passcode: applied

Advisors:  Co-Advisor:  Xueying Tang, Dept of Mathematics

                 Co-Advisor: Cristian Roman Palacios, School of Information

Zoom link for remote attendees: https://arizona.zoom.us/j/2882717142

Password: transright

This will be a 20 minute presentation with Q&A to follow. 

Day 3: Physical Symmetries and Atomistic Machine Learning,

Place: ENR2, S395 and Zoom

            0 - Symmetry, Invariance, and Equivariance with Graph Neural Networks

            1 - Example Notebooks: Learning Potential Energy Surfaces 

            2 - Code walkthrough: Writing custom GPU kernels with Numba

            3 - Code walkthrough: Cross-language integration: Pytorch to C++ and Julia

Zoom:            https://arizona.zoom.us/j/83738249833   

Passcode: applied

Abstract:         Biologically realistic models in neuroscience are challenging to build and to simulate due to the large numbers of neurons, their complex interactions, and the large number of unknown physiological parameters.  Reduced, or coarse-grained, models are more tractable, but it is not always clear how to make sense of models too far removed from neuroanatomy and physiology.  In this talk, I will report on a recently proposed approach to coarse-graining cortical models that aims to balance biological realism and computational efficiency, and illustrate its use on models of specific cortical circuits in primates.  This is joint work with Zhuo-Cheng Xiao (PhD UA Applied Math 2020, now at NYU-Shanghai) and Lai-Sang Young (NYU).

Mechanical metamaterials are many-body elastic systems that deform in unusual ways, due to the interactions of essentially rigid building blocks. Examples include origami patterns with many folds, or kirigami patterns made by cutting material from a thin elastic sheet. In either case, the local deformations of the pattern involve internal degrees of freedom which must be matched with the usual global Euclidean invariances --- e.g., groups of four origami panels move by coordinated rotations and translations, but it is still possible to bend the whole origami pattern into an overall curved shape. This talk will introduce the homogenization problem for kirigami and origami metamaterials to a broad audience and describe our recent results. Our goal is to explain the link between the design of the individual cuts/folds and the bulk deformations and geometries they can produce. This is joint work with Paul Plucinsky (U. Southern California, Aerospace and Mechanical Engineering) and Paolo Celli (Stony Brook U., Civil Engineering).

Day 4: PDE learning and Scientific  with Julia 

Place: Math, 402

           1 - Scientific Computing Integration with AD in Julia and Custom Adjoints

            2 - Example Notebooks: GPU Computing in Julia

            3 - Example Notebooks: Neural PDE Learning in Julia

            4 - Example Notebooks: Data Processing and Cross language Integration

Zoom:   https://arizona.zoom.us/j/83738249833   Passcode: applied

Abstract:  This talk will build a careful foundation for modeling knit objects using graphs in order to explore equivalences between knittable objects and well known graph structures and build theory for the complexity of determining knittability of graph structures. This discussion will also relate knitting properties with the topological genus of graphs and explore restrictions of general graph algorithms which are uniquely motivated by the knitting process. This foundation will be used to represent real-world knitting patterns and illustrate considerations of visualizing such patterns through force-directed graph layout algorithms.

Al Scott, a native of Worcester, Massachusetts, obtained a doctorate in Electrical Engineering from MIT in 1961. During the early 1970’s his research interests led to important contributions to the then emerging field of soliton mathematics and nonlinear wave propagation. He became one of the leading figures in the new field of nonlinear science and a founding editor of Physica D, the first journal devoted to the study of nonlinear phenomena. He was very interested in the role of nonlinear dynamics in modeling biological systems and, in particular, its applications to neuroscience.

In addition to many scholarly papers on a wide variety of topics he wrote several books on neuroscience and nonlinear science and was the editor of the comprehensive Encyclopedia of Nonlinear Science published in 2005. He joined the faculty of the Mathematics Department at the University of Arizona in 1985 and became a member of the University’s Program in Applied Mathematics. He retired from the University in 2000. His many contributions to the life of both the Program in Applied Mathematics and the Department of Mathematics were characterized by a civilized and good-humored approach to academic life. He was particularly encouraging of graduate students and it is this characteristic that is the basis for the Al Scott Prize and Lecture. The annually awarded prize is given to a senior student in the Program in Applied Mathematics and consists of a cash prize and a lecture in the Applied Mathematics Colloquium series.

Day 4: PDE learning and Scientific  with Julia 

Place: ENR2, S395

           1 - Scientific Computing Integration with AD in Julia and Custom Adjoints

            2 - Example Notebooks: GPU Computing in Julia

            3 - Example Notebooks: Neural PDE Learning in Julia

            4 - Example Notebooks: Data Processing and Cross language Integration

Zoom:   https://arizona.zoom.us/j/83738249833   Passcode: applied

For details, see https://science.arizona.edu/academics/graduation-convocation