Vector Maxwell Simulations

December 2003

For the past 6 months I have been working with the Arizona Center for Mathematical Sciences under Professor Moysey Brio    developing and expanding upon a set of Vector Maxwell Simulators.  . These simulators can model electro-magnetic waves over time (such as light, radiation, etc) and play an important role in developing optical devices as well as many other applications.

There are essentially four different simulators, three of them are closely related and one has been largely built from scratch.  The former three are all based on the computational code designed Aramis Zakharian and programmed purely in C.  There is essentially a 2D version, a 3D version, and an AMR (adaptive mesh refinement) version.  The 2D and 3D versions were equipped with a Python interface allowing the simulation to be set up in a simple Python script.  This allows us to change the simulation variables without having to recompile the whole thing again.  The Python script is also simple and uncluttered, making it possible for people with no programming experience to set up and run a simulation with only the simulator’s user manual in hand.

The fourth simulator was written by Colm Dineen in C++ and Fortran using the Chombo framework (designed at Lawrence Berkeley National Laboratory).  This version also uses AMR, but it allows for parallel processing which the C version does not due to the complexity of the data structures.  Both the AMR and parallel processing are implemented because of the amount of time it can take to run a detailed simulation.

These simulators define a 3d space as a matrix of cells over which the calculations can be preformed. The size (and therefore number) of cells determines the precision of the simulation. If there is a huge number of cells it can take days or weeks to process. If there are too few cells we get an inaccurate portrayal of what really happens. This was the motivation for 'Adaptive Mesh Refinement' or AMR. Essentially AMR allows us to use large cells in areas of the domain which are of little interest, and then smaller cells in the areas where we really need precision.

My first project was to create a library of functions in C++ that would draw some basic geometric shapes into the matrix of cells for Colm Dineen's Chombo based simulator.  These functions allow the user to simply specify the location, size, and material for the object, which is then drawn into the appropriate matrices for each level of mesh refinement.  I later expanded on these geometric primitives and designed functions which could automatically draw triangular and square lattice structures.  This library of functions is designed to simplify the setup of a simulation and eventually be linked to an interface which will eliminate the compile step.

By using the Chombo Visualization program we are able to view the matrix of cells in full 3d, and even view various levels of the AMR at the same time. The second project I have been working on also uses the Chombo library. This time we wanted a standalone converter that could read in binary files dumped by Aramis's simulator (written entirely in C) and then use the Chombo library to create a hdf5 file (Hierarchical Data Format Version 5) which could then be viewed in full 3d using ChomboVis. Before this code was written you could only view a 2d slice of a single level of mesh refinement, now we can view all levels of mesh refinement at the same time in full 3d.

An example of an HDF5 file created by my program.  You can see 3 levels of mesh

refinement at the same time, in full 3D

This project has proven itself useful to the researchers and so I did some work to make it a more complete and polished application.  I provided command line options that allow the user to select a range of time steps that they want to convert, which components to extract (various fields or geometry layout), and various other helpful functionalities.

The rest of my time has been spent updating the MX3D simulation (the non-AMR C version) into Version 1.5.  The first priority was to tidy up the interface as well as the general code.  At the same time I have been implementing some new geometries and features for researchers using the software, eliminated some bugs and inefficiencies, and finally updated the user manual to reflect these changes.  The goal has been to essentially make a more complete, efficient, and logically coded piece of software that we can distribute to the researchers.

Much of the code for the simulations was written by people with little programming experience.  I have been going through this code file by file trying to make it more logical and efficient. I have reorganized and simplified the observer and source functionalities, the geometric functions, the interface functions, and various helper functions.  In particular I have minimized the function calls that take place inside the simulation loop as well as the unnecessary logic operations. I have also worked on making the python interface more consistent both internally and with the other Vector Maxwell solvers. Finally I have been deleting the clutter of unused and outdated code from the core of the simulation.

I have also been adding a lot of new functionalities to the code.  One of the first things I did was set up the output function so that you can view each resulting time frame as they were output rather than having to wait for the simulation to finish in its entirety.  This allows researchers to look at preliminary results as the simulation runs to make sure it is set up correctly, this is especially helpful when the simulation can take over 12 hours to complete.  I changed the output model so that the output function is now accessed through a ‘Domain_Observer’.  This allows the user to create multiple Domain_Observers which switch on and off at different times and have their own temporal resolution (the number of time frames outputted in their lifetime).  I have also added interface code that will allow you to choose which fields to output, and if you want to show the observers, sources, and processor division inside the layout file.  Also you can output the magnitude of the field vectors rather than just the components in the x, y, and z directions.  In the interface you can now ‘name’ your observers and sources and then set up the field they work with, the shaping of the time pulse to use, when they turn on and off, etc.

While reorganizing the functions that draw geometries I have also added some new geometries for the researchers using the simulation.  I added a function that draws what is essentially a 2D parabolic curve stretched into 3D.  This can be used to simulate a parabolic mirror for optical research as well as many other things.  I also created a function that draws a 3D cone which can also be used in optical simulations as a guide for the light.  I have also looked into creating a set of functions which would rotate and translate the geometries so that they could be placed at any angle, while not yet implemented this is probably something I will do soon.

There was also a bug found in the generation of planes for the observers, for example when you want to calculate the flux going through a certain plane inside the domain there would be a line missing in the plane.  I created a simpler version where you can have a plane that is parallel to the XY, YZ, and ZX planes.  At the moment this is enough for researchers and I plan on implementing the code that will allow the planes to be at any arbitrary angle next semester.

Version 1.5 is now complete and ready for distribution with updated an updated user manual.  This is an ongoing process where the researchers feed me their ideas as to what is wrong or could use improvement in the simulations.  By the end of next semester I hope to have version 2.0 ready.  This would involve me debugging all of the observer functionalities, allowing planes and geometry to be placed at arbitrary angles and implementing and debugging the Non-uniform grid spacing interface.  Non-uniform grid spacing has the same motivations as AMR except that it uses a single data structure where some cells are define as being smaller than others.  This would also allow for a transition from large cells to small cells which helps reduce the unwanted high frequency noise that has been plaguing the AMR simulations.