home news images pubs c3dTeam
whatIsCart3D?
surfaceModeling
meshGen?
flowSolvers?
Tiger
flowCart
overview
reorder
mgPrep
running
Input/Output files
postprocess?
mailList?
betaTest?
licensing?

   Running flowCart


Input/Output files
Command line options and usage
Multi-processor running
Flux Function Options - and Adding Your Own!:
Robust Mode
Limiter Choice
How do I restart a calculation?
More information on flowCart I/O

  Input/Output files:
flowCart requires an input.cntl file, a mesh, and a Mesh.c3d.Info file to get running. If you want to map the solution back to the surface triangulation, then you'll also need to provide the configuration triangulation file which you used to create the mesh (in cubes).  Meshes can be any one of the following:
 
File type - default name
Contents
Platform and solution approach
Mesh.c3d Unordered mesh from cubes Single CPU, no multigrid or grid sequencing
Mesh.R.c3d Reordered mesh ready for partitioning Multiple CPU, no multigrid or grid sequencing
Mesh.mg.c3d Reordered, multilevel mesh file Multiple CPU, with multigrid or grid sequencing

Most often, you're feeding flowCart a "Mesh.mg.c3d" mesh. For more information on flowCart I/O command line options and files, click here.

   Command line options and Usage:

Usage:

 % flowCart -
 
   flowCart - 
    Usage: flowCart [ argument list ]
 Options:
      -- Runtime Options--
-N %d          Max Number of MultiGrid cycles to advance
-choice %d     1=do_mg_new, 2=jameson, 3=orig_do_mg_cycle
-mg %d         Number of multigrid levels
-gs %d         Use grid sequencing on # levels (no mg) 
-cfl   %f      CFL number
-rampUp %f     Ramp up factor from .01*CFL, default: off
-tm  %f        cut-cell Grad mod (1stOrd=0. -> 1.0=2ndOrd) def: 1.0
-limiter %d    0=None 1=BJ 2=vanLeer 3=SinLim 4=VanAlbada 5=Minmod
-flux  %d      Flux function: 0=VanLeer, 1=Colella, 2=HLLC (betatest)
-gamma %F      Ratio of specific heats, cp/cv, default: 1.4
-no_fmg        no full multigrid (start MG at finestLevel)

-subcell       Use subcell resolution on finest mesh
-gr            Use grads in MGrestrict (auto ON if gradEval all stages)
-y_is_spanwise Default assumes z_is_spanwise direction

      -- I/O Options --
-v         Verbose mode ON
-mem       Report memory usage (auto on with -v)
-i %s      Input  file name, default:<input.cntl>
-T         Dump surf triangulation in Tecplot format <surfName.dat>
-clic      Dump surf trianglulation for Clic <surfName.triq>
-his       write history files <converge.dat,forces.dat>
-Xcut %d   Num of X=const cut planes <cutPlanes.dat>
-Ycut %d   Num of Y=const cut planes <cutPlanes.dat>
-Zcut %d   Num of Z=const cut planes <cutPlanes.dat>
-version   Dump version info and exit
-Dmatrix   Dump i,j formatted connectivity Matrices

-Dcut      Dump tecplottable file <cutcells.dat>

-no_ckpt   Suppress checkPointing
-restart   Restart into any # of partitions <Restart.file>

top

  Multi-processor running:
 
flowCart uses OpenMP or MPI for explicit communication between subdomains in multi-CPU runs. This means that you can run it on any machine or cluster which supports either communication protocol. Since it explicitly decomposes the domain, it achieves extremely good scalability as is typical of message passing codes. Here is a plot of showing parallel speedup (current Oct 2003) for a typical case on about 4.7M cells.
More  information on the programming paradigm and details of the parallelization are availible in:
scalability OpenMP & MPI

To run on a multi-cpu system with OpenMP follow these steps (csh):

 
1% setenv OMP_NUM_THREADS nProc #  ...nProc is the integer 
#    number of CPU's you want
2% flowCart # ...run the executable, flowCart will interrogate 
#     the environmetn and set the number of subdomains
#     to nProc automatically
(top)
  Flux Function Options - and Adding Your Own!:
flowCart is designed to allow user-selectable flux functions. Currently, van Leer, Colella'89 and HLLC (beta-test) are availible. Of these van Leer is the most well excercised, and is the most robust. Most users should choose it as their default. To do this use the  FluxFun tag in the $_Solver_Control_Information category of the input control file and set it to "1". Alternatively, you can
set it on the flowCart command line  with "%flowCart -flux 1".
Curious users can experiment with the other choices.

We very much encourage users to submit flux functions that they'd like to see added or would like to work with. We'll gladly send you examples to follow, but here is the general function prototype.

int myFluxFunction(double * uL, double * uR, double *F, 
                 double gamma, double area){
}

Where:
   Inputs:
      uL = Left  state{rho,NormVel, XverseVel_0, XverseVel_1, pressure}
      uR = Right state{rho,NormVel, XverseVel_0, XverseVel_1, pressure}
    area = face area
   gamma = gas constant
   Outputs:
   F = Numerical Flux{rhoflux, norm Mom, Xverse0 Mom, Xverse1 Mom, RhoEflux}
top

  Robust Mode:

Starting with verions v1.3, flowCart has a new "robust mode" of operation. In general, the code is very robust running with the standard Runge-Kutta (RK) scheme sketched in the sample input.cntl file, This scheme works great most of the time, but its a compromise between robustness and speed. As is clear from the RK tags, typically you can get away with only evaluating the gradient and limiters at the first stage of the scheme, saving  this work on subsequent stages. In taking this shortcut, we give up some of the remarkable positivity property of some flux-vector-splittings. By being more careful - evaluating the gradient at every stage in the RK scheme, and re-evaluating it during multigrid restrictions, the scheme can be made extremely robust.
To invoke "robust mode" simply set up your RK scheme in the input.cntl file to evaluate the gradient (and limiter) at every stage in the RK, like this:
Entries in "$__Solver_Control_Information" block of "input.cntl" to invoke "robust mode"
RK    0.0695      1
RK    0.1602      1
RK    0.2898      1
RK    0.5060      1 
RK    1.0         1

Use "robust mode" sparingly. Nearly all calculations will run with the standard scheme (which is about 50% cheaper). Virtually all the simulations that you'll find on this website were run with the standard scheme.  
top

  Limiter Choice:

Starting with Cart3D_v1.3 the limiters have been completely re-coded to be better behaved than in earlier distributions. They are now linearity preserving regardless of mesh stretching, and there is a choice of 5 of them. Having said that, you should almost always use limiter 2 (van Leer). This is the most aggressive smooth limiter and is not excessively dissipative - especially in the k-exact implementation in flowCart. Furthermore, its among the fastest to compute. The limiter choices are (in order of increasing dissipation): (0) no limiter, (1) Barth-Jespersen (2) van Leer, (3) sin limiter, (4) van Albada (5) Minmod. To understand the differences between them,
Look at the figure below. The plot on the left shows the limiter value (0-1) vs. the normalized acceleration or deceleration of the data. (sketch on the left). Phi = 1 means "no limiting". The parameter f is equivalently either del+/delC or del-/delC. The Barth-Jespersen (BJ) limiter essentially follows the monotonicity boundary for a 2nd order monotone scheme, so you can think of this as an effective "upper bound" for the limiter value. More aggressive, and you're not TVD. Below this curve, and you're limiting more than is strictly necessary. All the limiters go through Phi=0 at f=0.5. This says that they recommend no limiting when the data is linear, which is necessary for a 2nd order scheme.
Relative dissipation of limiters
Basically, we have BJ as the most aggressive, and minmod as the most dissipative. Everybody else falls in between. This graph helps you understand how much dissipation you're buying when you choose something other than "-limiter 1" on the command line, or in input.cntl. About the only time you should even consider something else is for high mach number cases since the more aggressive limiters may "over compress" the shocks leading to excessive "staircasing".
I've made a small quicktime movie comparing the "sin limiter" to Barth-Jespersen. (The "sin limiter" is in green, and BJ is shown with the red line.) Notice how, in an effort to be smooth, the sin limiter retards the slope  pre-emptively as  u_i approaches either of its neighbors. Van Albada is even more dissipative, and you can see that it gives up its slope substantially more quickly than sin(). BJ, or course, holds onto its slope as long as it can without overshoots, but the transitions are abrupt.
(top)

  How do I restart a calculation?
    You can restart a calculation using the parameters in your input.cntl file, and your can override most of these, if you'd like, using command line options (preferred). Both are scanned upon restart.

    1.  When you run a flowCart simulation, a checkpoint file is automatically produced (you can suppress this with the "-no_ckpt" command line option, if you konw you dont want to restart).  This checkpoint file is automatically named using the scheme ckPtFileName = check.[nCycles], so if you ran 150 cycles the restart file would be named "check.00150". You can use this file to restart the calculation by soft-linking the special name "Restart.file" to the checkpoint file, and invoking the "-restart" command line flag. e.g.

       
      %ln -s check.00150 Restart.file # create a softlink
      %flowCart -restart -N 200 # run flowCart an additional 50 cycles
    2.  When you restart a computation, you are free to change both the number of partitions or the number of levels of multigrid that you're using. To change the number of partitions, just reset your OMP_NUM_THREADS environment variable in OpenMP, (or the "%mpirun -np %d" input in MPI), and flowCart will check it when it begins. To change the number of multigrid levels, use the -mg %d flag on the command line. 
(top)

last update Jul 2004, M. Aftosmis