|
|
Running flowCart
Input/Output
files
Command line options
and usage
Multi-processor
running (shared memory and MPI)
Flux
Function Options:
Robust
Mode
Limiter
Choice
How do I restart a calculation?
More information on flowCart I/O
Input/Output
files:
flowCart
requires an input.cntl
file, a mesh, and a Mesh.c3d.Info
file to get running. If you want to map the solution back to the
surface
triangulation, then you'll also need to provide the configuration
triangulation
file which you used to create the mesh (in cubes). Meshes
can be any one of the following:
|
File
type - default name
|
Contents
|
Platform
and solution approach
|
| Mesh.c3d |
Unordered
mesh from cubes |
Single
CPU, no
multigrid or grid sequencing |
| Mesh.R.c3d |
Reordered
mesh ready for
partitioning |
Multiple
CPU, no
multigrid or grid sequencing |
| Mesh.mg.c3d |
Reordered,
multilevel mesh
file |
Multiple
CPU, with multigrid or grid sequencing |
Most often, you're feeding flowCart a "Mesh.mg.c3d" mesh.
For
more information on flowCart I/O command line options and
files,
click here. If you're
using the new (Cart3D v1.4) "$__Force_Moment_Processing"
section in input.cntl
you will also have to have a GMP-style
Config.xml file, but flowCart
will automatically produce one of these if the only component you're
interested in is "entire".
Command line options and Usage:
Usage:
% flowCart
-
flowCart
-
Usage:
flowCart [ argument
list ]
Options:
-- Runtime
Options--
-N %d Max
Number of MultiGrid cycles to advance
-choice %d 1=do_mg_new, 2=jameson,
3=orig_do_mg_cycle
-mg %d Number of
multigrid levels
-gs %d Use grid
sequencing on # levels for startup (no mg)
-cfl %f CFL number
-rampUp %f ramp up factor from .01*CFL,
default: off
-tm %f cut-cell
Grad mod (1stOrd=0. -> 1.0=2ndOrd) def: 1.0
-limiter %d 0=None, 1=BJ, 2=VanLeer, 3=SinLim,
4=VanAlbada, 5=MinLim
-buffLim Buffer limiters to
smear shocks, Default: <false>
-flux %d Flux function
0=VanLeer, 1=VanLeer-Hanel, 2=Colella, 3=HLLC)
-no_fmg no full
multigrid (start MG at finestLevel)
-subcell Use subcell
resolution on finest mesh
-gr Use
grads in MGrestrict (auto ON if gradEval all stages)
-nPart %d Number of Sub Domains
for partitioning
-order %d SubDomain ordering, 0=No
Reordering, 1=RCM, 2=MLD
-y_is_spanwise Default assumes z_is_spanwise direction
-- I/O Options --
-v verbose
mode ON
-mem
Report memory usage (auto on with -v)
-i %s
Input file name, default:<input.cntl>
-T Dump
surf triangulation in Tecplot format <surfName.dat>
-clic Dump surf
trianglulation in Clic format <surfName.triq>
-his write
history files <history.dat,forces.dat>
-binaryIO Write postprocessing data in
binary (plotfiles etc.)
-Xcut %d Num of X=const cut planes
<disjointCutPlanes.dat>
-Ycut %d Num of Y=const cut planes
<disjointCutPlanes.dat>
-Zcut %d Num of Z=const cut planes
<disjointCutPlanes.dat>
-version Dump version info and exit
-Dmatrix Dump i,j formatted
connectivity Matrices
-Dcut Dump
tecplottable file <cutcells.dat>
-no_ckpt Suppress checkpointing
-restart Restart into any # of
partitions <Restart.file>
-Derr Output tau
error estimates for adapt
|
top
Multi-processor running:
flowCart
uses OpenMP or MPI for explicit
communication
between subdomains in multi-CPU/multi-core runs. This means that you
can run it on
any machine or cluster which supports either communication protocol.
Since it explicitly decomposes the domain,
it achieves extremely good scalability as is typical of message passing
codes. to the right is a plot of showing parallel speedup (current Oct
2003)
for a typical case on about 4.7M cells. Here
is a more recent comparison on NASA's Columbia
system
More information on the programming paradigm and details of the
parallelization are availible in:
|
 |
- To run on a
multi-cpu or multi-core system
with OpenMP (shared memory)
follow these steps
(csh):
| 1% setenv OMP_NUM_THREADS nProc |
#
...nProc is the integer
#
number of
CPU's you want |
2% flowCart -[flowCart Options]
|
# ...run the
executable, flowCart
will interrogate
#
the environmetn and set the number of subdomains
#
to
nProc automatically |
- To run on a
multi-cpu or multi-core system
with MPI do the following
(csh):
| 1% mpiexec -np nProc
flowCart -[flowCart Options] |
#
...nProc is the integer
#
number of
CPU's you want |
Note: We frequently get questions about MPI running. mpi_flowCart is built using mpich2 and
we distribute both static and dynamiclly linked executables. There
often turn out to be basic issues with the underlying mpich2
installation. If you're new to MPI, there are a couple of things
you should start with before jumping in...
- Run flowCart (shared memory) on more than
1 thread using OpenMP.
- Does "%
mpiexec -np 2 /bin/date"
return 2 copies of the date?
- Can you run any (other) mpi code?
- Can you run any of the mpich2 demos?
(top)
Flux Function Options:
flowCart
is designed to allow user-selectable flux functions. Currently, van
Leer, Colella'89 and HLLC (beta-test) are availible. Of these van Leer
("-flux 0") and van Leer-Hänel ("-flux 1") are the most well
excercised, and since they're positive flux functions, they're certain
to be the most robust. Most users should
choose "-flux 0" (van Leer) as their default. You can specify the flu
function on the command line using "-flux 0" or by using the FluxFun tag
in the $_Solver_Control_Information category
of the input control file and set it to
"0". Flux functions are ordered roughly in order of decreasing smearing
of contact discontinuities. HLLC gets an exact contact speed,
while this is the classic achillies heel of van Leer. Take a 2D
airfoil (like the naca0012 in $CART3D/cases/samples/naca0012)
and experiment.
top
Robust
Mode:
Starting with verions v1.3, flowCart
has a new "robust mode" of operation. In general, the code is very
robust running with the standard Runge-Kutta (RK) scheme sketched in
the sample input.cntl file, This
scheme works great most of the time, but its a compromise between
robustness and speed. As is clear from the RK tags, typically you can
get away with only evaluating the gradient and limiters at the first
stage of the scheme, saving this work on subsequent stages. In
taking this shortcut, we give up some of the remarkable positivity
property of some flux-vector-splittings. By being more careful -
evaluating the gradient at every stage in the RK scheme, and
re-evaluating it during multigrid restrictions, the scheme can be made extremely robust.
To invoke "robust mode" simply set up your RK scheme in the input.cntl file to evaluate the gradient
(and limiter) at every stage in the RK, like this:
RK
0.0695
1
RK
0.1602 1
RK
0.2898 1
RK
0.5060 1
RK
1.0 1
Use "robust mode" sparingly. Nearly
all calculations will run with the standard scheme (which is about 50%
cheaper). Virtually all the simulations that you'll find on this
website were run with the standard scheme.
top
Limiter
Choice:
Starting with Cart3D_v1.3 the limiters have been
completely re-coded to be better behaved than in earlier distributions.
They are now linearity preserving regardless of mesh stretching, and
there is a choice of 5 of them. Having said that, you should almost always use limiter 2
(van Leer). This is the most aggressive smooth limiter and is not
excessively dissipative - especially in the k-exact implementation in flowCart. Furthermore, its among
the fastest to compute. The limiter choices are (in order of increasing
dissipation): (0) no limiter, (1) Barth-Jespersen (2) van Leer, (3) sin
limiter, (4) van
Albada (5) Minmod. To understand the differences between them,
Look at the figure below. The plot on the left shows the limiter value
(0-1) vs. the normalized acceleration or deceleration of the data.
(sketch on the left). Phi = 1 means "no limiting". The parameter f is equivalently either del+/delC
or del-/delC. The Barth-Jespersen (BJ) limiter essentially follows the
monotonicity boundary for a 2nd order monotone scheme, so you can think
of this as an effective "upper bound" for the limiter value. More
aggressive, and you're not TVD. Below this curve, and you're limiting
more than is strictly necessary. All the limiters go through Phi=0 at
f=0.5. This says that they recommend no limiting when the data is
linear, which is necessary for a 2nd order scheme.

Basically,
we have BJ as the most aggressive, and minmod as the most dissipative.
Everybody else falls in between. This graph helps you understand how
much dissipation you're buying when you choose something other than "-limiter
1"
on the command line, or in input.cntl. About the only time you should
even consider something else is for high mach number
cases since the more aggressive limiters may "over compress" the shocks
leading to excessive "staircasing".
I've made a small quicktime movie
comparing the "sin limiter" to
Barth-Jespersen. (The "sin limiter" is in green, and BJ is shown
with the red line.) Notice how, in an effort to be smooth, the sin
limiter retards the slope pre-emptively as u_i approaches
either of its neighbors. Van Albada
is even more dissipative, and you can see that it gives up its
slope substantially more quickly than sin(). BJ, or course, holds onto
its slope as long as it can without overshoots, but the transitions are
abrupt.
(top)
How do I restart a calculation?
You
can restart a calculation using the parameters in your input.cntl file,
and your can override most of these, if you'd like, using command line
options (preferred).
Both are scanned upon restart.
1.
When you run a flowCart simulation, a checkpoint file is automatically
produced (you can suppress this with the "-no_ckpt"
command line option, if you konw you dont want to restart). This
checkpoint file is automatically named using the scheme ckPtFileName = check.[nCycles],
so if you ran 150 cycles the restart file would be named "check.00150".
You can use this file to restart the calculation by soft-linking the
special
name "Restart.file" to the checkpoint file, and invoking the "-restart"
command line flag. e.g.
| %ln -s check.00150
Restart.file |
# create a
softlink |
| %flowCart -restart -N
200 |
#
run flowCart an additional
50 cycles |
2.
When you restart a computation, you are free to change both the number
of partitions or the number of levels of multigrid that you're using.
To
change the number of partitions, just reset your OMP_NUM_THREADS
environment variable in OpenMP, (or the "% mpiexec -np %d" input in MPI),
and
flowCart will check it when it begins. To change
the number of multigrid levels, use the -mg %d
flag on the command line.
(top)
last update Dec. 2008, M. Aftosmis
|