Physical Parameters

Simulation Parameters

Fixed boundary (old code)

\lambda_P set such that R_P is equal for both runs.

TimeScale = P_orb
lScale = a

Contour key:

Not enough frames to capture evolution (where I suspect the tidal forces may be apparent). Requeued with 1/10th time. However, top view shows clear Coriolis effect. Looks perhaps a bit stronger but otherwise consistent with \tau = 0.08 and \Xi_P = 0.76 given ​Fig. 6.

Line Transfer

lScale = R_P

Contour key:

Unable to start from steady-state run, planet in incorrect position. At frame 760, there still appears to be some subsonic material at the edges, but this may just be because it hasn't yet blown out the entire ambient. The flow profile seems to be more strongly entrained up- and downwind than the above run.

Next?

May be best to run a (slightly larger?) box to steady state with the planet located correctly (using HD209458b orbital parameters - appropriate?), then add stellar gravity and rotation.

Finished Run

WT Run 001

• Previously overestimated temperature fixed.
• Setup: 3Msun AGB + 0.1Msun secondary. 10e12cm separation. (Henceforth 3_-1_12)
  1. Particle mass: 1.989e32 g
  2. Wind temperature: 5.56862e5 K
  3. Wind density: 0.0001 g/cm3
  4. Particle radius: 8.364e9 cm = 1.0 CU = 0.120 Rsun
  5. Bondi radius: 1.564 cm = 18.7 CU = 2.249 Rsun
  6. Run Time: 2 TS = 2 wind passing time of the box = 3.472e5 s ~ 4 days

• Sim parameters：
  1. Fix grid 200^3.
  2. Time Scale = 1.736e5 s.
  3. Length scale: 1 CU = 8.364e9 cm = 0.120 Rsun
  4. Length resolution:: 1 CU
  5. Time resolution : 0.02 CU = 3.472e3 s
  6. Accretion = off.
  7. BC: extrapolated.
  8. Gamma = 1
  9. Softening length = 4 CU = 3.346e10 cm = 0.481 Rsun
  10. Run time: 00:56:13

Preliminary Results

• Clearly defined shock wave.
• Significant accretion, density instability normal to the wind.
• Significant movement of the particle, drag force overwhelms box effect.
• ~ 80 CU displacement in 2 TS - let's end this sim here before the particle escapes the box.

Ongoing run

Test run for higher gamma: WT Heat Up 002

• Setup: 3_-1_12 Run time = 1 TS No Particle Placed
• Sim parameters：
  1. Fix grid 200^3.
  2. Time Scale = 1.736e5 s.
  3. Length scale: 1 CU = 8.364e9 cm = 0.120 Rsun
  4. Length resolution:: 1 CU
  5. Time resolution : 0.01 CU = 1.736e3 s
  6. Accretion = off.
  7. BC: extrapolated.
  8. Gamma = 1.2
  9. Softening length = 4 CU = 3.346e10 cm = 0.481 Rsun

Next steps

• More complete drag force & accretion analysis on its way.
• Higher time resolution, shorter timescale.
• Mesh refinement around the particle. Longer box.
• Different gamma numbers.
• Reconsider possibilities of periodic boundary conditions, fixed particle position. Or fixing the particle in place and measure accretion rate, then let go and measure drag.
• Larger softening length?
• Miscellaneous improvements for visualization.

Questions

• Monday Luke mentioned extracting information on the fly, wonder how that works?
• Sink particle radius defined as integer, which limits lowest resolution.
• Force low resolution on the rim of the box to save time?
• What's the optimal number of nodes & CPU per nodes to use?
• Luke's method of measuring accretion rate & drag - comparable?

Model with modest magnetic field

New Work

1. Now able to run code on bluehive 2.5.
2. Three runs in progress.
3. How to optimize refinement?
4. Edited code to be able to write some output to files.
5. What other data to output?
6. Reading and discussions conerning accretion onto secondary in CEE (presentation?).

Summary of New Results

1. Code is running on bh2.5 and seems to be faster than regular bh (which is nice).
2. New runs will be analyzed next week. Stampede queue time has increased (will eventually go to infinity as it is phased out)! Bluestreak queue is variable. No queue for Bluehive 2.5 as of now!
3. There are a few possibilities:
   • Refine to maxlevel up to refinement radius of min(5e12cm,1.5*separation) about primary point particle (trying now)?
   • Choose refinement radius by hand at each restart (trying now)?
   • Set some refinement criteria within the refinement radius and force to lowest level outside refinement radius?
   • Refine to maxlevel within refinement radius but make refinement radius very small and use large buffer zones?
4. Can now write accreted mass & momentum as well as time and acceleration on particles from gas gravity each time step.
5. Aside from those, other output is also desired, so need to decide what exactly, then spend time implementing that.
   • angular momentum
   • particle separation
   • integrals of damping force out to a given radius around secondary
   • gas potential and kinetic energies
6. Should I give a presentation at a CE meeting of what has been done so far on this topic of accretion in CEE, and also about what output needs to be generated?

Ongoing runs

Run to test making refinement zone decrease with time:

Binary run 133 similar to run 125 but now the refinement zone changes with time
Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame:
Total simulation time:
Machine and partition: Bluestreak standard (running, completed up to frame 181 in 4 days walltime)
Number of cores: 8192
Total wall time:
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within min(5e12cm, 1.5*particle_separation)
Buffer zones: 2 cells
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²
DeafaultAccretionRoutine=2 (Krumholz)

Run to test case where no accretion onto secondary is permitted:

Binary run 135 similar to run 133 but now DefaultAccretionRoutine=0 instead of 2. Also suppress generation of new sink particles.
Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame:
Total simulation time:
Machine and partition: bluehive2.5 standard (running, completed up to frame 196 in 2 days wall time)
Number of cores: 120
Total wall time: about
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within min(5e12cm, 1.5*particle_separation)
Buffer zones: 2 cells
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²
DeafaultAccretionRoutine=0 (No accretion)

Run that uses twice as high resolution and 10x lower ambient pressure as previous runs:

Binary run 132 with double max resolution, lower resolution in ambient medium, 10x smaller ambient pressure than run 116
Relaxation run: 129
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame:
Total simulation time:
Machine and partition: Stampde 1 normal (running, completed up to frame 153, or 18 sim-days)
Number of cores: 1024
Total wall time: 4 days so far (starting from frame 75 of relaxation run)
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 2.25 Rsun (256³ cells)
Highest resolution: 0.14 Rsun (4096³ cells, 4 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within 5e12 cm (71.87 Rsun) of primary center and within a cylinder of radius 20 Rsun and height 20 Rsun around secondary center. After t~9.6e5s (~frame 123) refinement radius around primary was halved to 2.5e12cm.
Buffer zones: 0 cells (no buffer zones)
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁵ dyne/cm²

Next steps

• Analysis of runs 133, 135, 132 and comparison with previous runs.
• Decide what data is needed to output and implement this in the code.
• Improve style of code in preparation for more serious runs.
• Run combining the merits of 132 and 133 that also outputs the relevant data.

• Ionizing planet: see previous post
  • Two more things to (possibly) examine:
    • Increase box size by R_p in each direction, to match size of box in ATHENA
    • Whether ramping time or box size has bigger effect on \tau=1 surface
      • Should just need one more run (high resolution, small box, long ramping time)
  • Also, doesn't seem to be resolving anywhere but where it's forced

• Testing planet with Coriolis and stellar gravity (point particle off grid - ok?): BlueHive problems, only at frame 60. Wind isn't far enough from planet to be affected significantly by Coriolis, but tidal forces are definitely working.

Side View

Top View

• Still working on 1D version of planet

• Charge exchange: Have a modified fixed boundary condition module that should be testable, but working through a strange issue with the code.

• Conferences: I sent an email with a few options, reproduced below. Ruth and John have not made any plans for next year yet - they're going to meet to discuss and get back to us.

​https://exoplanets.phy.cam.ac.uk/Meetings/exoplanets2 - Early July.

​http://rencontresduvietnam.org/conferences/2018/exoplanetary_science/ - Last week of February/first week of March.

​https://www.cospar-assembly.org/show_infopage.php?info=52 - Mid July.

​http://eas.unige.ch/EWASS2018/program.jsp - Early April.

New Work

• Performed and analyzed new run 125 that evolves binary for 3 times longer than old run 116 (60 days as opposed to 20 days), made possible by forcing ambient medium to have lower resolution.
• Compared run 125 with run 116 to check to what degree forcing the ambient medium to be low res affects the region of interest in the center.
• Compared run 125 with results of Ohlman+16a.
• Peformed new relaxation run 129 with resolution doubled for RG, made possible by making resolution of ambient medium much lower.
• Set up new high res binary run 132 that uses 129 and also the trick of run 125 which forces low resolution in the ambient medium [pending on stampede].
• Set up new run 133 that uses standard resolution (like 125) but now makes max resoln refinement zone a function of inter-particle separation (so evolves with time) [pending on bluehive].

Summary of New Results

• Run 125, which removes resolution in the surrounding medium, is reasonably consistent with run 116 in the highly resolved region of interest for the first 20 days (first few orbits). This suggests that it is okay to greatly reduce the resolution outside of the region of interest.
• Run 125 is qualitatively similar to the results of Ohlmann+16a for the first 60 days, but the amplitude and frequency of variation with time of the separation are smaller than in O+16a. This suggests that the gravitational force is still not being fully resolved in our sims (but a higher res version is pending).

Detailed Results

Binary run 116 (see also previous blog post) with half the softening length of old run binary run 088 (and relaxation run 062)
Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame: 168
Total simulation time: 3.36e6 s or ~4.2 RG sound-crossing times (93 frames)
Machine and partition: Stampede 1 normal
Number of cores: 1024
Total wall time: 96 hours
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set automatically by AstroBear
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²

Comments

• See last blog post for movies and discussion.
• Evidence of a disk around secondary.
• Problem: too computationally demanding.
• Solution: reduce resolution in outer regions that are not directly interesting. But will this affect region of interest?

Binary run 125 with longer simulation time and lower resolution in ambient medium
Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame: 335
Total simulation time: 5.2e6 s or 60 days ~6.5 RG sound-crossing times (260 frames)
Machine and partition: Bluehive standard
Number of cores: 120
Total wall time: about 8.5 days
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within 50 Rsun of primary and within a cylinder of radius 50 Rsun and height 50 Rsun around secondary
Buffer zones: 0 cells (no buffer zones)
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²

Movies
Density sliced at half-way point of box:
​face-on full box
​face-on zoom-in
​edge-on zoom-in
​face-on slice through secondary, extra zoom-in
​face-on slice through secondary, extra zoom-in with velocity vectors

Edge-on slice through secondary as viewed from primary point particle:
​viewed from P1 with P2 at center

Edge-on slice through secondary AND primary:
​slice through P1 (left side) and P2 (center)

Mach number:
​face-on full box

Particle mass:
​face-on zoom-in

Comparison of runs 116 and 125:
Comparison of density (116 on left, 125 on right):
​face-on slice through secondary, extra zoom-in

Comparison of particle mass (116 on left, 125 on right):
​face-on zoom-in

Comparison of separation-time graph and orbits (116 on left, 125 on right):

Comparison of density at 10 and 20 days (116 on left, 125 on right):

Density at 40 days (left) and 60 days (right) for run 125:

Ohlmann+16a Fig 3:

Comments

• The results compare well qualitatively to those of run 116. However, there are small differences:
  • The "disk" that forms arond the secondary is still slightly less circular and less extended than in run 116.
  • The separation variation AMPLITUDE is reduced by about 20% by t=20 days compared to run 116.
  • The mass accreted onto the secondary is increased by about 1% at t=21 days compared to run 116.
• The results compare well qualitatively with those of Ohlmann+16a. However, the differences are significant:
  • The separation-time plot has a smaller amplitude of oscillations.
  • The separation-time plot has a smaller frequency of oscillations.
  • The mean separation is larger by about 25% by t=60 days.

Relaxation run 129 with double max resolution, lower resolution in ambient medium, 10x smaller ambient pressure than run 096
First frame: 0
Last frame: 138 (2.76e6 s or 9.2 RG freefall times of primary star, with damping implemented up to frame 75)
Total simulation time: 2.76e6 s or 3.45 RG sound-crossing times
Machine and partition: Stampde 1 normal
Number of cores: 1024
Total wall time: 92.5 hours (about 49 hours to reach frame 75)
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 2.25 Rsun (256³ cells)
Highest resolution: 0.14 Rsun (4096³ cells, 3 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within 5e12 cm (71.87 Rsun) of primary point particle
Buffer zones: 2 cells
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁵ dyne/cm²

Movies
Density sliced at half-way point of box:
​zoom-in

Mach number sliced at half-way point of box:
​zoom-in

Comments

• We have run the relaxation run up to frame 138 but present the results up to frame 75 here.
• Frame 75 or 1.5e6s is the point at which damping has been reduced to zero (5 free-fall times).
• We see that the density is reasonably stable, though it is still evolving at t=1.5e6s.
• This suggets that the new refinement criteria used here are reasonable.
• The plot of Mach number shows that grid effects are present.
• In the future we may want to:
  • Increase the resolution, enabling higher resolution in the binary runs.
  • Increase the relaxation time from 5 freefall times to 5 sound-crossing times with damping + 5 sound-crossing times without damping, as in O+16a.
  • Increase the box size.
• Note that the relaxation run is quite costly (but several binary runs can be executed from a single relaxation run). This run up to frame 75 took 49 hours on stampede with 1024 cores.
• One way to speed it up would be to resolve the core and a shell that includes the surface, rather than the whole star. But then the resolution in between would still have to be reasonably high, so would need to use large buffer zones.
• Note that with the higher resolution, we were able to decrease the ambient pressure by 1 order of magnitude. This also helps to reduce the ambient temperature and sound speed, as the density is kept the same as for run 125. This may help to relax the CFL constraint.
• Maybe it would be worth trying a binary run that DOES NOT USE ANY RELAXATION RUN (i.e. no damping) and see how much difference there is.

Binary run 132 with double max resolution, lower resolution in ambient medium, 10x smaller ambient pressure than run 116
Relaxation run: 129
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame:
Total simulation time:
Machine and partition: Stampde 1 normal (running)
Number of cores: 1024
Total wall time:
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 2.25 Rsun (256³ cells)
Highest resolution: 0.14 Rsun (4096³ cells, 4 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within 5e12 cm (71.87 Rsun) of primary center and within a cylinder of radius 20 Rsun and height 20 Rsun around secondary center
Buffer zones: 0 cells (no buffer zones)
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁵ dyne/cm²

Binary run 133 similar to run 125 but now the refinement zone changes with time
Relaxation run: 096
First frame: 75 (5 RG freefall times, when velocity damping ended)
Last frame:
Total simulation time:
Machine and partition: Bluehive standard (pending)/Bluestreak standard (running)
Number of cores: 120(bh)/8192(bs)
Total wall time: about
Hydro BCs: extrapolated
Poisson BCs: multipole expansion
Box size: L=4e13 cm
Base resolution: 9.0 Rsun (64³ cells)
Highest resolution: 0.29 Rsun (2048³ cells, 5 levels AMR)
AMR implementation: set by hand to have max level around point particles
Max resolution zone: within min(5e12cm, 2.5*particle_separation)
Buffer zones: 2 cells
Softening length: 2.4 Rsun
Ambient density: 6.7e-9 g/cc
Ambient pressure: 10⁶ dyne/cm²

Discussion

• Refinement of the central region only works reasonably well and is the way forward.
• Should work toward making the refinement a function of the particle separation as attmpted in ongoing run 133.
• Comparison of separation vs time and orbit plots with those of O+16a suggests we still lack enough resolution.

Next Steps

• Analyze ongoing runs.
• Do run that combines merits of 132 and 133.
• Output diagnostics including accretion rate onto secondary, drag force on secondary
• Experiment with variations to "fiducial parameter values": initial separation, primary spin, accretion onto primary.

• Xsede Renewal proposal for Binary due on 10/15, working on code multi-threading optimization & scaling testings.
• ​Current allocation usage
• ​Wire paper

• Queued two more: -5 R_p to 5 R_p in all directions (same domain as ATHENA), at medium resolution (32 cells/R_p - BlueHive) and high resolution (64 cells/R_p) - BlueStreak.

• Also would like to compare a quick ramping time for one set of good runs, to determine if it's the ramping time or box size that has the most impact.

1 unit of computational time is 2 crossing times (for the planet - from 0 to Rp at the sound speed at the planet surface).

No movies for first two, due to storage difficulties.

64 zones/R_p, 2x10¹³ flux, quick ramping time (half flux by frame 30)

(Ignore the velocity data in this frame - I suspect it's corrupted).

64 zones/R_p, 2x10¹¹ flux, quick ramping time (half flux by frame 30)

64 zones/R_p, 2x10¹³ flux (total execution time 3.56 days)

32 zones/R_p, 2x10¹³ flux

32 zones/R_p, 2x10¹¹ flux

The low-flux case doesn't appear to have reached a steady state yet, possibly because the flow stays subsonic even with the larger box.

16 zones/R_p, 2x10¹³ flux

New Work

• I created movies of the previous run that follow the secondary point particle, keeping it at the centre of the frame.

Summary of New Results

• The new movies reveal structure around the accreting secondary.

Detailed Results

Binary run 96/116 with half softening length of old run 062/088
Damp116) Extrapolated hydro BCs, Multipole expansion Poisson BCs, ambient P=10^6 dyne/cm^2, ambient density 6.7\times10^{-9} g/cc.
(Stampede 1 normal 1024 cores, ~4 days wall time)
(L=4\times10^{13} cm, 64^3, 5 levels AMR)

Reference frame of secondary:
​2d density slice (zoomed)
​2d density slice (zoomed), edge-on

Note: I tried to make a similar movie of face-on temperature, but visit could not handle it (don't understand reason)

Now sliced through secondary:
​2d density slice (zoomed)
​2d density slice now with velocity vectors (zoomed)
​2d density slice edge-on (zoomed)

Now sliced through secondary, zoomed in 4x again:
​2d density slice (extra zoomed)
​2d density slice (extra zoomed)

Box frame (as shown in previous post):
​2d density slice zoomed
​2d density slice with mesh

Discussion

• Something resembling an accretion disk forms.

Next Steps

• Long run with maxlevel restricted to vicinity of primary and secondary (running on bluehive)
• Run with maxlevel restricted to vicinity of primary and secondary AND max resolution doubled (relaxation run almost completed on stampede)
• Output diagnostics including accretion rate onto secondary, drag force on secondary
• Make AMR maxlevel region a function of time
• Experiment with variations to "fiducial parameter values": initial separation, primary spin, accretion onto primary

Lyman-\alpha tests

2x10¹³

2x10¹³, no Lyman-\alpha

2x10¹³, X=0

Conclusion: Lyman-\alpha cooling is related to the problem (perhaps just exacerbating), but not due to the interior of the planet collapsing any more (since there's no cooling in the case X=0 inside the planet).

Bigger box with corrected? BCs

Some …interesting… grid-aligned effects, plus the jump in temperature and pressure of the ambient (HII and ne increase, but that would decrease rather than increase the temperature).