Ieee08 Time In Paraview

Working with Time in ParaView John Biddiscombe CSCS Swiss National Supercomputing Centre

03/06/2008

Contents Data Formats for time-dependent data Overview of GUI controls Animation Controls Connection between GUI and internals Comparative Visualization mode Temporal Pipeline Overview Time Dependent Filters (vtkTemporalXXX) Manipulating Time in Filters (C++ code)… Gotchas & Bugs Future Plans

Data Import

Formats Supported • •

Exodus (used by several US research facilities) vtkPVDReader (vtk XML Collection) •

• •

vtkFileSeriesReader vtkXML * Reader •

• • • •

PolyData, UnstructuredGrid, StructuredGrid, ImageData, RectilinearGrid, Multiblock, Heirarchical etc.

Legacy VTK files with 001, 002, 003 filenames Ensight (case files, ASCII/Binary data) OpenFOAM, SpyPlot, Phasta, STL, MFIX (untested by me) Xdmf (extensible data model format) • •

•

filename.pvd + Filename-00.vtu/vti/vtp etc

XML light data with geometry/time information inside Hdf5 – heavy – big data containing scalars/fields

(CSCS) custom readers include netCDF, H5Part (from CSCS web site)

Vtk (XML based) file format VTK Collection - Easy to create by hand if necessary A General purpose holder for vtk XML files of all types vtu=unstructured, vtp=polydata, vtr=rectilinear, vti=imagedata Each individual file can be binary/text, compressed or not

The VTK Collection is in fact a generic holder for MultiBlock composite datasets which can store time information too. The vtkXMLReader family is responsible for loading this kind of data. User can use vtkXML_xxx_Writer to write N time steps of any kind of data and then add a little XML meta data to describe it. Caveat : pvd time collections are not working well in parallel (NxM split of blocks)

.pvd versus File List • .pvd collection is a true Time compatible holder for data • Time steps can be specified with real time values • Don’t need to be contiguous or equally spaced

• FileSeriesReader : Selecting *.vtu/p/i/etc • • • • • • •

simple but efficient way of loading time series One step per file Can’t specify true time values vtkFileSeriesReader is clever Load one – get one Load *.ext, get time series No C++ changes required to reader

vtkFileSeriesReader : make reader time aware The XML Unstructured Grid reader reads the VTK XML unstructured grid data file format. The standard extension is .vtu. This reader also supports file series.



FileSeriesReader (slide 2) Put details of your existing reader in ExposedProperties <SubProxy> <ExposedProperties>



FileSeriesReader (slide 3) <StringVectorProperty name="FileName" clean_command="RemoveAllFileNames" command="AddFileName" animateable="0" number_of_elements="0" repeat_command="1"> The list of files to be read by the reader. If more than 1 file is specified, the reader will switch to file series mode in which it will pretend that it can support time and provide 1 file per time step.



FileSeriesReader (slide 4) The part which makes it ‘time aware’ <TimeStepsInformationHelper/> Available timestep values.



Old and New Time in ParaView In PV2.x Time was generally animated by changing the TimeStep value of a reader (or filter sometimes). In PV3.x Time is an information variable/object passed down the pipeline which makes it possible for filters to modify time before passing it to their source. For this reason, the GUI sets the time inside the Rendering/Mapping code and not via a TimeStep variable. Some older (custom) readers may still have a TimeStep variable, but this is being phased out and its use is not encouraged – unless you are trying to do something ‘a bit special’ (see later slides).

Reader XML For any reader If you omit this (this is all you need in the XML) Then you won’t see the time information. <TimeStepsInformationHelper/> Available timestep values.



It connects the TIME_STEPS key to the GUI

The old method (xml for reference) Set the current timestep.



And for convenience we usually provided a GetTimestepValues(vector)

C++ Reader : RequestInformation Scan files/data structures and build a list/vector/array of times Set TIME_STEPS for discrete time data (most readers really) outInfo->Set( vtkStreamingDemandDrivenPipeline::TIME_STEPS(), &this->TimeStepValues[0], this->TimeStepValues.size()); Set TIME_RANGE if continuous, but can also be set for discrete double timeRange[2]; timeRange[0] = this->TimeStepValues.front(); timeRange[1] = this->TimeStepValues.back(); outInfo->Set( vtkStreamingDemandDrivenPipeline::TIME_RANGE(), timeRange, 2);

C++ Reader : RequestData Find the correct time step (index into array of step values) if ( outInfo->Has(vtkStreamingDemandDrivenPipeline::UPDATE_TIME_STEPS())) { // usually only one actual step requested double requestedTimeValue = outInfo>Get(vtkStreamingDemandDrivenPipeline::UPDATE_TIME_STEPS())[0]; this->ActualTimeStep = vtkstd::find_if( this->TimeStepValues.begin(), this->TimeStepValues.end(), vtkstd::bind2nd( WithinTolerance( ), requestedTimeValue )) - this->TimeStepValues.begin(); }

C++ Reader : RequestData Time stamp the output DataObject using DATA_TIME_STEPS output->GetInformation()->Set(vtkDataObject::DATA_TIME_STEPS(), &requestedTimeValue, 1);

Nothing bad happens if you don’t – But filters which use the time information won’t get what they need. vtkTemporalPathLineFilter builds lists of data times for each frame and joins the dots. You can read the UPDATE_TIME_STEPS key, but it might not be the same, and is strictly only present during RequestUpdateExtent

GUI Controls Animating with Time

Animation GUI Controls Time value displayed in toolbar – Time step mode – user can change step Sequence mode – user can change time freely VCR style toolbar has play controls (play, step forward/back, jump to start/end) Data with Time support shows the timesteps in the information tab. (Lacks ability to click on timestep and jump to it)

Animation control has settings for time stepping and keyframes Abort button to stop movie generation not obvious

Time Modes in Animation Inspector Snap to TimeSteps When you click ‘play’ the animation will begin at whatever time step you are on and continue until it either reaches the end (or is stopped manually). When stopped the animation will continue from where it left off if restarted. Each frame played represents a single time step of the input data There is no need to set any keyframes or specify any particular property to animate. The Track selection is used to select or create keyframes

Time Modes in Animation Inspector Snap to TimeSteps : * * * Warning * * * When dataset 1 is loaded it may have time values 0, 1, 2, 3, …..N When dataset 2 is loaded it may have time values 0.5, 1.5, 2.5, 3.5, …..M Playing an animation in Snap To TimeSteps mode will traverse ALL KNOWN Timesteps – which means 0, 0.5, 1, 1.5, 2, 2.5 …..max(M,N) This can cause unexpected results! Future time support should allow you to ‘Snap to active pipeline steps’ so that only the times exported for a particular dataset/filter will be traversed. (In development). Fix : Use Sequence mode and ‘lock’ the start and end times to prevent changes

Time Modes in Animation Inspector Sequence Mode The start and end times correspond to the max and min values of time in the data you loaded. If you manually change the start/end times you can ‘Lock’ the new times – this prevents them being reset to the default start/end times if new data is loaded. Animation will now ignore the Time steps present in the data and use interval=(end/start)/(numFrames-1) This is frequently NOT WHAT YOU WANT

Time Modes in Animation Inspector Worked example interval=(end/start)/(numFrames-1) 60.001424 - 50.001424 = 10 Num Frames = 10 We should get our original time steps back (1s per step) In fact we get 10/(numFrames-1) = 1.11111111s/frame Need to use 11 frames to get 0,1,2,3,4,5,6,7,8,9,10 Note : In the some versions of PV3 selecting time values which are not exactly the same as those present will produce no update. It should snap to the nearest time step, but does not. See SnapToTimeSteps for a fix.

Time Modes in Animation Inspector Real Time Animates using the ‘Wall time’ Not always useful for the majority of filters/data sources that we use Useful to animate ‘as fast as we can’ if data was stored with time representing a real clock value and we wish to ‘play back’ data in real time. In this mode not all times may be displayed. Some will be skipped or some may be on screen for ages. Could also be very handy for integrating generators of data (eg sensor equipment) into the gui and displaying the real time update of the sensor (no examples of this yet though)

Comparative visualization

Comparative View Inspector • Allows MxN array of views on the same data • Allows a variable to be changed along the X axis. • Wrong - limitation used to be that time must be between 0.0-1.0 for the view to operate Solution : TemporalShiftScale with 1/N scale factor (or use Normalize flag set if present in CVS)

Same Data at multiple T Uses Time steps of selected object Avoid mismatched time display in same window. FilmStrip uses X Comparative : X&Y Works, but patience required

Comparative viz Summary

• Much easier than • • • • •

Creating multiple windows manually Setting up Shift/Scale Instantiating Pipelines Setting view/display properties Can also use other plot styles.

Time Dependent Algorithms/Filters Pipeline Introduction

Pipeline Introduction The pipeline is demand driven with data flowing downstream Information flowing up and down stream Data Data source

filter(s)

Display/GUI Renderer

UPDATE_TIME_STEPS

Information In PV3 Time is part of the Information flow.

How does it work • How does the pipeline actually fit together • All filters are connected together by Executives which receive information from downstream (and upstream) and decide • Are the data inputs valid • Are the data outputs valid • Is everything else valid • Has anything changed since I last updated

In PV3 Time is passed as information and the executives can LOOP portions of the pipeline over multiple time steps This means that filters can request multiple time steps This enables us to implement Time Dependent Algorithms

Pipeline Looping • vtkTemporalDataSet • When the pipeline is looped to generate multiple time steps, the executive generates a dataset collection • The collection is passed to the temporal algorithm • The algorithm can request any number of time steps • But usually 2 (linear interpolation) • In order to make looping work, some information keys are used internally

Information Keys During Updates, the pipeline makes 4 request passes… DataObject, Information, UpdateExtent, Data

Keys exported by time aware sources TIME_RANGE (continuous), TIME_STEPS (discrete) Time aware reader will declare N steps 0.0, 0.1, 0.2…..etc

Keys used during requests UPDATE_TIME_STEPS (Filter says ‘I want these’) Interpolator says I need times 0.1 and 0.2 for example

Keys set during execution DATA_TIME_STEPS (Source says ‘I made these’) Data generator says I generated times 0.1 and 0.2

Keys used internally by executives REQUIRES_TIME_DOWNSTREAM (looping will be needed) CONTINE_EXECUTING (multiple passes inside algorithm)

Time Dependent Algorithms/Filters Interpolation

(Linear) Interpolation • vtkTemporalInterpolator • Linearly interpolates between 2 time steps • When Time T.5 is requested it requests times T and T+1

• 2 Modes of Operation • Continuous • Discrete

• Continuous Mode • DiscreteTimeStepInterval=0.0 • Filter generates no TIME_STEPS on output – just a TIME_RANGE • GUI can request any time between min/max

• Discrete Mode • DiscreteTimeStepInterval>0 • Filter generates (max-min)/DiscreteTimeStepInterval steps • GUI sees discrete data with new TIME_STEPS values

(Linear) Interpolation Continuous mode Input data (left) has N discrete time steps Output data (right) has no time steps It does report a TIME_RANGE So the GUI knows that any time between min/max can be requested

Interpolation – Continuous mode

Data Courtesy : David Graham, Plymouth UK.

(Linear) Interpolation Discrete mode Example : DiscreteTimeStepInterval=0.01 New time steps are generated and the output data ‘looks’ like it has 10x as many time steps. Note that no interpolation of the data has been performed yet, only when something is actually rendered/requested will the interpolation take place.

Interpolation – Discrete mode

(Linear) Interpolation •

What can be interpolated • Any Dataset which does not change topology between time steps • Point positions are interpolated • Connectivity (cells are copied) • Point Data is interpolated • Cell Data is interpolated • ImageData/PolyData/Rectilinear/Unstructured • All can be interpolated if cell connectivity and number of cells remains the same • MultiBlock/MultiGroup/Heirarchical/AMR • If the tree structure remains the same between time steps and the individual leaves satisfy above conditions • Other interpolation methods could be added

Fixing Problems with the interpolator • Discrete mode • Given an input with N time steps, but a few are accidentally missing (lost of never generated) • Set DiscreteTimeStep Interval to the original time step size (say 0.1 or 0.01 etc) • Data saved every Nth frame to save IO time. • Output now seems to be exactly the same as the input – except that the steps that were missing from the input are recreated using interpolation when requested on the output • Animate using ‘Snap To TimeSteps’ mode • Animation will look like intended original

Problem Fixing Example (Interpolation) Before

After

Missing data, low sampling Problems for particle tracer

Lovely

Temporal DataSet Cache

Interpolation – Use a cache If animation from 0->t is performed using a TemporalInterpolator, the interpolator will request 2 time steps each time it updates Interpolating at 0.1 spacing between 2 steps causes Step 0 + Step 1 : Output Step 0.1 Step 0 + Step 1 : Output Step 0.2 Each step N times! Step 0 + Step 1 : Output Step 0.3

Use a TemporalDataSetCache to store 2 timesteps and prevent this repeated re-execution of the pipeline

Data source (time aware)

TemporalDataSet Cache

Interpolator

Display/GUI Renderer

Branching Time – Use a cache It’s not just the filter delivering the data at T, but the whole upstream pipeline that is protected… Consider the case where a pipeline branches and different T values are requested. The cache prevents updates from one section propagating too far upstream and forcing the other branch to be re-executed needlessly. Simple filter(s) (no special time requirements)

Data source (time aware)

Temporal DataSetCache

TemporalFilter (requires multiple time steps, e.g.. Particle tracer)

TemporalShiftScale (modifies time values)

Display/GUI Renderer

Branching Time – Caution in the GUI When different values to T are visible/manipulated Only display the leaf nodes of the pipeline Rendering intermediate portions can trigger unwanted/confusing updates Display/GUI Renderer

Simple filter(s) (no special time requirements)

UPDATE_TIME_STEPS

UPDATE_TIME_STEPS

Data source (time aware)

Temporal DataSetCache

TemporalFilter (requires multiple time steps, e.g.. Particle tracer)

TemporalShiftScale (modifies time values)

Display/GUI Renderer

Multiple Inputs

Related Cautionary Note #1

Multiple Inputs • Time dependent filters with multiple inputs • Trigger updates on all inputs • Particle Tracer is one example Avoid this. Save Seeds if possible

Data source (time aware)

UPDATE_TIME_STEPS

Temporal DataSetCache

UPDATE_TIME_STEPS

Slice – Seed Points (for example)

Particle Tracer

Display/GUI Renderer

Temporal Snap to TimeSteps

SnapToTimeSteps Example

Data source (time aware)

TemporalDataSet Cache

Interpolator

Time To Text

Snap To TimeStep

Time To Text

Display/GUI Renderer

Time Display Side note Source : Time Source Takes time from the UPDATE_TIME_STEPS Actually it takes it directly from the view which is responsible for setting the key on the filter/output

Filter->Temporal->Annotate Time Takes its time from DATA_TIME_STEPS Which is the actual time step output from the filter The two might not always be the same (snap to timestep for example)

Time Shifting

Temporal Shift Scale Changes time between the input and output of a filter Can be used to compare the same data at 2 different values of T (though the comparative viz view is easier to use in some ways) Invaluable for combining different datasets with different T values See example pipeline below Tout = PreShift + Tin*Scale + PostShift NB. +ve PostShift produces +ve delay of output time seems the wrong way around but time 0 is now time N, so it takes N seconds to get to it

Temporal Shift Scale Pre-Shift = -50.0014 Scale = 1/10 Post-Shift = 3

Before 50-0014 - 60.0014

After 3.0 – 4.0

Developer Tip : Hiding bad time data In RequestData if (outInfo->Has(vtkStreamingDemandDrivenPipeline::UPDATE_TIME_STEPS())) { double requestedTimeValue = outInfo-> Get(vtkStreamingDemandDrivenPipeline::UPDATE_TIME_STEPS())[0]; if (requestedTimeValueTimeStepValues.front() || requestedTimeValue>this->TimeStepValues.back()) { this->TimeOutOfRange = 1; } output->GetInformation()->Set(vtkDataObject::DATA_TIME_STEPS(), &requestedTimeValue, 1); } … if (this->TimeOutOfRange && this->MaskOutOfTimeRangeOutput) { // don't do anything, just return success return 1; }

Developer Tip : Normalize Time In RequestInformation if (this->NormalizeToUnitTime && this->TimeStepValues.size()>0) { double t1 = this->TimeStepValues.front(); double t2 = this->TimeStepValues.back(); double t3 = (t2-t1)>0 ? (1.0/(t2-t1)) : 1.0; for (unsigned int i=0; iTimeStepValues.size(); i++) { double t = this->TimeStepValues[i]; this->TimeStepValues[i] = t3*(t-t1); } }

Then everything comes out {0,1} Feature might be available in ShiftScale. Regional Variations 

Comparative Vis (using Time shift) • Time shift + Interpolation + Trails • 3 time dependent features in 1 go

Temporal Shift Scale Periodic Mode : Turns N time steps spanning t time units into N*T steps spanning T*t units Periodic End Correction If simulation steps are {0,1,2…N-1} and {N-1} is identical to {0} Periodic end correction OFF = {0,1,2…N-1,1,2,3..N-1} Otherwise {0,1,2…N-1,1,2,3..N-1} causes duplicated step If simulation steps are {0,1,2…N-1} and {N-1} is different from {0} Periodic end correction On Otherwise {0,1,2…N-1,0,2,3..N-1} causes duplicated step

Periodic example 2 Separate Datasets one 40 steps one 80 steps Duration 1s (approx) Need 5+hours 80*3600*5 = 1.44E6 Tried 10,000 Periods in ParaView (1 day?) Need a better model!

Particle Tracer Pipeline (one nasty example)

Temporal Cache

DataSet 1

PathLines

Particle Tracer

DataSet 2

Shift Scale

Interpolate

Temporal Cache

Seeds 1 GUI Seeds 2

Periodic Time + Resampling + Time Shift

Side Note : XML for Temporal Support <SourceProxy name="TemporalSnapToTimeStep" class="vtkTemporalSnapToTimeStep" label="Temporal Snap-to-Time-Step"> Will change in future : no way of saying : DataType : PolyData : Temporal_required

Particle Tracer TemporalStreamTracer

Particle Tracing All Inputs must have same vector field Multiple Multiblock Inputs supported (Temporal Required) Meshes can be dynamic

Parallel operation If input dataset has N time steps Particle tracer will generate N-1 output steps Output step 0 time value corresponds to input step {0,1} time Nothing can be generated until 1 time period has passed

Seed points are supplied as separate inputs Use SetTimeStep to combat odd time input combinations

Request Update Extent for (int i=0; iGetInformationObject(i); // our output timestep T is timestep T+1 in the source // so output inputTimeSteps[T], inputTimeSteps[T+1] inInfo-> Set(vtkStreamingDemandDrivenPipeline::UPDATE_TIME_STEPS(), &this->InputTimeValues[this->ActualTimeStep], 2); vtkDebugMacro(<< "requested 2 time values : " << this->InputTimeValues[this->ActualTimeStep] << " " << this->InputTimeValues[this->ActualTimeStep+1]); }

Particle Tracing (setup example) Source (seedpoints) LineSource 100 points

Time Step resolution Leave as 1 – this can be used to scale time if the input time was not stored correctly (e.g. it goes 0…N but should be 0.1 0.2 0.3 etc)

Time Step (output index) Set to zero initially – this will request steps, (0,1) from the input and the time actually corresponds to T=1 (but it will be the 0th step on the output)

Particle Tracing Force re-injection every 1 or 2 time steps Particles will be injected at the seed points every N steps

Input vectors Usually a velocity field, but could be another

Initial Integration step Used by Runge-Kutta integration to make incremental velocity field approximations Can be smaller but 0.25 usually OK

Static Seeds - Static Mesh Force optimizations GEOMETRY_NOT_MODIFIED (work in progress)

Particle Tracing Ignore pipeline time This is very important – the particle tracer generates less output steps than the input (-1) and so we cannot (yet) use the default animation controls due to bug mentioned earlier. NB. This works now, but the comment is useful We therefore instruct the particle tracer to ignore pipeline time and instead animate the Timestep property from 0 to N-2 (if the input has N steps, the Particle tracer has N-1 steps output, so we go from 0 to N-2 to get the correct number)

Particle filename (not shown) An option to store particles to disk will be included at a future date.

Side Note : Ignore Pipeline Time if (this->IgnorePipelineTime) { if (this->TimeStepOutputTimeValues.size()) { requestedTimeValue = this->OutputTimeValues[this->TimeStep]; } else { requestedTimeValue = this->OutputTimeValues.back(); } this->ActualTimeStep = this->TimeStep; vtkDebugMacro(<< "SetTimeStep

: requestedTimeValue “

<< requestedTimeValue << " ActualTimeStep " << this->ActualTimeStep);

} else { the usual stuff to get requested time }

Particle Tracing To generate animation of particles Use Sequence Mode Animation Use N-1 Frames (because there will be N-1 steps generated – this data had 51 steps on the input indexed as 0-50), so we use 50 frames for output Create a Keyframe on ParticleTracer TimeStep. Set Keyframe index 0 Time to start time = 0.0 here Value to TimeStep 0

Set Keyframe index 1 Time to end time = 7.99 in this example Value to 49 (last index = 51-2)

Animation can now be played/saved as movie

Francis Turbine

PathLineFilter

Pathline Filter • Connects to output of Particle tracer • Can be used on any dataset with points (not just particles) • Now supports selection of subset • But needs to be updated to use vtkSelection type • Uses Ids from selection dataset to choose points for line construction

• Is not strictly a vtkTemporalXXX filter because it does not loop time • Listents to DATE_TIME_STEPS on input and builds list from that

• See other slides for example of usage

Plotting Data over Time

Temporal Statistics

03/06/2008

Temporal Statistics Computes Mean Min Max Standard Deviation More can be added Can take time!

Selection+Time

03/06/2008

Selection over time • The same procedure can be used to query a particle over all available time • Extract Selection over time filter builds a spreadsheet of values for the selected IDs over all time steps • NB: Can’t set a sub region of time using the animation controls yet, but this feature will be coming

• Selection tutorial should cover possible types • Cell/Point/Global IDs can be used here

03/06/2008

Plot Selection over time

Which Point to display Which field(s)

03/06/2008

Plot Selection over Time

Animations with Non simple time

03/06/2008

How to animate a slice on a time-dependent dataset Problem When I try to animate, time changes as well as the slice I can’t set time start/end to constants and then animate slice alone ParaView hangs because of zero increment Trying to fool it with tiny increments is not safe. Show animation viewer and double click time track

Now animate the slice or other property using sequence mode And keyframes to suit.

Demo 1 • How to animate a slice, and then animate the data?

Double click on these tracks

Setup a slice keyframe • • • • •

Create Slice offset keyframe Animate a slice 50 frames one way 50 frames back again Note that extents are +/- 1.5 = 3 Check values when adding keyframes

Change the Time track itself • Double click Time track • • • •

Time is variable It has its own keyframes Fix time from frames 1-50 Animate time from 50 to 50+60

Sequence mode coordination • Make sure the Animation inspector has the same information as the animation viewer • And….the keyframes • When you add/remove them, or adjust the start/end time, they get rescaled • Could do with a ‘locking keyframe time’ option here

• The time here should be thought of as frames

Demo 2 Fixed, forward and reverse time in one animation

Animation (frame) time

View (data) time

We need tracks per dataset – to combine different sources

Animate Camera from Python phi = range(0, 360) for i in phi: view.CameraPosition = [2400*math.cos(i*math.pi*2.0/360), 2400*math.sin(i*math.pi*2.0/360), 300] view.StillRender() imgfile = "/path/to/snapshot.%03d.png" % i view.WriteImage(imgfile, "vtkPNGWriter", 1) To be Continued Python versions of TimeAnimationCue, Scene, View Exercise for the reader (Utkarsh) : recreate fancy time animations in Python

Conclusion Few Time dependent filters in ParaView so far But … Already powerful tool for animations/analysis Animation improvements and time step handling will make it hard to beat in terms of features and flexibility