TotalView delivers unbeatable platform, language, and compiler support for C, C++, and mixed-language Python – C/C++ applications.

GET LATEST RELEASE    TRY FOR FREE

TotalView 2022.4

AMD GPU Debugging Support 

TotalView 2022.4 adds support for debugging AMD GPUs in addition to its ability to debug NVIDIA GPUs. TotalView supports versions 5.1 to 5.2 of the ROCm software stack for GPU programming, including support for debugging HIP (Heterogeneous Interface for Portability) and MPI code running on AMD MI50, MI100, and MI200 series of GPUs.  TotalView supports the following debugging features on AMD GPUs: 

  • Process launch, attach, and detach 
  • Viewing scalar, vector, general, and special AMD GPU registers 
  • Instruction disassembly 
  • Breakpoint creation and deletion on AMD GPU code 
  • Single-stepping and fast smart-stepping 
  • Stack unwinding, including inlined functions 
  • Navigation controls for changing the logical workgroup / work-item focus or physical agent, queue, dispatch, wave, and lane focus 
  • Variable display with the ROCm 5.1+ compilers 
  • Data watchpoints on global memory variables 

Preparing your program for debugging is simple. Use the “-O0 -g” flags with the ROCm 5.1+ compilers to generate the necessary debug symbols for TotalView. 

To debug a program on AMD GPUs, use the “-rocm” flag when you start TotalView, for example: 

totalview -rocm a.out

Once started, simply set breakpoints in the HIP kernel code by selecting the line number where you would like to stop, then begin running your program.  TotalView will automatically plant breakpoints in the HIP code once it is loaded onto the GPU.  Use TotalView’s stepping commands to control execution, and dive on variables to view their values in your GPU code.  Reverse debugging will not be supported for code running on the AMD GPUs but is planned to work with reverse debug code running on the CPU in a combined CPU/GPU debugging session.  There currently is an issue while reverse debugging when the -rocm switch is used and debugging on the GPU.  We are working to resolve this issue for an upcoming release of TotalView. 

To learn more about specific AMD GPU debugging characteristics, see help for the drocm command in TotalView’s Command Line Interface (dhelp drocm). 

Array View for Array Debugging and Visualization 

A new Array View is available in TotalView 2022.4 which enables focused debugging of array data.  Through the Array View, you can examine statistical information about an array and visualize array data. 

To bring up the Array View, either activate it from the Window > Views > Array View menu item or select Add to Array View from the context menu in the Local Variables view or Data View when selecting an array variable. 

TotalView Release 2022.4 -1

Figure 1 - Add to Array View menu item 

The toolbar area of the Array View displays the process and thread for the array data being displayed.  The array expression is shown along with the slice of the array to debug.  In Figure 2, array expression floatArray[:] is shown and the [:] specifies to debug the whole array.  A portion of the array can be specified by providing the beginning and ending indices, such as [10:15]. 

The Array View displays array statistics information initially.  In previous versions of TotalView this was separate functionality available in the Array Statistics View.   

TotalView Release 2022.4 -2

Figure 2 - Array Statistics 

A histogram plot of the array data is shown when the bar plot icon is clicked in the toolbar.  Adjust the number of bins used in the plot using the field in the plot. 

TotalView Release 2022.4 -3

Figure 3 - Visualize Array in Histogram Plo

A line plot of the array data is also available when the line plot icon is selected in the toolbar. 

TotalView Release 2022.4 -4

Figure 4 - Visualize Array in Line Plot 

Watch for more array debugging capabilities to be added to the Array View in coming versions of TotalView! 

TotalView Startup Performance Enhancements 

For TotalView 2022.4, further startup performance improvements optimize how the debugger looks up symbols and entry point values in the runtime linker.  This can have a big impact on debugger startup performance for applications built with many shared libraries. 

DEBUGGING TIP:  A further startup performance optimization customers should consider using is to instruct TotalView to defer processing of shared libraries until later in the debug session.  By adding the TotalView command line option -no_dlopen_always_recalculate the debugger will delay processing shared library symbols until they are needed later in the debug session.  Learn more about tuning TotalView’s parallel debugging performance in the “Scalability in HPC Computing Environments” section of the TotalView User Guide, Part III Parallel Debugging chapter. 

TotalView Base Compilation System and Dependency Changes 

TotalView 2022.4 changes the base compilation for the following systems: 

Linux x86-64 

  • Base build system was changed from CentOS 7.0 to CentOS 7.9, resulting in a base OS of CentOS/RHEL 7.9 and above.  Ubuntu 18.04 and 20.04 are also supported.  
     
    Note that TotalView’s user interface requires have libxkbcommon and libXcomposite libraries, typically installed as part of a desktop graphical environment. 

Linux PowerLE 

  • Base build system was changed from Ubuntu 14.04 to CentOS 7.5.  The new base supported system changes to CentOS/RHEL 7.5 and above.  Ubuntu 18.04 and 20.04 are also supported.  

Note that TotalView’s user interface requires have libxkbcommon and libXcomposite libraries, typically installed as part of a desktop graphical environment. 

macOS 

  • Base build system was changed from macOS High Sierra to macOS Big Sur.  TotalView’s base compilation support stays the same with support for macOS Big Sur and Monterey. 

Bug fixes and performance improvements 

Numerous bug fixes and performance improvements have been addressed, including performance improvements for data transformations. 

View Full Release Notes

For more details please refer to the TotalView release notes, where you can see more details and specs on how to enable various features, what to expect from updates, and info about any features on hold.

Release Notes  Latest Release

TotalView 2022.3 

AMD GPU Debugging Preliminary Support 

TotalView 2022.3 expands its GPU debugging capabilities with support for debugging AMD GPUs, in addition to its ability to debug NVIDIA GPUs.  TotalView supports versions 5.1 and 5.2 of the ROCm software stack for GPU programming, including support for debugging HIP (Heterogeneous Interface for Portability) and MPI code running on AMD MI50, MI100, and MI200 series of GPUs.  With this release, TotalView supports the following debugging features on AMD GPUs: 

  • Process launch, attach, and detach 

  • Viewing scalar, vector, general, and special AMD GPU registers 

  • Instruction disassembly 

  • Breakpoint creation and deletion on AMD GPU code 

  • Single-stepping and fast smart-stepping 

  • Stack unwinding, including inlined functions 

  • Navigation controls for changing the logical workgroup / work-item focus or physical agent, queue, dispatch, wave, and lane focus 

  • Variable display with the ROCm 5.1+ compilers 

  • Data watchpoints on global memory variables 

Preparing your program for debugging is simple. Use the “-O0 -g” flags with the ROCm 5.1+ compilers to generate the necessary debug symbols for TotalView. 

To debug a program on AMD GPUs, use the “-rocm” flag when you start TotalView, for example: 

totalview -rocm a.out 

Once started, simply set breakpoints in the HIP kernel code by selecting the line number where you would like to stop, then begin running your program.  TotalView will automatically plant breakpoints in the HIP code once it is loaded onto the GPU.  Use TotalView’s stepping commands to control execution, and dive on variables to view their values in your GPU code.  Reverse debugging will not be supported for code running on the AMD GPUs but is planned to work with reverse debug code running on the CPU in a combined CPU/GPU debugging session.  There currently is an issue while reverse debugging when the -rocm switch is used and debugging on the GPU.  We are working to resolve this issue for the next release of TotalView. 

To learn more about specific AMD GPU debugging characteristics, see help for the drocm command in TotalView’s Command Line Interface (dhelp drocm). 

Watch for more AMD GPU debugging features in our 2022.4 release in November, including watchpoint support!  If you are attending Supercomputing 2022 this year, make sure to attend the “Debugging the Toughest Challenges with NVIDIA and AMD GPUs” Exhibitor Forum talk for more details about debugging AMD and NVIDIA GPUs with TotalView. 

Dive Stack Expansion State 

TotalView 2022.3 expands on the Dive Stack functionality introduced in version 2022.2.  The Dive Stack enables developers to easily explore data by traversing/diving into data structures and back out.  Version 2022.3 adds functionality to maintain the expansion and scroll state before the dive so that an undive operation restores the Data View as it was. 

NVIDIA OpenACC Support 

TotalView 2022.3 adds official support for debugging NVIDIA OpenACC compiled applications. 

Bug Fixes and Performance Improvements

Numerous bug fixes and performance improvements have been addressed, including performance improvements for data transformations. 

View Full Release Notes

For more details please refer to the TotalView release notes, where you can see more details and specs on how to enable various features, what to expect from updates, and info about any features on hold.

Watch the What's New Webinar

Bill Burns, Senior Director of Product Engineering will show you all the new features. Watch now >>

 release notes   Get Latest Release

TotalView 2022.2

Data Debugging Workflow Improvements With Dive Stacks 

The latest release incorporates a popular feature from TotalView’s Classic UI, allowing users to navigate a Dive Stack of expressions when using data structures in Data View. Navigating up and down the stack allows you to move more easily through the data and streamlines workflows.

Fedora 35 and Ubuntu 22.04 Preliminary Support

There have been many Fedora 35 and Ubuntu 22.04 system level changes that impact debuggers. Our latest release offers enhancements and fixes to support debugging on these platforms. Testing for full support in the 2022.3 release is ongoing.

Solaris Base System Change and Query About System Support

Due to infrastructure changes, we’ve moved from Solaris 11u3 to 11u4.  We are interested in hearing if customers still actively use Solaris for development and debugging. Please send an email to Bill Burns, TotalView product manager, at bburns@perforce.com if Solaris is still an important platform for you.

TotalView CLI dprint Command Timeout Option

The latest release supports the ability to define a timeout via CLI dprint commands. Simply add “-timeout nSeconds” to define the number of maximum number of seconds the command should run.

TotalView's Security Updates

TotalView 2022.2 upgrades its use of zlib to the latest version to eliminate a CVE security vulnerability.

TotalView Startup Performance Improvements

TotalView’s processing of debug symbols during startup have resulted in a 42x performance improvement and reduced processing time from minutes to seconds, in one use case, as compared to the 2021.4 version. In cases where an executable or shared library contains thousands of subroutines, this new processing logic greatly improves efficiency.

Utilize Library Build ID's to Improve TotalView Processing Performance

TotalView 2022.2 incorporates advancements to processing of libraries in parallel jobs.  Checksums are used to find similar libraries loaded by parallel processes across a cluster of nodes to determine which libraries are identical. Checksumming can incur a performance penalty as each library is checked. TotalView 2022.2 attempts to extract the Build ID from shared library image files, thereby improving overall processing.

New User Interface Improvements

TotalView’s new user interface is available by default for new users and is activated through the Display Preferences panel or using the -newUI command line option for existing TotalView users.

Share Your Feedback

If you have any feedback about new features, wish to make requests for new or missing features, or want to report any problems please send an email to tv-beta@perforce.com.

View Full Release Notes

For more details please refer to the TotalView release notes, where you can see more details and specs on how to enable various features, what to expect from updates, and info about any features on hold.

View Release Notes   Get Latest Release

TotalView 2022.1

Memory Debugging Block Notify

A particularly challenging memory debugging situation occurs when a specific memory block is free but a program accesses the memory later through a dangling pointer reference. Because the memory may or may not still contain valid data, sporadic crashes and data problems can occur. TotalView 2022.1 adds the ability to watch a specific allocated memory block and raise an event when it gets freed or deleted, effectively mitigating this debugging challenge.

MacOS Monterey Support

TotalView 2022.1 adds support for Apple’s latest version of macOS 12 Monterey.

CUDA 11.4 and 11.5 Support

TotalView 2022.1 adds official debugging support for CUDA 11.4 and 11.5.

Bug Fixes and Performance Improvements

Numerous bug fixes and performance improvements have been made to TotalView.  See the release notes for the full list of bug fixes.

Share Your Feedback

If you have any feedback about new features, wish to make requests for new or missing features, or want to report any problems please send an email to tv-beta@perforce.com.

View Full Release Notes

For more details please refer to the TotalView release notes, where you can see more details and specs on how to enable various features, what to expect from updates, and info about any features on hold.

Watch the What's New Webinar

Bill Burns, Senior Director of Product Engineering will show you all the new features. Watch now >>

View Release NotesGet Latest Release