# Cornerstone3D Documentation > Cornerstone3D is a modern, high-performance JavaScript library for medical imaging, designed for building web-based medical imaging applications. It provides tools for rendering, manipulating, and analyzing medical images in various formats including DICOM. This file contains the complete documentation for Cornerstone3D, concatenated for easy reference and searching. Each section is clearly marked with its source URL. # Root Documentation ## Core Concepts Source: https://cornerstonejs.org/docs/llm/concepts #### Rendering _index.html_ ```html ``` _app.js_ ```js import { RenderingEngine, // class ORIENTATION, // constant ViewportType, // enum } from 'vtkjs-viewport'; // RENDER const renderingEngine = new RenderingEngine('ExampleRenderingEngineID'); const volumeId = 'VOLUME_ID '; const viewports = []; const viewport = { sceneUID, viewportId: 'viewportUID_0', type: ViewportType.ORTHOGRAPHIC, canvas: document.querySelector('.target-canvas'), defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, background: [Math.random(), Math.random(), Math.random()], }, }; // Kick-off rendering viewports.push(viewport); renderingEngine.setViewports(viewports); // Render backgrounds renderingEngine.render(); // Create and load our image volume // See: `./examples/helpers/getImageIdsAndCacheMetadata.js` for inspiration const imageIds = [ 'wadors:https://wadoRsRoot.com/studies/studyInstanceUID/series/SeriesInstanceUID/instances/SOPInstanceUID/frames/1', 'wadors:https://wadoRsRoot.com/studies/studyInstanceUID/series/SeriesInstanceUID/instances/SOPInstanceUID/frames/2', 'wadors:https://wadoRsRoot.com/studies/studyInstanceUID/series/SeriesInstanceUID/instances/SOPInstanceUID/frames/3', ]; imageCache.makeAndCacheImageVolume(imageIds, volumeId); imageCache.loadVolume(volumeId, (event) => { if (event.framesProcessed === event.numFrames) { console.log('done loading!'); } }); // Tie scene to one or more image volumes const scene = renderingEngine.getScene(sceneUID); scene.setVolumes([ { volumeId, callback: ({ volumeActor, volumeId }) => { // Where you might setup a transfer function or PET colormap console.log('volume loaded!'); }, }, ]); const viewport = scene.getViewport(viewports[0].viewportId); // This will initialise volumes in GPU memory renderingEngine.render(); ``` For the most part, updating is as simple as using: - `RenderingEngine.setViewports` and - `Scene.setVolumes` If you're using clientside routing and/or need to clean up resources more aggressively, most constructs have a `.destroy` method. For example: ```js renderingEngine.destroy(); ``` #### Tools A tool is an uninstantiated class that implements at least the `BaseTool` interface. Tools can be configured via their constructor. To use a tool, one must: A tool is an uninstantiated class that implements at least the `BaseTool` interface. Tools can be configured via their constructor. To use a tool, one must: A tool is an uninstantiated class that implements at least the `BaseTool` interface. Tools can be configured via their constructor. To use a tool, one must: A tool is an uninstantiated class that implements at least the `BaseTool` interface. Tools can be configured via their constructor. To use a tool, one must: - Add the uninstantiated tool using the library's top level `addTool` function - Add that same tool, by name, to a ToolGroup The tool's behavior is then dependent on which rendering engines, scenes, and viewports are associated with its Tool Group; as well as the tool's current mode. #### Adding Tools The @Tools library comes packaged with several common tools. All implement either the `BaseTool` or `AnnotationTool`. Adding a tool makes it available to ToolGroups. A high level `.removeTool` also exists. ```js import * as csTools3d from '@cornerstonejs/tools'; // Add uninstantiated tool classes to the library // These will be used to initialize tool instances when we explicitly add each // tool to one or more tool groups const { PanTool, StackScrollMouseWheelTool, ZoomTool, LengthTool } = csTools3d; csTools3d.addTool(PanTool); csTools3d.addTool(StackScrollMouseWheelTool); csTools3d.addTool(ZoomTool); csTools3d.addTool(LengthTool); ``` #### Tool Group Manager Tool Groups are a way to share tool configuration, state, and modes across a set of `RengeringEngine`s, `Scene`s, and/or `Viewport`s. Tool Groups are managed by a Tool Group Manager. Tool Group Managers are used to create, search for, and destroy Tool Groups. ```js import { ToolGroupManager } from '@cornerstonejs/tools'; import { ctVolumeId } from './constants'; const toolGroupId = 'TOOL_GROUP_ID'; const sceneToolGroup = ToolGroupManager.createToolGroup(TOOL_GROUP_ID); // Add tools to ToolGroup sceneToolGroup.addTool(PanTool.toolName); sceneToolGroup.addTool(ZoomTool.toolName); sceneToolGroup.addTool(StackScrollMouseWheelTool.toolName); sceneToolGroup.addTool(LengthTool.toolName, { configuration: { volumeId: ctVolumeId }, }); ``` #### Tool Modes Tools can be in one of four modes. Each mode impacts how the tool responds to interactions. Those modes are:
Tool Mode Description
Active
  • Tools with active bindings will respond to interactions
  • If the tool is an annotation tool, click events not over existing annotations will create a new annotation.
Passive (default)
  • If the tool is an annotation tool, if it's handle or line is selected, it can be moved and repositioned.
Enabled
  • The tool will render, but cannot be interacted with.
Disabled
  • The tool will not render. No interaction is possible.
_NOTE:_ - There should never be two active tools with the same binding ```js // Set the ToolGroup's ToolMode for each tool // Possible modes include: 'Active', 'Passive', 'Enabled', 'Disabled' sceneToolGroup.setToolActive(StackScrollMouseWheelTool.toolName); sceneToolGroup.setToolActive(LengthTool.toolName, { bindings: [{ mouseButton: MouseBindings.Primary }], }); sceneToolGroup.setToolActive(PanTool.toolName, { bindings: [{ mouseButton: MouseBindings.Auxiliary }], }); sceneToolGroup.setToolActive(ZoomTool.toolName, { bindings: [{ mouseButton: MouseBindings.Secondary }], }); ``` #### Synchronizers The SynchronizerManager exposes similar API to that of the ToolGroupManager. A created Synchronizer has methods like `addTarget`, `addSource`, `add` (which adds the viewport as a "source" and a "target"), and equivelant `remove*` methods. A synchronizer works by listening for a specified event to be raised on any `source`. If detected, the callback function is called once for each `target`. The idea being that changes to a `source` should be synchronized across `target`s. Synchronizers will self-remove sources/targets if the viewport becomes disabled. Synchronizers also expose a `disabled` flag that can be used to temporarily prevent synchronization. ```js import { Events as RENDERING_EVENTS } from 'vtkjs-viewport'; import { SynchronizerManager } from '@cornerstonejs/tools'; const cameraPositionSyncrhonizer = SynchronizerManager.createSynchronizer( synchronizerName, RENDERING_EVENTS.CAMERA_MODIFIED, ( synchronizerInstance, sourceViewport, targetViewport, cameraModifiedEvent ) => { // Synchronization logic should go here } ); // Add viewports to synchronize const firstViewport = { renderingEngineId, sceneUID, viewportId }; const secondViewport = { /* */ }; sync.add(firstViewport); sync.add(secondViewport); ``` #### Next steps For next steps, you can: - [Check out the Usage documentation](#) - [Explore our example application's source code](#) --- ## Examples Source: https://cornerstonejs.org/docs/llm/examples #### Basic usage | Example | Description | | ------- | ----------- | | [Basic Stack Viewport Usage](pathname:///live-examples/stackBasic.html) | Displays a single image in a Stack Viewport. | | [Stack Viewport API](pathname:///live-examples/stackAPI.html) | Demonstrates how to interact with a Stack viewport (e.g. Set VOI Range, Next/Previous Images, Flip H/V, Rotate, Invert, Zoom/Pan, Reset) | | [Stack Viewport Positioning](pathname:///live-examples/stackPosition.html) | Demonstrates basic positioning of the viewport image using display area and flip/rotation | | [Stack Sigmoid LUT](pathname:///live-examples/stackVoiSigmoid.html) | Demonstrates the Sigmoid LUT Function instead of Linear | | [Stack Viewport Events](pathname:///live-examples/stackEvents.html) | Demonstrates the Events that are fired during interaction with a Stack Viewport | | [Stack Viewport Canvas-to-World](pathname:///live-examples/stackCanvasToWorld.html) | Demonstrates how to obtain the coordinates in the 3D world from a coordinate on the canvas. | | [Basic Volume Viewport Usage](pathname:///live-examples/volumeBasic.html) | Displays a set of DICOM images in a Volume Viewport. | | [Volume Viewport API](pathname:///live-examples/volumeAPI.html) | Demonstrates how to interact with a Volume viewport (e.g. Set VOI Range, Change Camera Position / Orientation, Change Slab Thickness, Flip H/V, Rotate, Invert, Zoom/Pan, Reset) | | [Volume Sigmoid LUT](pathname:///live-examples/volumeVoiSigmoid.html) | Demonstrates the Sigmoid LUT Function instead of Linear | | [3D Volume Rendering](pathname:///live-examples/volumeViewport3D.html) | Demonstrates how to 3D render a volume and apply a preset | | [Volume Viewport Events](pathname:///live-examples/volumeEvents.html) | Demonstrates the Events that are fired during interaction with a Volume Viewport | | [Multiple Volumes in a Volume Viewport](pathname:///live-examples/multiVolumeAPI.html) | Demonstrates how to interact with a Volume viewport when using multiple volumes (e.g. for PET/CT fusion). | | [Multiple Volume Canvas-to-World](pathname:///live-examples/multiVolumeCanvasToWorld.html) | Demonstrates how to use the canvasToWorld API to find the intensity value of each volume on mouse hover | | [Poly Data Actor in a Volume Viewport](pathname:///live-examples/polyDataActorAPI.html) | Demonstrates how to render poly data with a Volume viewport | | [Legacy DICOMweb (WADO-URI) Support](pathname:///live-examples/wadouri.html) | Demonstrates how to support retrieval of entire Part 10 DICOM files directly via URL | | [Basic Volume using streaming WADOURI](pathname:///live-examples/volumeBasicWadoUri.html) | Demonstrates how to displays a DICOM series (via URL) in a Volume viewport | | [Load Mesh of PLY, OBJ, STL or VTP format](pathname:///live-examples/meshLoader.html) | Demonstrates how to load a mesh in a volume3D viewport | | [Load Web images of PNG or JPG format](pathname:///live-examples/webLoader.html) | Demonstrates how render web images in a stack viewport | | [Load a dynamic 4D data](pathname:///live-examples/dynamicCINETool.html) | Demonstrates how you can render 4D data with cornerstone 3d | | [Render To Canvas](pathname:///live-examples/renderToCanvas.html) | Demonstrates how to use the api to render to a canvas directly | | [Change the colormap and adjusting the opacity](pathname:///live-examples/changeColorMap.html) | Demonstrate how to interact with a fusion viewport, specifically by changing the colormap and adjusting the opacity. | | [Prioritizing Slices during Volume Loading](pathname:///live-examples/volumePriorityLoading.html) | Demonstrates how to customize the slice loading order using the streaming-image volume loader | | [Programmatic Pan/Zoom](pathname:///live-examples/programaticPanZoom.html) | Demonstrates how to programmatically pan/zoom a stack viewport. It can be used for setting initial display area and presentation state. | | [DICOM P10 from the local file system](pathname:///live-examples/local.html) | Provides an interface to load a DICOM P10 image from your local file system to the Cornerstone3D | | [DICOM P10 with annotation and CPU choice](pathname:///live-examples/advancedLocal.html) | Annotation tools with drag and drop and CPU/GPU choice | | [Stack viewport default properties](pathname:///live-examples/stackProperties.html) | Demonstrates how you can set per image properties for a stack viewport that acts as default values for that specific image | | [Volume Slab Scroll](pathname:///live-examples/volumeSlabScroll.html) | Demonstrates how to use the slab scroll tool to scroll through a volume | | [Resize viewport and change aspect ratio](pathname:///live-examples/resize.html) | Resize the viewport and allow various aspect ratios/conditions | | [Apply view reference and/or presentation parameters](pathname:///live-examples/viewReferencePresentation.html) | Demonstrates how to apply various view/reference presentation parameters. | | [Viewport Projection Service](pathname:///live-examples/viewportProjection.html) | Demonstrates projection snapshots and presentation writes for Planar Next and Volume 3D Next viewports. | | [Stack to Volume Viewport](pathname:///live-examples/stackToVolume.html) | Demonstrates how to convert a Stack Viewport to a Volume Viewport | | [Custom Web Worker Function](pathname:///live-examples/webWorker.html) | Demonstrates how to use the web worker manager to register and execute custom web worker functions off the main thread | | [6x6 Grid with ContextPoolRenderingEngine](pathname:///live-examples/contextpoolrenderingengine.html) | Displays a 6x6 grid of viewports using ContextPoolRenderingEngine for better performance with large viewport counts | | [PET-CT Multi-Monitor Layout](pathname:///live-examples/ptctmultimonitor.html) | Demonstrates how to create a multi-monitor layout with PET-CT fusion using the ContextPoolRenderingEngine | | [WebGL Context Pooling](pathname:///live-examples/webGLContextPooling.html) | Demonstrates how to use WebGL context pooling to render many viewports in sync using ContextPoolRenderingEngine | | [Image Sharpening For Stack & Volume Viewports](pathname:///live-examples/sharpening.html) | Demonstrates how to apply image sharpening to stack and volume viewports | | [Image Smoothing For Stack & Volume Viewports](pathname:///live-examples/smoothing.html) | Demonstrates how to apply image smoothing to stack and volume viewports | | [Axis-based Image Stretching](pathname:///live-examples/axisBasedImageStretching.html) | Here we demonstrate axis based stretching with annotation and segmentation tools | #### Tools library | Example | Description | | ------- | ----------- | | [ETDRS Grid Tool](pathname:///live-examples/etdrsGrid.html) | Demonstrates how to use the ETDRS Grid tool. An ETDRS Grid (Early Treatment Diabetic Retinopathy Study Grid) is a standardized grid used in ophthalmology to assess macular thickness and retinal changes. | | [Image CPR Mapper](pathname:///live-examples/imageCPRMapper.html) | Demonstrates how to take a SplineROI annotation line from a Sagittal viewport and project the volume data along that centerline into a Stack viewport. | | [3D Volume Picking](pathname:///live-examples/cursor3D.html) | Demonstrates how to use the VTK.js vtkCellPicker object to pick 3D point in volume rendering scene. Also shows how to synchronize between 3D and 2D viewports. | | [3D Volume Cropping](pathname:///live-examples/volumeCroppingTool.html) | Demonstrates how to use the VolumeCropping and VolumeCroppingControl tools. | | [Multiple Tool Groups](pathname:///live-examples/multipleToolGroups.html) | Demonstrates the usage of multiple tool groups for a set of viewports. | | [Left Click and Right Click multi bindings](pathname:///live-examples/leftClickRightClickTools.html) | Demonstrates how to bind different annotation tools to left and right mouse buttons. Left click uses the Length tool, and right click uses the Bidirectional tool. Center/wheel+shift pan/zoom, and shift/left or right click to draw a rectangle or circle. | | [Stack Manipulation Tools](pathname:///live-examples/stackManipulationTools.html) | Demonstrates several manipulation tools (window/level, pan, zoom) as well as Stack Viewport-specific scrolling | | [Viewport Projection Synchronizer](pathname:///live-examples/viewportProjectionSynchronizer.html) | Demonstrates a custom tool-driven synchronizer for Planar Next viewports using the projection service. | | [Stack Manipulation Tools Touch](pathname:///live-examples/stackManipulationToolsTouch.html) | Demonstrates several manipulation tools (window/level, pan, zoom) as well as Stack Viewport-specific scrolling for mobile touch | | [Annotation Tool Modes](pathname:///live-examples/annotationToolModes.html) | Demonstrates the various tool modes for annotation tools (active, passive, enabled, disabled) | | [Stack Annotation Tools](pathname:///live-examples/stackAnnotationTools.html) | Demonstrates usage of various annotation tools (Probe, Rectangle ROI, Elliptical ROI, Bidirectional measurements) on a Stack Viewport. | | [Stack Range](pathname:///live-examples/stackRange.html) | Demonstrates use of a selection range for key image and other tools | | [Calibration Tools](pathname:///live-examples/calibrationTools.html) | Demonstrates usage of calibration tools on a Stack Viewport. | | [Volume Annotation Tools ](pathname:///live-examples/volumeAnnotationTools.html) | Demonstrates annotation using the Length tool in a Volume Viewport (on axial, sagittal, and oblique views) | | [Annotation Selection and Locking](pathname:///live-examples/annotationSelectionAndLocking.html) | Demonstrates how to toggle the Locked and Selected states for Annotations | | [Viewports Reset Camera](pathname:///live-examples/resetCamera.html) | Demonstrates various options that are available for resetting camera on viewports | | [Annotation changing visibility](pathname:///live-examples/annotationVisibility.html) | Demonstrates how to toggle the Visibility state for Annotations | | [Binding Tools with Modifier Keys](pathname:///live-examples/modifierKeys.html) | Demonstrates how to bind a tool to a keyboard and mouse combination (e.g. shift+click, ctrl+click) | | [Magnify Tool](pathname:///live-examples/magnifyTool.html) | Demonstrates the usage of the magnification tool | | [Advanced Magnify Tool](pathname:///live-examples/advancedMagnifyTool.html) | Demonstrates the usage of the advanced magnification tool on stack and volume viewports | | [CINE Tool](pathname:///live-examples/CINETool.html) | Demonstrates the usage of the CINE tool | | [Freehand ROI Tool](pathname:///live-examples/planarFreehandROITool.html) | Demonstrates drawing of both open and closed freehand ROIs (contour tool) on stack and volume viewports | | [Sculptor Tool](pathname:///live-examples/SculptorTool.html) | Demonstrates sculpting of freehand ROIs and FreehandContourSegmentations | | [Manipulation Tools with Poly Data in a Volume Viewport API](pathname:///live-examples/polyDataActorManipulationTools.html) | Demonstrates how to interact with a Volume viewport (Pan, Zoom, Rotate) by mouse events | | [Volume Viewport Orientation](pathname:///live-examples/volumeViewportOrientation.html) | Demonstrates you can switch between different orientation of a volume viewport | | [Referencing Cursors](pathname:///live-examples/referenceCursors.html) | Demonstrates how to synchronize the cursor between multiple viewports | | [Double Click With Stack Annotation Tools](pathname:///live-examples/doubleClickWithStackAnnotationTools.html) | Demonstrates double click detection before/during/after using various annotation tools on a stack viewport. | | [ColorBar](pathname:///live-examples/colorBar.html) | Demonstrates how to add an interactive color bar to stack viewport | | [Advanced ColorBar](pathname:///live-examples/advancedColorBar.html) | Demonstrates how to add an interactive color bar to stack and volume viewports with PT/CT volumes | | [Ultrasound Enhanced Region](pathname:///live-examples/ultrasoundenhancedregion.html) | Demonstrates several tools that can be used on Ultrasound data with Sequence of Ultrasound Regions Attributes | | [Window Level Region](pathname:///live-examples/windowLevelRegion.html) | Demonstrates how to use the window level region tool to adjust the window level of an image | | [Spline ROI Tools](pathname:///live-examples/splineROITools.html) | Demonstrates how to use spline ROI tools (Linear, Cardinal, Catmull-ROM and BSpline) | | [Livewire](pathname:///live-examples/livewireContour.html) | Demonstrates how to use the livewire tool to create ROIs | #### Segmentation | Example | Description | | ------- | ----------- | | [Click Segment Tool](pathname:///live-examples/clickSegment.html) | Click-to-segment lesions on PET: hover to scout candidates (plus/blocked cursor), click once to segment with a dynamically-derived one-sided threshold, then Shrink/Expand to fine-tune. No configuration. | | [Labelmap Segmentation Rendering](pathname:///live-examples/labelmapRendering.html) | Demonstrates how to add a Labelmap to the viewports for rendering | | [Labelmap Slice Rendering (useSliceRendering)](pathname:///live-examples/labelmapSliceRendering.html) | Demonstrates labelmap rendering with useSliceRendering to avoid 3D texture allocation | | [Labelmap Slice Rendering Tools (useSliceRendering)](pathname:///live-examples/labelmapSliceRenderingTools.html) | Demonstrates sphere brush painting with useSliceRendering enabled | | [Contour Segmentation Representation](pathname:///live-examples/contourRendering.html) | Demonstrates how to use the Contour Segmentation Representation | | [Surface Segmentation Representation](pathname:///live-examples/surfaceRendering.html) | Demonstrates how to use the Surface Segmentation Representation | | [Labelmap Segmentation Swapping](pathname:///live-examples/labelmapSwapping.html) | Demonstrate how to display segmentations on a volume viewport, and swap which segmentation is being displayed | | [Global Labelmap Segmentation Configuration](pathname:///live-examples/labelmapGlobalConfiguration.html) | Demonstrates how to set a global configuration for segmentation representations | | [Contour rendering configuration ](pathname:///live-examples/contourRenderingConfiguration.html) | Demonstrates how to set a configuration (such as line thickness) for contour rendering | | [Viewport Specific Labelmap Segmentation Configuration](pathname:///live-examples/labelmapViewportSpecificConfiguration.html) | Demonstrate how to change the configuration of how a specific tool group displays segmentations through via segmentation representations | | [Labelmap segment-specific Configuration](pathname:///live-examples/labelmapSegmentSpecificConfiguration.html) | Demonstrates how to change the configuration of a specific segment | | [Segmentation Tools (Labelmap) - Brush, Scissors](pathname:///live-examples/labelmapSegmentationTools.html) | Demonstrates how to use manual segmentation tools to modify the segmentation data | | [Labelmap Overlap Playground](pathname:///live-examples/labelmapOverlapPlayground.html) | Minimal overlap demo with brush and eraser variants, segment selection, and an Allow Overlap toggle for legacy, cpu=true, and type=next. | | [Labelmap Overlap PET-CT](pathname:///live-examples/labelmapOverlapPetCt.html) | Segmentation drawn on CT (top row) is also displayed on PET (bottom row). Demonstrates cross-volume labelmap rendering. | | [Labelmap Statistics](pathname:///live-examples/labelmapStatistics.html) | Show labelmap statistics | | [Labelmap Segmentation Dynamic Threshold and Preview](pathname:///live-examples/labelmapSegmentationDynamicThreshold.html) | Demonstrates how to use dynamic threshold with preview to modify the segmentation data | | [Labelmap Segment Color Change](pathname:///live-examples/labelmapSegmentColorChange.html) | Here we demonstrate how to change the color of a segment in a segmentation representation | | [Labelmap Segmentation Locking](pathname:///live-examples/labelmapSegmentLocking.html) | Demonstrate how a segment can be locked such that it cannot be edited by segmentation tools | | [Rendering Labelmap with Different Resolutions](pathname:///live-examples/labelmapRenderingDifferentResolutions.html) | Demonstrates that the segmentation resolution need not to be the same as the source data | | [Rectangle ROI Threshold Segmentation](pathname:///live-examples/rectangleROIThreshold.html) | Demonstrates how to use the rectangle roi tool to perform threshold segmentation | | [Stack Labelmap creation/edit for stack viewports](pathname:///live-examples/stackLabelmapSegmentation.html) | Demonstrates how to create and edit a segmentation labelmap for stack viewports | | [Stack Viewports with Segmentation Sync](pathname:///live-examples/stackViewportsWithSegmentationSync.html) | Demonstrates how to synchronize two stack viewports with a segmentation on one of them | | [Spline ROI Tools](pathname:///live-examples/splineROITools.html) | Demonstrates how to use spline ROI tools (Linear, Cardinal, Catmull-ROM and BSpline) | | [Interpolation of Contours between slices](pathname:///live-examples/interpolationContourSegmentation.html) | Demonstrates how to setup interpolation between frames for contour segmentations | | [Display surfaces in slices](pathname:///live-examples/surfaceContourRendering.html) | Demonstrates how 3D surfaces as displayed in different orientation viewports | | [Contour Segmentation Configuration](pathname:///live-examples/contourSegmentationConfiguration.html) | Demonstrates how to set a configuration for contour segmentations | | [Segmentation Bidirectional Tool](pathname:///live-examples/segmentBidirectionalTool.html) | Demonstrates the calculation of largest bidirectional diameters within segmented contours, akin to RECIST measurements for assessing changes in tumor sizes or anatomical structures over time in medical imaging. | | [Segment Select Tool](pathname:///live-examples/segmentSelect.html) | Demonstrates the segmentSelectTool capabilities which you can use to switch active segment by only hovering over them. | | [Segment Label Tool](pathname:///live-examples/segmentLabel.html) | Demonstrates the segmentLabelTool capabilities which you can use to see the labels of segments by only hovering over them. | | [Spline Segmentation Tools](pathname:///live-examples/splineContourSegmentationTools.html) | Demonstrates how to create contour segmentations using SplineROI tool | | [Advanced Spline Segmentation Tools](pathname:///live-examples/splineContourSegmentationToolsAdvanced.html) | Demonstrates how to create contour segmentations using SplineROI tool on multiple viewports (stack and volume), segmentations and different styles for active and inactive states | | [Freehand Segmentation Tool](pathname:///live-examples/planarFreehandContourSegmentationTool.html) | Demonstrates how to create contour segmentations using planarFreehandROITool tool | | [Livewire Segmentation Tool](pathname:///live-examples/livewireContourSegmentation.html) | Demonstrates how to create contour segmentations using livewireContour tool | | [sculptorTool Tool](pathname:///live-examples/sculptorTool.html) | Demonstrates how to have similar brush tool effects on the contour | | [Logical operators for contour segmentations](pathname:///live-examples/logicalOperators.html) | Demonstrates the logical operations that can be performed on contour segmentations, such as union, intersection, and subtraction. | | [Labelmap Editing with Contour](pathname:///live-examples/labelmapEditWithContour.html) | Use contour tools to edit the labelmap | | [Automatic Labelmap editing with contour Tool](pathname:///live-examples/labelmapEditWithContourAutomatic.html) | Apply contour tool automatic to efficiently modify and refine labelmap regions. | | [Labelmap Interpolation](pathname:///live-examples/labelmapInterpolation.html) | Interpolate between slices using labelmap | | [Labelmap MIP](pathname:///live-examples/labelmapMIP.html) | Demonstrates how to use the labelmap MIP tool | | [Contour Utility API](pathname:///live-examples/contourApi.html) | Demonstrates how to use various contour utility functions concerning simplification, smoothing, and hole removal. | #### Advanced Tools library | Example | Description | | ------- | ----------- | | [Maximum Intensity Projection (MIP) - Jump to Click](pathname:///live-examples/mipJumpToClick.html) | Demonstrates how to obtain the location of the maximum value along the ray in a MIP view, and then navigate another set of viewports to this location. | | [Crosshairs](pathname:///live-examples/crossHairs.html) | Here we demonstrate crosshairs linking three orthogonal views of the same data | | [Crosshairs Binding Modes](pathname:///live-examples/crossHairsBindings.html) | Demonstrates Crosshairs as a regular active tool binding on either primary or right click. | | [World Crosshair (Reference Point)](pathname:///live-examples/worldCrosshair.html) | Demonstrates the WorldCrosshairTool: a persistent world-space reference point that stays fixed while viewports scroll, pan and zoom, with off-slice projection display. | | [World Crosshair on MIP](pathname:///live-examples/worldCrosshairMip.html) | Demonstrates the WorldCrosshairTool on a PET maximum intensity projection viewport: clicking the MIP snaps the reference point to the hottest voxel along the line of sight and the CT viewports jump to that anatomy. | | [World Crosshair + Slice Intersections (PET/CT, Generic Viewports)](pathname:///live-examples/worldCrosshairSliceIntersectionsPetCt.html) | CT and PT rows of native PLANAR_NEXT viewports (2D planar stack plus axial/sagittal/coronal volume slices and a 3D bone rendering); one intersection line per plane group drives CT and PT together, with both tools independently toggleable. | | [Slice Intersections](pathname:///live-examples/sliceIntersections.html) | Demonstrates the SliceIntersectionTool rendering one true plane-plane intersection line per plane, with line drag, rotation and slab thickness handles. | | [World Crosshair + Slice Intersections](pathname:///live-examples/worldCrosshairAndSliceIntersections.html) | Demonstrates the WorldCrosshairTool and SliceIntersectionTool enabled together while remaining fully independent. | | [Ten Viewport Reference Point](pathname:///live-examples/tenViewportReferencePoint.html) | Demonstrates the WorldCrosshairTool reference point on a large ten viewport grid. | | [DICOM Reformats in MPR](pathname:///live-examples/mprReformat.html) | Aligns MPR viewports to dicom acquisition orientations | | [Overlay Grid](pathname:///live-examples/overlayGrid.html) | Demonstrate overlay grid tool usage with three viewports one for each orientation | | [Reference Lines](pathname:///live-examples/referenceLines.html) | Demonstrate reference line tool for rendering viewports location with respect to each other | | [Orientation Marker](pathname:///live-examples/orientationMarker.html) | Demonstrate orientation marker tool for viewports orientation it has cube, axis and custom actors | | [PET-CT Fusion + MIPLayout](pathname:///live-examples/petCT.html) | PT-CT fusion layout with Crosshairs, and synchronized cameras, CT W/L and PET threshold | | [Shared Tool State](pathname:///live-examples/sharedToolState.html) | Demonstrates that annotations are stored on frame of reference, and can therefore be shared between Stack and Volume Viewports. | | [StackViewport to and from VolumeViewport ](pathname:///live-examples/stackToVolumeWithAnnotations.html) | Demonstrates how annotations are preserved and rendered correctly even when a stack viewport is converted to a volume viewport and vice versa. This is an advanced usage for MPR | | [Volume Viewport Synchronization](pathname:///live-examples/volumeViewportSynchronization.html) | Demonstrates how to set up synchronization between viewports for viewport-level (e.g. camera) and actor-level (e.g. VOI) properties. | | [Cancel Annotation Drawing](pathname:///live-examples/cancelAnnotationDrawing.html) | Demonstrates how to use the keyboard (ESC) key to cancel annotation drawing. | | [Scale Overlay Tool](pathname:///live-examples/scaleOverlayTool.html) | Demonstrates the scale overlay tool for rendering a scale on a viewport showing the real world size of the image. | | [Generate 3D Volume From 4D Data](pathname:///live-examples/generateImageFromTimeData.html) | Demonstrates generating a 3D volume from 4D data using subtract, average or sum. | | [Dynamically Add Annotations](pathname:///live-examples/dynamicallyAddAnnotations.html) | Demonstrates how to dynamically add annotations to a viewport | | [Tool History](pathname:///live-examples/toolHistory.html) | Demonstrates how to use the tool history to undo and redo tool actions | | [Tool History Grouping](pathname:///live-examples/toolHistoryGrouping.html) | Demonstrates how to use the tool history grouping to undo and redo batched tool actions that are related to one another | #### GPU Segmentation Tools | Example | Description | | ------- | ----------- | | [Region Segment Tool](pathname:///live-examples/regionSegment.html) | Demonstrates how to create a segmentation after drawing a 3D sphere and running grow cut algorithm in the GPU | | [Region Segment Plus Tool](pathname:///live-examples/regionSegmentPlus.html) | Demonstrates how to create a segmentation with a single click and running grow cut algorithm in the gpu | | [Custom Brush Grow Cut](pathname:///live-examples/growCutLabelmap.html) | Demonstrates how to run grow cut algorithm in the GPU on a labelmap with positive and negative seeds | | [Whole Body Segment tool](pathname:///live-examples/wholeBodySegment.html) | Demonstrates how to segment the whole body of a region selected by the user that is processed in the gpu | | [Segmentation AI Assistance](pathname:///live-examples/SAMClientSide.html) | Demonstrates how to use AI assistance tools for segmentation creation using onnx runtime on the client side | #### Polymorph Segmentation | Example | Description | | ------- | ----------- | | [Convert contour segmentation to stack labelmap](pathname:///live-examples/PolySegWasmContourToStackLabelmap.html) | Demonstrates how to convert a contour segmentation to a stack labelmap | | [Convert contour segmentation to volume labelmap](pathname:///live-examples/PolySegWasmContourToVolumeLabelmap.html) | Demonstrates how to convert a contour segmentation to a volume labelmap | | [Convert contour segmentation to surface](pathname:///live-examples/PolySegWasmContourToSurface.html) | Demonstrates how to convert a contour segmentation to a closed surface | | [Convert stack labelmap to surface](pathname:///live-examples/PolySegWasmStackLabelmapToSurface.html) | Demonstrates how to convert a stack labelmap to a closed surface | | [Convert volume labelmap to surface](pathname:///live-examples/PolySegWasmVolumeLabelmapToSurface.html) | Demonstrates how to convert a volume labelmap to a closed surface | | [Convert surface to volume labelmap](pathname:///live-examples/PolySegWasmSurfaceToVolumeLabelmap.html) | Demonstrates how to convert a closed surface to a volume labelmap | | [Convert surface to stack labelmap](pathname:///live-examples/PolySegWasmSurfaceToStackLabelmap.html) | Demonstrates how to convert a closed surface to a stack labelmap | | [Convert volume labelmap to contour](pathname:///live-examples/PolySegWasmVolumeLabelmapToContour.html) | Demonstrates how to convert a volume labelmap to a contour segmentation | | [Convert surface to contour](pathname:///live-examples/PolySegWasmSurfaceToContour.html) | Demonstrates how to convert a closed surface to a contour segmentation | #### DICOM image loader | Example | Description | | ------- | ----------- | | [WADO-URI (DICOM P10)](pathname:///live-examples/dicomImageLoaderWADOURI.html) | WADO-URI (DICOM P10 via HTTP GET) with different codecs | | [HTJ2K Stack Basic Loading](pathname:///live-examples/htj2kStackBasic.html) | Demonstrates basic loading of HTJ2K | | [HTJ2K Volume Basic Loading](pathname:///live-examples/htj2kVolumeBasic.html) | Demonstrates basic loading of HTJ2K in MPR views | | [Stack Progressive Loading](pathname:///live-examples/stackProgressive.html) | Stack progressive loading using HTJ2K and/or other methods. | | [Volume Progressive Loading](pathname:///live-examples/volumeProgressive.html) | Volume progressive loading inter and intra image | | [Volume Decimated Loading](pathname:///live-examples/volumeDecimatedLoading.html) | Volume Decimated loading | #### Adapters | Example | Description | | ------- | ----------- | | [DICOM SEG Stack](pathname:///live-examples/segmentationStack.html) | Demonstrates how to import or export a segmentation to DICOM SEG from a Cornerstone3D stack | | [DICOM SEG Volume](pathname:///live-examples/segmentationVolume.html) | Demonstrates how to import or export a segmentation to DICOM SEG from a Cornerstone3D volume | #### Other Viewports (video,wsi) | Example | Description | | ------- | ----------- | | [Video Display](pathname:///live-examples/video.html) | Basic video display | | [Video Navigation](pathname:///live-examples/videoNavigation.html) | Navigation of video playback | | [Video Color Control](pathname:///live-examples/videoColor.html) | Video Color Correction and Brightness/Contrast | | [Video Tools](pathname:///live-examples/videoTools.html) | Video annotation tools | | [Video Annotation Grouping](pathname:///live-examples/videoGroup.html) | Annotation grouping tools | | [Video Labelmap Segmentation](pathname:///live-examples/videoSegmentation.html) | Video Labelmap Based Segmentation | | [Video Contour Segmentation](pathname:///live-examples/videoContourSegmentation.html) | Demonstrates spline and livewire contour segmentation on video viewports | | [Video Range Selection](pathname:///live-examples/videoRange.html) | Video range selection | | [Whole Slide Imaging](pathname:///live-examples/wsi.html) | Display WSI Series | | [ECG Viewport](pathname:///live-examples/ecg.html) | Displays a 12-lead ECG from DICOM Waveform data. | | [WSI Annotation Tools](pathname:///live-examples/wsiAnnotationTools.html) | WSI with length and other annotation tools | #### Nifti Volume loader | Example | Description | | ------- | ----------- | | [Load Nifti Volume](pathname:///live-examples/niftiBasic.html) | Demonstrates how to load and render a nifti volume | | [Tool Usage in Nifti](pathname:///live-examples/niftiWithTools.html) | Demonstrates how to use manipulation and annotation tools on a nifti volume | --- ## Frequently Asked Questions Source: https://cornerstonejs.org/docs/llm/faq #### Frequently Asked Questions #### What is the difference between Cornerstone (legacy) and Cornerstone3D (alpha) and react-vtkjs-viewport? Although Cornerstone (legacy) has gpu-accelerated rendering through webgl, it only handles 2D rendering of medical images. To address this issue, we created [react-vtkjs-viewport](https://github.com/OHIF/react-vtkjs-viewport) which enabled 3D rendering of medical images by moving the rendering functionalities to [`vtk.js`](https://github.com/kitware/vtk-js), a powerful rendering library. However, vtk.js uses WebGL instances per viewport, and this does not scale for situations like PET/CT hanging protocols which may require > 10 viewports on-screen simultaneously, due to GPU memory constraints (textures are not shared across canvases) and WebGL context limits (a maximum of 16 contexts can exist per browser tab). In addition, `vtk.js` does not provide support for SVG annotation tools. To satisfy complex imaging use cases, we have chosen to build Cornerstone rendering engine from the ground up for efficient GPU memory usage. This rendering engine abstracts many of the technicalities of `vtk.js`; it processes data offscreen in one WebGL canvas, and transfers the resulting images to on-screen canvases. This approach allows us to efficiently share GPU texture memory between different views/representations of the same data. For example, in a PET/CT Fusion MPR hanging protocol, only one PET volume is stored in the GPU memory and is used when rendering both the inverted PET and fusion PET viewports. #### What are the feature parity between Cornerstone and Cornerstone3D? The following will not be migrated at the current time
Feature Reason
Cornerstone Modules In CornerstoneTools these are namespaced plugins used to store tool-wide metadata in a custom manner, whilst also having initialization hooks for enabled/disabled events. They are not necessary for simple planar tools and as such will not be available in the first version.
Mixins Mixins are self registering addons for tools introduced in CornerstoneTools 3.0+. We found there are more useful design patterns for making tools by composition, such as wrapping common utility functions. We intend to deprecate this feature.
Registered third-party content other than tools (custom manipulators, utils, etc). We feel utils should just be wrapped up in NPM libraries and imported, and the old framework was probably too heavy for its use cases.
--- ## Help Source: https://cornerstonejs.org/docs/llm/help #### Help We all need a little help sometimes. Don't let a few roadblocks stand in the way of you building something awesome. #### Community Support If you're a developer looking to contribute code, documentation, or discussion; we are more than happy to help provide clarification and answer questions via [GitHub issues][gh-issues]. You can also join our [Slack Group](https://ohif.org/community) or post on the [Community Forum](https://community.ohif.org/). For bug reports and feature requests (including incomplete or confusing documentation), [GitHub issues][gh-issues] continue to be your best avenue of communication. Complex issues specific to your organization/situation are still okay to post, but they're less likely to receive a response. Unfortunately, we have limited resources and must be judicious with how we allocate them. If you find yourself in this situation and in need of assistance, it may be in your best interest to persue paid support. [gh-issues]: https://github.com/cornerstonejs/cornerstone3D/issues/ --- ## Test Coverage Source: https://cornerstonejs.org/docs/llm/test-coverage #### Test Coverage #### Playwright Here's the test coverage report for our Playwright tests. Keep in mind that this doesn't include our Karma tests, so our actual test coverage is likely higher than what's shown. We're focusing on Playwright for future Cornerstone3D testing, and we're really pushing to improve that coverage number. You can view our latest test coverage report here: - [Cornerstone3D Playwright Test Coverage Report](https://www.cornerstonejs.org/coverage) --- # Concepts ## Design Considerations Source: https://cornerstonejs.org/docs/llm/concepts/designConsiderations.md #### High level design considerations These libraries expand upon and update the interfaces `cornerstone.js` provided to better support volume rendering, 3D aware tools, and PET images support. These interfaces and functionality are broadly identified as: - Rendering / Renderer - Image Loading / Image Loader - Metadata Provider - Tools `@cornerstonejs/core` is a "rendering" library built on top of `vtk.js`. which leverages `cornerstone`'s existing plumbing to integrate with image loaders and metadata providers. This repository's `@cornerstonejs/tools` is a "tools" library that, once initialized, will listen for custom events emitted by `@cornerstonejs/core`. Please note, the event naming and handling overlaps the events and event handling in the `cornerstone-tools` library. If you attempt to use `cornerstone-tools` in tandem, you will likely encounter issues. As this is a possible use case, please don't hesitate to report any issues and propose potential solutions. --- ## Cornerstone-core ### Cache Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/cahce.md #### Cache The Cache API’s role is to keep track of created volumes, manage memory usage, and alert the host application when trying to allocate data that would exceed application defined limits. This module deals with Caching of images and volumes The cache has two main components: a volatile portion for images and a non-volatile portion for volumes. We will have a shared block of memory allocated for the entire cache, e.g. 1GB which will be shared for images and volumes. - Individual 2D images are volatile and will be replaced by new images hitting the cache. - When you allocate volumes, it tags the images used by the volume as non-volatile unless you release the volume. #### Utilities for the cache There are various utility functions you can use to manage the cache. - **isCacheable**: One of the many utility functions that the `Cache` API provides is `isCacheable` which you can use to check if there is enough free space before initiating the fetch for the volume or image. - **purgeCache**: Deletes all the images and volumes inside the cache. - **decacheIfNecessaryUntilBytesAvailable**: It purges the cache if necessary based on the requested number of bytes. #### Cache Optimizations All data in the cache is actually `image` objects. When you request a volume, we pass the images necessary for the volume to the GPU on demand, and at no point do we store the voxel data of the volume in the cache. If you need to access the voxel data of a volume, you can do so by using the `VoxelManager` class. If you ever actually need the full voxel data of a volume, you can use the `VoxelManager` class method `.getCompleteScalarDataArray()` to get the full voxel data. This new change that was introduced in `Cornerstone3D` 2.x is part of the new image-based approach that aims to improve performance, reduce memory usage, and provide more efficient data access, especially for large datasets. Here are other benefits of the new approach: 1. Single Source of Truth - Previously: Data existed in both image cache and volume cache, leading to synchronization issues. - Now: Only one source of truth - the image cache. - Benefits: Improved syncing between stack and volume segmentations. 2. New Volume Creation Approach - Everything now loads as images. - Volume streaming is performed image by image. - Only images are cached in the image cache. - For volume rendering, data goes directly from image cache to GPU, bypassing CPU scalar data. - Benefits: Eliminated need for scalar data in CPU, reduced memory usage, improved performance. 3. VoxelManager for Tools - Acts as an intermediary between indexes and scalar data. - Provides mappers from IJK to indexes. - Retrieves information without creating scalar data. - Processes each image individually. - Benefits: Efficient handling of tools requiring pixel data in CPU. 4. Handling Non-Image Volumes - Volumes without images (e.g., NIFTI) are chopped and converted to stack format. - Makes non-image volumes compatible with the new image-based approach. 5. Optimized Caching Mechanism - Data stored in native format instead of always caching as float32. - On-the-fly conversion to required format when updating GPU textures. - Benefits: Reduced memory usage, eliminated unnecessary data type conversions. 6. Elimination of SharedArrayBuffer - Removed dependency on SharedArrayBuffer. - Each decoded image goes directly to the GPU 3D texture at the correct size and position. - Benefits: Reduced security constraints, simplified web worker implementation. --- ### Geometry Loaders Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/geometryLoader.md #### Geometry Loaders This section describes the geometry loaders in Cornerstone Core. If you read the Segmentation rendering [section](../cornerstone-tools/segmentation/index.md) you can see that a Segmentation can be rendered as a Volume (Labelmap), or it can be rendered as a Contour, or Surface. :::note TIP Similar relationship structure has been adapted in popular medical imaging software such as [3D Slicer](https://www.slicer.org/) with the addition of [polymorph segmentation](https://github.com/PerkLab/PolySeg). ::: Geometry loaders are used to load and cache geometry data from a file or URL in general. #### Register Mesh Loader You can use [`registerGeometryLoader`](/docs/api/core/namespaces/geometryloader/functions/registerGeometryLoader) to make an external mesh loader available to the cornerstone library. This function accept a `scheme` which the mesh loader function (second argument) should act on. ```js import { geometryLoader, cornerstoneMeshLoader, Enums, Types, } from '@cornerstonejs/core'; geometryLoader.registerGeometryLoader('mesh', cornerstoneMeshLoader); ``` #### CornerstoneMeshLoader You can take a look at our sample code example for `cornerstoneMeshLoader` [here](https://github.com/cornerstonejs/cornerstone3D/tree/main/packages/core/examples/meshLoader) ```js const mesh1 = await geometryLoader.loadAndCacheGeometry( 'mesh:https://example.com/mesh.ply', { type: Enums.GeometryType.MESH, geometryData: { id: 'mesh1', format: Enums.MeshType.PLY, } as Types.MeshData, } ); const mesh2 = await geometryLoader.loadAndCacheGeometry( 'mesh:https://example.com/mesh.obj', { type: Enums.GeometryType.MESH, geometryData: { id: 'mesh2', format: Enums.MeshType.OBJ, materialUrl: 'https://example.com/material.mtl', } as Types.MeshData, } ); viewport.setActors([ { uid: mesh1.id, actor: (mesh1.data as Types.IMesh).actor }, { uid: mesh2.id, actor: (mesh2.data as Types.IMesh).actor }, ]); ``` #### Supported mesh formats The supported mesh formats for the `cornerstoneMeshLoader` are: - PLY - OBJ - STL - VTP #### Supported material formats The supported material formats for the `cornerstoneMeshLoader` are: - MTL - JPG - PNG - JPEG --- ### ImageId Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/imageId.md #### ImageId A `Cornerstone3D` `ImageId` is a URL which identifies a single image for cornerstone to display. The URL scheme in the `ImageId` is used by Cornerstone to determine which [Image Loader](./imageLoader.md) plugin to call to actually load the image. It should be noted that `Cornerstone3D` delegates loading of the images to the registered image loaders. This strategy allows Cornerstone to simultaneously display multiple images obtained with different protocols from different servers. For example, Cornerstone could display a DICOM CT image obtained via WADO alongside a JPEG dermatology image captured by a digital camera and stored on a file system. The ImageId format ![image-id-format](./../../assets/image-id-format.png) DICOM Persistent Objects (WADO) is a standard for storing and retrieving medical images using the DICOM protocol. WADO allows for retrieval (and storage) of images from a WADO-compliant server. Here are some examples of what an imageId would look like for different ImageLoader plugins: [**WADO-URI**](https://dicom.nema.org/dicom/2013/output/chtml/part18/sect_6.2.html) ``` http://www.medical-webservice.st/RetrieveDocument? requestType=WADO&studyUID=1.2.250.1.59.40211.12345678.678910 &seriesUID=1.2.250.1.59.40211.789001276.14556172.67789 &objectUID=1.2.250.1.59.40211.2678810.87991027.899772.2 &contentType=application%2Fdicom&transferSyntax=1.2.840.10008.1.2.4.50 ``` [**WADO-RS**](https://dicom.nema.org/dicom/2013/output/chtml/part18/sect_6.5.html) ``` https://d14fa38qiwhyfd.cloudfront.net/dicomweb/ studies/1.3.6.1.4.1.25403.345050719074.3824.20170126083429.2/ series/1.3.6.1.4.1.25403.345050719074.3824.20170126083454.5/ instances/1.3.6.1.4.1.25403.345050719074.3824.20170126083455.3/frames/1 ``` Cornerstone does not specify what the contents of the URL are - it is up to the Image Loader to define the contents and format of the URL so that it can locate the image. For example, a proprietary Image Loader plugin could be written to talk to a proprietary server and lookup images using a GUID, filename or database row id. Here are some examples of what an ImageId could look like for different Image Loader plugins: - `example://1` - `dicomweb://server/wado/{uid}/{uid}/{uid}` - `http://server/image.jpeg` - `custom://server/uuid` - `wadors://server/{StudyInstanceUID}/{SeriesInstanceUID}/{SOPInstanceUID}` --- ### Image Loaders Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/imageLoader.md #### Image Loaders An `ImageLoader` is a JavaScript function that is responsible for taking an [`ImageId`](./imageId.md) and returning an [`Image Object`](./images.md). Since loading images usually requires a call to a server, the API for image loading needs to be asynchronous. Cornerstone requires that Image Loaders return an Object containing a Promise which Cornerstone will use to receive the Image Object asynchronously, or an Error if one has occurred. #### Image Loader Workflow 1. `ImageLoaders` register themselves using [`registerImageLoader`](/docs/api/core/namespaces/imageloader/functions/registerimageloader) API with cornerstone to load specific ImageId URL schemes 2. The application requests to load an image using the `loadImage` API for stack or `createAndCacheVolume` API for volume. 3. Cornerstone delegates the request to load the image to the `ImageLoader` registered with the URL scheme of the imageId. 4. The ImageLoader will return an `Image Load Object` containing a Promise which it will resolve with the corresponding Image Object once it has obtained the pixel data. Obtaining the pixel data may require a call to a remote server using `XMLHttpRequest`, decompression of the pixel data (e.g. from JPEG 2000), and conversion of the pixel data into the format that Cornerstone understands (e.g. RGB vs YBR color). 5. The [Image Object](./images.md) passed back by the resolved Promise is then displayed using `renderingEngine` API. #### Register Image Loader You can use [`registerImageLoader`](/docs/api/core/namespaces/imageloader/functions/registerimageloader) to make an external image loader available to the cornerstone library. This function accept a `scheme` which the image loader function (second argument) should act on. #### Available Image Loaders | Image Loader | Used for | | ----------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | [Cornerstone DICOM Image Loader](https://github.com/cornerstonejs/cornerstone3D/tree/main/packages/dicomImageLoader) | DICOM Part 10 images; Supports WADO-URI and WADO-RS; Supports multi-frame DICOM instances; Supports reading DICOM files from the File objects | | [Cornerstone Web Image Loader](https://github.com/cornerstonejs/cornerstoneWebImageLoader) | PNG and JPEG | | [Cornerstone-nifti-image-loader](https://github.com/cornerstonejs/cornerstone3D/tree/main/packages/nifti-volume-loader) | NIFTI | #### CornerstoneDICOMImageLoader [`CornerstoneDICOMImageLoader`](https://github.com/cornerstonejs/cornerstone3D/tree/main/packages/dicomImageLoader) is a cornerstone image loader that loads DICOM images from a WADO-compliant server. You can install it and initialize to via the following code. Internally, `CornerstoneDICOMImageLoader` registers its `wado-rs` and `wado-uri` imageLoaders to `Cornerstone3D` and uses [`dicomParser`](https://github.com/cornerstonejs/dicomParser) to parse the the metadata and pixel data. ```js import { init } from '@cornerstonejs/dicom-image-loader'; init({ maxWebWorkers: navigator.hardwareConcurrency || 1, }); ``` After initialization of the `CornerstoneDICOMImageLoader`, any imageId using the `wado-uri` scheme will be loaded using the `CornerstoneDICOMImageLoader` `wado-uri` image loader and metadata provider (e.g., imageId = 'wado-uri: https://exampleServer.com/wadoURIEndPoint?requestType=WADO&studyUID=1.2.3&seriesUID=4.5.6&objectUID=7.8.9&contentType=application%2Fdicom'), and likewise for `wado-rs` imageIds which will use `CornerstoneDICOMImageLoader` `wado-rs` image loader and metadata provider (e.g., imageId = 'wado-rs: https://exampleServer.com/wadoRSEndPoint/studies/1.2.3/series/4.5.6/instances/7.8.9/frames/1'). #### CornerstoneWebImageLoader You can take a look at our sample code example for `CornerstoneWebImageLoader` [here](https://github.com/cornerstonejs/cornerstone3D/tree/main/packages/core/examples/webLoader) --- ### Image Object Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/images.md #### Image Object Cornerstone [Image Loaders](./imageLoader.md) return `Image Load Objects` which contain a Promise. The reason we have chosen to use an Object instead of solely returning a Promise is because now Image Loaders can also return other properties in their Image Load Objects. As an example, we intend to implement support for `cancelling` pending or ongoing requests using a `cancelFn` passed back by an Image Loader within an Image Load Object. Here is an interface of such Image Load Object. You can read more about each field in the [IImage section](/docs/api/core/namespaces/Types/interfaces/IImage) of API reference. ```js interface IImage { imageId: string sharedCacheKey?: string minPixelValue: number maxPixelValue: number slope: number intercept: number windowCenter: number[] windowWidth: number[] getPixelData: () => Array getCanvas: () => HTMLCanvasElement rows: number columns: number height: number width: number color: boolean rgba: boolean numberOfComponents: number columnPixelSpacing: number rowPixelSpacing: number sliceThickness?: number invert: boolean sizeInBytes: number scaling?: { PET?: { SUVlbmFactor?: number SUVbsaFactor?: number suvbwToSuvlbm?: number suvbwToSuvbsa?: number } } } ``` --- ### Cornerstone Core Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/index.md import DocCardList from '@theme/DocCardList'; import {useCurrentSidebarCategory} from '@docusaurus/theme-common'; #### Core Introduction This section describes the core concepts in the `Cornerstone3D` (`@cornerstonejs/core`). `Cornerstone3D` is more than a "rendering" library. It handles: - rendering of the image (both using GPU or CPU) - caching of the data and metadata - providing a framework for image/volume loader APIs - providing metadata API support. The purpose of this section is to give an overview of the core concepts in `Cornerstone3D`. --- ### Metadata Providers Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/metadataProvider.md #### Metadata Providers For package-level architecture and current 5.x metadata behavior, see [Metadata Module](../cornerstone-metadata/index.md). Medical images typically come with lots of non-pixel-wise metadata such as the pixel spacing of the image, the patient ID, or the scan acquisition date. With some file types (e.g. DICOM), this information is stored within the file header and can be read and parsed and passed around your application. With others (e.g. JPEG, PNG), this information needs to be provided independently from the actual pixel data. Even for DICOM images, however, it is common for application developers to provide metadata independently from the transmission of pixel data from the server to the client since this can considerably improve performance. A Metadata Provider is a JavaScript function that acts as an interface for accessing metadata related to Images in Cornerstone. Users can define their own provider functions in order to return any metadata they wish for each specific image. A Metadata Provider function has the following prototype: ``` function metadataProvider(type: string, ...queries: any): any ``` However, typically, providers implement the following, more simple prototype: ``` function metadataProvider(type: string, imageId: string): Record ``` This is because most metadata is provided for [ImageIds](./imageId.md), but Cornerstone provides infrastructure for the definition and usage of metadata providers for any information. #### Types of Metadata The `type` parameter to a metadata provider can be any string. You can call `cornerstone.metaData.get()` with any type, and if any metadata provider can provide that type for the given image ID, you get the response. You can use this, for example, to easily provide application-specific information such as ground truth or patient information. Cornerstone core and tools also automatically request various types of metadata for displayed images. A list of standard metadata modules can be found in the [MetadataModules section](/docs/api/core/namespaces/enums/enumerations/metadatamodules/) of the API reference. Many of these modules conform to the DICOM standard. If you want to implement them in a [custom metadata provider](../../how-to-guides/custom-metadata-provider.md), it is easiest to look at how an existing metadata provider implements them, such as the [WADOURI metadata provider](https://github.com/cornerstonejs/cornerstone3D/blob/main/packages/dicomImageLoader/src/imageLoader/wadouri/metaData/metaDataProvider.ts#L65). #### Priority of Metadata Providers Since it is possible to register more than one metadata provider, upon adding a provider you can define a priority number for it. When there is a time to request metadata, Cornerstone requests the metadata for `imageId` by the priority order of providers (if provider returns `undefined` for the imageId, Cornerstone moves to the next provider). For instance, if provider1 is registered with 10 priority and provider2 is registered with 100 priority, provider2 is asked first for the metadata for the imageId. #### Provided Computed Metadata There are a few metadata providers which either store metadata information updated/modified while running, or which transform the existing metadata into other formats, or provide static metadata. #### Transient Metadata #### Calibrated Pixel Spacing The `calibratedPixelSpacingMetadataProvider` allows storing of over-ride values for the calibration metadata, allowing a user or system to add calibration of spacing for an image independent of the original metadata. #### Computed Metadata Some metadata can be computed based on other metadata available in the system. For example, the adapters module can generate study module information in the 'Normalized' format from dcmjs based on the existing default metadata providers. It is suggested that any metadata that is computed just from straight DICOM data, but is modified in some way use a computed metadata provider. This pattern allows creating standard computed changes to the existing metadata across a variety of different types of underlying data such as multiframe instances, formatted data, or data used for producing new instances #### `referencedMetadataProvider` The adapters module provides referenced metadata useful when creating new instances based on an existing module. It also provides constants for the Part 10 prefix header and referenced objects. #### Study, Series, Instance Data The data modules provide the study or series level information in the 'Normal' format for dcmjs, without including the full instance header, and based on the underlying standard modules defined for WADO-URI, RS and from OHIF. #### Part 10 Constants The part 10 \_meta field in dcmjs can be hard coded, but that prevents changes to the generated object without modifying the object after creation or without modifying the creation code. The part 10 constants metadata provides the standard `0002` header module for dcmjs to use when encoding DICOM. #### Referenced and Predecessor Data The referenced data and predecessor sequence providers allow replacing the default instance in an SR or SEQ type series with a new one that references the previously used data. #### metaData helpers The metaData service has a few helper methods to deal with naming variations and computed results. #### `getNormalized` The get normalized module method will take a set of modules in the lower camel case version, and combine them into an upper (dcmjs NormalCase) version. This is used for creating an `instance` module from non-dcmjs data sources as well as for creating the study/series/instance data modules. #### `capitalizeTag` and `lowerTag` There are some specific rules needed for converting between UpperCamelCase names used in dcmjs `normalized` modules and the `lowerCamelCase` names used in the metaData modules. These are encapsulated into helper functions available on the metaData object exported from CS3D core. --- ### Rendering Engine Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/renderingEngine.md #### Rendering Engine A `RenderingEngine` allows the user to create Viewports, associate these Viewports with onscreen HTML elements, and render data to these elements using an offscreen WebGL canvas. It should be noted that `RenderingEngine` is capable of rendering multiple viewports, and you don't need to create multiple engines. However, multiple `RenderingEngine` instances can be created, e.g., if you wish to have a multiple monitor setup, and use a separate WebGL context to render each monitor’s viewports. In `Cornerstone3D` we have built the `RenderingEngine` from ground up, and we are utilizing [vtk.js](https://github.com/kitware/vtk-js) as the backbone of the rendering. `vtk.js` is a 3D rendering library capable of using WebGL for GPU-accelerated rendering. #### OnScreen and Offscreen Rendering Previously in Cornerstone (legacy), we processed data in each viewport with a WebGL canvas. This doesn't scale well, as the number of viewports increases and for complex imaging use cases (e.g., synced viewports), we will end up with lots of updates to onscreen canvases and performance degrades as the number of viewports increases. In `Cornerstone3D`, we process data in an offscreen canvas. This means that we have a big invisible canvas (offscreen) that includes all the onscreen canvases inside itself. As the user manipulates the data, the corresponding pixels in the offscreen canvas get updated, and at render time, we copy from offscreen to onscreen for each viewport. Since the copying process is much faster than re-rendering each viewport upon manipulation, we have addressed the performance degradation problem. #### Shared Volume Mappers `vtk.js` provides standard rendering functionalities which we use for rendering. In addition, in `Cornerstone3D` we have introduced `Shared Volume Mappers` to enable re-using the texture for any viewport that might need it without duplicating the data. For instance for PET-CT fusion which has 3x3 layout which includes CT (Axial, Sagittal, Coronal), PET (Axial, Sagittal, Coronal) and Fusion (Axial, Sagittal, Coronal), we create two volume mappers for CT and PET individually, and for the Fusion viewports we re-use both created textures instead of re-creating a new one. #### Rendering Engine Implementations Cornerstone3D provides two rendering engine implementations to handle different use cases and overcome technical limitations: #### TiledRenderingEngine The `TiledRenderingEngine` is the original implementation that uses a single, large offscreen canvas for all viewports. This approach: - Creates one massive offscreen canvas that grows horizontally as viewports are added - Renders all viewports to specific coordinates on this single offscreen canvas - Copies pixel data from the offscreen canvas to individual onscreen viewports **Limitations of TiledRenderingEngine:** - **Canvas Size Limits**: Browsers impose maximum canvas dimensions (e.g., 16,384px in Chrome). When the combined width of all viewports exceeds this limit, the offscreen canvas is silently cropped, causing severe visual artifacts, misaligned viewports, and blank viewports - **Performance Degradation**: As the offscreen canvas approaches size limits, performance degrades significantly, especially on high-resolution displays or layouts with many viewports - **Multi-Monitor Issues**: Practically impossible to use across multiple high-resolution monitors due to canvas size limitations - **Memory Consumption**: Allocates a huge, memory-intensive offscreen canvas regardless of actual viewport usage **Advantages of TiledRenderingEngine:** - **Simplicity**: Straightforward implementation that works well for small numbers of viewports - **Track Record**: Proven reliability for 5 years, and for most basic use cases, it performs adequately #### ContextPoolRenderingEngine (SequentialRenderingEngine) The `ContextPoolRenderingEngine` (internally called `SequentialRenderingEngine`) fundamentally solves the limitations of the tiled approach by using a different rendering strategy: - Renders each viewport individually to a viewport-sized offscreen canvas - Copies the result to the corresponding onscreen canvas - Proceeds sequentially to the next viewport, reusing the same offscreen canvas - Utilizes WebGL context pooling to render in batches (e.g., batches of 8 for 8 WebGL contexts) **Advantages of ContextPoolRenderingEngine:** - **No Canvas Size Limits**: The browser's maximum canvas size now applies to individual viewports, not the combined width - **Improved Performance**: Consistent performance regardless of the number of viewports or display resolution - **Better Memory Usage**: Avoids allocating massive offscreen canvases - **Multi-Monitor Support**: Enables smooth performance across multiple high-resolution monitors - **Enhanced Stability**: Reduces WebGL context loss associated with huge canvas surfaces #### Configuring the Rendering Engine The `ContextPoolRenderingEngine` is now the default in Cornerstone3D. If you need to use the legacy `TiledRenderingEngine`, you can configure it during initialization: ```js import { init } from '@cornerstonejs/core'; // To use the legacy TiledRenderingEngine init({ rendering: { renderingEngineMode: 'standard', }, }); // The ContextPoolRenderingEngine is used by default, or you can explicitly set it init({ rendering: { renderingEngineMode: 'next', }, }); ``` For `ContextPoolRenderingEngine` you can also configure the number of WebGL contexts to use for batch rendering: ```js import { init } from '@cornerstonejs/core'; // To use the ContextPoolRenderingEngine with a specific number of WebGL contexts init({ rendering: { renderingEngineMode: 'next', webGLContextCount: 7, // Default is 7, can be adjusted based on your needs }, }); ``` #### General usage After creating a renderingEngine, we can assign viewports to it for rendering. There are two main approach for creating `Stack` or `Volume` viewports which we will discuss now. #### Instantiating a `RenderingEngine` You can instantiate a `RenderingEngine` by calling the `new RenderingEngine()` method. ```js import { RenderingEngine } from '@cornerstonejs/core'; const renderingEngineId = 'myEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); ``` #### Viewport Creation You can then use two methods to create viewports: `setViewports` or `enable/disable` APIs. For both methods, a ViewportInput object is passed as an argument. ```js PublicViewportInput = { /** HTML element in the DOM */ element: HTMLDivElement /** unique id for the viewport in the renderingEngine */ viewportId: string /** type of the viewport VolumeViewport or StackViewport*/ type: ViewportType /** options for the viewport */ defaultOptions: ViewportInputOptions } ``` #### setViewports API `setViewports` method is suitable for creation of a set of viewports at once. After setting the array of viewports, the `renderingEngine` will adapt its offScreen canvas size to the size of the provided canvases, and triggers the corresponding events. ```js const viewportInput = [ // CT Volume Viewport - Axial { viewportId: 'ctAxial', type: ViewportType.ORTHOGRAPHIC, element: htmlElement1, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }, // CT Volume Viewport - Sagittal { viewportId: 'ctSagittal', type: ViewportType.ORTHOGRAPHIC, element: htmlElement2, defaultOptions: { orientation: Enums.OrientationAxis.SAGITTAL, }, }, // CT Axial Stack Viewport { viewportId: 'ctStack', type: ViewportType.STACK, element: htmlElement3, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }, ]; renderingEngine.setViewports(viewportInput); ``` #### Enable/Disable API For having a full control over enabling/disabling each viewport separately, you can use the `enableElement` and `disableElement` API. After enabling the element, `renderingEngine` adapts its size and state with the new element. ```js const viewport = { viewportId: 'ctAxial', type: ViewportType.ORTHOGRAPHIC, element: element1, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }; renderingEngine.enableElement(viewport); ``` You can disable any viewport by using its `viewportId`, after disabling, renderingEngine will resize its offScreen canvas. ```js renderingEngine.disableElement(viewportId: string) ``` --- ### Request Pool Manager Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/requestPoolManager.md #### RequestPool Manager The RequestPool Manager has been extensively reworked to provide two new features: 1) `asynchronous image retrieval and decoding` 2) `requests re-ordering`. #### ImageLoad and ImageRetrieval Queues Previously, there was just one loading queue for fetching and decoding an image. Once the image decoding was completed, a new request was initiated. This had a constraint for when decoding required time; thus, no new retrieval (fetch) requests would be sent, even if additional requests were permitted based on the configured maximum number of requests. To overcome this limitation, two distinct queues have been created for this purpose: `imageRetrievalPoolManager` and `imageLoadPoolManager`, each with their own configurable maximum concurrent jobs. They are separated and executed asynchronously from one another, allowing each retrieval request to be initiated instantly upon the availability of a request firing slot. Splitting the image retrieval request and decoding is enabled by default `Cornerstone-wado-image-loader` version `v4.0.0-rc` or above. ```js // Loading = Retrieval + Decoding imageLoadPoolManager.maxNumRequests = { interaction: 1000, thumbnail: 1000, prefetch: 1000, }; // Retrieval (usually) === XHR requests imageRetrievalPoolManager.maxNumRequests = { interaction: 20, thumbnail: 20, prefetch: 20, }; ``` #### Usage In your custom `imageLoader` or `volumeLoader`, to properly use the poolManagers inside cornerstone, you need to define a `sendRequest` function to make an load image request. ```js import { imageLoadPoolManager, loadAndCacheImage, RequestType, } from '@cornerstonejs/core'; function sendRequest(imageId, imageIdIndex, options) { return loadAndCacheImage(imageId, options).then( (image) => { // render successCallback.call(this, image, imageIdIndex, imageId); }, (error) => { errorCallback.call(this, error, imageIdIndex, imageId); } ); } const imageId = 'schema://image'; const imageIdIndex = 10; const requestType = RequestType.INTERACTION; const priority = -5; const additionalDetails = { imageId }; const options = { targetBuffer: { type: 'Float32Array', }, }; imageLoadPoolManager.addRequest( sendRequest.bind(this, imageId, imageIdIndex, options), requestType, additionalDetails, priority ); ``` #### Requests re-ordering You could have a certain sequence in mind for retrieving the images. For example, suppose you want to load a volume from the middle slice to the top and bottom. We have implemented such option in the `cornerstoneStreamingImageVolumeLoader`. You can read more about it in the [re-ordering requests](../streaming-image-volume/re-order) section. --- ### Stack Viewport Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/stackViewport.md #### Stack Viewport This documentation is under development. --- ### Viewport Image Selection Reference and Presentation Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/viewportReferencePresentation.md #### Viewport Image Selection Reference and Presentation The reference and presentation information for a viewport specify what image a viewport is displaying, and the presentation of the image. These are specified in several ways so that a view can be transfered from one viewport to another, or can be remembered in order to restore a view later. Getting a reference can be done either for the current image, or a specific image in the stack, ordered/numbered in the same way that the scrolling positions are numbered/ordered. Some specific use cases for this are: - Referencing a specific image for a tool - Uses `ViewReference` to specify an image to apply to - Uses `isReferenceCompatible` to determine if the tool should be displayed or not - Uses `isReferenceCompatible` to determine which of a set of viewports is best suited to navigating to an image - Uses `setViewReference(viewRef)` to navigate to the specified image - Restoring an earlier view or converting from stack to volume or vice-versa - Uses `ViewReference` and `ViewPresentation` to store the image information - Applying interpolation to image sets - Uses `getViewReference` with a specific image position to get references to images in between or related to nearby annotations to interpolate - Resizing and sync display presentation - Legacy viewports use `getViewPresentation` to get old presentation information and then restore with `setViewPresentation(viewPres)` - Direct Generic/Next viewports read presentation with `viewportProjection.getPresentation(viewport)` and apply the returned native view state from `viewportProjection.withPresentation(...)` with `viewport.setViewState(...)` #### View Reference A view reference specifies what image a view contains, typically identified as the referenced image id, as well as the frame of reference/focal point related information. Specifically, this allows correct correlation between viewports containing the same images or same frame of reference but in different orderings, stack image Ids or volumes. A very important use case for the view reference is as a base for the metadata for annotations where the annotation metadata specifies which image it applies to. The view reference in that case is used both to determine if an image is applicable to a given view, as well as to determine if a viewport could navigate to display the given annotation, either with or without navigation and/or orientation changes. Then, to navigate to the given reference, the `viewport.setViewReference` is called to apply the given navigation. This can apply to both orthographic and stack viewports. The `ViewReference` contains a number of fields that determine the view, the important ones being `referencedImageId` for stack views, and `volumeId` combined with `cameraFocalPoint, viewPlaneNormal, FrameOfReferenceUID` for volumes. Where possible, both the stack and volume viewports populate both sets of information in order to allow the view to apply to either image type. #### referencedImageId The referenced image id allows specifying non-frame of reference based stack type images. This is a single image typically, and can be used by stack viewports to navigate to a specific image. The value is provided by orthographic viewports when getting a reference to an acquisition orientation single image, so that those view references are compatible to stack viewports. #### `referencedImageId` and `sliceIndex` The stack viewport uses the sliceIndex and referencedImageId combined to try to quickly guess the `imageIdIndex` value for a given referencedImageId. If the referencedImageId is identical to the one at the given sliceIndex then it can directly use the sliceIndex, otherwise it needs to find the `imageIdIndex`. `sliceIndex` is never required. For video viewports, the referenced image id will be the video image id, while the slice index can be either a single frame or it can be an array range #### Frame of reference, focal point and normal The frame of reference and focal point/normal values can be used by orthographic viewports to specify other views than the acquisition plane views. The values are provided when available from the stack viewports and can be consumed by the volume viewport. Currently all three are required for applying to a volume viewport, although in the future it may become possible to specify views in other ways than providing a normal. #### `volumeId`, `sliceIndex` and `viewPlaneNormal` When a orthographic viewport creates a view reference, it includes the volume id, slice index and view plane normal. This allows for quick identification of whether a viewport is showing a given reference, as well as navigating quickly to the given view. This is primarily used in `isReferenceCompatible` which can be called many times on orthographic views in order to determine annotation tool views. Note that a stack viewport will not provide the `volumeId`, so this optimization cannot be used for those references. These values are not required for navigation, but for annotation display detection they are required to detect the view applicability. #### Stack Viewport References The stack viewport creates references containing: - referencedImageId and sliceIndex - Frame of reference, focal point and normal when available It can do this for both the currently displayed image, and images referenced by slice index, where the slice index is the index into the imageIds. _warning_ do not assume that the slice index for volumes in any way correlates to slice indices for stacks, or that two stacks displaying the SAME image use corresponding slice indices, or that the frame number has ANY correlation to slice index or vice-versa. The stack viewport can only navigate to a view reference containing a referencedImageId, it will (cannot in fact because of missing information) navigate or discover the appropriate images based on volume/camera etc. The isReferenceCompatible for stack viewports will additionally use the slice index for a quick check of whether the image is found at the given location, but does not rely on the slice index for that, it is just faster that way. #### Volume Viewport References Volume viewports create references with: - referencedImageId appropriate for an acquisition view - Frame of reference, focal point and normal Additionally, orthographic viewports add: - volumeId and slice index for the view in focus. The orthographic viewport will first use any volume id, slice index and normal to determine whether the reference applies or to navigate to it. Both volume viewports will then apply the frame of reference/focal/normal. Specific additional behaviour for detecting 1d and 2d points maybe added in the future (to allow lines and points to appear on view other than original). #### View Presentation The view presentation specifies the pan, zoom and VOI information for a viewport. The pan and zoom are specified as percentage values relative to the viewport size and the original display area (which is included if specified). This allows applying the same view presentation to a variety of viewport sizes that may or may not display the same image instance. The VOI is relative to the base LUT specified in the image data. That is, it excludes modality and presentation LUT transforms. Currently only window width/center is specified, although full lookup tables may be allowed later. Some typical uses cases for view presentation are: - Remembering how an image is presented to allow displaying the same presentation later, e.g., when a viewport is used to display another stack and then is returned to the original stack. - Syncing similar but not identical viewports, for example, syncing some or all presentation attributes between different CT views. - Resizing of viewports, used to remember the relative positions so that the image remains in the same "relative" position. #### `setViewReference` and View Presentation The `viewport.setViewReference` API navigates to the specified reference. Legacy viewport classes and temporary compatibility adapters also expose `viewport.setViewPresentation` to apply presentation directly, but that legacy presentation mutation helper should be expected to be removed from the compatibility layer in a later breaking release. Direct Generic/Next viewports do not expose that mutation API; use `viewportProjection.withPresentation(...)` to translate a presentation patch to the viewport family's native `ViewState`, then call `viewport.setViewState(...)`. If both reference and presentation are being applied, then the view reference must be applied first. A render is required afterwards to complete the view change since multiple parts of the view may be affected. Some example code is shown below for various uses. This assumes that `viewports` is an array of viewports of various types, and that `viewport` is a specific one to apply a change to. The reference and presentation are in `viewRef` and `viewPres` respectively. #### Navigate to a given annotation ```javascript const { metadata } = annotation; if (viewport.isReferenceCompatible({ withNavigation: true })) { viewport.setViewReference(metadata); } else { // throw error indicating view isn't compatible or other behaviour // such as changing to a volume or display a different set of images ids etc } ``` #### Finding the best viewport for displaying an annotation ```javascript function findViewportForAnnotation(annotation, viewports) { const { metadata } = annotation; // If there is a viewport already displaying this, then just return it. const alreadyDisplayingViewport = viewports.find((viewport) => viewport.isReferenceCompatible(metadata) ); if (alreadyDisplayingViewport) return alreadyDisplayingViewport; // If there is a viewport that just needs navigation, then return it const navigateViewport = viewports.find((viewport) => viewport.isReferenceCompatible(metadata, { withNavigation: true }) ); if (navigateViewport) return navigateViewport; // If there is a viewport showing the volume that could have orientation changed, use it const orientationViewport = viewports.find((viewport) => viewport.isReferenceCompatible(metadata, { withOrientation: true }) ); if (orientationViewport) return orientationViewport; // If there is a stack viewport that could be converted to volume to show this, then do so const stackToVolumeViewport = viewports.find((viewport) => viewport.isReferenceCompatible(metadata, { withOrientation: true, asVolume: true, }) ); if (stackToVolumeViewport) { // convert stack to volume viewport here return stackToVolumeViewport; } // Might also look for viewport showing same frame of reference, but different volume // Find the set of image ids or volumeId from the metadata and apply that // to the viewport at position 0 and display it. } ``` #### Resize the viewport(s) ```javascript const resizeObserver = new ResizeObserver(() => { if (resizeTimeout) { return; } resizeTimeout = setTimeout(resize, 100); }); function resize() { resizeTimeout = null; const renderingEngine = getRenderingEngine(renderingEngineId); if (renderingEngine) { // Legacy viewport path: store the presentation from before for after. const presentations = viewports.map((viewport) => viewport.getViewPresentation() ); // Apply the resize renderingEngine.resize(true, false); // Restore the presentations as this will reset the relative positions // rather than resetting to null. viewports.forEach((viewport, idx) => { viewport.setViewPresentation(presentations[idx]); }); } } resizeObserver.observe(viewportGrid); ``` For direct Generic/Next viewports, keep presentation reads and writes on the projection service: ```javascript const presentations = viewports.map((viewport) => viewportProjection.getPresentation(viewport) ); renderingEngine.resize(true, false); viewports.forEach((viewport, index) => { const nextViewState = viewportProjection.withPresentation( viewport, presentations[index] ); if (nextViewState) { viewport.setViewState(nextViewState); } }); ``` --- ### Viewports Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/viewports.md #### Viewports A viewport can be thought of as: - A camera viewing an image from a specific perspective. - A canvas to display the output of this camera. - A set of transforms from the image data to viewable data (LUT, Window Level, Pan etc) In `Cornerstone3D` viewports are created from HTML elements, and the consumer should pass the `element` for which the viewport should be created. For example, a CT series can be viewed via 4 viewports in a “4-up” view: Axial MPR, Sagittal MPR, Coronal MPR, A 3D perspective volume render. See [Viewport Reference and Presentation](./viewportReferencePresentation.md) for more details on the reference and presentation details that select which image and how that image is presented. Generic/Next viewport implementations use semantic `ViewState` as their mutation source of truth. Code that needs cross-viewport pan, zoom, rotation, scale, or renderer-camera output should use [`Viewport Projection`](./generic-viewport/viewport-projection.md) instead of copying camera objects between viewport families.
![](../../assets/viewports.png)
#### StackViewport - Suitable for rendering a stack of images, that might or might not belong to the same image. - Stack can include 2D images of various shapes, size and direction #### VolumeViewport - Suitable for rendering a volumetric data which is considered as one 3D image. - Having a VolumeViewport enables Multi-planar reformation or reconstruction (MPR) by design, in which you can visualize the volume from various different orientations without addition of performance costs. - For having image fusion between two series #### 3D Viewport - Sutiable for actual 3D rendering of a volumetric data. - For having different types of presets such as Bone, Soft Tissue, Lung, etc. :::note Both `StackViewport` and `VolumeViewport`, `VolumeViewport3D` are created via the `RenderingEngine` API. ::: #### VideoViewport - Suitable for rendering video data - Video can include MPEG 4 encoded vide streams. In theory, MPEG2 is also supported, but practically the browser doesn't support that. #### Whole Slide Image Viewport - Suitable for rendering whole slide images #### Initial Display Area All viewports inherit from the Viewport class which has a `displayArea` field which can be provided. This field can be used to programmatically set the initial zoom/pan on an image. By default, the viewport will fit the dicom image to the screen. The `displayArea` takes a `DisplayArea` type which has the following fields. ```js type DisplayArea = { imageArea: [number, number], // areaX, areaY imageCanvasPoint: { imagePoint: [number, number], // imageX, imageY canvasPoint: [number, number], // canvasX, canvasY }, storeAsInitialCamera: boolean, }; ``` Zoom and pan are all relative to the initial "fit to screen" view. In order to zoom into the image 200%, we would set the `imageArea` to [0.5, 0.5]. Panning is controlled by a provided `imagePoint` and a provided `canvasPoint`. You can imagine the canvas as a sheet of white paper and the image as another sheet of paper like a chest x-ray. Mark a point in the canvas paper with a pen and then mark another point on your chest x-ray image. Now try to "pan" your image so the point so the `imagePoint` matches the `canvasPoint`. This is what the API design of `imageCanvasPoint` represents. Thus if you wanted to left align you image, you could provide the following value: ```js imageCanvasPoint: { imagePoint: [0, 0.5], // imageX, imageY canvasPoint: [0, 0.5], // canvasX, canvasY }; ``` This means the left (0) middle (0.5) point on the canvas needs to align with the left (0) middle (0.5) point on the image. Values are based on % size of the full image. In this example, if we had a 1024 x 1024 x-ray image. The imagePoint would be [0, 512]. Let's say we were on a mobile iPhone in landscape mode (844 x 390). The canvasPoint would be [0, 195]. --- ### Volume Loaders Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/volumeLoader.md #### Volume Loaders Similar to the [`Image Loaders`](./imageLoader.md), a volume loader takes a `volumeId` and other information that is required to load a volume and returns a `Promise` that resolves into a `Volume`. This `Volume` can be a constructed from a set of 2D images (e.g., `imageIds`) or can be from one 3D array object (such as `NIFTI` format). We have added [`cornerstoneStreamingImageVolumeLoader`](/docs/concepts/streaming-image-volume/streaming) library to support streaming of the 2D images (`imageIds`) into a 3D volume and it is the default volume loader for streaming volumes. #### Register Volume Loaders You can use [`registerVolumeLoader`](/docs/api/core/namespaces/volumeloader/functions/registervolumeloader) to define a volume loader which should be called on a particular `scheme`. Below you can see a simplified code for our `cornerstoneStreamingImageVolumeLoader` in which: 1. Based on a set of imageIds, we compute volume metadata such as: spacing, origin, direction, etc. 2. Instantiate a new [`StreamingImageVolume`](/docs/api/core/classes/streamingimagevolume/) - `StreamingImageVolume` implements methods for loading (`.load`) - It implements load via using `imageLoadPoolManager` - Each loaded frame (imageId) is put at the correct slice in the 3D volume 3. Return a `Volume Load Object` which has a promise that resolves to the `Volume`. ```js function cornerstoneStreamingImageVolumeLoader( volumeId: string, options: { imageIds: Array, } ) { // Compute Volume metadata based on imageIds const volumeMetadata = makeVolumeMetadata(imageIds); const streamingImageVolume = new StreamingImageVolume( // ImageVolume properties { volumeId, metadata: volumeMetadata, dimensions, spacing, origin, direction, scalarData, sizeInBytes, }, // Streaming properties { imageIds: sortedImageIds, loadStatus: { loaded: false, loading: false, cachedFrames: [], callbacks: [], }, } ); return { promise: Promise.resolve(streamingImageVolume), cancel: () => { streamingImageVolume.cancelLoading(); }, }; } registerVolumeLoader( 'cornerstoneStreamingImageVolume', cornerstoneStreamingImageVolumeLoader ); // Used for any volume that its scheme is not provided registerUnknownVolumeLoader(cornerstoneStreamingImageVolumeLoader); ``` As seen above, since the `cornerstoneStreamingImageVolumeLoader` is registered with the scheme `cornerstoneStreamingImageVolume`, we can load a volume with the scheme `cornerstoneStreamingImageVolume` by passing the `volumeId` as shown below: ```js const volumeId = 'cornerstoneStreamingImageVolume:myVolumeId'; const volume = await volumeLoader.createAndCacheVolume(volumeId, { imageIds: imageIds, }); ``` #### Default unknown volume loader By default if no `volumeLoader` is found for the scheme, the `unknownVolumeLoader` is used. `cornerstoneStreamingImageVolumeLoader` is the default unknown volume loader. :::info Even if you don't provide the scheme, the `cornerstoneStreamingImageVolumeLoader` will be used by default. So the following code will work as well: ```js const volumeId = 'myVolumeId'; const volume = await volumeLoader.createAndCacheVolume(volumeId, { imageIds: imageIds, }); ``` --- ### Volume Viewport Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/volumeViewport.md #### Volume Viewport This documentation is under development. --- ### Volumes Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/volumes.md #### Volumes A volume is a 3D data array that has a physical size and orientation in space. It can be built by composing pixel data and metadata of a 3D imaging series, or can be defined from scratch by the application. A volume has a `FrameOfReferenceUID`, `voxelSpacing (x,y,z)`, `voxel dimensions (x,y,z)`, `origin`, and `orientation` vectors which uniquely define its coordinate system with respect to the patient coordinate system. #### ImageVolume In `Cornerstone3D` we use the `ImageVolume` base class to represent a 3D image volume. All volumes are derived from this class. For instance the `StreamingImageVolume` which is used to represent a volume that is being streamed image by image. We will discuss the `StreamingImageVolume` class in more detail later. ```js interface IImageVolume { /** unique identifier of the volume in the cache */ readonly volumeId: string /** volume dimensions */ dimensions: Point3 /** volume direction */ direction: Float32Array /** volume metadata */ metadata: Metadata /** volume origin - set to the imagePositionPatient of the last image in the volume */ origin: Point3 /** volume scaling metadata */ scaling?: { PET?: { SUVlbmFactor?: number SUVbsaFactor?: number suvbwToSuvlbm?: number suvbwToSuvbsa?: number } } /** volume size in bytes */ sizeInBytes?: number /** volume spacing */ spacing: Point3 /** number of voxels in the volume */ numVoxels: number /** volume image data as vtkImageData */ imageData?: vtkImageData /** openGL texture for the volume */ vtkOpenGLTexture: any /** loading status object for the volume containing loaded/loading statuses */ loadStatus?: Record /** imageIds of the volume (if it is built of separate imageIds) */ imageIds?: Array /** volume referencedVolumeId (if it is derived from another volume) */ referencedVolumeId?: string // if volume is derived from another volume /** voxel manager */ voxelManager?: IVoxelManager } ``` #### Voxel Manager The `VoxelManager` is responsible for managing the voxel data of a volume. In previous version of `Cornerstone3D` we used to include `scalarData` in the `ImageVolume` object. However, this approach had several limitations in memory usage and performance. Therefore, we now delegate the voxel data management to the `VoxelManager` class which is a stateful class that keeps track of the voxel data in a volume. You can read more about the `VoxelManager` class [here](./voxelManager.md). --- ### Voxel Manager Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/voxelManager.md #### VoxelManager Documentation The VoxelManager is a key component of the Cornerstone library’s new architecture for handling voxel data and volume management. This updated design streamlines data flow and enhances performance, providing a single source of truth for image caching and data access, with a focus on reducing memory usage and improving performance in handling large image datasets. #### Overview With the integration of VoxelManager, voxel data handling shifts from relying on large scalar arrays to using individual images and targeted voxel data access methods. VoxelManager serves as an adapter for tools and functions that interact with voxel data, providing efficient methods for accessing, modifying, and streaming voxel information. #### Key Features - **Single Source of Truth**: Only the image cache is used, eliminating the need for separate volume caches and reducing synchronization issues. - **Efficient Volume Streaming**: Loads image by image, caching only what’s necessary and streaming data directly to the GPU. - **Optimized Caching**: Data is stored in its native format and converted only as needed, minimizing memory and processing overhead. - **Simplified Web Worker Implementation**: Removed `SharedArrayBuffer` dependencies, simplifying security and worker requirements. #### VoxelManager API The VoxelManager API replaces direct scalar data access with methods that provide precise control over voxel data without generating large data arrays. Here are the primary methods and usage patterns: #### Accessing Voxel Data - **`getScalarData()`**: Returns the scalar data array for individual images (applicable only to `IImage`). - **`getScalarDataLength()`**: Provides the total voxel count, replacing `scalarData.length`. - **`getAtIndex(index)`**: Retrieves the voxel value at a specific linear index. - **`setAtIndex(index, value)`**: Sets the voxel value at a specific linear index. - **`getAtIJK(i, j, k)`**: Gets the voxel value at IJK coordinates. - **`setAtIJK(i, j, k, value)`**: Sets the voxel value at IJK coordinates. - **`getArrayOfModifiedSlices()`**: Lists modified slice indices. #### Data Manipulation - **`forEach(callback, options)`**: Iterates over voxels with a callback for processing or modifying data. - **`toIndex(ijk)`**: Converts IJK coordinates to a linear index. - **`toIJK(index)`**: Converts a linear index back to IJK coordinates. #### Volume Information - **`getConstructor()`**: Returns the scalar data type constructor. - **`getBoundsIJK()`**: Fetches the volume bounds in IJK coordinates. #### Specialized Methods - **`setTimePoint(timePoint)`**: For 4D datasets, sets the current time point. - **`getAtIndexAndTimePoint(index, timePoint)`**: Retrieves the voxel value at a specified index and time point. #### Example: Migrating Data Access and Manipulation Instead of accessing `scalarData` directly, use VoxelManager for data manipulation. Here’s a migration example: #### Before ```javascript function processVolume(volume) { const scalarData = volume.getScalarData(); for (let i = 0; i < scalarData.length; i++) { if (scalarData[i] > 100) { scalarData[i] = 100; } } } ``` #### After ```javascript function processVolume(volume) { const voxelManager = volume.voxelManager; const length = voxelManager.getScalarDataLength(); for (let i = 0; i < length; i++) { const value = voxelManager.getAtIndex(i); if (value > 100) { voxelManager.setAtIndex(i, 100); } } } ``` #### Handling Image Volume Construction When creating volumes, `scalarData` is no longer required. Instead, use `VoxelManager` internally: #### Before ```typescript const streamingImageVolume = new StreamingImageVolume({ volumeId, metadata, dimensions, spacing, origin, direction, scalarData, sizeInBytes, imageIds, }); ``` #### After ```typescript const streamingImageVolume = new StreamingImageVolume({ volumeId, metadata, dimensions, spacing, origin, direction, imageIds, dataType, numberOfComponents, }); ``` #### Best Practices - **Data Access Optimization**: Use `getAtIndex` and `setAtIndex` for bulk operations due to their efficiency. Use `forEach` for large-volume iteration. - **Memory Management**: Avoid `getCompleteScalarDataArray()` as it rebuilds large data arrays and can degrade performance. - **Handling RGB Data**: `getAtIndex` and `getAtIJK` return `[r, g, b]` arrays for RGB volumes. #### Conclusion The VoxelManager is central to Cornerstone’s new volume management strategy, offering a flexible, efficient API for voxel data access and manipulation. This migration to VoxelManager allows for more efficient memory usage, faster performance, and improved compatibility with large datasets, ensuring a smoother workflow for developers working with complex medical imaging data. --- ### Web Workers Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/web-worker.md WebWorkers provide a way to run scripts in background threads, allowing web applications to perform tasks without interfering with the user interface. They are particularly useful for executing computationally intensive tasks or those that require a lot of processing time. Generally, working with workers requires a lot of boilerplate code, postMessage calls, and event listeners. Cornerstone provides a simple API to create and use workers, hiding all the complexity for you. #### Requirements You need to install [`comlink`](https://www.npmjs.com/package/comlink) as a dependency to your application. That is all. `comlink` is a library that allows you to use WebWorkers as if they were local objects, without having to worry about the underlying messaging. Although it doesn't handle priority queues, load balancing or worker lifecycle, it provides a simple API to communicate with workers which is used by Cornerstone to create a more robust and user-friendly API. #### Usage Example It would be easier for us to explain the WebWorker API by using an example. Let's say you have a set of functions that you want to run in the background. You need to write an object that exposes these functions via comlink. ```js // file/location/my-awesome-worker.js import { expose } from 'comlink'; const obj = { counter: 69, inc() { obj.counter++; console.debug('inc', obj.counter); }, fib({ value }) { if (value <= 1) { return 1; } return obj.fib({ value: value - 1 }) + obj.fib({ value: value - 2 }); }, }; expose(obj); ``` :::note As you can see above, our object can contain any number of functions and can hold a local state. The only requirement for these functions is that the arguments SHOULD BE serializable. This means that you can't pass DOM elements, functions, or any other non-serializable objects as arguments. We use objects for arguments. So, in the above we use `fib({value})` instead of `fib(value)` (`value` is just an argument name; you can use any name you want for the argument.) ::: Now, the key is to inform Cornerstone about this function so that it can run smoothly in the background. Let's dive in. #### WebWorker Manager The WebWorkerManager plays a crucial role in the WebWorker API. Its main function is to create and supervise workers. By assigning tasks with different priorities and queue types, you can rely on the manager to effectively execute them in the background, based on the specified priority. Furthermore, it handles the lifecycle of workers, distributes the workload, and provides a user-friendly API for executing tasks. #### `registerWorker` Registers a new worker type with a unique name and a function to let the manager know about it. Arguments are - `workerName`: the name of the worker type (should be unique) and we use this later to invoke functions. - `workerFn`: a function that returns a new Worker instance (more on this later) - `options` an object with the following properties: - `maxWorkerInstances(default=1)`: the maximum number of instances of this worker type that can be created. More instance mean if there are multiple calls to the same function they can be offloaded to the other instances of the worker type. - `overwrite (default=false)`: whether to overwrite an existing worker type if already registered - `autoTerminateOnIdle` (default false) can be used to terminate a worker after a certain amount of idle time (in milliseconds) has passed. This is useful for workers that are not used frequently, and you want to terminate them after a specific period of time. on the manager. The argument for this method is the object of `{enabled: boolean, idleTimeThreshold: number(ms)}`. :::tip Note that if a worker is terminated it does not mean the worker is destroyed from the manager. In fact any subsequent call to the worker will create a new instance of the worker and everything would worker as expected. ::: So to register the worker we created above, we would do the following: ```js import { getWebWorkerManager } from '@cornerstonejs/core'; const workerFn = () => { return new Worker( new URL( '../relativePath/file/location/my-awesome-worker.js', import.meta.url ), { name: 'ohif', // name used by the browser to name the worker } ); }; const workerManager = getWebWorkerManager(); const options = { // maxWorkerInstances: 1, // overwrite: false }; workerManager.registerWorker('ohif-worker', workerFn, options); ``` In the above as you see you need to create a function that returns a new Worker instance. In order for the worker to work, it should lie in a directory that is accessible by the main thread (it can be relative to the current directory). :::note There are two names that you can specify: 1. The `name` in the workerFn which is used by the browser to show the worker name in the debugger 2. The registration name, which we later use to invoke the functions ::: #### `executeTask` Until now, the manager only knows about the workers that are available, but it doesn't know what to do with them. the `executeTask` is used to execute a task on a worker. It takes the following arguments: - `workerName`: the name of the worker type that we registered earlier - `methodName`: the name of the method that we want to execute on the worker (the function name, in the above example `fib` or `inc`) - `args` (`default = {}`): the arguments that are passed to the function. The arguments should be serializable which means you cannot pass DOM elements, functions, or any other non-serializable objects as arguments (check below on how to pass non-serializable functions) - `options` an object with the following properties: - `requestType (default = RequestType.COMPUTE)` : the group of the request. This is used to prioritize the requests. The default is `RequestType.COMPUTE` which is the lowest priority. Other groups in order of priority are `RequestType.INTERACTION` and `RequestType.THUMBNAIL`, `RequestType.PREFETCH` - `priority` (`default = 0`): the priority of the request within the specified group. The lower the number the higher the priority. - `options` (`default= {}`): the options to the pool manager (you most likely don't need to change this) - `callbacks` (`default = []`): pass in any functions that you want to be called inside the worker. Now to execute the `fib` function on the worker we would do the following: ```js import { getWebWorkerManager } from '@cornerstonejs/core'; const workerManager = getWebWorkerManager(); workerManager.executeTask('ohif-worker', 'fib', { value: 10 }); ``` The above will execute the `fib` function on the worker with the name `ohif-worker` with the argument `{value: 10}`. Of course this is a simplified example, often you need to perform some actions when the task is completed or failed. Since the return of the `executeTask` is a promise, you can use the `then` and `catch` methods to handle the result. ```js workerManager .executeTask('ohif-worker', 'fib', { value: 10 }) .then((result) => { console.log('result', result); }) .catch((error) => { console.error('error', error); }); ``` or simply you can await the result ```js try { const result = await workerManager.executeTask('ohif-worker', 'fib', { value: 10, }); console.log('result', result); } catch (error) { console.error('error', error); } ``` #### `eventListeners` Sometimes, it is necessary to provide a callback function to the worker. For instance, if you wish to update the user interface when the worker makes progress. As mentioned earlier, it is not possible to directly pass a function as an argument to the worker. However, you can overcome this issue by utilizing the `callbacks` property in the options. These `callbacks` are conveniently passed as arguments to the function based on their position. Real Example from the codebase: ```js const results = await workerManager.executeTask( 'polySeg', 'convertContourToSurface', { polylines, numPointsArray, }, { callbacks: [ (progress) => { console.debug('progress', progress); }, ], } ); ``` Above as you can see we pass a function to the worker as a callback. The function is passed as the NEXT argument to the worker after the args. In the worker we have ```js import { expose } from 'comlink'; const obj = { async convertContourToSurface(args, ...callbacks) { const { polylines, numPointsArray } = args; const [progressCallback] = callbacks; await this.initializePolySeg(progressCallback); const results = await this.polySeg.instance.convertContourRoiToSurface( polylines, numPointsArray ); return results; }, }; expose(obj); ``` #### `terminate` For terminating a worker you can use `webWorkerManager.terminate(workerName)`. Stops all instances of a given worker and cleans up resources. --- ## Cornerstone-core/generic-viewport ### API Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/generic-viewport/api.md #### API The Generic Viewport API is centered on logical display set ids. Register the display set once, mount it into a viewport, then update view state and data presentation independently. #### Create A Planar Generic Viewport Use `ViewportType.PLANAR_NEXT` for stack-like and volume-slice 2D workflows. The viewport infers the render path from the registered dataset shape, requested orientation, rendering configuration, WebGL support, and segmentation slice-rendering configuration. ```ts import { Enums, RenderingEngine, viewportProjection, utilities, type PlanarViewport, } from '@cornerstonejs/core'; const renderingEngine = new RenderingEngine('renderingEngineId'); renderingEngine.enableElement({ viewportId: 'CT_AXIAL', type: Enums.ViewportType.PLANAR_NEXT, element, defaultOptions: { background: [0, 0, 0], }, }); const viewport = renderingEngine.getViewport('CT_AXIAL') as PlanarViewport; ``` Planar render-path selection is internal. Stack-like image-id data uses an image path; volume-backed data or reformatted orientations use a volume slice path. CPU/GPU choice is made by the planar render-path decision service from runtime rendering configuration and thresholds. #### Add Stack Data Register stack-like data with the metadata provider, then mount it with `setDisplaySets()`. ```ts const stackDisplaySetId = 'ct-stack'; utilities.genericViewportDisplaySetMetadataProvider.add(stackDisplaySetId, { kind: 'planar', imageIds, initialImageIdIndex: 0, }); await viewport.setDisplaySets({ displaySetId: stackDisplaySetId, }); viewport.setDisplaySetPresentation(stackDisplaySetId, { voiRange: { lower: -1500, upper: 2500 }, }); viewport.render(); ``` `setDisplaySets()` is variadic; the first entry becomes the source binding unless a role is provided explicitly. Each call replaces all currently mounted display sets. `setDisplaySets()` and `addDisplaySet()` do not return runtime rendering ids. Use the `displaySetId` you provided for later presentation updates, removal, and view-reference operations. #### Add Volume Slice Data Volume slice data uses the same viewport API. The registered data includes a `volumeId`, so the viewport selects a volume slice render path. ```ts const ctDataId = 'ct-volume-source'; utilities.genericViewportDisplaySetMetadataProvider.add(ctDataId, { kind: 'planar', imageIds: ctImageIds, initialImageIdIndex: Math.floor(ctImageIds.length / 2), volumeId: ctVolumeId, }); await viewport.setDisplaySets({ displaySetId: ctDataId, options: { orientation: Enums.OrientationAxis.SAGITTAL, }, }); ``` The same calls work for CPU or GPU volume slicing. Configure CPU/GPU preference through rendering configuration and thresholds instead of passing a render mode with the data. #### Add An Overlay Overlays are additional data bindings mounted with `role: 'overlay'`. They use the same viewport view state as the source but keep their own data presentation. ```ts const ptDataId = 'pt-volume-overlay'; utilities.genericViewportDisplaySetMetadataProvider.add(ptDataId, { kind: 'planar', imageIds: ptImageIds, initialImageIdIndex: Math.floor(ptImageIds.length / 2), volumeId: ptVolumeId, }); await viewport.addDisplaySet(ptDataId, { orientation: Enums.OrientationAxis.SAGITTAL, role: 'overlay', }); viewport.setDisplaySetPresentation(ptDataId, { colormap: { name: 'hsv', opacity: 0.4, }, }); viewport.render(); ``` `setDisplaySets()` can also mount source and overlays together: ```ts await viewport.setDisplaySets( { displaySetId: ctDataId, options: { orientation: Enums.OrientationAxis.SAGITTAL, role: 'source', }, }, { displaySetId: ptDataId, options: { orientation: Enums.OrientationAxis.SAGITTAL, role: 'overlay', }, } ); ``` Use `reference` only when the registered data is semantically derived from another object. Source stack and volume data usually do not need it because their `displaySetId`, `imageIds`, and optional `volumeId` are already the public identity. ```ts utilities.genericViewportDisplaySetMetadataProvider.add(labelmapDataId, { kind: 'planar', imageIds: labelmapImageIds, reference: { kind: 'segmentation', segmentationId, representationUID, labelmapId, }, }); ``` #### Update View State Navigation and view appearance are separate from per-dataset appearance. For direct Next viewports, `ViewState` is the only mutable viewport source of truth. Use `setViewState()` for patches and `updateViewState()` for read-modify-write changes. Use `resetViewState()` when you want the viewport family's default navigation reset. ```ts viewport.setViewState({ flipHorizontal: true, rotation: 90, }); viewport.updateViewState(({ rotation = 0 }) => ({ rotation: rotation + 30, })); viewport.resetViewState(); ``` Use viewport projection when the input is a portable presentation patch rather rather than native view state. The projection service is pure: it returns the next native `ViewState`, and the caller applies it. ```ts const nextViewState = viewportProjection.withPresentation(viewport, { zoom: 1.5, pan: [40, -20], }); if (nextViewState) { viewport.setViewState(nextViewState); } viewport.render(); ``` Read presentation through the same service: ```ts const presentation = viewportProjection.getPresentation(viewport, { selector: { pan: true, zoom: true, rotation: true, }, }); ``` Do not call `viewport.getViewPresentation()` or `viewport.setViewPresentation()` on direct Next viewports. Those methods are kept only on temporary legacy compatibility adapters and should be expected to be removed from that compatibility layer in a later breaking release. Use `setImageIdIndex()` for index-style navigation. For volume-backed data, the viewport resolves the requested index into a volume slice point internally. ```ts await viewport.setImageIdIndex(viewport.getCurrentImageIdIndex() + 1); ``` #### Update Display Set Presentation Display set presentation is scoped to one mounted display set id. Call `setDisplaySetPresentation` with just `props` to apply the update to the current source binding, or with an explicit `displaySetId` to target a specific binding. ```ts viewport.setDisplaySetPresentation(ctDataId, { voiRange: { lower: -1500, upper: 2500 }, }); viewport.setDisplaySetPresentation(ptDataId, { visible: false, }); // Apply to the current source binding when the id is omitted. viewport.setDisplaySetPresentation({ voiRange: { lower: -1000, upper: 1000 }, }); viewport.render(); ``` This is the preferred place for VOI, opacity, colormap, invert, blend mode, interpolation, and visibility. #### Labelmap Segmentations Segmentations are still added through `@cornerstonejs/tools`. For Next planar volume-slice viewports, labelmaps can use slice rendering by setting `config.useSliceRendering`. ```ts import * as cornerstoneTools from '@cornerstonejs/tools'; const { segmentation, Enums: csToolsEnums } = cornerstoneTools; const { SegmentationRepresentations } = csToolsEnums; const segmentationId = 'segmentation-volume-id'; segmentation.addSegmentations([ { segmentationId, representation: { type: SegmentationRepresentations.Labelmap, data: { volumeId: segmentationId, }, }, }, ]); await segmentation.addLabelmapRepresentationToViewportMap({ CT_AXIAL: [ { segmentationId, type: SegmentationRepresentations.Labelmap, config: { useSliceRendering: true, }, }, ], }); ``` With `useSliceRendering`, a compatible volume labelmap is rendered through an image/slice path instead of allocating and drawing it as a full 3D labelmap volume. This is useful for planar slice workflows, especially when the source viewport is using a volume slice path. The segmentation display tool registers each labelmap layer as overlay data in the viewport: ```ts await viewport.addDisplaySet(labelmapDataId, { orientation: viewport.getViewState().orientation, role: 'overlay', }); viewport.setDisplaySetPresentation(labelmapDataId, { blendMode: Enums.BlendModes.COMPOSITE, visible: true, }); ``` Application code usually does not need to call this lower-level overlay path directly for segmentations; it is shown here to explain how segmentations map onto the Generic Viewport binding model. #### View References Use view references when transferring spatial location between viewports or restoring a remembered view. ```ts const reference = viewport.getViewReference(); otherViewport.setViewReference(reference); otherViewport.render(); ``` Use projection presentation when only pan, zoom, rotation, flips, and display area should be copied between compatible viewport families. The presentation shape is adapter-specific, so this is appropriate for Planar Next to Planar Next. Do not treat it as a universal cross-family camera copy; use view references or a synchronizer that explicitly maps scale and position semantics for that case. ```ts const presentation = viewportProjection.getPresentation(viewport, { selector: { displayArea: true, flipHorizontal: true, flipVertical: true, pan: true, rotation: true, zoom: true, }, }); if (!presentation) { return; } // `withPresentation` is pure: it translates the presentation for the target // viewport, but it does not mutate the target or schedule rendering. const nextViewState = viewportProjection.withPresentation( otherViewport, presentation ); if (nextViewState) { // `setViewState` remains the single mutation path for Next viewports. otherViewport.setViewState(nextViewState); otherViewport.render(); } ``` --- ### Camera Model Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/generic-viewport/camera.md #### Camera Model Generic viewports use `viewState`, not VTK-style camera fields, as the clean source of truth. The model is: ```text Viewport viewState -> ResolvedView -> renderer projection -> runtime engine state ``` Only `viewState` is durable viewport navigation state. `ResolvedView` is a computed snapshot for the current canvas, data, and state. Renderer projections are commands sent to VTK, CPU canvas, DOM, OpenLayers, or another runtime. Runtime engine state is private to that renderer. Clean Next viewport instances do not expose `getViewPresentation()` or `setViewPresentation()`. Presentation is a projection-service concern: ```ts import { viewportProjection } from '@cornerstonejs/core'; const presentation = viewportProjection.getPresentation(viewport, { selector: { pan: true, zoom: true, rotation: true, }, }); const nextViewState = viewportProjection.withPresentation(viewport, { zoom: 2, pan: [20, -10], }); if (nextViewState) { viewport.setViewState(nextViewState); } ``` `viewportProjection.withPresentation()` is pure. It translates the presentation patch into the viewport family's native `ViewState`, but it does not mutate the viewport and it does not render. `setViewState()` and `updateViewState()` remain the clean Next paths for arbitrary view-state changes; `resetViewState()` is the clean reset helper. For cross-viewport tooling and synchronizers, use the Viewport Projection construct instead of treating `ICamera` as a universal camera model. Viewport Projection exposes capability-checked transforms, semantic scale and position, and optional renderer-camera output. See [Viewport Projection](./viewport-projection.md). > **Naming note.** "Viewport Projection" uses _projection_ in the mathematical > sense — projecting semantic viewport state onto presentation, transforms, and > renderer output. It is distinct from VTK's parallel-vs-perspective projection > (`parallelProjection`), which is a renderer-matrix setting carried on the > resolved `ICamera`. The two concepts coexist in the same code paths but > describe different layers. #### Contract Matrix | Concept | Owns | Does Not Own | | ------------------ | -------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- | | `ViewState` | Mutable viewport-local navigation and layout source of truth. | Cross-viewport persistence by itself. | | `ViewPresentation` | Persistable look state: pan, zoom or scale, rotation, flips, and display area. | Data identity, slice identity, VOI, opacity, or colormap. | | `ViewReference` | Persistable spatial pointer: frame of reference, data id, volume id, image id, slice locator, and plane restriction. | Pan, zoom, rotation, flips, VOI, opacity, or colormap. | | `ResolvedView` | Ephemeral world/canvas transforms, resolved presentation, and renderer geometry. | Durable state or persistence. | | `DataPresentation` | Per-binding appearance such as VOI, opacity, colormap, interpolation, and visibility. | Viewport navigation. | #### Planar View State `PlanarViewState` is semantic. It does not extend `ICamera`, and it does not store `focalPoint`, `position`, `parallelScale`, `viewPlaneNormal`, or `viewUp` as source truth. It stores fields such as: - `orientation` - `slice` - `anchorWorld` - `anchorCanvas` - `scale` - `scaleMode` - `rotation` - `flipHorizontal` - `flipVertical` - `displayArea` Planar slice identity is explicit: - Stack and image paths use `slice.kind === 'stackIndex'`. - Volume paths use `slice.kind === 'volumePoint'`. `setImageIdIndex()` remains a convenience API. For stack data it stores a stack index. For volume data it resolves the requested index into a world point and stores a `volumePoint` slice locator. Crosshairs and navigation tools should use `ViewReference` or `sliceWorldPoint`, not raw camera position. #### Resolved Planar View Planar render code derives VTK-compatible fields from the resolved view. This includes the focal point, position, parallel scale, view plane normal, view up, presentation scale, and slice metadata needed by CPU and VTK paths. Those fields are renderer projection data. They are not copied back into `PlanarViewState` as durable truth. #### Video And ECG Video and ECG viewports also use semantic state as the source of truth. Their rendering code resolves a canvas mapping from: - viewport state - canvas or element dimensions - intrinsic media or waveform metrics - object-fit or signal layout rules The resolved canvas mapping supplies pan, zoom, and canvas/world conversion for tools and renderers. It is not persisted as a camera. Video projection reports intrinsic media-pixel coordinates: - `ProjectionPosition.kind === 'mediaPoint'` - `ProjectionScale.kind === 'nativePixel'` ECG projection reports signal coordinates: - world tuples are `[sampleIndex, amplitudeValue, channelIndex]` - `ProjectionPosition.kind === 'signalPoint'` - `ProjectionScale.kind === 'signal'` #### 3D And WSI Exceptions 3D viewports are runtime-camera-backed. The VTK active camera remains the source of truth, `getViewState()` reads from VTK, and `setViewState()` applies to VTK. Whole-slide image viewports have semantic `WSIViewState`, but they synchronize with OpenLayers before reads and after map interactions. Their projection adapter exposes slide/world transforms, zoom, rotation, and renderer-camera output through `viewportProjection`. #### Camera Patch Migration Legacy code often wrote durable camera fields: ```ts viewport.setCamera({ focalPoint, position, parallelScale, }); ``` For direct Next viewports, prefer native state or projection writes: ```ts viewport.updateViewState((viewState) => ({ ...viewState, anchorWorld: [x, y, z], })); ``` ```ts const nextViewState = viewportProjection.withPresentation(viewport, { zoom: 2, }); if (nextViewState) { viewport.setViewState(nextViewState); } ``` Use `ViewReference` for spatial navigation across slices or datasets: ```ts const reference = sourceViewport.getViewReference(); targetViewport.setViewReference(reference); targetViewport.render(); ``` Use `setCamera()` only on legacy compatibility adapters. Position-only camera patches are not a stable Next-state operation because Next view state stores semantic anchors, slice locators, and scale, not durable renderer position. #### Legacy Compatibility Legacy adapters are the temporary migration boundary for `ICamera`. They exist to keep older applications running while code moves to direct Next viewports, and they should not be treated as the long-term Next API surface. Plan for these compatibility camera methods to be removed in a later breaking release. Clean Generic viewports expose `getViewState()`, `setViewState()`, `updateViewState()`, `resetViewState()`, and `getResolvedView()`. Legacy adapters expose `getCamera()`, `setCamera()`, `resetCamera()`, `getViewPresentation()`, and `setViewPresentation()` for old APIs and legacy camera events. For planar adapters: - `getCamera()` derives an `ICamera` from `getResolvedView()`. - `parallelScale` maps to semantic scale using the current resolved fit scale. - In-plane focal-point deltas map to pan and anchor state. - Normal-direction focal-point deltas map to volume `sliceWorldPoint` navigation. - `position` may disambiguate legacy movement but is not stored. - Position-only planar patches are unsupported and should not mutate clean state. Tools that still need an `ICamera`-compatible shape should use the bridge utility that derives that shape from `getResolvedView()` first and falls back to legacy `getCamera()` only when needed. --- ### Data Bindings And Loading Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/generic-viewport/data-bindings-and-loading.md #### Data Bindings And Loading Generic viewports separate loading, binding, viewport navigation, and render appearance. The viewport asks its `DataProvider` to load a logical data id. The loaded data is passed to the selected render path, and the render path returns a `ViewportDataBinding`. The binding contains the mounted runtime rendering plus callbacks for view state, data presentation, rendering, resize, and cleanup. #### Data Provider A data provider converts an application-level data id into loaded data. For planar viewports this may resolve image ids, volumes, acquisition orientation, metadata, and the internally selected effective render path. The loaded object describes the data; it does not own viewport navigation. #### Viewport Data Binding A binding represents one mounted data/render-path pair. It has a role: - `source` defines the active view used by the viewport. - `overlay` draws additional data aligned to the source. Bindings receive `applyViewState(viewState)` whenever the viewport navigation state changes. This replaces the older `updateCamera()` language because the binding is applying viewport state, not owning camera truth. The viewport keeps bindings keyed by display set id. That means tools and application code can update a single mounted display set without reaching into actors or mapper objects: ```ts viewport.setDisplaySetPresentation(petDataId, { visible: false, }); viewport.render(); ``` `reference` is an optional semantic relationship on registered data. It is used when a binding renders something derived from another object, such as a segmentation labelmap, volume, image, geometry, or another data id. It is not an actor id and it is not used as runtime actor identity. ```ts utilities.genericViewportDisplaySetMetadataProvider.add(labelmapDataId, { kind: 'planar', imageIds: labelmapImageIds, reference: { kind: 'segmentation', segmentationId, representationUID, labelmapId, }, }); ``` #### Presentation Split Viewport state and data presentation have different ownership: - `viewState` is local viewport navigation and layout state. - `DataPresentation` is per-binding appearance such as VOI, opacity, colormap, interpolation, visibility, or playback presentation. This split lets one viewport pan, zoom, rotate, and navigate once while multiple bindings render with their own appearance settings. #### Segmentation Bindings Labelmap segmentations use the same binding model. The segmentation display tool creates or resolves labelmap data, registers it as planar data, and mounts it as an overlay binding. When `useSliceRendering` is enabled, compatible volume labelmaps render through the slice/image overlay path instead of the legacy volume-labelmap actor path. ```ts await segmentation.addLabelmapRepresentationToViewportMap({ [viewportId]: [ { segmentationId, config: { useSliceRendering: true, }, }, ], }); ``` Internally, that representation maps to overlay data: ```ts await viewport.addDisplaySet(labelmapDataId, { orientation: viewport.getViewState().orientation, role: 'overlay', }); ``` The segmentation remains owned by the tools segmentation state. The viewport only owns the mounted overlay binding used to render it. #### Loading Flow The typical flow is: 1. The viewport infers the render path from dataset shape, orientation, and rendering configuration. 2. The viewport calls `dataProvider.load(dataId, options)` with that internal decision. 3. The render path resolver selects the runtime path for the loaded data. 4. The render path mounts runtime resources and returns a binding. 5. The viewport stores the binding with its data id and role. 6. The viewport pushes current `viewState` and data presentation into the binding. 7. The binding projects that state into renderer commands during render or resize. The viewport remains the owner of navigation state throughout this flow. --- ### Generic Viewport Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/generic-viewport/index.md import DocCardList from '@theme/DocCardList'; import {useCurrentSidebarCategory} from '@docusaurus/theme-common'; #### Generic Viewport The Generic Viewport architecture is a cleanup of the old split between `StackViewport`, `VolumeViewport`, CPU rendering, VTK image rendering, VTK volume rendering, and segmentation overlays. The old model worked, but the ownership boundaries were not clear enough. Stack and volume viewports each had their own loading path, camera path, presentation path, actor path, and overlay path. CPU stack image rendering, GPU image rendering, GPU volume rendering, and volume slice rendering solved similar problems in different places. As features were added, behavior became spread across viewport classes, mapper helpers, synchronizers, segmentation display tools, and compatibility code. That made common workflows harder than they needed to be: - A stack image and a volume slice could represent the same plane but travel through different viewport APIs. - Fusion overlays had to know whether the base viewport was stack-like, volume-like, CPU-backed, or VTK-backed. - Segmentation labelmaps had separate paths for image actors, volume actors, and special overlay renderers. - Camera fields were used both as user-facing navigation state and as renderer commands, which made ownership ambiguous. - Adding a new render mode meant touching more viewport behavior than the renderer actually needed. Generic Viewport keeps the existing rendering power, but moves the ownership to a smaller set of concepts. #### What Changed The new shape is: ```text logical data id -> DataProvider -> RenderPath -> ViewportDataBinding -> viewState + DataPresentation -> renderer command ``` The viewport owns navigation and binding order. The data provider owns logical data lookup. A render path owns only the runtime implementation for one data shape and internal render-path decision. A binding owns one mounted dataset in the viewport. For planar imaging, one `PlanarViewport` can now display stack-like data, volume slice data, CPU image data, VTK image data, and VTK volume slice data behind the same clean API. The render path is inferred from dataset shape, orientation, rendering configuration, and segmentation slice-rendering needs instead of being passed by application code or hardwired into a separate stack or volume viewport class. #### Source And Overlay Data Every mounted dataset has a binding role: - `source` is the active dataset that defines the view. - `overlay` is drawn in the same view, aligned to the source. `setDisplaySets()` makes the first entry the source by default and later entries overlays. `addDisplaySet()` can explicitly add an overlay later. This replaces a lot of the old "stack vs volume vs actor overlay" branching with one binding model. #### Render Paths Render paths are the rendering implementations. Planar render paths include CPU image, CPU volume slice, VTK image, and VTK volume slice paths. Video, ECG, WSI, and 3D viewports use the same controller pattern but provide their own render paths and state models. The important rule is that render paths do not own viewport navigation. They receive state from the viewport and project it into renderer-specific commands. #### Presentation Split Generic Viewport separates two kinds of presentation: - View presentation: pan, zoom or scale, rotation, flips, and display area. Direct Next viewports read and write this through `viewportProjection`. - Data presentation: VOI, opacity, colormap, blend mode, interpolation, and visibility for one mounted dataset. This is what makes a CT source and PET overlay share the same view while still having independent VOI, color, and opacity. #### Camera In Brief Clean Generic viewports prefer semantic state over durable VTK-style camera fields. For planar, video, ECG, and WSI, the viewport state is the source of truth and the runtime camera, canvas transform, media transform, signal transform, or OpenLayers view is derived from it. 3D Next is runtime-camera backed, but still participates in the projection service. Legacy camera APIs remain available through temporary compatibility adapters; those adapters are not the long-term Next API surface and their legacy helpers are expected to be removed in a later breaking release. The full camera contract is covered in the camera page. --- ### Migration Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/generic-viewport/migration.md #### Migration This migration guide is local to the Generic Viewport architecture. It is not a general Cornerstone migration guide. The goal is to move application code from viewport-class-specific behavior to logical data ids, inferred render paths, bindings, view state, and data presentation. If you need to add custom viewport types from extensions, use `Enums.ViewportTypes` (register with `registerViewportType`, then `Enums.ViewportTypes.`) as documented in the [5.x Generic Viewport migration guide](../../../../migration-guides/5x/2-generic-viewport.md#extending-viewport-types-new-pattern). #### Stack Or Volume Viewport Selection Before, the viewport type usually encoded the data shape: ```ts renderingEngine.enableElement({ viewportId, type: Enums.ViewportType.STACK, element, }); await viewport.setStack(imageIds); ``` ```ts renderingEngine.enableElement({ viewportId, type: Enums.ViewportType.ORTHOGRAPHIC, element, }); await viewport.setVolumes([{ volumeId }]); ``` Now, planar 2D viewing uses `PLANAR_NEXT`, and the data/render path decides whether the source is stack-like or volume-slice-like: ```ts renderingEngine.enableElement({ viewportId, type: Enums.ViewportType.PLANAR_NEXT, element, }); const viewport = renderingEngine.getViewport(viewportId) as PlanarViewport; ``` #### Stack Data Before: ```ts await stackViewport.setStack(imageIds, 0); stackViewport.setProperties({ voiRange: { lower: -1500, upper: 2500 }, }); stackViewport.render(); ``` Now: ```ts const displaySetId = 'ct-stack'; utilities.genericViewportDisplaySetMetadataProvider.add(displaySetId, { kind: 'planar', imageIds, initialImageIdIndex: 0, }); await viewport.setDisplaySets({ displaySetId, }); viewport.setDisplaySetPresentation(displaySetId, { voiRange: { lower: -1500, upper: 2500 }, }); viewport.render(); ``` #### Volume Slice Data Before: ```ts await volumeViewport.setVolumes([ { volumeId, callback: ({ volumeActor }) => { volumeActor.getProperty().setRGBTransferFunction(0, cfun); }, }, ]); ``` Now: ```ts const displaySetId = 'ct-volume'; utilities.genericViewportDisplaySetMetadataProvider.add(displaySetId, { kind: 'planar', imageIds, initialImageIdIndex: Math.floor(imageIds.length / 2), volumeId, }); await viewport.setDisplaySets({ displaySetId, options: { orientation: Enums.OrientationAxis.AXIAL, }, }); viewport.setDisplaySetPresentation(displaySetId, { voiRange, colormap, }); viewport.render(); ``` #### Fusion Overlays Before, fusion often depended on volume actors, blend mode setup, and renderer state owned by the viewport: ```ts await volumeViewport.setVolumes([ { volumeId: ctVolumeId }, { volumeId: ptVolumeId }, ]); volumeViewport.setProperties( { colormap: { name: 'hsv' }, voiRange: ptVoiRange, }, ptVolumeId ); ``` Now, source and overlay are explicit data bindings: ```ts await viewport.setDisplaySets( { displaySetId: ctDataId, options: { orientation: Enums.OrientationAxis.SAGITTAL, role: 'source', }, }, { displaySetId: ptDataId, options: { orientation: Enums.OrientationAxis.SAGITTAL, role: 'overlay', }, } ); viewport.setDisplaySetPresentation(ptDataId, { colormap: { name: 'hsv', opacity: 0.4, }, }); ``` #### Adding Overlay Images Before: ```ts viewport.addImages([{ imageId }]); ``` Now, prefer registering overlay data and using data presentation: ```ts utilities.genericViewportDisplaySetMetadataProvider.add(overlayDataId, { kind: 'planar', imageIds: [imageId], initialImageIdIndex: 0, }); await viewport.addDisplaySet(overlayDataId, { role: 'overlay', }); viewport.setDisplaySetPresentation(overlayDataId, { opacity: 0.5, visible: true, }); ``` The compatibility `addImages()` path still exists for image overlays, but new code should use display set ids and bindings directly. #### VOI, Colormap, Opacity, And Visibility Before: ```ts viewport.setProperties({ voiRange, colormap, invert: true, }); ``` Now: ```ts viewport.setDisplaySetPresentation(displaySetId, { voiRange, colormap, invert: true, visible: true, }); ``` This makes presentation explicitly per display set binding, which matters when the viewport has both source and overlay display sets. #### Pan, Zoom, Rotation, And Flips Before, code often patched a camera object: ```ts const camera = viewport.getCamera(); viewport.setCamera({ ...camera, parallelScale: camera.parallelScale * 0.8, }); ``` Now, use semantic viewport APIs: ```ts viewport.setScale(viewport.getScale() * 1.25); viewport.setPan([40, -20]); viewport.updateViewState(({ rotation = 0 }) => ({ rotation: rotation + 30, })); const nextViewState = viewportProjection.withPresentation(viewport, { rotation: 90, }); if (nextViewState) { viewport.setViewState(nextViewState); } viewport.setViewState({ flipHorizontal: true }); viewport.render(); ``` Legacy adapters still support camera-style calls for older code. They are a temporary migration layer and should be expected to be removed in a later breaking release, not treated as a durable Next control surface. Clean Next code should use view state and viewport projection APIs. Direct Next viewports do not expose `getViewPresentation()`, `setViewPresentation()`, `getCamera()`, or `setCamera()` as durable control APIs. Before: ```ts const presentation = viewport.getViewPresentation(); viewport.setViewPresentation({ ...presentation, zoom: presentation.zoom * 2, }); ``` Now: ```ts const presentation = viewportProjection.getPresentation(viewport); const nextViewState = viewportProjection.withPresentation(viewport, { zoom: (presentation?.zoom ?? 1) * 2, }); if (nextViewState) { viewport.setViewState(nextViewState); } ``` Before: ```ts viewport.setCamera({ focalPoint, position, }); ``` Now, use a view reference for spatial navigation or a presentation patch for display navigation: ```ts targetViewport.setViewReference(sourceViewport.getViewReference()); targetViewport.render(); ``` ```ts const nextViewState = viewportProjection.withPresentation(viewport, { zoom: 1.5, }); if (nextViewState) { viewport.setViewState(nextViewState); } ``` If you are changing a native field such as planar orientation, a video media anchor, or ECG signal range, update the native view state directly with `setViewState()` or `updateViewState()`. Use `resetViewState()` for the clean Next reset operation; `resetCamera()` belongs to temporary legacy adapters and should be expected to be removed in a later breaking release. #### Slice Navigation Before: ```ts await stackViewport.setImageIdIndex(index); ``` ```ts volumeViewport.setCamera({ focalPoint, position, }); ``` Now: ```ts await viewport.setImageIdIndex(index); ``` For stack-backed data, this stores a stack index. For volume-backed data, the viewport resolves the index into a volume slice point so the state has one slice locator. For spatial navigation across viewports: ```ts const viewReference = sourceViewport.getViewReference(); targetViewport.setViewReference(viewReference); targetViewport.render(); ``` #### Segmentations Before, volume labelmaps commonly rendered as volume actors. That was useful for some workflows, but it could allocate full 3D labelmap textures even for a single-slice planar workflow. ```ts await segmentation.addSegmentationRepresentations(viewportId, [ { segmentationId, type: SegmentationRepresentations.Labelmap, }, ]); ``` Now, for compatible planar volume-slice workflows, enable slice rendering: ```ts await segmentation.addLabelmapRepresentationToViewportMap({ [viewportId]: [ { segmentationId, type: SegmentationRepresentations.Labelmap, config: { useSliceRendering: true, }, }, ], }); ``` With `useSliceRendering`, the labelmap representation is projected through the image/slice overlay path. It follows the source view state as an overlay binding, instead of requiring a separate volume-rendering overlay path. #### Recommended Migration Order 1. Move viewport creation to `ViewportType.PLANAR_NEXT` for planar 2D stack and volume-slice workflows. 2. Register each source or overlay as a logical display set id. 3. Replace `setStack()` and `setVolumes()` with `setDisplaySets()` or `addDisplaySet()`. 4. Move VOI, colormap, opacity, blend mode, and visibility to `setDisplaySetPresentation(displaySetId, ...)`. 5. Replace clean-code camera patches with view state, viewport projection, pan, zoom, and view reference APIs. 6. Enable `useSliceRendering` for labelmap segmentation overlays that should render through the slice path. --- ### Render Backends Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/generic-viewport/render-backends.md #### Render Backends A render backend is the named rendering implementation a planar Generic Viewport mounts data through. Cornerstone ships two concrete backends, `'gpu'` (VTK/WebGL) and `'cpu'`, plus the `'auto'` preference that resolves to one of them from the capability detection performed at `init()` (WebGL availability, texture-format probes, and the deprecated `useCPURendering` flag). Backends are addressed by plain wire strings. The `Enums.RenderBackends` constants object maps readable names to those strings (`RenderBackends.GPU === 'gpu'`) and, unlike a TypeScript enum, grows at runtime as extension backends register themselves. #### Selecting A Backend The backend used for a mounted dataset is resolved with this precedence: 1. The per-mount `renderBackend` option on `setDisplaySets()` / `addDisplaySet()`. A concrete backend pins that dataset; `'auto'` resolves from capability detection even when the global backend is pinned. 2. The global configuration at `rendering.planar.renderBackend`, set at `init()` or changed at runtime with `setRenderBackend()`. 3. The `'auto'` resolution: CPU when no usable WebGL context was detected, GPU otherwise. `setRenderBackend(backend, reason?)` live-switches all mounted viewports in place — viewport ids, mounted data, cameras, presentation state, and tool annotations are preserved; only the render paths are rebuilt. It emits `RENDER_BACKEND_CHANGED` on the eventTarget. Cornerstone never switches backends on its own: applications listening to degradation events (`WEBGL_CONTEXT_LOST`, `RENDER_PATH_ERROR`) are expected to call it, typically after prompting the user. `getRenderBackend()` returns the configured preference; `getEffectiveRenderBackend()` returns the resolved concrete backend. #### Registering A Custom Backend :::caution Experimental Registered custom backends are not fully functional yet, and the registration API is intentionally incomplete. `registerRenderBackend()` currently captures the wiring the planar viewport needs to select and mount a backend, but it is intended to grow additional parameters describing the backend-specific changes and behaviours it registers — for example participation in the `'auto'` capability resolution, backend-owned canvas/surface creation instead of drawing to an existing surface, and per-backend context-loss/degradation handling. Expect the `RegisterRenderBackendOptions` shape to change. ::: `registerRenderBackend()` follows the same extensible-enum model as `registerViewportType`: the backend id becomes a valid value for `setRenderBackend()`, the global `rendering.planar.renderBackend` configuration, and per-mount `renderBackend` options. ```ts import { registerRenderBackend, setRenderBackend, Enums, } from '@cornerstonejs/core'; registerRenderBackend({ name: 'WEBGPU', backend: 'myOrg:webgpu', renderModes: { image: { id: 'myOrg:webgpuImage', createDefinition: () => new WebGPUImageSlicePath(), }, volume: { id: 'myOrg:webgpuVolume', createDefinition: () => new WebGPUVolumeSlicePath(), }, }, }); setRenderBackend(Enums.RenderBackends.WEBGPU); ``` The definition carries the semantic wiring the viewport needs today: - `backend` — the wire id, e.g. `'myOrg:webgpu'`. Prefix custom ids with an organization namespace; `'auto'` is reserved. - `renderModes` — the render mode the backend resolves to per dataset kind, co-locating each mode's wire `id` with the `createDefinition` factory for the planar render path definition that implements it (see [Render Paths](./render-paths.md) for what a path implements). The factory is called once per viewport (each planar viewport owns its render path resolver), so it must return a fresh definition instance on every call. `image` is required and its id must differ from `volume`'s; omit `volume` when the backend cannot render volume-backed datasets, in which case selecting it for such a dataset fails with a descriptive error. - `surface` — which existing composited canvas the backend's render modes draw to, `'vtk'` (default) or `'cpu'`. Custom backends cannot yet register their own surface; this is one of the planned extension points noted above. - `name` — optional constant name added to `Enums.RenderBackends`, e.g. `RenderBackends.WEBGPU`. #### TypeScript Augmentation The backend string type stays open through two augmentable interfaces in `@cornerstonejs/core`. Augment them in your extension's `.d.ts` to get the new wire string and constant name into completions and checks: ```ts declare module '@cornerstonejs/core' { interface RenderBackendRegistry { 'myOrg:webgpu': 'myOrg:webgpu'; } interface RenderBackendConstants { readonly WEBGPU: 'myOrg:webgpu'; } } ``` `RenderBackendRegistry` feeds the `RenderBackendValue` string union accepted by `setRenderBackend()` and `renderBackend` options; `RenderBackendConstants` types the properties of `Enums.RenderBackends`. --- ### Render Paths Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/generic-viewport/render-paths.md #### Render Paths A render path is the runtime implementation that knows how to draw one logical data type in one render mode. The viewport chooses a render path when data is added, then the binding returned by that path receives view and presentation updates from the viewport. Examples include: - CPU image slice rendering for stack images. - CPU volume slice rendering for volume data. - VTK image mapper rendering. - VTK volume slice rendering. - DOM video element rendering. - Canvas ECG waveform rendering. #### Selection Render path selection starts from the requested viewport type, logical data, dataset shape, requested orientation, and rendering configuration. A viewport-specific decision service chooses the effective path, then the viewport family's render path resolver returns the first catalogued path that matches the loaded data and internal decision. The selected path can also narrow the root viewport render context into the runtime context it needs. For planar viewports, stack-like image ids select an image path, while volume-backed data and reformatted orientations select a volume slice path. CPU/GPU selection comes from rendering configuration, thresholds, and runtime support. Source vs overlay role is not used to decide the render path. #### Projection Render paths do not own viewport navigation truth. They receive the viewport `viewState` through `applyViewState()` and use the viewport-resolved data to apply renderer-specific commands. Planar render paths project a resolved planar view into: - VTK camera fields for VTK image and volume slice rendering. - CPU transform information for CPU image and CPU volume rendering. - A shared active source `ICamera` used by overlays for alignment and sampling. - Slice/image overlay commands for compatible labelmap segmentation rendering. Video and ECG render paths resolve a canvas mapping from semantic state, element or canvas dimensions, and data metrics. That mapping provides canvas/world transforms and DOM or canvas drawing offsets. #### Source And Overlay Ownership The source binding defines the active resolved view for the viewport. Overlay bindings receive the same `viewState`, but they do not replace the active source view. In planar rendering, only the source binding writes `ctx.view.activeSourceICamera`; overlays read it only to align sampling or actors to the source. This is also how segmentation slice rendering works. The source volume slice defines the active view. The labelmap representation is mounted as an overlay binding and rendered through the slice path when `useSliceRendering` is enabled. It follows source navigation without becoming the source view. #### Adding A Render Path When adding a new render path: - Match only the data type and internal render-path decision the path can actually draw. - Add the path definition to the owning viewport family's render path catalogue so production setup is explicit. - Keep persistent navigation state on the viewport. - Implement `applyViewState()` as a projection from semantic state to runtime commands. - Keep appearance settings in data presentation, not view state. - Return cleanup through `removeData()` so runtime resources are owned by the binding that created them. --- ### Viewport Projection Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-core/generic-viewport/viewport-projection.md #### Viewport Projection Viewport Projection is the Generic Viewport construct for asking how a viewport's semantic state maps into presentation, coordinate transforms, and renderer output. It exists because `zoom`, `scale`, `pan`, and camera fields do not mean the same thing for every viewport family. A planar viewport has a semantic anchor, slice geometry, display area, and derived renderer camera. A 3D viewport is runtime-camera-backed. Video, ECG, and WSI have their own mapping rules. The shared abstraction is therefore not `ICamera`; `ICamera` remains renderer output when a renderer needs it. #### What "projection" means here In this codebase, "projection" is the mathematical sense — the act of _projecting_ a viewport's semantic state onto a presentation, a set of coordinate transforms, and (when applicable) a renderer camera. A `Projection` is the adaptation seam between a viewport's internal model and the cross-family surface that synchronizers and tools consume. It is **not** VTK's parallel-vs-perspective projection (`parallelProjection`, `setParallelProjection`). That concept is a renderer-matrix setting and lives inside the resolved `ICamera` payload (`rendererCamera.parallelProjection`). Both terms can coexist in the same code path: ```ts // Cross-viewport projection adapter — this file's subject. const snapshot = viewportProjection.get(viewport); // VTK projection matrix — unrelated, set on the renderer camera. snapshot?.rendererCamera?.parallelProjection; // boolean ``` If you are reading code that mentions "projection," check the noun: a `ProjectionSnapshot`, `ViewportProjectionAdapter`, or `viewportProjection` refers to the cross-viewport seam below. A `parallelProjection` flag or `setParallelProjection` call refers to VTK's render-matrix mode. #### Public Contract And Stability The stable entry point is the projection service and the generic projection types: - `viewportProjection` - `ViewportProjectionService` - `ViewportProjectionTypes.ts` - `ProjectionSnapshot` - `ProjectionPresentation` - `ProjectionScale` - `ProjectionPosition` - `ViewportProjectionAdapter` The family namespaces are intentionally exported as advanced helpers: - `planarProjection` - `volume3DProjection` - `videoProjection` - `ecgProjection` - `wsiProjection` Use those namespaces when building a custom synchronizer, tool, test, or new viewport family that needs lower-level snapshot or renderer-camera behavior. They are a tier below the core viewport methods and may change while the Generic Viewport API is still settling. Application code should prefer `viewportProjection.getPresentation()` and `viewportProjection.withPresentation()` for presentation reads and writes. Direct Next viewport instances intentionally do not expose `getViewPresentation()` or `setViewPresentation()`. #### Core Types The projection interface lives in `ViewportProjectionTypes.ts`. ```ts interface ViewportProjectionAdapter { id: string; viewportTypes: string[]; getSnapshot(request: ProjectionRequest): ProjectionSnapshot | undefined; getPresentation( snapshot: ProjectionSnapshot, selector?: ViewPresentationSelector ): TPresentation; withPresentation( snapshot: ProjectionSnapshot, presentation: Partial, options?: ProjectionWriteOptions ): TViewState; applyToRenderer?(snapshot: ProjectionSnapshot, target: unknown): void; } ``` A `ProjectionSnapshot` is capability-based: ```ts interface ProjectionSnapshot { kind: string; frameOfReferenceUID?: string; spaces: { canvas?: boolean; world?: boolean; image?: boolean; renderer?: boolean; }; transforms?: { canvasToWorld?(point: Point2): Point3; worldToCanvas?(point: Point3): Point2; }; presentation: ProjectionPresentation; rendererCamera?: ICamera; } ``` If a viewport cannot provide a transform, it should omit that capability. Do not add placeholder transforms just to satisfy a universal shape. #### Semantic Scale And Position Projection scale and position are tagged so callers can read intent before using values: ```ts type ProjectionScale = | { kind: 'fit'; value: number } | { kind: 'fitWidth'; value: number } | { kind: 'fitHeight'; value: number } | { kind: 'displayArea'; value: number; area: DisplayArea } | { kind: 'nativePixel'; pixelsPerCanvasPixel: number } | { kind: 'physical'; mmPerCanvasPixel: number } | { kind: 'signal'; samplesPerCanvasPixel: number; valueUnitsPerCanvasPixel: number; }; type ProjectionPosition = | { kind: 'anchor'; worldPoint?: Point3; canvasPoint: Point2 } | { kind: 'imagePoint'; imagePoint: Point2; canvasPoint: Point2 } | { kind: 'mediaPoint'; mediaPoint: Point2; canvasPoint: Point2 } | { kind: 'signalPoint'; sampleIndex: number; value: number; channelIndex: number; canvasPoint: Point2; } | { kind: 'focalPoint'; worldPoint: Point3 }; ``` Do not treat `presentation.zoom`, `presentation.scale`, or `presentation.pan` as universal values. Use the tag first, then branch on the semantics your tool or synchronizer supports. #### Projection Service The package-level projection service is exported as `viewportProjection`. ```ts import { viewportProjection } from '@cornerstonejs/core'; const projection = viewportProjection.get(viewport, { kind: 'planar', dataId, }); ``` The service is package/global, not per rendering engine. That keeps custom synchronizers and advanced tools independent from rendering-engine ownership. Built-in viewport types and explicit `kind` requests have typed helper aliases for downstream code: ```ts import type { ProjectionPresentationForKind, ProjectionSnapshotForKind, ProjectionViewStateForKind, } from '@cornerstonejs/core'; type PlanarSnapshot = ProjectionSnapshotForKind<'planar'>; type PlanarPresentation = ProjectionPresentationForKind<'planar'>; type PlanarViewState = ProjectionViewStateForKind<'planar'>; ``` When the viewport instance has a literal Next viewport type, the service can infer those same types from the viewport argument. Explicit `kind` requests are available for custom synchronizers that only know a viewport as `unknown`. Built-in adapters are registered for: - `planarProjection` - `volume3DProjection` - `videoProjection` - `ecgProjection` - `wsiProjection` The advanced namespaces expose lower-level helpers for code that intentionally works below the core viewport API. `createZoomPanSynchronizer` in `@cornerstonejs/tools` already uses the service when both source and target viewports expose projection adapters, then falls back to the older `getZoom`/`setZoom` and `getPan`/`setPan` capability checks for legacy viewport families. #### When To Care Most application code should use `setViewState` for native viewport mutation, `resetViewState` for the viewport family's default navigation reset, `getViewReference` / `setViewReference` for spatial navigation, and `canvasToWorld` / `worldToCanvas` for coordinate conversion. Use projection when code needs a portable presentation layer across viewport families. Use Viewport Projection when you are writing: - a custom synchronizer that needs to work across viewport families - a tool that must inspect capabilities before transforming points - a new Generic Viewport family - a bridge between semantic state and renderer-specific camera output #### Reading A Projection Check capabilities before using transforms: ```ts const projection = viewportProjection.get(viewport); if (projection?.spaces.canvas && projection.spaces.world) { const worldPoint = projection.transforms?.canvasToWorld?.([100, 120]); } ``` Check scale and position semantics before applying them: ```ts const scale = projection?.presentation.scale; if (scale?.kind === 'displayArea') { syncDisplayArea(scale.area); } if (scale?.kind === 'physical') { syncPhysicalSpacing(scale.mmPerCanvasPixel); } ``` #### Writing Presentation Use `withPresentation` when you need the adapter to translate a presentation patch back into semantic state: ```ts const nextState = viewportProjection.withPresentation(viewport, { zoom: 2, pan: [10, -5], }); if (nextState) { viewport.setViewState(nextState); } ``` Next viewports intentionally do not expose `setViewPresentation`. The projection service is the portable write layer, and `setViewState` remains the only Next viewport mutation primitive. They also do not expose `getViewPresentation`; use `viewportProjection.getPresentation(viewport, { selector })` instead. Legacy compatibility adapters may still expose `getViewPresentation` and `setViewPresentation` for older code, but those adapters are a temporary migration layer and their legacy camera/presentation methods should be expected to disappear in a later breaking release. Before, legacy or compatibility code might do this: ```ts const presentation = viewport.getViewPresentation({ pan: true, zoom: true, }); viewport.setViewPresentation({ zoom: presentation.zoom * 2, }); ``` Direct Next code should do this: ```ts const presentation = viewportProjection.getPresentation(viewport, { selector: { pan: true, zoom: true, }, }); const nextViewState = viewportProjection.withPresentation(viewport, { zoom: (presentation?.zoom ?? 1) * 2, }); if (nextViewState) { viewport.setViewState(nextViewState); } ``` Do not make a custom projection adapter mutate its viewport. It should return native view state and let the caller decide whether to call `setViewState`. #### Adding A New Adapter For a new Generic Viewport family: 1. Define the family-specific snapshot and presentation types. 2. Implement `getSnapshot` from current semantic state and resolved geometry. 3. Implement `getPresentation` for the existing public view-presentation shape. 4. Implement `withPresentation` as a pure translation back to semantic state. 5. Implement `applyToRenderer` only if the viewport can produce renderer output. 6. Register the adapter in the Generic Viewport projection setup. The adapter should not mutate the viewport in `withPresentation`. Return the next semantic state and let the viewport decide how to apply it. #### Current Adapters Planar projection uses: - `PlanarViewState` as semantic state - `PlanarSliceBasis` and resolved view geometry for data/world/canvas mapping - `PlanarResolvedICamera` only as renderer output - compatibility helpers for legacy `getZoom`, `getPan`, and `getScale` Volume3D projection uses: - the current runtime VTK camera as its state source - focal point as semantic position when available - physical canvas spacing derived from `parallelScale` - `ICamera` as renderer output Video projection uses: - `VideoViewState` as semantic state - intrinsic media-pixel coordinates for world/canvas conversion - `mediaPoint` position tags - `nativePixel` scale tags - optional renderer-camera output for legacy interop ECG projection uses: - `ECGViewState` as semantic state - signal tuples shaped as `[sampleIndex, amplitudeValue, channelIndex]` - `signalPoint` position tags - `signal` scale tags with samples and value units per canvas pixel - optional renderer-camera output for legacy interop WSI projection uses: - `WSIViewState` synchronized from OpenLayers - slide/world transforms from the WSI resolved view - `anchor` position tags - physical scale when renderer-camera output can provide it This gives cross-viewport callers one projection interface while preserving the real differences between viewport families. --- ## Cornerstone-metadata ### Display Sets Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-metadata/display-sets.md #### Display Sets A **display set** is the unit a viewport renders. It groups the instances of a series that should be shown together and records which viewport type(s) can render them. This mirrors the OHIF "display set" concept, but lives in `@cornerstonejs/metadata` as a framework-agnostic, **data-shaped** object (`IDisplaySet`) so any application — not just OHIF — can reuse it. A series does not always map to a single display set. The classic case is the fix this module was extracted for: a **diffusion MR (DWI)** series that mixes 4D b-value frames with trailing frames that have no b-value. Those undefined b-value frames are not part of the 4D data set, so rendering them as one volume applies the wrong window/level. The `mixedDimensionalityBValue` split rule separates them into their own display set (see [Split rules](#split-rules)). #### The split → create → consume pipeline The end-to-end flow has three stages: 1. **Split** a series' image ids into instance groups with `splitImageIdsBySplitRules` using a set of split rules. 2. **Create** an `IDisplaySet` for each group with `createDisplaySetFromGroup`. 3. **Consume** each display set — render it on a viewport, and/or cache it in the metadata layer so downstream code can resolve it by image id. For examples, the demo helper `splitDisplaySetsFromImageIds(imageIds)` performs stages 1–2 for you (it normalizes frame image ids to their base form, dedupes to one instance per SOP, and re-attaches the frame-level image ids). Under the hood it is just: ```ts import { splitImageIdsBySplitRules, createDisplaySetFromGroup, defaultDisplaySetSplitRules, metaData, type IDisplaySet, type NaturalizedInstance, } from '@cornerstonejs/metadata'; // Resolve one (base) imageId to its naturalized DICOM instance. In a real app // this reads the metadata cache, e.g. metaData.get('instance', imageId), with // the imageId normalized to its base (frame 1) form. function getNaturalizedInstance( imageId: string ): NaturalizedInstance | undefined { return metaData.get('instance', imageId) as NaturalizedInstance | undefined; } const groups = splitImageIdsBySplitRules(seriesImageIds, { getNaturalizedInstance, splitRules: defaultDisplaySetSplitRules, }); const displaySets: IDisplaySet[] = groups.map((group) => createDisplaySetFromGroup(group) ); ``` #### Driving a viewport from a display set Each display set exposes the viewport type(s) it can be shown in (`viewportTypes`, with `preferredViewportType` being the first). A viewport's `setDisplaySets({ displaySetId })` is the single entry point that loads a display set: it resolves `displaySetId` to renderable data, calls the viewport's native setter (`setStack` / `setVolumes` / `setVideo` / `setWSI` / `setEcg`), and records the mounted entry so `getDisplaySets()` reflects it. The viewport/registry `displaySetId` is the same value as the display set's `displaySetId` field — there is one identifier for a display set, used on both the metadata object and the viewport API. For the legacy viewports, `setDisplaySets` resolves `displaySetId` through the **generic-viewport display-set provider**, so you register the renderable data there first. The registered shape depends on the viewport family: ```ts import { Enums, utilities } from '@cornerstonejs/core'; const { ViewportType } = Enums; const HINT_TO_VIEWPORT_TYPE: Record = { stack: ViewportType.STACK, volume: ViewportType.ORTHOGRAPHIC, volume3d: ViewportType.VOLUME_3D, video: ViewportType.VIDEO, wholeslide: ViewportType.WHOLE_SLIDE, ecg: ViewportType.ECG, }; const displaySetId = displaySet.displaySetId; // 1. Register the renderable data so the viewport can resolve `displaySetId`. // stack/volume use { imageIds }; video/ecg use { kind, sourceDataId }; // wsi uses { kind: 'wsi', imageIds, options: { webClient } }. utilities.genericViewportDisplaySetMetadataProvider.add(displaySetId, { imageIds: [...displaySet.imageIds], }); // 2. Enable a viewport of the display set's preferred type, then mount it. const viewportType = HINT_TO_VIEWPORT_TYPE[displaySet.preferredViewportType] ?? ViewportType.STACK; renderingEngine.enableElement({ viewportId, type: viewportType, element }); const viewport = renderingEngine.getViewport(viewportId); await viewport.setDisplaySets({ displaySetId }); viewport.getDisplaySets(); // [{ displaySetId }] — reflects what was mounted ``` `getDisplaySets()` is available on both the legacy `Viewport` and the generic viewport, so mounted display sets can be read uniformly across either hierarchy. The runnable end-to-end version (all five viewport families, plus a dropdown to switch a display set among its allowed viewport types) is the **Display Sets** example under `packages/core/examples/displaySets`. #### Caching display sets in the metadata layer Independently of rendering, a display set can be stored in the typed metadata cache so any consumer (tools, measurements, custom UI) can resolve it from any of its image ids: ```ts import { registerDisplaySetProviders, registerDisplaySetMetadata, Enums, metaData, } from '@cornerstonejs/metadata'; // Once at app init (after registerDefaultProviders): registerDisplaySetProviders(); // After creating a display set, cache it keyed by its (underlying) image ids: registerDisplaySetMetadata(seriesImageIds, displaySet); // Anywhere downstream, resolve the display set from one of its image ids: const ds = metaData.getTyped(Enums.MetadataModules.DISPLAY_SET, imageId); ds?.instances; // the full IDisplaySet — including instances and split-rule ds?.numImageFrames; // attributes such as isClip / numImageFrames / splitNumber ``` `getTyped(MetadataModules.DISPLAY_SET, …)` returns the full `IDisplaySet` that was registered, not a narrowed projection, so the cached shape and the typed read never drift apart. #### Split rules Split rules decide how a series' instances are grouped into display sets and which viewport types each group supports. `defaultDisplaySetSplitRules` covers the common DICOM cases (video, ECG, whole-slide, single-image modalities, multi-frame clips, mixed-b-value DWI, volumetric series, and a fallback image rule). Rules are evaluated **in order, first match wins per instance**. A `SplitRule` has up to five parts: | Field | Purpose | | ------------------ | ----------------------------------------------------------------------------------------------------------------------------- | | `matches` | Returns true if an instance belongs to this rule. Omit to match everything. | | `groupBy` | Keys (tag names or functions) that partition matched instances into separate display sets. | | `series` | Optional. Runs once per rule per split and **returns** that rule's derived facts; `matches`/`groupBy` read them via `series`. | | `viewportTypes` | Allowed viewport types for the produced display sets; index `0` is preferred. | | `customAttributes` | Returns extra attributes spread flat onto the display set (e.g. `isClip`, `numImageFrames`). | Most rules only need `matches` and `groupBy`: ```ts { matches: (instance) => isVideoInstance(instance), groupBy: ['SOPInstanceUID'], } ``` Reach for `series` only when a rule needs a value computed from the **whole series** and reused by `matches` or `groupBy`. It is optional, runs **once per rule per split operation**, and returns derived facts for that rule — it should **not** mutate shared state. The DWI fix is the worked example: `series` decides whether the series mixes b-value and non-b-value frames, and `groupBy` then separates them into two display sets: ```ts import type { SplitRule } from '@cornerstonejs/metadata'; const mixedDimensionalityBValue: SplitRule = { id: 'mixedDimensionalityBValue', viewportTypes: ['volume', 'volume3d', 'stack'], // Computed once over the whole series; returned, not mutated onto shared state. series: ({ instances }) => ({ mixedBValue: instances[0]?.Modality === 'MR' && instances.some((i) => i.DiffusionBValue !== undefined) && instances.some((i) => i.DiffusionBValue === undefined), }), // Reads this rule's own derived facts. matches: (_instance, { series }) => series.mixedBValue, // Two display sets: undefined-b-value frames split off from the rest. groupBy: [ 'SeriesInstanceUID', (instance) => instance.DiffusionBValue === undefined, ], }; ``` To customize splitting, prepend your own rules to (or replace) the defaults and pass the result as `splitRules`. `customAttributes` may set any attribute, but the resolved data fields a display set is built from — `imageIds`, `underlyingImageIds`, `instances`, and `displaySetId` — are reserved and cannot be overwritten, so the underlying-vs-frame image id invariant the viewports rely on always holds. A few engine guarantees worth knowing when writing rules: - **Buckets are namespaced by rule.** Two different rules can never merge into one display set even if their `groupBy` values coincide. - **Group order is deterministic.** Groups come back sorted by a stable, rule-namespaced key, so a series' display sets — and any id derived from their position — are stable regardless of the order the image ids were passed in. - **`series` samples `instances[0]`** for some facts (e.g. multi-frame, volumetric), so those rules assume a homogeneous series. A heterogeneous series needs a dedicated rule (as `mixedDimensionalityBValue` does for DWI) to separate it. - **`series` is scoped to its own rule.** A rule only ever sees the facts its own `series` hook returned; it cannot read another rule's facts, and it must not mutate shared state. - **Unmatched instances are dropped.** An instance that matches no rule (e.g. a non-image SOP) produces no display set; pass `onUnmatchedInstance` to `splitImageIdsBySplitRules` to observe them. - **`buildSeriesInfo` is safe on an empty instance list** — it returns zeroed counts. It aggregates series statistics only and is independent of split rules. #### Instance classifiers The default rules rely on small SOP-class/modality heuristics that are also exported for reuse, so you can detect a series' kind without re-hardcoding UID lists: - `isImageInstance(instance)` — the SOP class carries renderable pixel data. - `isVideoInstance(instance)` — video transfer syntax (reusing the shared `videoUIDs` list), a video SOP class, or a long multi-frame secondary capture. - `isEcgInstance(instance)` — an ECG / waveform SOP class. - `isWsiInstance(instance)` — VL Whole Slide Microscopy storage, or modality `SM`. #### Display set attributes (`IDisplaySet`) A display set implements `IDisplaySet`, which declares the **common attributes** read from a display set as plain data — not accessor methods — so it behaves like the OHIF display set object: ```ts const displaySet = createDisplaySetFromGroup(group); displaySet.displaySetId; displaySet.viewportTypes; // readonly ViewportTypeHint[] displaySet.preferredViewportType; // viewportTypes[0] displaySet.instances; // readonly NaturalizedInstance[] displaySet.imageIds; // frame-level, renderable image ids displaySet.underlyingImageIds; // SOP-level image ids (one per instance) ``` #### Adding new display set attributes - **Shared / common attributes** belong on `IDisplaySet` directly. Declare them optional unless every display set populates them. Many are produced by a split rule's `customAttributes` callback and spread flat onto the display set in `createDisplaySetFromGroup` (for example `isMultiFrame`, `isClip`, `numImageFrames`, `splitNumber`). - **App- or extension-specific attributes** that are not part of the common model should be added through **TypeScript module augmentation**, so they stay type-checked without widening the shared surface: ```ts // my-extension.ts — in an extension or the consuming app import '@cornerstonejs/metadata'; declare module '@cornerstonejs/metadata' { interface IDisplaySet { /** Whether this display set supports window/level. */ supportsWindowLevel?: boolean; } } ``` Keep augmented attributes optional — not all display set types define them. #### Related docs - [Cornerstone Metadata](./index.md) - [Metadata Providers](../cornerstone-core/metadataProvider.md) --- ### Cornerstone Metadata Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-metadata/index.md #### Metadata Module `@cornerstonejs/metadata` is the canonical metadata layer for current Cornerstone3D. It centralizes metadata ingestion, typed provider resolution, and shared cache behavior so applications do not need to duplicate source-specific metadata conversion logic. #### Current package role - Owns metadata provider registration and typed provider orchestration. - Normalizes source metadata into common module outputs used by core/tools. - Provides metadata cache coordination across source and derived metadata types. - Exposes utilities for tag mapping, normalized object handling, and metadata organization flows. #### Import path guidance - Recommended: import metadata APIs from `@cornerstonejs/metadata`. - Legacy compatibility: `@cornerstonejs/core` still re-exports metadata APIs via `core/src/metaData.ts`, but this path is deprecated. #### Provider model The module supports two complementary provider patterns: - **General provider chain** (`addProvider`): priority-ordered providers, highest priority first. - **Typed provider chain** (`addTypedProvider`): per-type provider composition with a typed provider bridge in the general chain. This allows applications to keep legacy provider integrations while adopting typed providers incrementally. #### Add-path ingestion and NATURALIZED Current metadata changes add explicit ingestion handlers through the add path: - `metaData.addMetaData(type, query, options)` routes to typed `typeAdd` providers. - `NATURALIZED` is the canonical base metadata state for DICOM source data. - Callers can provide source payloads (for example DICOMweb JSON or Part10 data) and let the metadata layer naturalize and cache them consistently. #### Cache and imageId model (current behavior) - Shared typed caches support read-through and in-flight de-duplication. - Source metadata (especially `NATURALIZED`) should be keyed by canonical base imageId. - Derived frame-specific modules resolve on frame imageIds. - Frame/base normalization and frame-image expansion are handled by metadata providers (including `FRAME_IMAGE_IDS`) rather than scattered call-site logic. #### Initialization and provider registration `registerDefaultProviders()` wires the default typed provider stack and related helpers. If the provider chain is reset or re-initialized by application startup flow, required providers must be re-registered after init. This is especially important during migration from older code paths where provider registration happened once and relied on persistent global state. #### Display sets The metadata layer also organizes a series' instances into **display sets** — the unit a viewport renders — via framework-agnostic split rules, exposed as the data-shaped `IDisplaySet`. Because this is a large topic (the split pipeline, driving a viewport, caching, the split-rule model, and the data model), it has its own page: - **[Display sets](./display-sets.md)** — `splitImageIdsBySplitRules` → `createDisplaySetFromGroup` → driving a viewport via `setDisplaySets` / caching via `registerDisplaySetMetadata`, the split-rule model (with the DWI worked example), and the `IDisplaySet` attributes + module-augmentation pattern. #### Package boundaries - `@cornerstonejs/metadata`: metadata ingestion, provider chains, normalized module resolution, metadata-specific cache orchestration. - `@cornerstonejs/core`: rendering/runtime primitives and rendering-focused caches and loaders. - `@cornerstonejs/dicom-image-loader`: retrieve/decode pipeline and source data handoff into metadata. - adapters: conversion between parsed metadata and tool/segmentation representations. #### Related docs - [Metadata Providers](../cornerstone-core/metadataProvider.md) - [Custom Metadata Provider](../../how-to-guides/custom-metadata-provider.md) - [5.x Migration Guides](../../migration-guides/5x/index.md) --- ## Cornerstone-tools ### Cornerstone Tools Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/index.md import DocCardList from '@theme/DocCardList'; import {useCurrentSidebarCategory} from '@docusaurus/theme-common'; #### Tools Introduction #### Tools With `Cornerstone3D` core library where each image renders in physical space (even our stack viewports are rendered at the actual position and normal direction in space), rather than any arbitrary 2D plane, we built a `Tools` library to be able to create and manipulate tools in 3D space. In `Cornerstone3DTools`, annotations are now stored in 3D patient space in a particular DICOM Frame of Reference (FoR). In general, all images in a single DICOM study exist in the same FoR (e.g. both PET and CT in a PET/CT acquisition). Let's take a look at some concepts that we will use in this library. --- ### State Management Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/stateManagement.md #### State Management We will shift from using an image ID-based default annotations manager to a FrameOfReference annotations manager, where annotations use world coordinates for points. Under the hood, the annotations manager will have a very familiar structure to current cornerstoneTools annotations managers: ```js const annotations = { myFrameOfReferenceUID: { myToolID: [ { viewPlaneNormal: [0, 0, 1], // The normal on which the tool was drawn toolUID: 'someUniqueIdentifier.1.231.4.12.5', // A unique identifier for this annotations. FrameOfReferenceUID: 'myFrameOfReference.1.2.3', toolName: 'myToolID', // properties specific to that annotation. }, // ... Other annotation entries for myToolID ], // Other annotation present on the frameOfReference }, //... other FramesOfReference }; ``` Where an individual annotations entry will look something like this: ```js // Example length annotation entry: const annotation = { viewPlaneNormal: [0, 0, 1], // Drawn on an axial plane. uid: 'someUniqueIdentifier.1.231.4.12.5', // A unique identifier for this annotations. FrameOfReferenceUID: 'myFrameOfReference.1.2.3', // The FrameOfReferenceUID toolName: LengthTool.toolName, // The tool name handles: { points: [ // Two points in world space that define the line. [23.54, 12.42, -27.6], [13.54, 14.42, -27.6], ], }, }; ``` Annotation may have properties specific to their own tools, but must contain viewPlaneNormal, UID and tool. Developers will be able to interact with the annotations manager with the following API: ```js // Adds annotation annotationManager.addAnnotation(annotation); // Remove the annotations given the annotation reference. annotationManager.removeAnnotation(annotation.annotationUID); // Returns the full annotations for a given Frame of Reference. // Optional: If a toolName is given only returns the annotations for that tool. // Optional: If a annotationUID is given, only that specific annotation is returned. annotationManager.getAnnotationsByFrameOfReference( FrameOfReferenceUID, toolName, annotationUID ); // A helper which returns the single annotation entry matching the UID. // Less efficient than getAnnotationsByFrameOfReference with all arguments, but allows // you to find the annotation if you don't have all the information. annotationManager.getAnnotation(annotationUID); // Deletes the annotation found by the given UID. // Less efficient than removeAnnotation, but can be called if you have only the UID. annotationManager.removeAnnotation(annotationUID); ``` --- ### Synchronizers Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/synchronizers.md #### Synchronizers Synchronizers can be used to link particular actions across viewports (e.g. sync pan/zoom interaction), but they can also be used to tie any callback to a particular event. Synchronizers require: - An [`Event`](/docs/api/core/namespaces/enums/enumerations/events) to listen for - A function to call when that event is raised on a source viewport - An array of `source` viewports - An array of `target` viewports The provided function receives the event, source viewports, and target viewports, and is often used to check “some value” on the source viewport. The function then updates the target viewports, often using public API exposed by the core library, to match that state/value. #### Usage The `SynchronizerManager` exposes similar API to that of the `ToolGroupManager`. A created Synchronizer has methods like `addTarget`, `addSource`, `add` (which adds the viewport as a "source" and a "target"), and equivalent `remove*` methods. Synchronizers will self-remove sources/targets if the viewport becomes disabled. Synchronizers also expose a `disabled` flag that can be used to temporarily prevent synchronization. ```js import { Enums } from '@cornerstonejs/core'; import { SynchronizerManager } from '@cornerstonejs/tools'; const cameraPositionSynchronizer = SynchronizerManager.createSynchronizer( 'synchronizerName', Enums.Events.CAMERA_MODIFIED, ( synchronizerInstance, sourceViewport, targetViewport, cameraModifiedEvent ) => { // Synchronization logic should go here } ); // Add viewports to synchronize const firstViewport = { renderingEngineId, viewportId }; const secondViewport = { /* */ }; sync.addSource(firstViewport); sync.addTarget(secondViewport); ``` #### Built-in Synchronizers We have currently implemented two synchronizers that can be used right away, #### Position Synchronizer It synchronize the camera properties including the zoom, pan and scrolling between the viewports. ```js const ctAxial = { viewportId: VIEWPORT_IDS.CT.AXIAL, type: ViewportType.ORTHOGRAPHIC, element, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }; const ptAxial = { viewportId: VIEWPORT_IDS.PT.AXIAL, type: ViewportType.ORTHOGRAPHIC, element, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, background: [1, 1, 1], }, }; const axialSync = createCameraPositionSynchronizer('axialSync')[ (ctAxial, ptAxial) ].forEach((vp) => { const { renderingEngineId, viewportId } = vp; axialSync.add({ renderingEngineId, viewportId }); }); ``` Internally, upon camera modified event on the source viewport, `cameraSyncCallback` runs to synchronize all the target viewports. For direct Generic/Next viewports, synchronizers should avoid treating `ICamera` as a universal state object. Read portable presentation through `viewportProjection.getPresentation(sourceViewport, { selector })`, translate it for the target with `viewportProjection.withPresentation(targetViewport, presentation)`, and then apply the returned native state with `targetViewport.setViewState(nextViewState)`. The projection service is pure and does not mutate either viewport. #### VOI Synchronizer It synchronizes the VOI between the viewports. For instance, if in the 3x3 layout of PET/CT, the CT image contrast gets manipulated, we want the fusion viewports to reflect the change as well. ```js const ctWLSync = createVOISynchronizer('ctWLSync'); ctViewports.forEach((viewport) => { const { renderingEngineId, viewportId } = viewport; ctWLSync.addSource({ renderingEngineId, viewportId }); }); fusionViewports.forEach((viewport) => { const { renderingEngineId, viewportId } = viewport; ctWLSync.addTarget({ renderingEngineId, viewportId }); }); ``` Internally, `voiSyncCallback` runs after the `VOI_MODIFIED` event. --- ### ToolGroups Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/toolGroups.md #### Introduction As discussed in the [`Tools`](./tools.md) section, in order to use a Tool you should fist add the tool via `CornerstoneTools3D.addTool()` AND then add and set them active on viewports via `Tool Groups`. Tool Groups are a new concept in the `Cornerstone` libraries. The goal of `ToolGroup` is to define a simple way to define tool behavior in a per viewport/per tool fashion. In addition, via a common `ToolGroup` viewports can share the same configuration, modes and tools. Consider the following set of viewports, and the desired behavior for scrolling and panning.
![](../../assets/toolGroup-intro.png)
For `ct-axial` and `ct-sagittal` viewports, we want to enable scrolling by mouse wheel and panning by mouse middle button drag. However, for the `pt-coronal` which is a Maximum Intensity Projection (MIP) viewport, scrolling through slices has no meaning, and desired behavior is to rotate the MIP volume by mouse wheel and disable panning.
![](../../assets/toolGroup-Annotated.png)
:::note Important There is a one-to-one relationship between viewports and tool groups. In other words, no viewport can be part of more than one tool group. ::: #### ToolGroup Creation and Tool Addition `ToolGroups` are managed by a `ToolGroupManager`. Tool Group Managers are used to create, search for, and destroy Tool Groups. > Currently ToolGroups are not optional, and in order to use a tool you should create a toolGroup and add it to the toolGroup. ToolGroupManager can be utilized to create a tool group using `createToolGroup`. ```js import { ToolGroupManager } from '@cornerstonejs/tools'; const toolGroupId = 'ctToolGroup'; const ctToolGroup = ToolGroupManager.createToolGroup(toolGroupId); // Add tools to ToolGroup // Manipulation tools ctToolGroup.addTool(PanTool.toolName); ctToolGroup.addTool(ZoomTool.toolName); ctToolGroup.addTool(ProbeTool.toolName); ``` #### Adding Viewports to ToolGroups Viewports should be added to the `ToolGroup` using `addViewport`. ```js // Apply tool group to viewport or all viewports rendering a scene ctToolGroup.addViewport(viewportId, renderingEngineId); ```
Why we need to pass `renderingEngineId`? The reason is `viewportId`s are unique to a rendering engine. You can have multiple rendering engines that include different viewports with the same `viewportId`.
#### Activating a Tool You can use `setToolActive` for each toolGroup to activate a tool providing a corresponding mouse bindings key. ```js // Set the ToolGroup's ToolMode for each tool // Possible modes include: 'Active', 'Passive', 'Enabled', 'Disabled' ctToolGroup.setToolActive(LengthTool.toolName, { bindings: [{ mouseButton: MouseBindings.Primary }], }); ctToolGroup.setToolActive(PanTool.toolName, { bindings: [{ mouseButton: MouseBindings.Auxiliary }], }); ctToolGroup.setToolActive(ZoomTool.toolName, { bindings: [{ mouseButton: MouseBindings.Secondary }], }); ctToolGroup.setToolActive(StackScrollMouseWheelTool.toolName); ``` Other Tool modes can also be set using `setToolEnabled`, `setToolPassive`, and `setToolDisabled`. #### ToolGroup Manager Other methods for managing ToolGroups are available via `ToolGroupManager`. #### `getToolGroupForViewport` returns the ToolGroup for a given viewport, read more [here](/docs/api/tools/namespaces/toolgroupmanager/functions/gettoolgroupforviewport) #### `getToolGroup` returns the ToolGroup for a given toolGroupId #### `destroyToolGroup` destroys a ToolGroup --- ### Tools Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/tools.md #### Introduction A tool is an uninstantiated class that implements at least the `BaseTool` interface. Tools can be configured via their constructor. To use a tool, one must: - Add the uninstantiated tool using the library's top level `addTool` function - Add that same tool, by name, to a ToolGroup Here we will introduce several concepts about tools (annotation and segmentation tools) inside `Cornerstone3DTools`. #### Tools #### Manipulation Tools `Cornerstone3DTools` provides a set of tools that can be used to manipulate the images in the viewports. These include: - enabling zooming in and out of the image (`ZoomTool`) - performing panning and navigation of the image (`PanTool`) - scrolling through the image (`StackScrollMouseWheelTool`) - manipulating the windowLevel of the image (`WindowLevelTool`) #### Annotation Tools `Cornerstone3DTools` provide a set of annotation tools. You can use these tools to create and edit annotations for use cases such as: - Measuring distance between two points (Length Tool) - Measuring height between two points (Height Tool) - Measuring width and length for a structure (Bidirectional Tool) - Measuring area and statistics for a rectangular area (RectangleRoi Tool) - Measuring volume and statistics for a ellipsoid (EllipseRoi Tool) - Getting the underlying value for a voxel (Probe Tool) Below, you can see a screenshot of the annotation tools that are available in `Cornerstone3DTools`.
![](../../assets/annotation-tools.png)
#### Dynamic tool statistics `Cornerstone3DTools` is capable of calculating dynamic statistics based on the modality of the volume being rendered. For instance, for CT volumes a `ProbeTool` will give Hounsfield Units and for PET it will calculate SUV stats.
![](../../assets/dynamic-stats.png)
#### Annotation sharing in Frame of Reference Since, annotations are stored in the patient physical space, if there are two viewports that are displaying the same frame of reference, they will share the same annotations. #### Segmentation Tools `Cornerstone3D` also provides segmentation tools. This includes3D segmentation editing tools such as brush, rectangle and circle scissors, and 3d sphere tools. We will discuss in length the different types of segmentation tools and how they are used in `Cornerstone3DTools` in [`Segmentation`](./segmentation/index.md) section.
How tools work internally mouse and keyboard fire events, these events are captured and normalized by `Cornerstone3DTools`. The normalized events are then fired and handled by tools either as `mouseDown`, `mouseDrag` and `mouseUp` events.
![](../../assets/segmentation-tools-intro.png)
#### Adding Tools The `Cornerstone3DTools` library comes packaged with several common tools. All implement either the `BaseTool` or `AnnotationTool`. In order to be able to use the tools, you must first add them to the `Cornerstone3DTools`. You can do this by using the `addTool` function. ```js import * as csTools3d from '@cornerstonejs/tools'; const { PanTool, ProbeTool, ZoomTool, LengthTool } = csTools3d; csTools3d.addTool(PanTool); csTools3d.addTool(ZoomTool); csTools3d.addTool(LengthTool); csTools3d.addTool(ProbeTool); ``` :::note warning Adding a tool to the library will only let the library know about the tool. It will not automatically add the tool to any tool groups, nor will it instantiate the tool for usage. ::: #### Tool Modes Tools (in their toolGroup) can be in one of four modes. Each mode impacts how the tool responds to interactions. > There should never be two active tools with the same binding
Tool Mode Description
Active
  • Tools with active bindings will respond to interactions
  • If the tool is an annotation tool, click events not over existing annotations will create a new annotation.
Passive (default)
  • If the tool is an annotation tool, if it's handle or line is selected, it can be moved and repositioned.
Enabled
  • The tool will render, but cannot be interacted with.
Disabled
  • The tool will not render. No interaction is possible.
#### Annotation in stretched viewport The tool prioritizes physical distance in World Coordinates. The **Circle ROI** is defined by a center point and a radius. If the viewport is stretched, the tool will automatically render as an ellipse. This ensures that if you draw a circle in stretched view, it still represents a true physical circle in patient's body. #### Segmentation Circular Brush Cursor in stretched viewport When you move your mouse, the tool calculates the cursor shape in Canvas Coordinates. It draws a circle with a radius defined in pixels. The cursor remains a perfect circle even if the image behind it is stretched 2x vertically. #### Segmentation Circular/Sphere Brush/Eraser/Scissor Tools in stretched viewport The tool maps the Canvas-space circle to the underlying image pixels. The brush should draw perfect circles, not ellipses, regardless of image stretching. When the image is stretched or shrunk, the drawn segments should stretch or shrink proportionally with the image. --- ### TouchEvents Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/touch.md #### Touch Events Touch events are fired when the user touches device with one or more touch points such as a finger or stylus. The flow of touch points are the following: 1. `TOUCH_START` 2. `TOUCH_START_ACTIVATE` 3. optional: `TOUCH_PRESS` 4. optional: `TOUCH_DRAG` 5. `TOUCH_END` Every time a user places a finger down and lifts it up, the touch order flow will always follow the above. Touch events are not mutually exclusive from click events. Other touch events that can occur independently are the `TOUCH_TAP` event and `TOUCH_SWIPE` event. A `TOUCH_TAP` will trigger a `TOUCH_START` -> `TOUCH_END` event flow. If the user taps successively, only one `TOUCH_TAP` event will fire with the count of how many times the user tapped. A `TOUCH_SWIPE` event occurs whenever the user moves more the `48px` in the canvas within a single drag cycle. Additionally, `TOUCH_SWIPE` will only activate if this movement occurs within the first `200ms` of touching the screen. If the user moves diagonal, both a `LEFT`/`RIGHT` and `UP`/`DOWN` swipe will trigger. | EVENT | Description | | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `TOUCH_START` | Triggers if the user places their touchpoint down. | | `TOUCH_START_ACTIVATE` | Triggers only if no tools decided to stop propagation from the `TOUCH_START` event. It is useful to differentiate between touching an existing annotaton, vs needing to create a new annotaiton. | | `TOUCH_PRESS` | Triggers if the user places their touchpoint down and does not move it for > 700ms | | `TOUCH_DRAG` | Triggers anytime the user moves their touchpoint, may occur before `TOUCH_PRESS` since the `TOUCH_PRESS` event will tolerate some motion. | | `TOUCH_END` | Triggers when the user lifts one or more of their touchpoints. | | `TOUCH_TAP` | Triggers when the user makes contact with screen for less than `300ms` and moves less than canvas `48px` from `TOUCH_START`. | | `TOUCH_SWIPE` | Triggers when the user user moves more than `48px` within a single drag cycle, less than `200ms` after touching the screen. | #### Multitouch Touch events natively support multitouch which is provided as a list of [`ITouchPoints[]`](api/tools/namespace/Types#ITouchPoints). In order for touch events to be compatiable with mouse events, these `ITouchPoints[]` need to be reduced into a single `ITouchPoint`. The current strategy for array reduction is taking the mean coordinate values. Other strategies can be implemented such as first point, median point, etc. This can be implemented in the [`touch` utilities codebase](https://github.com/cornerstonejs/cornerstone3D/main/packages/tools/src/utilities/touch/index.ts) The structure of `ITouchPoints` are the following: ```js type ITouchPoints = { /** page coordinates of the point */ page: Types.Point2, /** client coordinates of the point */ client: Types.Point2, /** canvas coordinates of the point */ canvas: Types.Point2, /** world coordinates of the point */ world: Types.Point3, /** Native Touch object properties which are JSON serializable*/ touch: { identifier: string, radiusX: number, radiusY: number, force: number, rotationAngle: number, }, }; ``` #### Multitouch Drag Calculations `TOUCH_DRAG` events have the following structure: ```js type TouchDragEventDetail = NormalizedTouchEventDetail & { /** The starting points of the touch event. */ startPoints: ITouchPoints, /** The last points of the touch. */ lastPoints: ITouchPoints, /** The current touch position. */ currentPoints: ITouchPoints, startPointsList: ITouchPoints[], /** The last points of the touch. */ lastPointsList: ITouchPoints[], /** The current touch position. */ currentPointsList: ITouchPoints[], /** The difference between the current and last points. */ deltaPoints: IPoints, /** The difference between distances between the current and last points. */ deltaDistance: IDistance, }; ``` `deltaPoints` is the difference between the mean coordinate point of `lastPointsList` and `currentPointsList`. `deltaDistance` is the difference between the average distance between points in `lastPointsList` vs `currentPointsList` #### Usage You can add an event listener to the element for the event. ```js import Events from '@cornerstonejs/tools/enums/Events'; // element is the cornerstone viewport element element.addEventListener(Events.TOUCH_DRAG, (evt) => { // my function on drag console.log(evt); }); element.addEventListener(Events.TOUCH_SWIPE, (evt) => { // my function on swipe console.log(evt); }); // within the chrome console in a deployed OHIF application cornerstone .getEnabledElements()[0] .viewport.element.addEventListener(Events.TOUCH_SWIPE, (evt) => { // my function on swipe console.log('SWIPE', evt); }); ``` A full example can be found by running `yarn run example stackManipulationToolsTouch` whose source is [here](https://github.com/gradienthealth/cornerstone3D/blob/gradienthealth/added_touch_events/packages/tools/examples/stackManipulationToolsTouch/index.ts) #### Binding Touch tools have bindings depending on the number of pointers that are placed down. In the future, bindings can be filter based on force, as well as radius (stylus detection). The `numTouchPoints` can be as many as is supported by hardware. ```js // Add tools to Cornerstone3D cornerstoneTools.addTool(PanTool); cornerstoneTools.addTool(WindowLevelTool); cornerstoneTools.addTool(StackScrollTool); cornerstoneTools.addTool(ZoomTool); // Define a tool group, which defines how mouse events map to tool commands for // Any viewport using the group const toolGroup = ToolGroupManager.createToolGroup(toolGroupId); // Add tools to the tool group toolGroup.addTool(WindowLevelTool.toolName); toolGroup.addTool(PanTool.toolName); toolGroup.addTool(ZoomTool.toolName); toolGroup.addTool(StackScrollTool.toolName); // Set the initial state of the tools, here all tools are active and bound to // Different touch inputs // 5 touch points are possible => unlimited touch points are supported, but is generally limited by hardware. toolGroup.setToolActive(ZoomTool.toolName, { bindings: [{ numTouchPoints: 2 }], }); toolGroup.setToolActive(StackScrollTool.toolName, { bindings: [{ numTouchPoints: 3 }], }); toolGroup.setToolActive(WindowLevelTool.toolName, { bindings: [ { mouseButton: MouseBindings.Primary, // special condition for one finger touch }, ], }); ``` The `MouseBindings.Primary` is a special binding type which will automatically bind single finger touch. #### Touch and Mouse Event Analogs Touch and Mouse Events share a lot of overlapping inheritance. Most touch events have a mouse event analog. See the below: | TOUCH EVENT | MOUSE_EVENT | | ---------------------- | --------------------- | | `TOUCH_START` | `MOUSE_DOWN` | | `TOUCH_START_ACTIVATE` | `MOUSE_DOWN_ACTIVATE` | | `TOUCH_PRESS` | N/A | | `TOUCH_DRAG` | `MOUSE_DRAG` | | `TOUCH_SWIPE` | N/A | | `TOUCH_END` | `MOUSE_UP` | | `TOUCH_TAP` | `MOUSE_CLICK` | The main difference between touch events and mouse events are that touch events can have multiple pointers (multi-touch). Touch events will automatically reduce multiple pointers into a single point value. The default way these points are reduced is taking the weighted average. This reduced point can be used as a `IPoints` or `ITouchPoints` depending if touch information is needed. In the case multiple touch points are needed, they are accessible in list form. ```js type MousePointsDetail = { /** The starting points of the mouse event. */ startPoints: IPoints, /** The last points of the mouse. */ lastPoints: IPoints, /** The current mouse position. */ currentPoints: IPoints, /** The difference between the current and last points. */ deltaPoints: IPoints, }; type TouchPointsDetail = { /** The starting points of the touch event. */ startPoints: ITouchPoints, /** The last points of the touch. */ lastPoints: ITouchPoints, /** The current touch position. */ currentPoints: ITouchPoints, startPointsList: ITouchPoints[], /** The last points of the touch. */ lastPointsList: ITouchPoints[], /** The current touch position. */ currentPointsList: ITouchPoints[], /** The difference between the current and last points. */ deltaPoints: IPoints, /** The difference between distances between the current and last points. */ deltaDistance: IDistance, }; ``` --- ## Cornerstone-tools/annotation ### Annotation Groups Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/annotationGroups.md #### Annotation Groups In order to indicate that annotations are related to each other, there is an `AnnotationGroup` class that can be used to group annotations. Currrently, the grouping is very basic and isn't saved/restored automatically within the adapters. The requirements for enhanced grouping are still being gathered, but the basic capability is there to use. Annotations can be added to a group, and navigated between them by finding the next/previous annotation. #### Creating a new group To create a new annotation group, just create an instance of AnnotationGroup. #### Adding an annotation to a group Annotations can automatically be added to a group if the group is active, and has had the addListeners method called on it. Alternatively, they can be added manually by calling the add method on the annotation group. For example: ```javascript const group = new cornerstoneTools.annotation.AnnotationGroup(); group.add(annotation.annotationUID); ``` #### Setting visibility of annotations Annotations can be shown/hidden by calling the setVisibility method on the annotation group. This takes an optional second parameter which will prevent hiding for any filtered elements (those where the filter function returns false). There is a default filter function provided that excludes any members of the current group which are visible because of visibility flags in the group. This allows overlapping groups to be used, with the annotations only being hidden when all annotations groups are not visible. ```javascript // Toggle visibility of group members only. // Need the other information to fire events group.setVisibility(!group.isVisible, { viewportId, renderingEngineId }); ``` --- ### Annotation Manager Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/annotationManager.md The Annotation Manager is a singleton class that manages annotations in Cornerstone Tools. We use the Annotation Manager to store annotations, retrieve annotations, and save and restore annotations. #### Default Annotation Manager The default Annotation Manager, `FrameOfReferenceSpecificAnnotationManager`, stores annotations based on the FrameOfReferenceUID. This means that annotations are stored separately for each FrameOfReferenceUID. Currently in our rendering pipeline, if two VolumeViewports share the same FrameOfReferenceUID, they will share the same annotations. However, StackViewports works on the per imageId basis, so annotations are not shared between StackViewports. #### GroupKey Annotation groups are identified by a groupKey. The groupKey is a string that is used to identify the group of annotations. As mentioned above, the default Annotation Manager stores annotations based on the FrameOfReferenceUID, so the groupKey is the `FrameOfReferenceUID`. #### Custom Annotation Manager You can create your own custom Annotation Manager by implementing the `IAnnotationManager` interface: ```ts interface IAnnotationManager { getGroupKey: (annotationGroupSelector: any) => string; getAnnotations: ( groupKey: string, toolName?: string ) => Annotations | GroupSpecificAnnotations | undefined; addAnnotation: (annotation: Annotation, groupKey?: string) => void; removeAnnotation: (annotationUID: string) => void; removeAnnotations: (groupKey: string, toolName?: string) => void; saveAnnotations: ( groupKey?: string, toolName?: string ) => AnnotationState | GroupSpecificAnnotations | Annotations; restoreAnnotations: ( state: AnnotationState | GroupSpecificAnnotations | Annotations, groupKey?: string, toolName?: string ) => void; getNumberOfAllAnnotations: () => number; removeAllAnnotations: () => void; } ``` To use the Annotation Manager, you can set it as the default Annotation Manager using ```js import { annotation } from '@cornerstonejs/tools'; import myCustomAnnotationManager from './myCustomAnnotationManager'; annotation.state.setAnnotationManager(myCustomAnnotationManager); ``` The most important method in a custom Annotation Manager is the `getGroupKey` method. This method is used to determine the groupKey for a given element. For instance, if you have a usecase to show two separate annotations (e.g. two different readers) on two viewports that share the same FrameOfReferenceUID, you can use the `getGroupKey` method to return a different groupKey for each viewport given the element. (certainly you don't want to share the same annotations between the two viewports). --- ### Config Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/config.md In this section we will explain various ways you can change the tool styles. This includes various properties such as `color` when `selected`, `highlighted`, or `locked`; textbox color, line dash style and thickness and more. #### Style Hierarchy We will start by looking at the style hierarchy. The style hierarchy is as follow. - Annotation-level settings (with UID) **set/getAnnotationToolStyle** - Viewport-level tool settings **set/getViewportToolStyle** - Per-tool this layer: Length on this viewport - Global this layer: All tools in this viewport - toolGroup settings (for any tool specified in this toolGroup in all viewports of the toolGroup) **set/getToolGroupToolStyle** - Per-tool layer: Angle on this toolGroup in all viewports - Global this layer: All tools in this toolGroup in all viewports - Default level: **set/getDefaultToolStyle** - Per-tool layer (Length) settings - Global (app-level) settings (we provide a default). In annotation rendering loop, upon getting a style for a certain property (`color`, `lineDash`, `lineThickness`) we check whether the style is set at the annotation level (highest priority). If not, we check whether any viewport-level setting is set (for the viewport annotation is drawing on); however, in the viewportLevel, we first check whether the tool-level setting is set. If not, we check in the "global" (all tools in the viewport) level. If not found, we move to the next level for toolGroup level. If not found, we move to the next level for global level which is last level to check. ![configs](../../../assets/configs.png) #### Default Setting `Cornerstone3DTools` initializes a default settings for toolsStyles class that can be found in `packages/tools/src/stateManagement/annotation/config/ToolStyle.ts` ```js { colorHighlighted: 'rgb(0, 255, 0)', colorSelected: 'rgb(0, 220, 0)', colorLocked: 'rgb(209, 193, 90)', lineWidth: '1', lineDash: '', shadow: true, textBoxVisibility: true, textBoxFontFamily: 'Helvetica Neue, Helvetica, Arial, sans-serif', textBoxFontSize: '14px', textBoxColor: 'rgb(255, 255, 0)', textBoxMargin: '0', textBoxBorderRadius: '0', textBoxColorHighlighted: 'rgb(0, 255, 0)', textBoxColorSelected: 'rgb(0, 255, 0)', textBoxColorLocked: 'rgb(209, 193, 90)', textBoxLinkLineWidth: '1', textBoxLinkLineDash: '2,3', textBoxShadow: true, markerSize: '10', angleArcLineDash: '', }; ``` However, you can adjust each of the above parameters along with other styles that we will discuss next. #### Set styles Each level of the style hierarchy has a set of styles that can be set. The styles are as follow. #### Annotation-level settings ```js import { annotation } from '@cornerstonejs/tools'; // Annotation Level const styles = { colorHighlighted: 'rgb(255, 255, 0)', }; annotation.config.style.setAnnotationStyles(annotationUID, style); ``` #### Viewport-level tool settings ```js import { annotation } from '@cornerstonejs/tools'; // Viewport Level const styles = { LengthTool: { colorHighlighted: 'rgb(255, 255, 0)', }, global: { lineWidth: '2', }, }; annotation.config.style.setViewportToolStyle(viewportId, styles); ``` #### ToolGroup-level tool settings ```js import { annotation } from '@cornerstonejs/tools'; const styles = { LengthTool: { colorHighlighted: 'rgb(255, 255, 0)', }, global: { lineWidth: '2', }, }; annotation.config.style.setToolGroupToolStyles(toolGroupId, styles); ``` #### Global(Default)-level tool settings ```js import { annotation } from '@cornerstonejs/tools'; const styles = annotation.config.style.getDefaultToolStyle(); const newStyles = { ProbeTool: { colorHighlighted: 'rgb(255, 255, 0)', }, global: { lineDash: '2,3', }, }; annotation.config.style.setDefaultToolStyle(deepMerge(styles, newStyles)); ``` #### Configurable Styles Currently we have the following styles that can be configured. ```js color; colorActive; colorHighlighted; colorHighlightedActive; colorHighlightedPassive; colorLocked; colorLockedActive; colorLockedPassive; colorPassive; colorSelected; colorSelectedActive; colorSelectedPassive; lineDash; lineDashActive; lineDashHighlighted; lineDashHighlightedActive; lineDashHighlightedPassive; lineDashLocked; lineDashLockedActive; lineDashLockedPassive; lineDashPassive; lineDashSelected; lineDashSelectedActive; lineDashSelectedPassive; lineWidth; lineWidthActive; lineWidthHighlighted; lineWidthHighlightedActive; lineWidthHighlightedPassive; lineWidthLocked; lineWidthLockedActive; lineWidthLockedPassive; lineWidthPassive; lineWidthSelected; lineWidthSelectedActive; lineWidthSelectedPassive; textBoxBackground; textBoxBackgroundActive; textBoxBackgroundHighlighted; textBoxBackgroundHighlightedActive; textBoxBackgroundHighlightedPassive; textBoxBackgroundLocked; textBoxBackgroundLockedActive; textBoxBackgroundLockedPassive; textBoxBackgroundPassive; textBoxBackgroundSelected; textBoxBackgroundSelectedActive; textBoxBackgroundSelectedPassive; textBoxColor; textBoxColorActive; textBoxColorHighlighted; textBoxColorHighlightedActive; textBoxColorHighlightedPassive; textBoxColorLocked; textBoxColorLockedActive; textBoxColorLockedPassive; textBoxColorPassive; textBoxColorSelected; textBoxColorSelectedActive; textBoxColorSelectedPassive; textBoxMargin; textBoxMarginActive; textBoxMarginHighlighted; textBoxMarginHighlightedActive; textBoxMarginHighlightedPassive; textBoxMarginLocked; textBoxMarginLockedActive; textBoxMarginLockedPassive; textBoxMarginPassive; textBoxMarginSelected; textBoxMarginSelectedActive; textBoxMarginSelectedPassive; textBoxBorderRadius; textBoxBorderRadiusActive; textBoxBorderRadiusHighlighted; textBoxBorderRadiusHighlightedActive; textBoxBorderRadiusHighlightedPassive; textBoxBorderRadiusLocked; textBoxBorderRadiusLockedActive; textBoxBorderRadiusLockedPassive; textBoxBorderRadiusPassive; textBoxBorderRadiusSelected; textBoxBorderRadiusSelectedActive; textBoxBorderRadiusSelectedPassive; textBoxFontFamily; textBoxFontFamilyActive; textBoxFontFamilyHighlighted; textBoxFontFamilyHighlightedActive; textBoxFontFamilyHighlightedPassive; textBoxFontFamilyLocked; textBoxFontFamilyLockedActive; textBoxFontFamilyLockedPassive; textBoxFontFamilyPassive; textBoxFontFamilySelected; textBoxFontFamilySelectedActive; textBoxFontFamilySelectedPassive; textBoxFontSize; textBoxFontSizeActive; textBoxFontSizeHighlighted; textBoxFontSizeHighlightedActive; textBoxFontSizeHighlightedPassive; textBoxFontSizeLocked; textBoxFontSizeLockedActive; textBoxFontSizeLockedPassive; textBoxFontSizePassive; textBoxFontSizeSelected; textBoxFontSizeSelectedActive; textBoxFontSizeSelectedPassive; textBoxLinkLineDash; textBoxLinkLineDashActive; textBoxLinkLineDashHighlighted; textBoxLinkLineDashHighlightedActive; textBoxLinkLineDashHighlightedPassive; textBoxLinkLineDashLocked; textBoxLinkLineDashLockedActive; textBoxLinkLineDashLockedPassive; textBoxLinkLineDashPassive; textBoxLinkLineDashSelected; textBoxLinkLineDashSelectedActive; textBoxLinkLineDashSelectedPassive; textBoxLinkLineWidth; textBoxLinkLineWidthActive; textBoxLinkLineWidthHighlighted; textBoxLinkLineWidthHighlightedActive; textBoxLinkLineWidthHighlightedPassive; textBoxLinkLineWidthLocked; textBoxLinkLineWidthLockedActive; textBoxLinkLineWidthLockedPassive; textBoxLinkLineWidthPassive; textBoxLinkLineWidthSelected; textBoxLinkLineWidthSelectedActive; textBoxLinkLineWidthSelectedPassive; // nb: textBoxLinkLineColor falls back to the corresponding textBoxColor if not set textBoxLinkLineColor; textBoxLinkLineColorActive; textBoxLinkLineColorHighlighted; textBoxLinkLineColorHighlightedActive; textBoxLinkLineColorHighlightedPassive; textBoxLinkLineColorLocked; textBoxLinkLineColorLockedActive; textBoxLinkLineColorLockedPassive; textBoxLinkLineColorPassive; textBoxLinkLineColorSelected; textBoxLinkLineColorSelectedActive; textBoxLinkLineColorSelectedPassive; ``` --- ### Annotations Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/index.md import DocCardList from '@theme/DocCardList'; import {useCurrentSidebarCategory} from '@docusaurus/theme-common'; #### Annotations In `Cornerstone3DTools`, Annotation Tools keep their state in a `state` object. This object is a plain JavaScript object that is used to store the state of the annotation instance. Information such as the statistics of the annotation, its data and camera position are stored in this object. There are various methods for adding/removing, selection, locking and unlocking of annotations. They can be accessed via the `annotations` name space in the `Cornerstone3DTools` by calling: ```js import { annotation } from '@cornerstonejs/tools'; // All methods to deal with annotation state can be accessed via annotation.state.XYZ; // All methods for annotation selection can be accessed via annotation.selection.XYZ; // All methods for annotation locking can be accessed via annotation.locking.XYZ; // All methods for annotation styling can be accessed via annotation.config.XYZ; // The AnnotationGroup class allows for grouping of annotations annotation.AnnotationGroup; ``` Let's start by looking deeper into each of these methods. --- ### Locking Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/locking.md #### Locking Annotations can be locked to avoid accidental changes. You can use the locking API to lock/unlock annotations. #### API There are various APIs for locking and unlocking annotations along with get/set methods ```js import { annotation } from '@cornerstonejs/tools'; // locking of an annotation annotation.locking.setAnnotationLocked(annotationUID, (locked = true)); // get all the locked annotations annotation.locking.getAnnotationsLocked(); // unlock all annotations annotation.locking.unlockAllAnnotations(); ``` #### Read more :::note TIP Read more about the locking API [here](/docs/api/tools/namespaces/annotation/namespaces/locking) ::: --- ### Measurement Targets Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/measurementTargets.md #### Measurement Targets Annotation tools that calculate statistics (such as `CircleROITool` and `RectangleROITool`) store their results in the annotation's `cachedStats`, keyed by a **targetId** identifying the image data the statistics were computed on. On a stack viewport the targetId is derived from the imageId; on a volume viewport it is derived from the volumeId. A viewport can display more than one set of image data at once — a PT/CT fusion viewport has both a CT and a PT volume. Each displayed volume is a candidate **measurement target**, and the tool configuration decides which of them the tool computes and displays statistics for. **By default, the ROI statistics tools (`CircleROITool`, `RectangleROITool`) compute and display the statistics of every display set containing pixel values** — on a CT/PT fusion viewport both the HU and the SUV statistics are shown at once. Display sets whose modality does not carry measurable pixel values (SEG, RTSTRUCT, SR, ...) are never included, even when they are the only thing shown. #### The `targetsFilter` and `targetPredicate` configuration Target selection is split into two composable halves, so each stays a simple function: - **`targetsFilter`** — the _chooser_. It decides the **cardinality** (first vs all) and receives the whole candidate array plus an options object (the viewport and the tool configuration), returning the subset to measure. - **`targetPredicate`** — the _per-candidate predicate_. It decides whether **one** candidate is eligible, returning `true`/`false`. The chooser calls it once per candidate; the predicate never has to know how many targets are wanted. ```ts type MeasurementTargetsFilter = ( candidates: MeasurementTargetCandidate[], options: MeasurementTargetOptions // { viewport, configuration, data } ) => MeasurementTargetCandidate[]; type MeasurementTargetPredicate = ( candidate: MeasurementTargetCandidate, options: MeasurementTargetOptions ) => boolean; ``` Because the two decisions are independent, the same `forModality('PT')` predicate means _"the first PT"_ under the `firstPixelData` chooser and _"every PT"_ under `allPixelData`, without writing a bespoke combined filter. The display set related parameters of each candidate include: - `displaySet` — the display set being shown, where registered (an `IDisplaySet` from `@cornerstonejs/metadata` via the `displaySetModule` metadata module, or the display set registered with the generic viewports) - `displaySetUID` — the uid of the display set, where known - `instance` — an exemplar (first) instance of the display set: naturalized DICOM metadata (with `Modality`, `Rows`, `SeriesInstanceUID`, ...), if available - `index` — the index of this display set within the viewport - plus convenience fields: `modality`, `imageIds` and `referencedId` (the backing volume/image id) #### The two built-in choosers The ready-made choosers apply the pixel-data test **first** (so segmentations etc are never measured), and then the configured `targetPredicate` when one is set: - **`allPixelData`** (the ROI tools' default) — every eligible candidate, via `filter`. On a CT/PT fusion viewport this measures both volumes at once. - **`firstPixelData`** — just the first eligible candidate, via `find` (it stops at the first match instead of building an intermediate array), or an empty array when nothing is eligible. There are also the raw `first`/`all` choosers, which ignore the predicate and the pixel-data test — an escape hatch for measuring literally the first or all candidates. The included candidates drive both: - **The primary targetId** — `getTargetId` returns the first included target, so the chooser controls which statistics are stored/read by default. - **Multi-target statistics** — every included target has its statistics computed and displayed. On a single fusion viewport of CT and PT, a chooser including both display sets makes that one viewport compute the statistics for both volumes, each over its own pixel data, even if no other viewport has computed them. The predicate's decision should be based on the **modality of the display set** where available. When the display set is unknown — for example a stack viewport using the legacy set image ids — the candidate has no `displaySet`/`instance`/`imageIds`/`modality`, and the predicate can choose whether to include it based on those being undefined. A configured chooser's result is **authoritative**: when it includes no candidates (a PT-only predicate on a CT viewport, or the default pixel-data test when only a SEG is shown), the annotation is still drawn but no statistics are computed or displayed for that viewport. Ready-made choosers and predicates are the pure functions exported from `measurementTargetFilters`, defined once outside any tool: ```ts import { measurementTargetFilters } from '@cornerstonejs/tools'; ``` #### Examples All display sets with pixel values — this is the default configuration for the ROI tools, made explicit. Candidates with a non-pixel modality (non-pixel modalities: SEG, RTSTRUCT, RTPLAN, SR, PR, KO) are excluded, while candidates with an unknown display set (legacy stacks) are included: ```ts toolGroup.addTool(CircleROITool.toolName, { targetsFilter: measurementTargetFilters.allPixelData, }); ``` CT statistics only — nothing is shown on viewports without a CT. The chooser takes every candidate the predicate keeps: ```ts toolGroup.addTool(CircleROITool.toolName, { targetsFilter: measurementTargetFilters.allPixelData, targetPredicate: measurementTargetFilters.forModality('CT'), }); ``` PT statistics only (on a fusion viewport this shows the SUV statistics; nothing is shown on viewports without a PT): ```ts toolGroup.addTool(CircleROITool.toolName, { targetsFilter: measurementTargetFilters.allPixelData, targetPredicate: measurementTargetFilters.forModality('PT'), }); ``` Both CT and PT explicitly — like the default, but restricted to exactly those two modalities: ```ts toolGroup.addTool(CircleROITool.toolName, { targetsFilter: measurementTargetFilters.allPixelData, targetPredicate: measurementTargetFilters.forModality('CT', 'PT'), }); ``` Just the first pixel-data target (the pre-5.x single-target behaviour): ```ts toolGroup.addTool(CircleROITool.toolName, { targetsFilter: measurementTargetFilters.firstPixelData, }); ``` The first PT target only — the `firstPixelData` chooser with the PT predicate: ```ts toolGroup.addTool(CircleROITool.toolName, { targetsFilter: measurementTargetFilters.firstPixelData, targetPredicate: measurementTargetFilters.forModality('PT'), }); ``` A specific volume by id (a substring match, so a series UID contained in the id also works). This replaces the deprecated `isPreferredTargetId` configuration: ```ts toolGroup.addTool(RectangleROITool.toolName, { targetsFilter: measurementTargetFilters.allPixelData, targetPredicate: measurementTargetFilters.forId(ptVolumeId), }); ``` Predicates are plain functions, so any per-candidate selection logic is possible — including deciding from the exemplar instance or handling unknown display sets explicitly: ```ts toolGroup.addTool(CircleROITool.toolName, { // Custom predicate: PT only, decided from the exemplar instance targetsFilter: measurementTargetFilters.firstPixelData, targetPredicate: (candidate) => candidate.instance?.Modality === 'PT', }); toolGroup.addTool(CircleROITool.toolName, { // PT statistics where the display set is known, plus anything whose // display set is unknown (no imageIds array, eg legacy stacks) targetsFilter: measurementTargetFilters.allPixelData, targetPredicate: (candidate) => !candidate.imageIds || candidate.modality === 'PT', }); ``` The `tmtv` example wires several of these as separately labelled dropdown entries (default both, PT SUV only, CT HU only) on a PT/CT fusion layout, and the `petCt` example shows the `forId` variant. #### How it behaves #### Candidate derivation The candidates passed to the filter are built by `BaseTool.getMeasurementTargetCandidates` from the viewport's actors: - one candidate per actor whose `referencedId` is a volume present in the cache — the `displaySet`/`displaySetUID` come from the viewport's registered display sets where one matches the volume, the exemplar `instance` from the display set's instances or the `instance` metadata of the first image id, and `modality` from the instance or the volume metadata; - actors not derived from a cached volume (tool/canvas actors) are skipped. Segmentation representations (labelmaps etc) are **not** skipped here — they are included as candidates carrying a `representationUID`, and it is the configured chooser/predicate that decides whether to include them. The default `isPixelData` predicate (applied by the `firstPixelData`/ `allPixelData` choosers) excludes any candidate with a `representationUID`, so segmentations are never measured by default, but a custom predicate can opt to include them; - when no actor produces a candidate (for example on a stack viewport), a single candidate for the viewport's default view reference is used. If the viewport displays a registered display set (`setDisplaySets` on the generic viewports), the candidate's display set fields are resolved from the `displaySetModule` metadata (an `IDisplaySet` from `@cornerstonejs/metadata`) or the generic viewport display set registration; a legacy stack (`setStack` with plain image ids) has no display set, so the candidate carries none of the display set fields — filters can detect this via the missing `imageIds`/`instance`. #### targetId computation and reuse Statistics are keyed in `cachedStats` by view reference IDs. Volume candidates use IDs of the form `volumeId:?sliceIndex=...&viewPlaneNormal=...`; stack candidates keep using imageId-derived target IDs. For each volume candidate, if the annotation already has a `cachedStats` key whose embedded volume ID exactly matches the candidate, that existing key is reused. The annotation's world-space geometry, and therefore its statistics, do not depend on the viewing orientation, so recomputing per view would only duplicate work and display entries. Only when no key exists for the volume is a new targetId generated from this viewport's view reference for that volume. #### Seeding and computing multiple statistics The statistics calculators of the tools iterate the keys of the annotation's `cachedStats`. When a tool renders on a viewport, it seeds (via `BaseTool.ensureCachedStatsTargets`) a `cachedStats` entry for every filtered target that does not have one yet, and then recalculates — so a fusion viewport seeds and computes both its CT and PT statistics itself, even if the annotation was originally drawn on (and computed by) a single-volume viewport. Conversely, an annotation drawn on a fusion viewport with a multi-target filter carries the statistics of both volumes to every other viewport displaying it. The text box then renders one line per metric with the values of each target, for example `Mean: 34 HU 2.3 SUV`, skipping duplicated values. #### Relation to `isPreferredTargetId` The older `isPreferredTargetId` configuration (and the `BaseTool.isSpecifiedTargetId` helper) could only choose the preferred targetId among statistics that some viewport had already computed. It is deprecated in favour of `targetsFilter`, which selects among the actual display sets of the viewport and can therefore also cause statistics to be computed for targets no viewport has computed yet. For backward compatibility, a configured `isPreferredTargetId` is honoured **before** the filter (including the ROI tools' default filter), so configurations that predate `targetsFilter` keep their behaviour. :::note Multi-target selection currently only works for volumes displayed on screen. Stack-based fusion and targets not currently displayed are not yet supported. ::: --- ### Selection Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/selection.md #### Selection Annotations can be selected and deselected. This is achieved by holding down the `Shift` key (by default) and clicking on annotations. #### API There are various APIs for selecting and deselecting annotations along with get/set methods ```js import { annotation } from '@cornerstonejs/tools'; // selection of an annotation annotation.selection.setAnnotationSelected( annotationUID, (selected = true), (preserveSelected = false) ); // get all the selected annotations annotation.selection.getAnnotationsSelected(); // get all selected annotations from a specific tool annotation.selection.getAnnotationsSelectedByToolName(toolName); ``` #### Read more :::note TIP Read more about the selection API [**here**](/docs/api/tools/namespaces/annotation/namespaces/selection/) ::: --- ### State Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/state.md #### State Management `Cornerstone3DTools` implements a `FrameOfReference` annotations state manager, where annotations use world coordinates for points. #### Annotation Data When a new annotation is created (`addNewAnnotation` method in the annotation tools), a new annotation data is created based on the metadata and the current state of the tool, and it gets added to the global annotation state. In the following we show the annotation data for a `ProbeTool` instance. Other tools, basically follow the same pattern. ```js // ProbeTool Annotation Data const annotation = { invalidated: boolean, // Whether the annotation data has been invalidated by e.g., moving its handles highlighted: boolean, // Whether the annotation is highlighted by mouse over annotationUID: string, // The UID of the annotation metadata: { viewPlaneNormal: Types.Point3, // The view plane normal of the camera viewUp: Types.Point3, // The view up vector of the camera FrameOfReferenceUID: string, // viewport's FrameOfReferenceUID the annotation has been drawn on referencedImageId?: string, // The image ID the annotation has been drawn on (if applicable) toolName: string, // The tool name }, data: { handles: { points: [Types.Point3], // The handles points in world coordinates (probe tool = 1 handle = 1 x,y,z point) }, cachedStats: {}, // Stored Statistics for the annotation }, } ``` #### Annotation State Annotations state keeps track of the annotations for each FrameOfReference. The state is composed of a `FrameOfReference`-specific state object, in which each annotation-specific state is stored. Below, you can see a high-level overview of the state object.
![](../../../assets/annotation-state.png)
#### API You can get/add annotations using the following API: ```js // Adds annotation cornerstone3DTools.annotation.state.addAnnotation( annotation, element, suppressEvents ); // Remove the annotations given the annotation reference. cornerstone3DTools.annotation.state.removeAnnotation( annotationUID, suppressEvents ); // Returns the full annotations for a given Tool cornerstone3DTools.annotation.state.getAnnotations(toolName, element); // A helper which returns the single annotation entry matching the UID. cornerstone3DTools.annotation.state.getAnnotation(annotationUID); ``` #### Read more :::note TIP Read more about the state API [here](/api/tools/namespace/annotation#state) ::: --- ### Visibility Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/visibility.md #### Visibility Annotations can have their visibility changed. You can use the visibility API to show/hide annotations. #### API There are various APIs for showing and hiding annotations along with get/set methods ```js import { annotation } from '@cornerstonejs/tools'; // changing an annotation visibility to be visible (implicit visible param). annotation.visibility.setAnnotationVisibility(annotationUID); // changing an annotation visibility to NOT be visible. annotation.visibility.setAnnotationVisibility(annotationUID, false); // show all annotation(hidden) annotation.visibility.showAllAnnotations(); // get if an annotation is visible or not. // Possible results are: undefined if there is no annotation for given UID, true if visible and false if not. annotation.visibility.isAnnotationVisible(annotationUID); ``` #### Read more :::note TIP Read more about the visibility API [here](/docs/api/tools/namespaces/annotation/namespaces/visibility) ::: --- ### Voxel Statistics and Oblique Views Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/annotation/voxel-statistics.md #### Voxel Statistics and Oblique Views #### The problem A tool draws a shape on a viewport, and that shape stands for a set of voxels. An area annotation is a **prism**: the outline sweeps along the view normal, and the prism holds every voxel inside the outline and within the annotation's own thickness. Some tools draw a **solid** instead, such as a sphere or a box, and the solid holds every voxel inside it. Either way the tool needs the same thing: every voxel of that set, exactly once each. Producing that set is harder than it looks, and Cornerstone3D already contains four separate attempts at it: | tool | how it walks the voxels | | ----------------------- | --------------------------------------------------------------------------------------------------------------- | | `RectangleROITool` | the index-space bounding box of the two corner handles, and no shape test at all | | `EllipticalROITool` | the same bounding box, plus `pointInEllipse` on every voxel in the box | | `CircleROITool` | the same bounding box, plus a sphere test on every voxel in the box | | `PlanarFreehandROITool` | the same bounding box, plus `worldToCanvas` on every voxel, and a crossing count that carries state across rows | Every one of the four is a bounding box paired with a per-voxel test, and every one of the four is wrong in a different way. The rectangle omits the test, so it is exact for an axis-aligned rectangle and it over-counts the corners of a rotated one. The freehand tool works in canvas coordinates, so the answer moves when the user zooms. All four build the box from one slice, so all four return a sheet one voxel thick. Three failure modes follow, and no choice of sample step size reaches any of the three. **A nearest-neighbour sample cannot cover an integer lattice under rotation.** At a 45° in-plane oblique angle, samples along `(0.707, 0.707)` in IJK round to `(0,0), (1,1), (2,2)…` and never visit `(1,0)` or `(0,1)`. That is about half the voxels. The sub-pixel phase of the camera decides which half the code skips, so a half-pixel pan changes the reported maximum. **A sampled set derived from the canvas depends on the display.** Such a set is a function of the zoom, the pan, the canvas size and `devicePixelRatio`. The same annotation over the same data then reports a different mean, because the display state differed when the tool recalculated the statistics. **A single-plane traversal returns a sheet one voxel thick.** Draw a freehand annotation on an NM series with 1 mm slices, and fuse that series with a CT series at 0.5 mm in the same orientation. The correct CT maximum must examine two CT voxels for each in-plane location. Extra in-plane samples never produce the second voxel, because every sample lies on the same plane. The four traversals are also slow, and each one carries its own special cases. The bounding box of a disc holds `4 / π` times as many voxels as the disc, so a quarter of the per-voxel tests are wasted before the plane is even oblique. Tilt the plane and the box becomes the 3D box around the tilted prism, which holds many times the voxels of the prism itself. #### The solution: one iterator `iterateVoxelsInShape` walks the voxel set directly, and no tool needs a traversal of its own. It works in index space, where the depth test is exactly linear in the integer voxel indices. For each row it therefore solves two closed intervals in closed form, one from the slab and one from the shape, intersects the two, and emits the integers inside. It tests no voxel that it does not emit, and it reads nothing from the display. An area statistic becomes a loop over that iterator and an accumulator. The tool supplies the shape and the thickness, the iterator supplies the voxels, and the mean, the maximum and the count follow from one pass. Rule M and Rule D below define the set that the iterator produces, so a tool that uses the iterator gets the defined answer without knowing the arithmetic. Rule M and Rule D are **normative**. Issue [#2889](https://github.com/cornerstonejs/cornerstone3D/issues/2889) states them as well. The index-space arithmetic that evaluates Rule M quickly is an implementation detail, and anyone may change it as long as it selects the same voxels. #### The base case is the base of the iterator The ordinary case is a non-oblique view of a single layer: the plane lies on the acquisition axis, and the annotation is one voxel thick. The iterator does not special-case that view. It **is** the base of the iterator: the outer axis becomes the slice axis, the depth interval resolves to one layer, and the inner loop emits runs along `i` for each `j`, in memory order. The general oblique case is the same three loops with a depth interval that moves. Even in that base case the iterator is generally faster than the four traversals, because a shape that supplies runs needs no per-voxel test. The older code tests every voxel of the bounding box and rejects most of them. The iterator solves the row once and emits an interval, so the count of shape tests falls from the size of the box to zero. The gap widens as the geometry gets harder. In a stretched space, where the spacing is anisotropic, a circle in world coordinates is an eccentric ellipse in index space, and a rectangle rotated in the plane is a rotated box in index space. The bounding box of either grows faster than its content, so a bounding-box traversal wastes more of its work. The closed form does not care: an ellipsoid solves one quadratic per row, and a box solves one linear inequality per axis, at every angle and at every aspect ratio. #### Rule M: voxel membership A voxel belongs to an area annotation when the voxel obeys two conditions: 1. The voxel centre lies within `(T + T_v) / 2` of the annotation plane, measured along the normal. 2. The projection of that centre along the normal onto the plane falls inside the 2D shape. `T` is the thickness of the annotation. `T_v` is the voxel thickness along the normal. The viewport slab thickness `t` does not appear in Rule M. That absence is the purpose of the rule: the statistics cannot change because a user zoomed the viewport, resized the canvas, or increased the slab. The `T_v` term widens the slab by half a voxel on each side, so a voxel qualifies exactly when the voxel itself overlaps the slab. The term has no effect in the default case of `T = T_v` anchored on a voxel centre, which gives one layer either way. The term matters for a plane that misses the voxel centres, which would otherwise select nothing, and for a thicker slab, where an unwidened test asked for two voxels of thickness would select one. :::note A plane exactly midway between two voxel centres selects **both** layers. Both voxels overlap the slab by equal amounts, so no principled way to choose one exists, and a choice would make the count depend on a rounding tie. A mean over two layers is not the same number as a mean over one, and MPR at a half-slice position is the ordinary way to reach this state. ::: #### Rule D: display A viewport shows a plane when the distance from the plane point to the focal point, along the normal, is within `(t + T) / 2`. The effects across modalities are intended. An annotation on one thick NM slice can correctly appear on two thin CT slices, and an annotation that spans two CT slices can correctly appear on one NM slice. As in Rule M the comparison is strict and tightened by a relative epsilon, because the common case puts the neighbouring slice exactly on the boundary and must exclude it. A viewport shows the annotations created on its own slice, not those on the next one. Here the epsilon is relative to the half width, and not to the voxel thickness `T_v` that Rule M uses, because `T_v` needs a volume and a display decision is made without one. A reference that records no thickness falls back to an exact plane match to within `isEqual`. Any annotation created before `PlaneRestriction.referencePlaneThickness` existed records no thickness, and a wider visibility would change which slices those annotations appear on. #### Where `T` comes from A new annotation takes `T` once, at creation, from the slab thickness of the viewport that the user drew in. `Viewport.getReferencePlaneThickness` supplies the value, and `BaseVolumeViewport` overrides that method. After creation, `T` belongs to the annotation and lives on `PlaneRestriction.referencePlaneThickness`. When a reference records no thickness, `T` defaults to one voxel along the normal. A stack viewport uses that default, and so does every annotation that predates the field. A `T` of 0 or less also counts as unrecorded and takes the same default. A planar shape reports 0 from `getRequiredThickness`, and a caller may pass that value straight to the iterator, so 0 has to mean "the shape asks for no depth of its own". Reading the slab once at creation does not contradict the independence of Rule M from `t`. The code reads the slab when it creates the reference, and never when it recalculates the statistics. #### Two conversions on the volume viewport `BaseVolumeViewport.getReferencePlaneThickness` applies two conversions that a reader of the raw slab value would miss. **It doubles the value.** `getSlabThickness` returns the number passed to `setOrientationOfClippingPlanes`, which places the clipping planes at `focalPoint ± slabThickness`. The stored number is therefore a _half_ thickness on that render path, and the geometric thickness is twice it. The generic planar path uses `vtkImageResliceMapper`, where the same field is already a full thickness, so the doubling belongs on the volume viewport and not in the shared reference code. **It maps the rendering minimum to undefined.** A slab at `RENDERING_DEFAULTS.MINIMUM_SLAB_THICKNESS` means "no slab was requested", not "a 0.05 mm slab was requested". Recording it literally would give `T = 0.1 mm`, which is thinner than any real voxel and would break the guarantee that an annotation always covers at least one layer. Mapping it to undefined lets `T` fall back to one voxel along the normal. #### Using the iterator A tool builds a shape, then walks the voxels: ```ts import { utilities } from '@cornerstonejs/core'; const { createPolylineShape, iterateVoxelsInShape } = utilities.voxelSlab; const shape = createPolylineShape({ volume, // { dimensions, direction, spacing, origin } planePoint, // the annotation plane anchor viewPlaneNormal, // unit length polyline, // the outline in world coordinates }); for (const { ijk, center } of iterateVoxelsInShape({ volume, planePoint, viewPlaneNormal, referencePlaneThickness: shape.getRequiredThickness() || referencePlaneThickness, getShapeRuns: shape.getRuns, })) { // accumulate statistics } ``` A tool with a different outline replaces `createPolylineShape` with `createEllipseShape` or `createRectangleShape`, and changes nothing else. Two details matter for a consumer: - `ijk` and `center` are **reused between iterations**. Copy either one before you retain it. - The shape is intersected with the slab, and not unioned with it. Pass `getRequiredThickness()` as the `referencePlaneThickness` unless you deliberately want the slab to clip the shape. Every shape exposes `containsPoint` as its definition beside `getRuns` as the optimisation. Replace `getShapeRuns: shape.getRuns` with `isInShape: shape.containsPoint` and the voxel set must stay identical, only slower. That replacement is the cheapest way to debug a shape. #### Sampling the values The iterator yields indices and centres, and no values. A tool that measures needs the value of every voxel as well, so `sampleVoxelsInShape` wraps the iterator and reads it: ```ts const { createPolylineShape, sampleVoxelsInShape } = utilities.voxelSlab; const samples = sampleVoxelsInShape({ volume, planePoint, viewPlaneNormal, referencePlaneThickness: shape.getRequiredThickness() || referencePlaneThickness, bounds, // the index box the annotation can reach getShapeRuns: shape.getRuns, voxelManager, onSample: statsCallback, storePointData, }); ``` `onSample` receives `{ value, pointLPS, pointIJK }` for every voxel that has a value, in iteration order, which is what a statistics calculator consumes. A voxel the `voxelManager` holds no value for is skipped. `storePointData` also collects the samples and returns them, at the cost of one object per voxel, so a caller that only accumulates statistics leaves it off and uses `onSample`. The `bounds` must allow for the thickness that `referencePlaneThickness` resolves to. A caller that dilates its bounds for the annotation's own thickness, and then hands a solid shape's larger depth to the iterator, loses every layer past the first. #### The tools share one path Every area annotation tool in `@cornerstonejs/tools` reaches the sampler through `utilities.sampleAreaAnnotationVoxels`. That function takes the annotation and the target image, and derives the plane, the normal, the thickness and the index bounds. Only the shape differs between the tools: ```ts const pointsInShape = utilities.sampleAreaAnnotationVoxels({ annotation, image, // the target's IImageData, and not image.imageData voxelManager, points, // the world points the shape is built from boundsMargin, // how far the shape reaches past those points createShape: ({ volume, planePoint, viewPlaneNormal }) => createCircleShape({ volume, planePoint, viewPlaneNormal, centerWorld, radius, }), onSample: statsCallback, storePointData, }); ``` Two arguments carry the whole of what a tool must get right: - `points` and `boundsMargin` give the index bounds. The outline of a polyline, and the four corners of a rectangle, enclose the shape, so those tools leave `boundsMargin` at 0. A circle and an ellipse only touch their handles, so they pass the largest radius: the outline bulges out of the box of the handles as soon as that box is not aligned with the index axes, and a circle drawn with `simplified` handles keeps one handle only. - `createShape` returns nothing for a degenerate annotation, such as a circle of no radius, and the sampler then measures no voxel. The shape factories throw on one, and an exception inside the render loop stops the whole viewport. The plane comes from `annotation.metadata`, and never from a viewport. Every tool records `viewPlaneNormal` when the user draws the annotation. An annotation that arrives from a DICOM SR records no normal, because an SR stores no camera, and `updatePlaneRestriction` records two in-plane vectors instead; the sampler then crosses those two vectors, which describes the same plane. Two points give one in-plane vector and no plane, so a two point annotation that arrives without a normal reports no statistics. #### The shape contract Every shape implements `VoxelSlabShape`, which has three members. `containsPoint(point)` is the **definition** of the shape. It takes a voxel centre in world coordinates and answers whether the shape contains it. `getRuns(outer, row, depthRun, slab)` is the **optimisation**. It yields inclusive `[min, max]` runs along the slab's column axis for one `(outer, row)` position, and it must select the same voxels that `containsPoint` does. Yielding nothing means the shape does not reach that row. A provider works at one of three levels of precision: | level | contract | example | | -------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------- | | exact | one run that is exactly the covered voxels | a rectangle, or an axis-aligned row of an ellipse | | exact-multiple | several disjoint runs, for a row that enters and leaves the shape more than once | a non-convex freehand polygon | | approximate | a superset run, with `isInShape` supplied so the iterator tests each voxel inside it | any new shape, before it is optimised | `depthRun` is the run the depth test already permits. A provider may clip to it but need not, because the iterator intersects the results either way. `getRequiredThickness()` returns the smallest `T` for which the slab contains the whole shape. A planar shape returns 0, because it has no extent along the normal and any `T` works. A shape with depth returns that depth, and a smaller `referencePlaneThickness` will clip it. `createEllipseShape` and `createRectangleShape` each carry a depth, and each applies its own. `createPolylineShape` is always planar and returns 0, so the caller's `referencePlaneThickness` alone decides how far the slab reaches along the normal. A caller that wants a polyline prism passes the prism depth as that thickness. #### Polyline rings and holes `createPolylineShape` accepts either a single ring or an array of rings. Each ring is closed, so do not repeat the first point at the end. Points are projected onto the annotation plane, which handles an outline that carries a little depth error, as a drawn one always does. `planePoint` is optional for this shape, because every point of the outline lies in the plane already, and the first point of the first ring is the default. Pass the annotation's own anchor when you have one: a drawn vertex carries rounding error that the anchor does not. Whichever anchor you use, give the shape and the iterator the **same** one, or the two describe different slabs. The interior is the even-odd rule over every edge of every ring, and the parity accumulates across the rings rather than per ring. That single rule gives: - **internal holes** — give the hole as its own ring and it is excluded; - **nesting to any depth** — a ring inside a hole is solid again; - **disjoint regions** — separate rings describe separate regions. Winding direction does not matter, so a hole ring need not be wound opposite to its parent. A single ring need be neither convex nor simple, because even-odd resolves a self-intersecting one too. :::warning Do not flatten multiple rings into one array. Flattening inserts an edge from the end of each ring to the start of the next. That does not raise an error; it quietly measures a different shape. ::: #### Why the runs are exact The depth half of Rule M is exactly linear in the integer voxel indices. A voxel at index `p` has its centre at `origin + M p` in world space, where `M` is the index-to-world matrix, so: ``` depth(p) = (centre - P0) . n = p . g + c0 g = Mᵀ n (the index space normal) c0 = (origin - P0) . n ``` `g` and `c0` are constants, so along any single axis the voxels that satisfy `|depth(p)| < halfWidth` form a closed-form interval. The iterator therefore emits exact integer runs instead of a test for each voxel, and this holds at every orientation, oblique included. `g` is deliberately not normalised. Its components are the change in world depth per unit step of each index, which is what the run arithmetic needs. In acquisition orientation `g` comes out parallel to `(0, 0, 1)`, because the normal is the k axis, so `d0 . n` and `d1 . n` vanish and only `s2 * (d2 . n)` survives. #### Axis roles Iteration nests outer → row → column. - `outerAxis` is `argmax |g|`, the axis whose index step moves the depth most. A sweep of that axis outermost makes each outer step cover a thin band of the volume, and it leaves the two axes that lie closest to the annotation plane. - `rowAxis` and `columnAxis` are those two remaining axes. A 2D shape expresses its spans naturally as runs along `columnAxis` for each `rowAxis` value, which is why the shape constraint belongs innermost. For each `(outer, row)` pair the depth constraint gives one interval along `columnAxis`, the shape gives one or more, and the iterator emits their intersection. The depth interval along `columnAxis` is often unbounded. In acquisition orientation `g[columnAxis]` is zero, so the depth does not vary along that axis and the shape is the only binding constraint. Each shape reaches its exact runs by its own route: | shape | route | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------- | | ellipse, ellipsoid | a line substituted into the quadratic form gives a quadratic in the column index, whose real roots bound one interval | | rectangle, box | each face is a linear constraint, so each gives one interval, and their intersection is one interval because a box is convex | | polyline outline | the crossings of the line with every edge of every ring, sorted, with consecutive pairs bounding the inside intervals | A non-convex polyline therefore yields several runs, which is the exact-multiple case the iterator supports. #### Boundary handling **A voxel centre that lies on a shape outline is inside the shape.** Three independent cases made that rule necessary: - A circle of radius 5 on an integer grid puts voxel centres exactly on its outline, at `(5, 0)` and at every Pythagorean point such as `(3, 4)`. `containsPoint` adds the squares and can give a little more than 1, while `getRuns` solves for the roots and gives exactly 5. Both therefore compare against a boundary that a relative epsilon widens. - The even-odd rule gives the interior of a polyline, but `containsPoint` casts a ray along one plane axis while `getRuns` intersects a line along the direction that the column axis projects to. The two tie rules degenerate at different geometry. A rectangular polyline drawn on voxel boundaries kept a row at one end and lost it at the other. - The crossing test cannot see an outline edge that runs along a run line, because both end points lie on the same side of a line that holds them. Such an edge, and any vertex that touches the line, supplies its extent directly, and the code merges every contribution so that no voxel is emitted twice. The depth test moves in the opposite direction. `SLAB_RELATIVE_EPSILON` tightens it rather than widens it, because there the neighbouring layer must be excluded. #### Why the depth test is strict The slab tests use `<` and not `<=`, because the default `T = T_v` places the neighbouring voxel centres exactly on the slab boundary, and an acquisition-orientation annotation must cover exactly one layer. Signed distances come from dot products over world coordinates, so a value that is mathematically on the boundary lands on either side of it. Without a tolerance the most common case in the whole system would pick up two extra layers at random. The tolerance is relative to the voxel thickness, and not absolute, because spacings in medical imaging range from microns to centimetres. The value 1e-5 sits comfortably above float32 error, which is roughly 1e-7 relative, and most inputs carry float32 error because gl-matrix vectors and the rest of the rendering geometry are float32. The strict rule has one visible consequence. A thickness that exceeds an exact voxel multiple by less than `2 * SLAB_RELATIVE_EPSILON * T_v` still selects the smaller number of layers. At `T_v = 1 mm` that dead band is 20 nm wide, so it is unreachable in practice, but `T = T_v + 1e-6` does behave as `T = T_v` rather than pull in both neighbours. #### Voxel thickness along the normal `T_v` is the support width of the voxel box along the normal: ``` T_v = Σᵢ |dᵢ · n| * sᵢ ``` This is an L1 length, and it is deliberately not the L2 length that `getSpacingInNormalDirection` returns. Only the L1 length answers "how far does this voxel reach along the normal", which is what a voxel/slab overlap test needs. | function | formula | answers | | ----------------------------------- | -------------------------------- | ---------------------------------------------------- | | `getSpacingInNormalDirection` | L2, `sqrt(Σ (d·aᵢ·sᵢ)²)` | how far the camera dollies before it sees new voxels | | `getVoxelThicknessAlongNormal` | L1, `Σ \|d·aᵢ\|·sᵢ` | how far one voxel reaches along the direction | | `getEffectiveSpacingAlongDirection` | harmonic, `1/sqrt(Σ (d·aᵢ/sᵢ)²)` | how far to step to cross one voxel | All three agree whenever the normal is parallel to a voxel axis, which covers any acquisition-orientation view, and they diverge for an oblique normal. For 1×1×3 mm voxels viewed at 45 degrees between an in-plane axis and the slice axis, the L1 value is `2*sqrt(2) ≈ 2.83 mm` against `sqrt(5) ≈ 2.24 mm` for L2, and `≈ 1.34 mm` for the harmonic form. Rule M uses `T_v` and nothing else. The harmonic form belongs to a tool that walks a line, such as the sub-pixel resampler of the freehand ROI: it needs a step that crosses one voxel per step, and neither of the other two measures answers that. #### Cost The cost is proportional to the voxels that the iterator emits, plus the rows that it touches. The cost is not proportional to the volume of a bounding box, and not to the canvas area. An ROI at 8× magnification costs what it costs at fit-to-window. Supply `bounds` when the tool already knows the index-space bounding box of the annotation. Bounds only narrow: each axis intersects the volume extent, so a box that reaches outside the volume still yields no index outside it. #### API Everything here is exported under `utilities.voxelSlab`. | export | purpose | | ---------------------------------------------------------------------- | ---------------------------------------- | | `iterateVoxelsInShape`, `collectVoxelsInShape` | the traversal | | `sampleVoxelsInShape` | the traversal, with the values read | | `createEllipseShape`, `createCircleShape` | ellipse in-plane, ellipsoid out-of-plane | | `createRectangleShape` | rectangle in-plane, box out-of-plane | | `createPolylineShape` | a planar polyline, with internal holes | | `getVoxelThicknessAlongNormal` | `T_v` | | `isPlaneDepthViewable` | the depth half of Rule D | | `buildIndexSpaceSlab`, `getDepthRun`, `getSlabAxisBound` | the index-space run arithmetic | | `isVoxelCenterInSlab`, `getMembershipHalfWidth`, `getDisplayHalfWidth` | the Rule M and Rule D predicates | Two more exports sit outside that namespace: | export | purpose | | ----------------------------------------------------------------- | --------------------------------------------- | | `utilities.getEffectiveSpacingAlongDirection`, in core | the step that crosses one voxel along a line | | `utilities.sampleAreaAnnotationVoxels`, in `@cornerstonejs/tools` | the one path every area annotation tool takes | --- ## Cornerstone-tools/segmentation ### Active Segmentation Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/active.md #### Active Segmentation ![](../../../assets/active-segmentation.png) Each viewport can display multiple segmentation representations simultaneously, but only one segmentation can be active per viewport. The active segmentation is the one that will be modified by segmentation tools. You can have different styles for active and inactive segmentations. For instance, you can configure different fill and outline properties for active versus inactive segmentations in each viewport. As shown in the image above, you can display multiple labelmap segmentations in the same viewport. By default, active segmentations have a higher outline width to make them more visually distinct from inactive segmentations. #### Viewport-Specific Active Segmentations An important concept in version 2.x is that active segmentations are viewport-specific. This means: - Each viewport can have its own active segmentation - The same segmentation can be active in one viewport and inactive in another - Segmentation tools will only modify the active segmentation in the viewport they're being used in #### API The Active Segmentation API provides methods to get and set the active segmentation for each viewport: ```js import { segmentation } from '@cornerstonejs/tools'; // Get the active segmentation for a viewport const activeSegmentation = segmentation.getActiveSegmentation(viewportId); // Set the active segmentation for a viewport segmentation.setActiveSegmentation(viewportId, segmentationId); ``` #### Getting Active Segmentation Data Once you have the active segmentation, you can access various properties: ```js const activeSegmentation = segmentation.getActiveSegmentation(viewportId); ``` #### Working with Multiple Viewports Different viewports can have different active segmentations: ```js // Set different active segmentations for different viewports segmentation.setActiveSegmentation('viewport1', 'segmentation1'); segmentation.setActiveSegmentation('viewport2', 'segmentation2'); // Check active segmentations const activeInViewport1 = segmentation.getActiveSegmentation('viewport1'); const activeInViewport2 = segmentation.getActiveSegmentation('viewport2'); ``` Remember that tools will respect these viewport-specific active segmentations when performing operations. --- ### Config Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/config.md #### Configuration In version 2.x, segmentation configurations are managed through a unified style system that can be applied at different levels of specificity using a specifier object. #### Style System Styles can be applied at multiple levels: - Global styles for all segmentations - Type-specific styles (e.g., all Labelmaps) - Viewport-specific styles - Segmentation-specific styles - Segment-specific styles The style configuration object structure depends on the representation type: ```js // Labelmap Style Example { renderFill: true, renderOutline: true, outlineWidth: 3, fillAlpha: 0.7, outlineAlpha: 0.9 } // Contour Style Example { renderFill: true, renderOutline: true, outlineWidth: 2 } // Surface Style Example { renderFill: true, fillAlpha: 0.7 } ``` #### Style API The new style API uses a specifier object to target specific configurations: ```js import { segmentation } from '@cornerstonejs/tools'; // Get style for a specific context const style = segmentation.getStyle({ viewportId: 'viewport1', // optional segmentationId: 'segmentation1', // optional type: Enums.SegmentationRepresentations.Labelmap, // required segmentIndex: 1, // optional }); // Set style for a specific context segmentation.setStyle( { viewportId: 'viewport1', segmentationId: 'segmentation1', type: Enums.SegmentationRepresentations.Labelmap, }, { renderFill: true, renderOutline: true, outlineWidth: 3, } ); // Reset to global style segmentation.resetToGlobalStyle(); // Check if a context has custom style const hasCustomStyle = segmentation.hasCustomStyle({ viewportId: 'viewport1', segmentationId: 'segmentation1', type: Enums.SegmentationRepresentations.Labelmap, }); ``` #### Inactive Segmentations The rendering of inactive segmentations is now controlled per viewport: ```js // Set whether to render inactive segmentations in a viewport segmentation.setRenderInactiveSegmentations('viewport1', true); // Get whether inactive segmentations are rendered in a viewport const renderInactive = segmentation.getRenderInactiveSegmentations('viewport1'); ``` #### Color Management The color API has been updated to be viewport-specific and use more consistent naming: ```js import { segmentation } from '@cornerstonejs/tools'; // Add a new color LUT const colorLUTIndex = segmentation.addColorLUT(colorLUT); // Set color LUT for a segmentation in a viewport segmentation.setColorLUT('viewport1', 'segmentation1', colorLUTIndex); // Get color for a specific segment const color = segmentation.getSegmentIndexColor( 'viewport1', 'segmentation1', segmentIndex ); // Set color for a specific segment segmentation.setSegmentIndexColor( 'viewport1', 'segmentation1', segmentIndex, [255, 0, 0, 255] // RGBA color ); ``` #### Style Hierarchy Styles are applied in the following order of precedence (highest to lowest): 1. Segment-specific style (when segmentIndex is provided) 2. Viewport-specific style (when viewportId is provided) 3. Segmentation-specific style (when segmentationId is provided) 4. Type-specific style (when only type is provided) 5. Global style Example: ```js // Set global style for all labelmaps segmentation.setStyle( { type: Enums.SegmentationRepresentations.Labelmap }, { renderOutline: true } ); // Override style for a specific viewport segmentation.setStyle( { viewportId: 'viewport1', type: Enums.SegmentationRepresentations.Labelmap, }, { renderOutline: false } ); // Set style for a specific segment segmentation.setStyle( { viewportId: 'viewport1', segmentationId: 'segmentation1', type: Enums.SegmentationRepresentations.Labelmap, segmentIndex: 1, }, { outlineWidth: 5 } ); ``` :::note Tip For detailed information about available style options for each representation type, refer to the API documentation. ::: --- ### Contour Representation Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/contour.md #### Contour Segmentation Representation A Contour Segmentation Representation is a collection of Contour Sets. Each Contour Set is a collection of Contours. Each Contour is a collection of Points. Each Point is a collection of 3D coordinates. ![](../../../assets/contourSet.png) #### Contour Set Since usually a segmentation is a collection of multiple structures, each Contour Set represents a single structure. For example, a segmentation can have multiple Contour Sets, each representing a different structure. Each Contour Set has a unique ID, and a name. The name is used to display the structure name in the UI. #### Contour A Contour includes the information about the points that make up the contour. Each Contour has data, type (closed or open), and a color. #### Loading Contour as Segmentation Representation ```js // load each contour set and cache the geometry const promises = contourSets.map((contourSet) => { return geometryLoader.createAndCacheGeometry(contourSet.id, { type: GeometryType.CONTOUR, geometryData: contourSet as Types.PublicContourSetData, }); }); await Promise.all(promises); // Add the segmentations to state segmentation.addSegmentations([ { segmentationId, representation: { // The type of segmentation type: csToolsEnums.SegmentationRepresentations.Contour, // The actual segmentation data, in the case of contour geometry // this is a reference to the geometry data data: { geometryIds: contourSets.map((contourSet) => contourSet.id), }, }, }, ]); // Add contour representation to a specific viewport await segmentation.addContourRepresentationToViewport(viewportId, [ { segmentationId, type: Enums.SegmentationRepresentations.Contour, }, ]); ``` --- ### Custom Cursor Geometry & Fill Strategies Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/cursor-strategies.md #### Custom Cursor Geometry & Fill Strategies Segmentation tools expose `BrushStrategy` hooks that let you draw any cursor footprint and reuse the exact same geometry while painting. The cursor is not just a visual hint—`BrushTool` copies the geometry calculated during hover into the `operationData` that powers the fill strategy. This section explains how to customize both halves so a new cursor footprint (e.g. a square, diamond, or oblique polygon) fills exactly the pixels that the user expects. #### Lifecycle Overview 1. `BrushTool` builds `hoverData` (see `LabelmapBaseTool.createHoverData`) whenever the pointer moves. 2. The active `BrushStrategy` is asked to run `StrategyCallbacks.CalculateCursorGeometry`. This callback can populate `hoverData.brushCursor.data.handles` with world-space points that describe the cursor. 3. Immediately afterwards `StrategyCallbacks.RenderCursor` receives the same `operationData` as well as an `SVGDrawingHelper`. Use this callback to render the cursor in canvas space. 4. When the user paints, `LabelmapBaseTool.getOperationData` copies `brushCursor.data.handles.points` into `operationData.points` and forwards the original `hoverData` to the active strategy. 5. The strategy's `StrategyCallbacks.Initialize` implementation maps those points into index space, computes `operationData.isInObject`, and optionally updates `operationData.strokePointsWorld` so the fill and cursor stay aligned even while dragging. Keeping the cursor computation and the fill predicate in sync ensures that the sweep volume painted into the labelmap matches what was rendered on screen. #### Step 1: Calculate World-Space Geometry Implement a composition that handles `StrategyCallbacks.CalculateCursorGeometry`. The callback receives the enabled element, the tool configuration, and the latest `hoverData`: ```ts import { Enums } from '@cornerstonejs/tools'; import type { Types } from '@cornerstonejs/core'; const { StrategyCallbacks } = Enums; export const hexCursorComposition = { [StrategyCallbacks.CalculateCursorGeometry]: (enabledElement, operationData) => { const { viewport } = enabledElement; const { configuration, hoverData } = operationData; const { brushCursor, centerCanvas } = hoverData; const camera = viewport.getCamera(); const brushRadius = configuration.brushSize; const centerWorld = viewport.canvasToWorld(centerCanvas) as Types.Point3; const polygonWorld = createHexagonCorners( centerWorld, camera.viewUp, camera.viewPlaneNormal, brushRadius ); // BrushTool automatically copies handles.points into operationData.points. brushCursor.data.handles = { points: buildOrthogonalHandles(polygonWorld), polygonWorld, }; brushCursor.data.invalidated = false; }, }; ``` Guidelines: - Normalize `viewUp`/`viewPlaneNormal` before deriving `viewRight` so oblique planes behave consistently. - Always populate `handles.points` in the `[bottom, top, left, right]` order. Existing strategies (e.g. `fillCircle.ts`) expect that ordering when computing centers and radii. - Attach any extra data you need (such as `polygonWorld` or precomputed normals) on `brushCursor.data.handles`. It will be available through `operationData.hoverData` inside your fill strategy. #### Step 2: Render the Custom Cursor The render callback is responsible for drawing canvas-space overlays based on the world-space geometry calculated earlier. Leverage the shared SVG helpers in `packages/tools/src/drawingSvg` to keep the output consistent with the rest of the tooling: ```ts import { Enums, drawing } from '@cornerstonejs/tools'; const { StrategyCallbacks } = Enums; const { drawPolyline: drawPolylineSvg } = drawing; hexCursorComposition[StrategyCallbacks.RenderCursor] = ( enabledElement, operationData, svgDrawingHelper ) => { const { viewport } = enabledElement; const { brushCursor } = operationData.hoverData; const polygonWorld = brushCursor.data.handles?.polygonWorld ?? []; if (polygonWorld.length === 0) { return; } const polygonCanvas = polygonWorld.map((point) => viewport.worldToCanvas(point) ); const annotationUID = brushCursor.metadata?.brushCursorUID; drawPolylineSvg(svgDrawingHelper, annotationUID, 'hexagon', polygonCanvas, { color: `rgb(${brushCursor.metadata.segmentColor?.slice(0, 3) ?? [0, 255, 0]})`, lineDash: operationData.centerSegmentIndexInfo.segmentIndex === 0 ? [1, 2] : undefined, closed: true, }); }; ``` Keep the render step lightweight—`BrushTool` triggers it on every mouse move. Avoid re-computing world-space data here; cache everything during `CalculateCursorGeometry`. #### Step 3: Build a Matching Fill Strategy A fill strategy is a `BrushStrategy` instance that wires together reusable compositions. The class lives in `packages/tools/src/tools/segmentation/strategies/BrushStrategy.ts` (or `@cornerstonejs/tools/dist/tools/segmentation/strategies/BrushStrategy` when consuming the npm package). The `StrategyCallbacks.Initialize` portion is where you convert the cursor geometry into the predicate used by `compositions.regionFill`: ```ts import BrushStrategy from '@cornerstonejs/tools/dist/tools/segmentation/strategies/BrushStrategy'; import { Enums, utilities } from '@cornerstonejs/tools'; import { utilities as csUtils } from '@cornerstonejs/core'; const { StrategyCallbacks } = Enums; const { getBoundingBoxAroundShapeIJK } = utilities.boundingBox; const { transformWorldToIndex } = csUtils; const { regionFill, setValue, determineSegmentIndex, preview, labelmapStatistics, } = BrushStrategy.COMPOSITIONS; const initializeHexagon = { [StrategyCallbacks.Initialize]: (operationData) => { const { segmentationImageData, hoverData } = operationData; const worldPolygon = hoverData?.brushCursor?.data?.handles?.polygonWorld; if (!Array.isArray(worldPolygon) || worldPolygon.length === 0) { return; } const polygonIJK = worldPolygon.map((worldPoint) => transformWorldToIndex(segmentationImageData, worldPoint) ); operationData.isInObject = createPointInPolygon(worldPolygon, segmentationImageData); operationData.isInObjectBoundsIJK = getBoundingBoxAroundShapeIJK( polygonIJK, segmentationImageData.getDimensions() ); // Preserve stroke continuity for drag operations. operationData.strokePointsWorld = [ ...(operationData.strokePointsWorld ?? []), ...worldPolygon, ]; }, }; export const HEXAGON_STRATEGY = new BrushStrategy( 'Hexagon', regionFill, setValue, initializeHexagon, determineSegmentIndex, preview, labelmapStatistics, hexCursorComposition ); export const fillInsideHexagon = HEXAGON_STRATEGY.strategyFunction; ``` `createPointInPolygon` above represents whichever predicate you implement to classify voxels. Many strategies cache both the polygon plane and its normal so the predicate can avoid redundant transforms. Important details: - `operationData.isInObject` must be an efficient point-in-shape predicate because it runs on every candidate voxel. - Always update `operationData.isInObjectBoundsIJK`; `regionFill` short-circuits iteration using this bounding box. - Reuse `operationData.strokePointsWorld` to describe the swept volume of a drag. Strategies such as `fillCircle.ts` densify the stroke to avoid holes when the cursor moves faster than the event rate. - Compose existing helpers such as `getBoundingBoxAroundShapeIJK`, `pointInSphere`, or custom polygon math to keep the predicates deterministic. #### Wiring the Strategy into BrushTool Register the new strategy function with the `BrushTool` configuration inside your `ToolGroup`: ```ts import { addTool, BrushTool, ToolGroupManager, Enums } from '@cornerstonejs/tools'; import { fillInsideHexagon } from './strategies/fillHexagon'; addTool(BrushTool); const toolGroup = ToolGroupManager.createToolGroup('segmentationGroup'); toolGroup.addTool(BrushTool.toolName); const brushConfig = toolGroup.getToolConfiguration(BrushTool.toolName) ?? {}; toolGroup.setToolConfiguration( BrushTool.toolName, { ...brushConfig, strategies: { ...(brushConfig.strategies ?? {}), FILL_INSIDE_HEXAGON: fillInsideHexagon, }, defaultStrategy: 'FILL_INSIDE_HEXAGON', activeStrategy: 'FILL_INSIDE_HEXAGON', }, true ); toolGroup.setToolActive(BrushTool.toolName, { bindings: [{ mouseButton: Enums.MouseBindings.Primary }], strategy: 'FILL_INSIDE_HEXAGON', }); ``` Use `setToolConfiguration` if you need to swap strategies at runtime: ```ts toolGroup.setToolConfiguration(BrushTool.toolName, { activeStrategy: 'FILL_INSIDE_HEXAGON', }); ``` `BrushStrategy` automatically falls back to the default circular cursor when a composition does not implement the cursor callbacks, so you can selectively apply the custom cursor only to the strategies that require it. #### Matching Cursor and Fill Logic: Best Practices - **Share world-space data**: Write every geometry primitive you need into `brushCursor.data.handles`. The fill strategy can read it back from `operationData.hoverData` without recomputing. - **Stay idempotent**: Callbacks may run multiple times per frame. Avoid mutating shared instances; clone vectors with `vec3.clone` before caching. - **Obey coordinate systems**: `CalculateCursorGeometry` works in world coordinates, `RenderCursor` works in canvas coordinates, and `Initialize` must convert to IJK using `transformWorldToIndex`. - **Guard performance**: Keep predicates branch-light and memoize expensive transforms. `BrushStrategy` executes inside tight voxel loops. - **Test point-in-shape functions**: Jest unit tests similar to `packages/tools/src/tools/segmentation/strategies/__tests__/fillCircle.spec.ts` help catch regressions. - **Handle fast drags**: Populate `operationData.strokePointsWorld` (and densify long segments) so the predicate covers every point swept by the cursor. By following these steps you can confidently deliver new cursor footprints together with matching fill behavior, ensuring artists see exactly what will be written into their segmentations. --- ### Segmentations Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/index.md import DocCardList from '@theme/DocCardList'; import {useCurrentSidebarCategory} from '@docusaurus/theme-common'; #### Segmentations In `Cornerstone3DTools`, we have decoupled the concept of a `Segmentation` from a `Segmentation Representation`. This means that from one `Segmentation` we can create multiple `Segmentation Representation`s. For instance, a `Segmentation Representation` of a 3D Labelmap, can be created from a `Segmentation` data, and a `Segmentation Representation` of a Contour (not supported yet) can be created from the same `Segmentation` data. This way we have decouple the presentational aspect of a `Segmentation` from the underlying data. ![](../../../assets/segmentation-representation.png) :::note TIP Similar relationship structure has been adapted in popular medical imaging softwares such as [3D Slicer](https://www.slicer.org/) with the addition of [polymorph segmentation](https://github.com/PerkLab/PolySeg). ::: #### API `Segmentation` related functions and classes are available in the `segmentation` module. ```js import { segmentation } from '@cornerstonejs/tools'; // segmentation state holding all segmentations and their toolGroup specific representations segmentation.state.XYZ; // active segmentation methods (set/get) segmentation.activeSegmentation.XYZ; // locking for a segment index (set/get) segmentation.locking.XYZ; // segment index manipulations (set/get) segmentations.segmentIndex.XYZ; ``` Let's start by looking deeper into each of these methods. --- ### Segment Locking Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/locking.md #### Segment Locking ![](../../../assets/segment-locking.png) You can lock specific segments in a segmentation to prevent them from being modified by any tools. For example, consider the following image with an overlaid labelmap: - Left image: shows `segment index 1` - Middle image: shows the result when `segment index 2` is drawn on top of `segment index 1` - Right image: shows the result when `segment index 1` is locked and `segment index 2` is drawn on top of `segment index 1` As shown in the locked scenario (right image), when segment index 1 is locked, it cannot be modified by new drawings. ![segment-locking-example] #### API The locking API has been updated in version 2.x to provide clearer method names and functionality: ```js import { segmentation } from '@cornerstonejs/tools'; // Lock/unlock a segment index in a segmentation segmentation.locking.setSegmentIndexLocked( segmentationId, segmentIndex, locked ); // Get all locked segment indices for a segmentation const lockedIndices = segmentation.locking.getLockedSegmentIndices(segmentationId); // Check if a segment index is locked const isLocked = segmentation.locking.isSegmentIndexLocked( segmentationId, segmentIndex ); ``` #### Example Usage ```js // Lock segment 1 in a segmentation segmentation.locking.setSegmentIndexLocked('segmentation1', 1, true); // Check if segment 1 is locked const isLocked = segmentation.locking.isSegmentIndexLocked('segmentation1', 1); console.log(`Segment 1 is ${isLocked ? 'locked' : 'unlocked'}`); // Get all locked segments const lockedIndices = segmentation.locking.getLockedSegmentIndices('segmentation1'); console.log('Locked segment indices:', lockedIndices); // Unlock segment 1 segmentation.locking.setSegmentIndexLocked('segmentation1', 1, false); ``` #### Key Changes in Version 2.x 1. Renamed `getLockedSegments` to `getLockedSegmentIndices` for clarity 2. The locked state is now stored in the segment data structure: ```js { segments: { [segmentIndex]: { locked: boolean, // other segment properties... } } } ``` Note that the locking state applies to the segmentation as a whole, not to specific representations or viewports. If a segment is locked, it will be locked across all viewports and representations. --- ### Saving and Replacing Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/saving.md #### Saving and Replacing A segmentation is stored as a derived DICOM object: a SEG instance built from the images it was drawn on. A structure set (RTSTRUCT) and a measurement report (SR) are derived objects in the same way, and everything on this page applies to all three. There are two ways to store one, and the difference is a single option. #### A new series With no predecessor, the adapter builds the object on a dcmjs derivation, and that derivation invents the series it belongs to: a fresh `SeriesInstanceUID`, the description `Research Derived series`, the number `99`, and the current UTC date and time. The stored object is the first instance of a series of its own. This is what you want for a segmentation the user has just created. #### A revision of an existing series Pass `predecessorImageId` — the image id of the instance the new object supersedes — and the object joins that instance's series instead: ```js const { dataset } = generateSegmentation( referencedImages, labelmaps3D, metaData, { predecessorImageId } ); ``` `generateRTSSFromRepresentation` and `MeasurementReport.generateReport` take the same option. All three read it through the `PredecessorSequence` metadata module, which supplies three things: | What | Taken from | Effect | | ------------------------------ | -------------------------------------------- | --------------------------------------------------------------------------- | | The series attributes | the predecessor's General Series module | The revision lands in the same series, with the same number and description | | `InstanceNumber` | the predecessor's, plus one | The revision sorts after the instance it supersedes | | `PredecessorDocumentsSequence` | the predecessor's study, series and SOP UIDs | The revision names what it supersedes | Only the attributes the predecessor actually carries are copied, so a gap in the predecessor never clears a value the derivation has already set. #### Which instance a viewer shows Nothing in the stored object marks one instance as "the" segmentation of a series. What the revision carries is enough for a viewer to decide: - it is in the **same series** as the predecessor, so a viewer that lists one entry per series has a single entry for the whole chain; - it has a **higher `InstanceNumber`**; - it names its predecessor in **`PredecessorDocumentsSequence`**, so the order of a chain of revisions is recoverable however the instances are numbered. A viewer that shows the most recently created instance of the series therefore shows the revision, and the original stays retrievable behind it. The application that stores the object is responsible for stamping the instance level creation date and time on every save; the series level `SeriesDate` and `SeriesTime` belong to the series that already exists and must not be restamped. #### Recording the predecessor `Segmentation.predecessorImageId` holds the instance a segmentation was loaded from, or was last stored as, and `Annotation.predecessorImageId` does the same for an annotation. Read it when the user saves, and a second save writes a second revision rather than a second series. #### When the predecessor cannot be used The module answers `undefined` if no metadata provider holds the image id — a stale id, or an instance that was never ingested. Every caller merges that answer as a no-op, so the save still succeeds, but the object starts a new series and names no predecessor. The module logs a warning naming the image id, and that warning is the only report the caller gets. The module throws if a provider holds the image id but carries no `StudyInstanceUID`, or no `SeriesInstanceUID`. Both attributes are Type 1 in `PredecessorDocumentsSequence`. The module cannot write the sequence without them, and a stored object that names an empty predecessor is worse for the user than a save that fails and says why. --- ### Segment Index Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/segmentIndex.md #### Segment Index When drawing with segmentation tools, you can specify which segment index to use. Below, we have used the SegmentIndex API to change the `segmentIndex` to draw the second segment.
![](../../../assets/segment-index.png)
#### API ```js import { segmentation } from '@cornerstonejs/tools'; // get active segment index for the segmentation Id segmentation.segmentIndex.getActiveSegmentIndex(segmentationId); // set active segment index for the segmentation Id segmentation.segmentIndex.setActiveSegmentIndex(segmentationId, segmentIndex); ``` --- ### Segmentation Tools Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/segmentation-tools.md #### Segmentation Tools `Cornerstone3DTools` provides a set of tools to modify segmentations. These include the `BrushTool`, Scissors (such as `RectangleScissor`, `CircleScissor`, `SphereScissor`), and `RectangleRoiThresholdTool`. We will cover each tool in more detail below. :::note Tip All Segmentation tools can edit the segmentation in all 3D views (axial, coronal, and sagittal). ::: #### Brush Tool `BrushTool` is the most commonly used tool for segmentation. It allows you to draw segmentations by clicking and dragging (as seen below). To use this tool, you need to add it to your toolGroup like any other tools. Read more on how to activate a tool in [Tools](../tools.md#adding-tools) and [ToolGroup](../toolGroups.md#toolgroup-creation-and-tool-addition) sections. ![](../../../assets/brush-tool.gif) #### Rectangle Scissor Tool `RectangleScissorTool` can be used to create a rectangular segmentation. ![](../../../assets/rectangle-scissor.gif) #### Circle Scissor Tool `CircleScissorTool` can be used to create a circular segmentation. ![](../../../assets/circle-scissor.gif) #### Sphere Scissor Tool `SphereScissorTool` can be used to create a spherical segmentation. It draws a 3D sphere around the mouse pointer. ![](../../../assets/sphere-scissor.gif) #### Threshold Tool `RectangleROIThresholdTool` can be used to create a segmentation by thresholding the drawn area by the user. (in images below, a certain threshold is set to create a segmentation) ![](../../../assets/threshold-segmentation-tool.gif) --- ### State Source: https://cornerstonejs.org/docs/llm/concepts/cornerstone-tools/segmentation/state.md #### State `SegmentationState` stores all the information regarding the current state of `Segmentation`s and `SegmentationRepresentation`s in the library. In version 2.x, we've decoupled `Segmentation`s from their representations and made the system viewport-specific rather than toolGroup-specific. From a `Segmentation`, various representations can be created (currently supporting Labelmap, Contour, and Surface). #### ColorLUT `SegmentationState` stores an array of `colorLUT`s used to render segmentation representations. `Cornerstone3DTools` initially adds 255 colors (`[[0,0,0,0], [221, 84, 84, 255], [77, 228, 121, 255], ...]`) as the first index of this array. By default, all segmentation representations use the first colorLUT. However, using the color API in the config, you can add more colors to the global colorLUT and/or change the colorLUT for specific segmentation representations in specific viewports. #### Segmentations `SegmentationState` stores all segmentations in an array. Each Segmentation Object stores the required information for creating `SegmentationRepresentation`s. Each segmentation object has the following properties: ```js { segmentationId: 'segmentation1', label: 'segmentation1', segments: { 0: { segmentIndex: 0, label: 'Segment 1', active: true, locked: false, cachedStats: {} }, 1: { segmentIndex: 1, label: 'Segment 2', active: false, locked: false, cachedStats: {} } }, representationData: { Labelmap: { volumeId: 'segmentation1' }, Contour: { geometryIds: ['contourSet1', 'contourSet2'] }, Surface: { geometryId: 'surface1' } } } ``` - `segmentationId`: A required field provided by the consumer. This is the unique identifier for the segmentation. - `label`: The label of the segmentation. - `segments`: An object containing information about each segment, including its label, active state, locked state, and cached statistics. - `representationData`: **THE MOST IMPORTANT PART**, this is where the data for creation of each type of `SegmentationRepresentation` is stored. For instance, in `Labelmap` representation, the required information is a cached `volumeId`. #### Adding Segmentations to the State Since `Segmentation` and `SegmentationRepresentation` are separated, first we need to add the `segmentation` to the state using the top-level API: ```js import { segmentation, Enums } from '@cornerstonejs/tools'; segmentation.addSegmentations([ { segmentationId, representation: { type: Enums.SegmentationRepresentations.Labelmap, data: { imageIds: segmentationImageIds, }, }, }, ]); ``` :::note Important Adding a `Segmentation` to the state WILL NOT render the segmentation. You need to add `SegmentationRepresentation`s to specific viewports where you want to render them. ::: #### Viewports #### Adding a SegmentationRepresentation to a Viewport To render a segmentation, you need to add its representation to specific viewports. This can be done using the `addSegmentationRepresentation` method: ```js import { segmentation, Enums } from '@cornerstonejs/tools'; await segmentation.addSegmentationRepresentations(viewportId, [ { segmentationId, type: Enums.SegmentationRepresentations.Labelmap, }, ]); ``` #### Representation-Specific Methods Cornerstone3D v2 provides dedicated methods for adding different types of segmentation representations: ```js // Add labelmap representations await segmentation.addLabelmapRepresentationToViewport(viewportId, [ { segmentationId, config: {} } ]); // Add contour representations await segmentation.addContourRepresentationToViewport(viewportId, [ { segmentationId, config: {} } ]); // Add surface representations await segmentation.addSurfaceRepresentationToViewport(viewportId, [ { segmentationId, config: {} ]); ``` #### Multiple Viewport Operations You can also add representations to multiple viewports simultaneously using the viewport map methods: ```js const viewportInputMap = { viewport1: [ { segmentationId: 'seg1', type: Enums.SegmentationRepresentations.Labelmap, }, ], viewport2: [ { segmentationId: 'seg1', type: Enums.SegmentationRepresentations.Labelmap, }, ], }; await segmentation.addLabelmapRepresentationToViewportMap(viewportInputMap); ``` --- ## Progressive-loading ### Advance Options Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/advance-retrieve-config.md There are more advanced options both for `retrieve stages` and also for `retrieve options` that can be used to customize the behavior of the progressive loading. :::tip You can skip this section if you are not interested in the advanced options (yet) and still move to the [`usage` section](./usage). Basically, some of these options (position, decimate, offset, priority, and nearbyFrames) are used in the "volume progressive" example, which you can revisit later. ::: #### Advanced Retrieve Stages Options #### positions?: number[]; Used for volume-progressive loading, where we need to specify the exact image index we want to retrieve. This is generally true in general hanging protocols, as the initial image is usually in the middle, top, or bottom of the stack. You can use absolution positions, or relative positions between [0, 1]. Positions less than 0 are relative to the end, so you can use -1 to indicate the last image in the stack. Example ```js stages: [ { id: 'initialImages', positions: [0.5, 0, -1], retrieveType: 'initial', // arbitrary naming as discussed }, ]; ``` in the above example, we are requesting the middle image, the first image, and the last image in the stack. :::tip To retrieve another initial image automatically based on initial display positions, copy the stages, and add a new stage with your desired position, putting that stage first. This can be used to ensure the initial image is fetched. ::: #### decimate?: number & offset?: number; By utilizing the decimate and offset features, we can enhance the flexibility of specifying the desired images for retrieval. For example, if a volume comprises 100 images, applying a decimate value of 2 and an offset of 0 will retrieve images 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, and so on. Similarly, employing a decimate value of 2 and an offset of 1 will retrieve images 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, and so forth. This demonstrates how we can effectively interleave the images by leveraging different offsets and decimate values. It is safe to repeat image fetches, as the fetches will be discarded when the image quality status is already better than that of the specified fetch. ```js stages: [ { id: 'initialImages', positions: [0.5, 0, -1], retrieveType: 'initial', // arbitrary naming as discussed }, { id: 'initialPass', decimate: 2, offset: 0, retrieveType: 'fast', // arbitrary naming as discussed }, { id: 'secondPass', decimate: 2, offset: 1, retrieveType: 'fast', // arbitrary naming as discussed }, ]; ``` Above we have three stages where we first retrieve the initial images, then we retrieve the rest of the images in two passes. The first pass will retrieve images 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, and so on. The second pass will retrieve images 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, and so forth. #### priority?: number & requestType Using combination of requestType (thumbnail, prefetch, interaction) and priority (the lower the higher) you can effectively prioritize the requests. For example, you can set the priority of the initial images to be higher (lower number) than the rest of the images. This will ensure that the initial images are retrieved first in the queue. ```js stages: [ { id: 'initialImages', positions: [0.5, 0, -1], retrieveType: 'initial', requestType: RequestType.INTERACTION, priority: -1, }, { id: 'initialPass', decimate: 2, offset: 0, retrieveType: 'fast', priority: 2, requestType: RequestType.PREFETCH, }, { id: 'secondPass', decimate: 2, offset: 1, retrieveType: 'fast', priority: 3, requestType: RequestType.PREFETCH, }, ]; ``` :::tip Set the maximum number of requests to run to a lower value to ensure that your required requests are performed first. For example: ```javascript imageLoadPoolManager.setMaxSimultaneousRequests(RequestType.INTERACTION, 6); ``` ::: #### nearbyFrames?: NearbyFrames[]; Using nearby frames, you have the option to fill in the nearby frames to instantaneously fill and render the empty spaces in the volume. Example ```js stages: [ { id: 'initialPass', decimate: 2, offset: 0, retrieveType: 'fast', priority: 2, requestType: RequestType.PREFETCH, nearbyFrames: [ { offset: +1, imageQualityStatus: ImageQualityStatus.ADJACENT_REPLICATE, }, ], }, { id: 'secondPass', decimate: 2, offset: 1, retrieveType: 'fast', priority: 3, requestType: RequestType.PREFETCH, }, ]; ``` In the above, we are specifying that we would like to replicate the adjacent frames to the current frame (+1). This way, until the next stage (secondPass) arrives, we will have the adjacent frames ready to be rendered and displayed. The secondPass will overwrite them with actual data. #### Advanced Retrieve Options #### urlArguments - urlArguments - is a set of arguments to add to the URL - This distinguishes this request from other requests which cannot be combined with this one - The DICOMweb standard allows for the `accept` parameter to specify a content type - The HTJ2K content type is `image/jhc` The configuration for this is (assuming standards based DICOMweb support): ```js retrieveOptions: { default: { urlArguments: 'accept=image/jhc', rangeIndex: -1, }, multipleFast: { urlArguments: 'accept=image/jhc', rangeIndex: 0, decodeLevel: 0, }, }, ``` :::warning You MUST repeat the same framesPath and urlArguments for each stage in a range request, otherwise the assumption is that the data retrieved in the first range is NOT the same data retrieved in the second range, and the second range request will just retrieve the entire request. ::: #### framePath - framesPath - to update the URL path portion This is useful for fetching another available path such as the thumbnail, JPIP or rendered endpoints for lossy encoded retrieves as they are located on different paths than the lossless encoded images. This is also useful for integration with fixed path alternate encoding servers which choose the response to return based on the URL path, storing various lossy renderings on alternate paths. #### imageQualityStatus - imageQualityStatus - used to set the retrieve status to lossy or sub-resolution This is typically used when the URL or retrieve parameters specify a lossy final rendering of the given path such as for a lossy encoded HTJ2K image. #### Separate URL For Sub-Resolution Images An alternative to a byte range request is to make an different request for a complete, but lossy/low resolution image. This can be standards based assuming the DICOMweb supports `JPIP`, or more likely is non-standards based using a separate path for the low resolution fetch. For the `JPIP` approach shown here, the `JPIP` server must expose an endpoint identical in path to the normal pixel data endpoint, except ending in `/jpip?target=`, and supporting the `fsiz` parameter. See [Part 5](https://dicom.nema.org/medical/dicom/current/output/html/part05.html#sect_8.4.1) and [Part 18](https://dicom.nema.org/medical/dicom/current/output/html/part18.html#sect_8.3.3.1) of the DICOM standard. For the non-standard path approach, the assumption is that there are other endpoints related to the normal `/frames` endpoint, except that the `/frames/` part of the URL is replaced by another value. For example, this could be used to fetch a `/jlsThumbnail/` data as used in the `stackProgressive` example. An example configuration for `JPIP`: ```js retrieveOptions: { default: { // Need to note this is a lossy encoding, as it isn't possible to // detect based on the general configuration here. imageQualityStatus: ImageQualityStatus.SUBRESOLUTION, // Hypothetical JPIP server using a path that is the normal DICOMweb // path but with /jpip?target= replacing the /frames path // This uses the standards based target JPIP parameter, and assigns // the frame number as the value here. framesPath: '/jpip?target=', // Standards based fsiz parameter retrieves a sub-resolution image urlArguments: 'fsiz=128,128', }, }, ``` --- ### Encoding Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/encoding.md #### Types of Partial Resolution There are a few types of partial resolution image: - `lossy` images are original resolution/bit depth, but lossy encoded - `thumbnail` images are reduced resolution images - `byte range` images are a prefix of the full resolution, followed by retrieving the remaining data. This only works for images like HTJ2K encoded in resolution first ordering. #### Creating Partial Resolution Images [Static DICOMweb](https://github.com/RadicalImaging/Static-DICOMWeb) repository has been enhanced to add the ability to create partial resolution images, as well as to serve up byte range requests. Some example commands for a Ct dataset are below: ```bash #### Create HTJ2K as default and write HTJ2K lossy to .../lossy/ mkdicomweb create -t jhc --recompress true --alternate jhc --alternate-name lossy d:\src\viewer-testdata\dcm\Juno #### Create JLS and JLS thumbnail versions mkdicomweb create -t jhc --recompress true --alternate jls --alternate-name jls /src/viewer-testdata/dcm/Juno mkdicomweb create -t jhc --recompress true --alternate jls --alternate-name jlsThumbnail --alternate-thumbnail /src/viewer-testdata/dcm/Juno #### Create HTJ2K lossless and thumbnail versions (this is not required in general #### when the top item is already lossless) mkdicomweb create -t jhc --recompress true --alternate jhcLossless --alternate-name htj2k /src/viewer-testdata/dcm/Juno mkdicomweb create -t jhc --recompress true --alternate jhc --alternate-name htj2kThumbnail --alternate-thumbnail /src/viewer-testdata/dcm/Juno ``` Any other tools creating multipart/related encapsulated data can be used, as can using accept headers or parameters for a standard DICOMweb server. Note the data path for these is, in general the normal DICOMweb path with `/frames/` replaced by some other name. --- ### Progressive Loading Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/index.md import DocCardList from '@theme/DocCardList'; import {useCurrentSidebarCategory} from '@docusaurus/theme-common'; #### Progressive Loading We have added a new progressive loader for both stack and volume images. For stack images, the progressive loader can load a smaller or lossy image, while for volumes, both smaller/lossy images and fully interleaved versions can be loaded to speed up the loading process. --- ### Progressive Loading for non-HTJ2K Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/non-htj2k-progressive.md #### Progressive Loading for non-HTJ2K Progressive Encoded Data #### JLS Thumbnails JLS thumbnails can be created using the static-dicomweb toolkit, for example, by doing: ``` #### Create a JLS directory containing JLS encoded data in the /jls sub-path mkdicomweb create -t jhc --recompress true --alternate jlsLossless --alternate-name jls "/dicom/DE Images for Rad" #### Create a jlsThumbnail sub-directory containing reduce resolution data mkdicomweb create -t jhc --recompress true --alternate jls --alternate-name jlsThumbnail --alternate-thumbnail "/dicom/DE Images for Rad" ``` This can then be used by configuring: ```javascript cornerstoneDicomImageLoader.configure({ retrieveOptions: { default: { default: { framesPath: '/jls/', }, }, singleFast: { default: { imageQualityStatus: ImageQualityStatus.SUBRESOLUTION, framesPath: '/jlsThumbnail/', ``` #### Sequential Retrieve Configuration The sequential retrieve configuration has two stages specified, each of which applies to the entire stack of image ids. The first stage will load every image using the `singleFast` retrieve type, followed by the second stage retrieving using `singleFinal`. If the first stage results in lossless images, the second stage never gets run, and thus the behaviour is identical to previous behaviour for stack images. This configuration can also be used for volumes, producing the old/previous behaviour for streaming volume loading. The configuration is: ```javascript stages: [ { id: 'lossySequential', retrieveType: 'singleFast', }, { id: 'finalSequential', retrieveType: 'singleFinal', }, ], ``` Images for the stack viewport can be loaded with a lower resolution/lossy version first, followed by increasingly higher resolutions, and finally the final version being a lossless representation. For HTJ2K, this is done automatically when the image is encoded in progressive resolution order by using a streaming reader that returns lower resolution versions of the image as they are available. For other image types, a separate lower resolution/lossy version is required. The Static DICOMweb toolkit includes some options to create such images. #### Performance In general, about 1/16-1/10th of the image is retrieved for the lossy/first version of the image. This results in a significant speed improvement to first images. It is affected fairly strongly by overall image size, network performance and compression ratios. The full size images are 3036 x 3036, while the JLS reduced images are 759 x 759 | Type | Network | Size | First Render | Final Render | | ---------------- | ------- | ------ | ------------ | ------------ | | JLS | 4g | 10.6 M | | 4586 ms | | JLS Reduced | 4g | 766 K | 359 ms | 4903 ms | | HTJ2K | 4g | 11.1 M | 66 ms | 5053 ms | | HTJ2K Byte Range | 4g | 128 K | 45 ms | 4610 ms | - JLS Reduced uses 1/16 size JLS 'thumbnails' - HTJ2K uses streaming data - HTJ2K Byte Range uses 64k initial retrieve, followed by remaining data #### Interleave performance that none of the times include time to load the decoder, which can be a second or more, but is only seen on first render. These times are similar for both types. | Type | Size | Network | First Render | Complete | | ---------------- | ----- | ------- | ------------ | -------- | | JLS | 30 M | 4g | 2265 ms | 8106 ms | | JLS Reduced | 3.6 M | 4g | 1028 ms | 8455 ms | | HTJ2K | 33 M | 4g | 2503 ms | 8817 ms | | HTJ2K Byte Range | 11.1M | 4g | 1002 ms | 8813 ms | | JLS | 30 M | local | 1322 ms | 1487 ms | | JLS Reduced | 3.6 M | local | 1084 ms | 1679 ms | | HTJ2K | 33 M | local | 1253 ms | 1736 ms | | HTJ2K Byte Range | 11.1M | local | 1359 ms | 1964 ms | The HTJ2K byte range is very slightly slower than straight JLS, but can be done against any DICOMweb server supporting HTJ2K and byte range requests. - 4g speed - 30 mbit/s down, 5 mbit/s up, 10 ms latency - Complete time for the JLS and HTJ2K was essentially identical to baseline non-progressive - Full size images are 512x512 - Reduce resolution images are 128x128 and lossy compressed #### Configuration See the stackProgressive example for stack details. Stack viewports need to be configured for progressive streaming by registering metadata for the imageId or the default `stack` metadata as an `IRetrieveConfiguration` value. This value contains the stages to run, as well as the retrieve configuration for each stage. In specific, the `streaming` value needs to be set on the retrieve configuration for the value `single` retrieveType. The retrieve configuration has two pieces, the stages and the retrieve options (additionally, it can completely replace the retriever with a custom one). The stages are used to select the image ID's to retrieve, and provide the retrieve type to use. Then, the retrieve options map the retrieve type to the actual options to use. That allows multiple stages to use the same retrieve type for different purposes. The two retrieve types used for the progressive rendering for stack (which is defined in `sequentialRetrieveConfiguration`) are `singleFast` and `singleFinal`. This allows differing requests to be made for a fast initial request and a final, lossless request. The example `stackProgressive` shows several possible configurations for this which demonstrate how to load different URL paths or different parts of the image across repeated requests using byte range retrieves. --- ### Server Requirements Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/requirements.md #### Server Requirements Fast initial display of images requires a method to retrieve just a portion of an image or volume that can be rendered as a complete but lossy image. For instance, an image could be rendered using partial data (resolution), or images in a volume could be interpolated to generate an alternative image. These images are initially retrieved for rapid display, followed by retrieving a full-resolution image, resulting in a progressively improved display as more data is loaded. The DICOM Standards Committee just added support in DICOM for a new encoding method called High Throughput JPEG 2000 (HTJ2K). This encoding method enables progressive decoding of images, meaning that if the first `N bytes` of the image encoding are available, they can be decoded into a lower resolution or lossy image. The configuration that enables this feature is called `HTJ2K Progressive Resolution (HTJ2K RPCL)` or `High Throughput JPEG 2000 Resolution Position Component Layer`. Finally, some servers can be configured to serve up reduced (partial) resolution versions of images on other URL endpoints. The progressive loading will improve the display of stacked images by supporting HTJ2K progressive resolution encoded data. Meanwhile, volumetric data will be enhanced in terms of the time it takes to load the first volume for all backends, unless they are specifically configured for custom load order. However, the support for different types of reduced resolution and streaming responses varies significantly among DICOMweb implementations. Therefore, this guide provides additional details on how to configure various configurations. #### Server Requirements As HTJ2K is a new encoding (and still not merged into the DICOM standard, although approved for merging), it is not yet widely supported by DICOMweb servers. The various ways that servers support it might change in the future. However, we envision two main ways that this will be implemented in most servers but both require the server to support the DICOMWeb standard and HTJ2K RPCL encoding. - **HTJ2K Support**: For HTJ2K encoded images, the server must support the streaming of image data in a way that respects the HTJ2K RPCL configuration, allowing the client to decode partial data into a displayable image. #### Respond with Streaming Data XHR (XMLHttpRequest) streaming is an extension of the XHR browser-level API that enables the client to retrieve pieces of data as it arrives, instead of waiting for the entire response. XHR streaming works by keeping a persistent connection between the client and server and sending data incrementally as it becomes available. #### Respond with Byte Range Request An XHR byte range request is a feature of the XMLHttpRequest object in JavaScript that allows for retrieving only a specific range of bytes from a server. This feature is typically used for downloading large files in chunks or resuming interrupted downloads. By specifying the starting and ending byte positions, the server can send only the requested portion of the file, reducing bandwidth usage and improving download efficiency. - **Partial Content Delivery**: The server must support HTTP Range requests, allowing the client to request and receive specific byte ranges of the image data. This is crucial for handling large images or volumes by fetching and rendering portions of the data progressively. :::info The existing JPEG 2000 encoding and the new [HTJ2K in the standard](https://dicom.nema.org/medical/dicom/Supps/LB/sup235_lb_HTJ2K.pdf) also have a format that specifies a partial resolution endpoint. The exact endpoint needs to be specified in the JPIP referenced data URL. The options data could be used to provide the exact URL required in a future revision. ::: --- ### Retrieve Configuration Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/retrieve-Configuration.md #### Retrieve Configuration Progressive loading works in steps called `stages`. Each stage is part of **which images load with which settings**, and you can set each stage with different settings, known as `retrieve options`. Together (stages and options) make up the `retrieve configuration`, which manages how images are loaded step by step. Let's dive in. ```ts interface IRetrieveConfiguration { stages: RetrieveStage[]; retrieveOptions: Record; } ``` #### Retrieve Stages As the name of progressive loading suggests, the loading process is done in `stages`. Each `stage` can be configured to use a different retrieve method (streaming or byteRange) and its common or specific retrieve options. :::info Since you can have multiple stages, the two methods (streaming and byte range) can be combined and utilized at different stages. For instance you can create a configuration that 1. start **streaming** of specific initial slices (typically the first, middle, or last slice) for immediate viewing. 2. Subsequently, in the second stage, **byte range requests** (only couple of `kb`) can be made for the rest of the slices to efficiently render the complete volume as quickly as possible (even if lossy). 3. Finally, you perform supplementary **byte range requests** for the remaining segments that have not yet been requested, following the initial byte range request in step 2. This approach is actually employed in the volume loading process, which will be further elaborated upon in our subsequent discussion. ::: So, in summary, the Retrieve Stage is a configuration that specifies which images load with which settings. For the simplicity of this document and to not lose focus, we will only talk about the `retrieveType`, which is just a reference to the retrieve options. We will discuss more advanced options, such as selecting images for strategy, prioritizing, and queuing loading, later. ![](../../assets/retrieve-stages.png) As seen above, the retrieve stages can be as simple a list of objects, each with an `id` and a `retrieveType` (which is a reference to the retrieve options which we will talk next). :::tip The `retrieveType` is an optional string that is only used for referencing the option to be used. You can use any string as long as you are consistent in using it in the retrieve options **as well**. Use `'lkajsdflkjaslfkjsadlkfj'` if you wish (but then you should have an object with key of `'lkajsdflkjaslfkjsadlkfj'` in the `retrieveOptions` object as we will see below). :::
What would happen if we reference a retrieve type that is not defined in the retrieve options? Cornerstone will check if a `default` retrieve options is specified, if true, it will use that otherwise will ignore the progressive loading configuration and will load the image as if progressive loading is not enabled (like before)
#### Retrieve Options Now we an talk about retrieve options for each of the methods (streaming or byte range) in more detail. Let's dive into the common options first. #### Common Options There are more advanced options for the retrieval configuration that can be used to handle more use cases. We will talk about them later in another section. #### Decode Level (quality) One natural question that might arise is, regardless of the method (stream or byte range) how often the image is decoded and when we decode what is the resolution of the image that we should decode to? The resolution of decoding is controlled by `decodeLevel` configuration and it can be - 0 = full resolution - 1 = half resolution - 2 = quarter resolution - 3 = eighth resolution - ... So if the decodeLevel for a stage is set to 0, then the image will be decoded to full resolution. If it is set to 1, then the image will be decoded to half resolution (x/2, y/2) and so on. :::tip For volume viewports, we currently don't allow decoding into sub-resolution because it would require reallocating the volume in memory, which is inefficient. Therefore, if the data is partial and can't be decoded into full resolution, we simply replicate it (inside a web worker for performance) to fill the entire volume. However, for the stack viewport, we do allow decoding into sub-resolution since this re-allocation is cheaper than the whole volume. Additionally, in this scneario, future enhanced qualities of the image will wipe out the old image and create a new image with new size until full resolution is reached. ::: We will talk about `frequency` in the each method's section below. #### Streaming Options #### Options For streaming requests, you can configure the following options: - `streaming`: whether to use streaming or not #### Decoding Frequency Most often, when the stream is coming from a server, the server lets the client know about the final size of the data. So, at each point in time, we can identify the percentage of the data that has been downloaded and decode the image to the relevant resolution so in the streaming scenario you really don't have to set it manually. Different levels are, if the downloaded portion at the time of decoding is - \< 8 \% of the total data, then decode to level 3 - 8 \< x \< 13 \% of the total data, then decode to level 2 - 13 \< x \< 27 \% of the total data, then decode to level 1 - \< 100 (means it is not finished) then decode to level 0 - 100 \% of the total data (stream is finished), then decode to level 0 :::tip How did we come up with these levels? It is kind of simple. For instance, if we have only downloaded 1/16th of the total data, it means we have downloaded 6.25% of the data (8% is 6.25% with some offset). This means we can decode the image to 1/16th of the original size, which is level 4. However, the interpolation provided by the decoder is slightly better than that provided by straight image rendering, and thus one can decode to a slightly lower level, using level 3 instead. The same goes for the rest of the levels. ::: To answer the question of how many decoding levels will occur, it totally depends on the initial data that is downloaded and how the stream progresses. But at any given time when the data is downloaded, we check the progress against the above levels and decode the image to the relevant resolution if possible. If an error is thrown or the image is not decoded, we simply wait for the next progress event to occur. #### Example For the simple streaming scenario (streaming true) you should expect the following behavior: ![](../../assets/streaming-decode.png) #### Use cases using the streaming method is suitable for the scenarios that you eventually require the full resolution of the data and you want to start viewing the data as soon as possible. #### Byte Range Options #### Options - `chunkSize`: byte range value to retrieve for initial decode (default is 64kb). Ignored for all but the first range request (regardless of rangeIndex). - `rangeIndex`: is the range number (index) that you want to fetch, -1 for remaining data Note that there is no guarantee that the rangeIndex will actually fetch another range since it will discontinue fetching once all the data has been fetched. Also, -1 is used to flag the "remaining" data. #### Decoding Frequency There are two scenarios for byte range requests: - If the server sends back the total size of the data in the header of the response for the byte range in which we use our automatic decoding frequency (similar to the streaming scenario). - The server does not send back the total size of the data in the header of the response for the byte range in which we wait until the range request is finished and then decode the image. :::tip The server should send the cors header `Access-Control-Expose-Headers: *` to enable reading the `Range-Response` header required for seeing the total size. Otherwise, the range request is finished when the multipart/related header is complete OR the returned data is smaller than the requested data. ::: #### Example For instance for the options of ```js { rangeIndex: 0, chunkSize: 256000, // 256kb } ``` ![](../../assets/range-0.png) another example ```js { rangeIndex: 0, decodeLevel: 3 } // chunkSize is default 64kb ``` ![](../../assets/range-0-decode-3.png) :::tip You can fetch the remaining data by using `rangeIndex: -1`. In addition, `rangeIndex = 0` will always be the first chunk. For instance, if you have 4 ranges, then your ranges would be - `rangeIndex 0`: `0` to `chunkSize-1` (in bytes) - `rangeIndex 5`: `chunkSize` to `5 * chunkSize-1` (in bytes) - `rangeIndex 25`: `5 * chunkSize` to `25 * chunkSize-1` (in bytes) - `rangeIndex -1`: `25 * chunkSize` to `totalSize` (in bytes) - the rest of the data This use of rangeIndex allows retrieving larger increments to agree with the amount of data required for decodeLevel values. :::
What if I start with a range 1 instead of 0? Cornerstone will automatically fetch the range 0 combined with range 1 as a single request. This avoids needing to perform multiple intermediate requests.
#### Use cases Other than we can use the range request to progressively request and load a better quality images there are some other usecases - Thumbnails: Often, for thumbnails, we want to load the image as quickly as possible but don't need the full resolution. We can use a byte range request to fetch a lower resolution version of the data. - CINE: For certain imaging needs, the frame rate is absolutely essential in the cine mode. Often, the gross anatomy is desired in these scenarios, not the details, but the frame rate is of greater importance. We can use the byte range request to fetch the subresolution of the data, guaranteeing that we can achieve the target frame rate. In the future, a separate memory cache might be used for range request details, but right now the intermediate data is held alongside the image data. Storing it in a cache would allow for CINE display with only the cost of decoding the image. #### Conclusion So, we learned that a "retrieve configuration" is composed of at least one (can be more) "retrieve stage" and an accompanying "retrieve options" that have the keys referenced in the "retrieve stages." We also learned that each "retrieve stage" can be configured to use a different method (streaming or byte range) and has common or specific retrieve options. Let's look at an example of one of the examples that we have in the stackProgressive demo ```js const retrieveConfiguration = { stages: [ { id: 'initialImages', retrieveType: 'single', }, ], retrieveOptions: { single: { streaming: true, }, }, }; ``` :::tip Note the common use of 'single' in both the `stages` and `retrieveOptions` objects. This is just a reference to the retrieve options that we have defined in the `retrieveOptions` object. ::: Now your question might be, how do we [use this configuration](./usage)? We will talk about that in the next section. But curious readers can move to the advanced configuration section to learn more about the advanced options that we have for the retrieve configuration. --- ### Stack Progressive Loading Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/stackProgressive.md Here, we will explore the progressive loading of stackViewports as an example use case for progressive loading and benchmark it compared to regular loading. We will discuss this in more detail, including scenarios that involve multiple stages of progressive loading and different retrieval types. :::tip For stacked viewports, larger images can be decoded using a streaming method, where the HTJ2K RPCL image is received as a stream, and parts of it are decoded as they become available. This can significantly improve the viewing of stacked images, without requiring any special server requirements other than support for the HTJ2K RPCL transfer syntax. ::: #### Benchmark In general, about 1/16th to 1/10th of the image is retrieved for the lossy/first version of the image. This results in a significant speed improvement for the first images. It is fairly strongly affected by the overall image size, network performance, and compression ratios. **The full size test image is 3036 x 3036 and 11.1 MB in size. ** | Type | Network | Size | First Render | Final Render (baseline) | | --------------------------- | ------- | ------ | ------------ | ----------------------- | | HTJ2K streaming (1 stage) | 4g | 11.1 M | 66 ms | 5053 ms | | HTJ2K Byte Range (2 stages) | 4g | 128 K | 45 ms | 4610 ms | The configuration for the above test is as follows #### HTJ2K Streaming (1 stage) This configuration will retrieve an image using a single stage streaming response. It is safe to use for both streaming and non-streaming transfer syntaxes, but will only activate for the decoding portion when used with HTJ2K transfer syntaxes. For HTJ2K decoding, if the image is NOT in RPCL format, then other decoding progressions may occur, such as decoding by by region (eg top-left, top-right, bottom-left, bottom-right), or decoding may fail until the full data is available. :::tip You can use `urlParameters: accept=image/jhc` to request HTJ2K in a standards compliant fashion. ::: ```js const retrieveConfiguration = { // stages defaults to singleRetrieveConfiguration retrieveOptions: { single: { streaming: true, }, }, }; ``` #### HTJ2K Byte Range (2 stages) This sequential retrieve configuration has two stages specified, each of which applies to the entire stack of image ids. The first stage will load every image using the `singleFast` retrieve type, followed by the second stage retrieving using `singleFinal`. Note that this retrieve configuration requires support for byte-range requests on the server side. It MAY be safe for servers not supporting byte range requests, but the requests may also fail when attempted. Read your DICOM Conformance Statement. :::tip You can add a third, error recovery stage removing any byte range requests. This stage will only end up being run if the previous stages fail. This allows dealing with unknown server support. ::: ```js const retrieveConfiguration = { // This stages list is available as sequentialRetrieveStages stages: [ { id: 'lossySequential', retrieveType: 'singleFast', }, { id: 'finalSequential', retrieveType: 'singleFinal', }, ], retrieveOptions: { singleFast: { rangeIndex: 0, decodeLevel: 3, }, singleFinal: { rangeIndex: -1, }, }, }; ``` --- ### Static dicom web Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/static-wado.md standard and non-standard options, as well as instructions on setting it up in the [Static DICOMweb](https://github.com/RadicalImaging/Static-DICOMWeb) repository, mainly as an illustrative example. --- ### Usage Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/usage.md Now that we have learned about the retrieve configuration, let's see how we can use it in Cornerstone3D. #### `imageRetrieveMetadataProvider` This is a new metadata provider that we have added to the Cornerstone3D library. It is responsible for retrieving the metadata for the image (or volume, as we will explore later). So, in order to perform progressive loading on a set of imageIds, you need to add your retrieve configuration to this provider. #### Stack Viewport You can specify an imageId-specific retrieve configuration by including the imageIds as the key for your metadata. Considering our one stage retrieve configuration from the previous section we have the following: ```js import { utilities } from '@cornerstone3d/core'; const retrieveConfiguration = { stages: [ { id: 'initialImages', retrieveType: 'single', }, ], retrieveOptions: { single: { streaming: true, }, }, }; utilities.imageRetrieveMetadataProvider.add('imageId1', retrieveConfiguration); ``` If you don't need to define an imageId-specific retrieve configuration, you can then scope your metadata to `stack` in order for it to be applied to all imageIds. ```js utilities.imageRetrieveMetadataProvider.add('stack', retrieveConfiguration); ``` #### Volume Viewport For loading a volume as progressive loading, you can use the `volumeId` as the key for your metadata. ```js import { utilities } from '@cornerstone3d/core'; const volumeId = ....get volume id.... utilities.imageRetrieveMetadataProvider.add(volumeId, retrieveConfiguration); ``` Or you can scope your metadata to `volume` in order for it to be applied to all volumeIds. ```js utilities.imageRetrieveMetadataProvider.add('volume', retrieveConfiguration); ``` :::tip That is all you need to do! Everything else for loading the image progressively is handled by the Cornerstone3D library. ::: --- ### Volume Progressive Loading Source: https://cornerstonejs.org/docs/llm/concepts/progressive-loading/volumeProgressive.md #### Volume Viewport Interleaved Decode Since, for volume viewports, we mostly deal with rendering the reconstructed views (MPR) of the actual volume, the ideal scenario would be to have the initial images of the volume (even if lossy) as quickly as possible to avoid rendering a gray volume. We can achieve this by interleaving the requests. Interleaving the images applies to any encoding for a volume. That is, fetching every Nth image first allows a 1/Nth frequency image to be displayed. The interleave code then simply replicates the images to the missing positions to produce a low resolution in the longitudinal direction. This interleaving can then be combined with any discrete fetch for a lossy version of an image - that is, a non-streamed decoding version of an image that returns an entire request at once. #### Performance The performance gains on using progressive loading on volume viewports vary quite a bit depending on size of data and capabilities of the DICOMweb server components. Note that none of the times include time to load the decoder, but is only seen on first render. These times are similar for both types. | Type | Size | Network | First Render | Complete | | ---------------- | ---- | ------- | ------------ | -------- | | HTJ2K Stream | 33 M | 4g | 2503 ms | 8817 ms | | HTJ2K Byte Range | 33 M | 4g | 1002 ms | 8813 ms | The HTJ2K byte range is very slightly slower than straight JLS, but can be done against any DICOMweb server supporting HTJ2K and byte range requests. - 4g speed - 30 mbit/s down, 5 mbit/s up, 10 ms latency - Full size images are 512x512x174 - Reduce resolution images are 128x128 and lossy compressed #### HTJ2K Streaming Note that this stage model will interleave requests across different viewports for the various stages, by the selection of the queue and the priority of the requests. The interleaving isn't perfect, as it interleaves stages rather than individual requests, but the appearance works reasonably well without complex logic being needed to work between volumes. As learned in the [advanced retrieve configuration](./advance-retrieve-config), we saw that we can make use of `decimate`, `offset` and different priorities to achieve the interleaving. Decimation is a selection of every `N`th' image at the `F` offset, described as `N/F`, eg `4/3` is positions `3,7,11,...` This is done by retrieving, in order, the following stages: - Initial images - images at position 0, 50%, 100% - Decimated 4/3 image using multipleFast retrieve type - Displays a full volume at low resolution once this is complete - Decimated 4/1 image using multipleFast retrieve type - Updates the initial volume with twice the resolution - Decimated 4/2 and 4/0 images using multipleFinal - Replaces the replicated images with full resolution images - Decimated 4/3 and 4/1 using multipleFinal - Replices the low resolution images with full resolution The configuration looks like: ```javascript stages: [ { id: 'initialImages', // positions selects specific positions - middle image, first and last positions: [0.5, 0, -1], // Use teh default render type for these, which should retrieve full resolution retrieveType: 'default', // Use the Interaction queue requestType: RequestType.INTERACTION, // Priority 10, do first priority: 10, // Fill nearby frames from this data nearbyFrames: {....}, }, { id: 'quarterThumb', decimate: 4, offset: 3, retrieveType: 'multipleFast', priority: 9, nearbyFrames, }, ... other versions // Replace the first data with final data { id: 'finalFull', decimate: 4, offset: 3, priority: 4, retrieveType: 'multipleFinal', }, ], ``` 1. Fetch images shown initially at full resolution (first and last) 2. Fetch every 4th image first `initialByteRange` bytes - Fetch byte range [0,64000] - Display partial resolution version immediately - Use partial resolution version to display nearby slices 3. Other steps - There are other partial and full resolution views here to fill in data 4. Fetch remaining data for #2 (do not refetch original data) - Replaces the low resolution data from #2 with full data #### HTJ2K Byte Range The volume progressive loading extends the basic stack loading with the ability to interleave various images, interpolating them from a reduced resolution version both intra and inter image. That is, individual images might be fetched initially at 1/4 size (256x256 for a CT), and then only the initially displayed image plus every 4th image, with other images being interpolated. In this case, replicate interpolation is used to minimize interpolation overhead. Finally, after the lossy initial versions are fetched, the remaining images are fetched. The default retrieve ordering is below, where Decimate is described as the interval between images included, and the offset in that set. - Initial images, full resolution - Decimate 4/3 partial resolution - Interpolate images -2...+1 (nearest neighbors) - Decimate 4/1 partial resolution - Decimate 4/2 full resolution - Decimate 4/4 full resolution - Decimate 4/3 full resolution - Decimate 4/1 full resolution The same ordering is done if partial resolution is not configured, except that the last two stages are never run because the partial resolution has already loaded those. This DOES allow the interpolation of results to appear very quickly. --- ## Streaming-image-volume ### Streaming ImageVolume Source: https://cornerstonejs.org/docs/llm/concepts/streaming-image-volume/index.md #### Streaming ImageVolume We have added a new volume loader which implements a progressive loading of volumes to the GPU. You can read more in this section. --- ### Re-ordering Image Requests Source: https://cornerstonejs.org/docs/llm/concepts/streaming-image-volume/re-order.md #### Re-ordering Image Requests As mentioned in the [`Streaming of Volume Data`](./streaming.md) section, creation and caching of a volume is separated from the loading of the image data. This gives us the flexibility of loading images in any order, and the ability to re-order the image requests to load the images in the correct order. #### getImageLoadRequests After you create the `StreamingImageVolume` instance, you can call `getImageLoadRequests` to get the image load requests. You can then re-order (or interleave the serries request with another series) the image requests to load the images in desired order. --- ### Streaming of Volume Data Source: https://cornerstonejs.org/docs/llm/concepts/streaming-image-volume/streaming.md #### Streaming of Volume data With the addition of [`Volumes`](../cornerstone-core/volumes.md) to `Cornerstone3D`, we are adding and maintaining `Streaming-volume-image-loader` which is a progressive loader for volumes. This loader is designed to accept imageIds and load them into a `Volume`. #### Creating Volumes From Images Since 3D `Volume` is composed of 2D images (in `StreamingImageVolume`), its volume metadata is derived from the metadata of the 2D images. Therefore, an initial call to fetch images metadata is required for this loader. This way, not only we can pre-allocate and cache a `Volume` in memory, but we also can render the volume as the 2D images are being loaded (progressive loading). ![](../../assets/volume-building.png) By pre-fetching the metadata from all images (`imageIds`), we don't need to create the [`Image`](../cornerstone-core/images.md) object for each imageId. Instead, we can just insert the pixelData of the image is directly inserted into the volume at the correct location. This guarantees speed and memory efficiency (but comes at minimal cost of pre-fetching the metadata). #### Converting volumes from/to images `StreamingImageVolume` loads a volume based on a series of fetched images (2D), a `Volume` can implement functions to convert its 3D pixel data to 2D images without re-requesting them over the network. For instance, using `convertToCornerstoneImage`, `StreamingImageVolume` instance takes an imageId and its imageId index and return a Cornerstone Image object (ImageId Index is required since we want to locate the imageId pixelData in the 3D array and copy it over the Cornerstone Image). This is a process that can be reverted; `Cornerstone3D` can create a volume from a set of `imageIds` if they have properties of a volume (Same FromOfReference, origin, dimension, direction and pixelSpacing). #### Usage As mentioned before, a pre-cache volume should be created before hand from the image metadata. This can be done by calling the `createAndCacheVolume`. ```js const ctVolumeId = 'cornerstoneStreamingImageVolume:CT_VOLUME'; const ctVolume = await volumeLoader.createAndCacheVolume(ctVolumeId, { imageIds: ctImageIds, }); ``` Then the volume can call its `load` method to actually load the pixel data of the images. ```js await ctVolume.load(); ``` #### imageLoader Since the volume loader does not need to create the [`Image`](../cornerstone-core/images.md) object for each imageId in the `StreamingImageVolume`, it will use the `skipCreateImage` option internally to skip the creation of the image object. Otherwise, the volume's image loader is the same as wadors image loader written in `cornerstone-wado-image-loader`. ```js const imageIds = ['wadors:imageId1', 'wadors:imageId2']; const ctVolumeId = 'cornerstoneStreamingImageVolume:CT_VOLUME'; const ctVolume = await volumeLoader.createAndCacheVolume(ctVolumeId, { imageIds: ctImageIds, }); await ctVolume.load(); ``` #### Alternative implementations to consider Although we believe our pre-fetching method for volumes ensures that the volume is loaded as fast as possible, There can be other implementations of volume loaders that don't rely on this prefetching. #### Creating Volumes without pre-fetching metadata In this scenario, each image needs to be created separately, which means each image needs to be loaded and a Cornerstone [`Image`](../cornerstone-core/images.md) should be created. This is a costly operation as all the image objects are loaded in memory and a separate creation of a [`Volume`](../cornerstone-core/volumes.md) is required from those images. Advantages: - Not need for a separate metadata call to fetch the image metadata. Disadvantages: - Performance cost - Cannot progressively load the image data, as it requires creating a new volume for each image change --- # Contribute ## Writing Documentation Source: https://cornerstonejs.org/docs/llm/contribute/documentation.md #### Writing Documentation We strongly recommend for each Pull Request you make you ask yourself the following questions: - Does this change require change of documentation too? - Is this a new feature? If so, does it need to be documented? If the answer is Yes, it is recommended to document it. #### Running Documentation Page To run documentation you need to execute ```sh cd packages/docs/ yarn run start ``` This will open up port `3000` and start the documentation server. Then you can visit `http://localhost:3000` to see the documentation page. :::note Important Running the documentation server for the first time will probably fail complaining about the `example.md` file not being found. This is because the `example.md` file is created at build time and is not available in the repository. To fix this, for the first time, you can run `yarn docs:dev` to build and to run the documentation server. After the first time, You can just run `yarn docs` to run the documentation server. ::: #### Potential problems you may encounter #### Side bar not showing up There is a bug in your markdown file, likely in the way you are using the markdown syntax. --- ## Writing Karma Tests Source: https://cornerstonejs.org/docs/llm/contribute/karma-tests.md #### Writing Karma Tests To make sure our rendering and tools don't break upon future modifications, we have written tests for them. Rendering tests includes comparing the rendered images with the expected images. Tools tests includes comparing the output of the tools with the expected output. #### Running Karma Tests Locally You can run `yarn run test` to run all tests locally. By default, `karma.conf.js` will run the tests in a headless chrome browser to make sure our tests can run in any servers. Therefore, you cannot visualize it by default. In order to run the tests and visually inspect the results, you can run the tests by changing the `karma.conf.js` file to have `browsers: ['Chrome']` instead of `browsers: ['ChromeHeadless']`. ![renderingTests](../assets/tests.gif) #### Generating HTML Review Reports For local review, use the repository wrapper instead of reading terminal output only: ```bash ./scripts/run-karma.sh ./scripts/run-karma.sh --compat ./scripts/run-karma.sh --cpu ./scripts/run-karma.sh --next ``` The wrapper runs `npx karma start --single-run`, captures the log, and generates timestamped output under `reports/`. Examples: ```bash reports/legacy-karma//legacy-karma.log reports/legacy-karma-/index.html reports/compat-karma//compat-karma.log reports/compat-cpu-karma//compat-cpu-karma.log ``` Supported wrapper flags: - `--compat`: force compatibility mode for the Karma run. - `--cpu`: force CPU rendering for the Karma run. - `--next`: convenience mode that runs two passes, `--compat` and then `--compat --cpu`. Any other arguments are passed directly to `karma start`, so you can keep using normal Karma CLI options: ```bash ./scripts/run-karma.sh --browsers Chrome --no-single-run ./scripts/run-karma.sh --reporters spec ``` Useful environment variables: - `KARMA_GREP=""`: filter tests via Karma client args. - `KARMA_PACKAGE=core|tools`: load only the selected package's Karma tests. - `FORCE_COMPAT=true` and `FORCE_CPU_RENDERING=true`: direct overrides when running `karma start` without the wrapper. Examples: ```bash ./scripts/run-karma.sh ./scripts/run-karma.sh --compat ./scripts/run-karma.sh --compat --browsers Chrome --no-single-run KARMA_GREP="flip a stack viewport vertically" ./scripts/run-karma.sh --browsers Chrome --no-single-run KARMA_PACKAGE=core ./scripts/run-karma.sh ``` #### Baseline Images Karma uses two baseline locations depending on the mode: - Legacy mode compares against committed PNGs in `packages/core/test/groundTruth/` and `packages/tools/test/groundTruth/`. - Compatibility-mode runs use generated PNGs in `karma-baselines//`. To refresh the committed legacy ground-truth images, run: ```bash node utils/updateGroundTruth.js ``` Compatibility-mode baseline behavior is different: - Missing baselines are created automatically after `./scripts/run-karma.sh` finishes. - After a new compatibility baseline is created, rerun the same command to compare against it. - If you intentionally want to replace an existing compatibility baseline, remove or overwrite the PNG in `karma-baselines/` and rerun the wrapper. #### Reviewing Image Comparisons When Karma tests use `compareImages()`, the HTML report includes persisted image artifacts for review. This now applies to passing and failing image comparisons, not only failures. For each comparison artifact, the report shows: - `Expected` - `Actual` - `Compare` - `Diff Mask` The `Compare` panel overlays expected and actual with a slider, and each tile includes direct open links so you can inspect the raw generated images in a separate tab. The HTML report also supports filtering by status, and failed tests are rendered first in the report. #### Running Only One Karma Test Locally Use `KARMA_GREP` when you want to keep the wrapper but filter the suite: ```bash KARMA_GREP="flip a stack viewport vertically" ./scripts/run-karma.sh KARMA_GREP="flip a stack viewport vertically" ./scripts/run-karma.sh --browsers Chrome --no-single-run ``` For ad hoc debugging, you can also still use Jasmine helpers such as `fdescribe` and `fit`. --- ## Linking Cornerstone Libraries Source: https://cornerstonejs.org/docs/llm/contribute/linking.md #### Linking Cornerstone Libraries with OHIF for Development Often time you will want to link to a package to Cornerstone3D, this might be to develop a feature, to debug a bug or for other reasons. Also, sometimes you may want to link the external packages to include libraries into your build that are not direct dependencies but are dynamically loaded. See the externals/README.md file for details. #### Yarn Link There are various ways to link to a package. The most common way is to use [`yarn link`](https://classic.yarnpkg.com/en/docs/cli/link). This guide explains how to link local Cornerstone libraries for development with OHIF. #### Prerequisites - Local clone of OHIF Viewer - Local clone of desired Cornerstone libraries (@cornerstonejs/core, @cornerstonejs/tools, etc.) - Yarn package manager #### Steps to Link Libraries 1. **Prepare the Cornerstone Library** Navigate to the Cornerstone library directory you want to link (e.g., @cornerstonejs/core): ```bash cd packages/core ``` Unlink any existing links first: ```bash yarn unlink ``` Create the link: ```bash yarn link ``` Build the package to ensure latest changes: ```bash yarn dev ``` 2. **Link in OHIF** In your OHIF project directory: ```bash yarn link @cornerstonejs/core ``` Start OHIF: ```bash yarn dev ``` #### Working with Multiple Libraries You can link multiple Cornerstone libraries simultaneously. For example, to link both core and tools: ```bash #### In cornerstone/packages/core yarn unlink yarn link yarn dev #### In cornerstone/packages/tools yarn unlink yarn link yarn dev #### In OHIF yarn link @cornerstonejs/core yarn link @cornerstonejs/tools ``` #### Verifying the Link 1. Make a visible change in the linked library (e.g., modify a line width in tools) 2. Rebuild the library using `yarn dev` 3. The changes should reflect in OHIF automatically #### Important Notes - Always run `yarn dev` in the Cornerstone library after making changes - Due to ESM migration in Cornerstone 3D 2.0, linking process is simpler than before - Remove links when finished using `yarn unlink` in both projects #### Troubleshooting If changes aren't reflecting: 1. Ensure the library is rebuilt (`yarn dev`) 2. Check the console for any linking errors 3. Verify the correct library version is linked using the browser console #### Video Tutorials #### Tips 1. `yarn link` is actually a symlink between packages. If your linking is not working, check out the `node_modules` in the `Cornerstone3D` directory to see if the symlink has been created (the updated source code - not the dist - is available in the `node_modules`). 2. If your `debugger` is not hitting, you might want to change the `mode` setting in the webpack to be `development` instead of `production`. This ensures, minification is not applied to the source code. 3. Use a more verbose source map for debugging. You can read more [here](https://webpack.js.org/configuration/devtool/) --- ## Writing Playwright Tests Source: https://cornerstonejs.org/docs/llm/contribute/playwright-tests.md #### Writing PlayWright Tests Our Playwright tests are written using the Playwright test framework. We use these tests to test our examples and ensure that they are working as expected which in turn ensures that our packages are working as expected. In this guide, we will show you how to write Playwright tests for our examples, create new examples and test against them. #### Testing against existing examples If you would like to use an existing example, you can find the list of examples in the `utils/ExampleRunner/example-info.json` file. You can use the `exampleName` property to reference the example you would like to use. for example, if you would like to use the `annotationToolModes` example, you can use the following code snippet: ```ts import { test } from '@playwright/test'; import { visitExample } from './utils/index'; test.beforeEach(async ({ page }) => { await visitExample(page, 'annotationToolModes'); }); test.describe('Annotation Tool Modes', async () => { test('should do something', async ({ page }) => { // Your test code here }); }); ``` #### Testing against new examples Our playwright tests run against our examples, if you would like to add a new example, you can add it to the `examples` folder in the root of of the respective package, for example, `packages/tools/examples/{your_example_name}/index.ts`, and then add then register it in `utils/ExampleRunner/example-info.json` file under it's correct category, for example if its tool related, it can go into the existing `tools-basic` category. If you don't find a category that fits your example, you can create a new category and add it to the `categories` object in the `example-info.json` file. ```json { "categories": { "tools-basic": { "description": "Tools library" }, "examplesByCategory": { "tools-basic": { "your_example_name": { "name": "Good title for your example", "description": "Good description of what your example demonstrates" } } } } } ``` Once this is done, you can write a test against the example by using the `visitExample` function in the `tests/utils/visitExample.ts` file. For example, if you would like to write a test against the `your_example_name` example, you can use the following code snippet: ```ts import { test } from '@playwright/test'; import { visitExample } from './utils/index'; test.beforeEach(async ({ page }) => { await visitExample(page, 'your_example_name'); }); test.describe('Your Example Name', async () => { test('should do something', async ({ page }) => { // Your test code here }); }); ``` This will also make your example appear in our docs page, so that users can see how to use the example, so you are adding double value by adding a new example. #### Screenshots A good way to check your tests is working as expected is to capture screenshots at different stages of the test. You can use our `checkForScreenshot` function located in `tests/utils/checkForScreenshot.ts` to capture screenshots. You should also plan your screenshots in advance, screenshots need to be defined in the `tests/utils/screenshotPaths.ts` file. For example, if you would to capture a screenshot after a measurement is added, you can define a screenshot path like this: ```ts const screenShotPaths = { your_example_name: { measurementAdded: 'measurementAdded.png', measurementRemoved: 'measurementRemoved.png', }, }; ``` It's okay if the screenshot doesn't exist yet, this will be dealt with in the next step. Once you have defined your screenshot path, you can use the `checkForScreenshot` function in your test to capture the screenshot. For example, if you would like to capture a screenshot of the `cornerstone-canvas` element after a measurement is added, you can use the following code snippet: ```ts import { test } from '@playwright/test'; import { visitExample, checkForScreenshot, screenshotPath, } from './utils/index'; test.beforeEach(async ({ page }) => { await visitExample(page, 'your_example_name'); }); test.describe('Your Example Name', async () => { test('should do something', async ({ page }) => { // Your test code here to add a measurement const locator = page.locator('.cornerstone-canvas'); await checkForScreenshot( page, locator, screenshotPath.your_example_name.measurementAdded ); }); }); ``` The test will automatically fail the first time you run it, it will however generate the screenshot for you, you will notice 3 new entries in the `tests/screenshots` folder, under `chromium/your-example.spec.js/measurementAdded.png`, `firefox/your-example.spec.js/measurementAdded.png` and `webkit/your-example.spec.js/measurementAdded.png` folders. You can now run the test again and it will use those screenshots to compare against the current state of the example. Please verify that the ground truth screenshots are correct before committing them or testing against them. #### Simulating mouse drags If you would like to simulate a mouse drag, you can use the `simulateDrag` function located in `tests/utils/simulateDrag.ts`. You can use this function to simulate a mouse drag on an element. For example, if you would like to simulate a mouse drag on the `cornerstone-canvas` element, you can use the following code snippet: ```ts import { visitExample, checkForScreenshot, screenShotPaths, simulateDrag, } from './utils/index'; test.beforeEach(async ({ page }) => { await visitExample(page, 'stackManipulationTools'); }); test.describe('Basic Stack Manipulation', async () => { test('should manipulate the window level using the window level tool', async ({ page, }) => { await page.getByRole('combobox').selectOption('WindowLevel'); const locator = page.locator('.cornerstone-canvas'); await simulateDrag(page, locator); await checkForScreenshot( page, locator, screenShotPaths.stackManipulationTools.windowLevel ); }); }); ``` Our simulate drag utility can simulate a drag on any element, and avoid going out of bounds. It will calculuate the bounding box of the element and ensure that the drag stays within the bounds of the element. This should be good enough for most tools, and better than providing custom x, and y coordinates which can be error prone and make the code difficult to maintain. #### Running the tests After you have wrote your tests, you can run them by using the following command: ```bash ./scripts/run-playright.sh ./scripts/run-playright.sh --compat ./scripts/run-playright.sh --cpu ./scripts/run-playright.sh --next ``` The wrapper runs `npx playwright test`, auto-selects the test files for the chosen mode, and writes timestamped logs and artifacts under `reports/`. Examples: ```bash reports/legacy-playwright// reports/compat-playwright// reports/compat-cpu-playwright// reports/generic-viewport-playwright// ``` Supported wrapper flags: - `--compat`: open example pages with `?type=next`. - `--cpu`: open example pages with `?cpu=1`. - `--next`: run only `tests/genericViewport/**/*.spec.ts`. `--next` on Playwright is different from `--next` on Karma. Playwright uses it to select the `tests/genericViewport` suite only. Karma uses it as a convenience mode that runs compatibility and CPU passes. Any other arguments are passed directly to `playwright test`, so you can still use the normal Playwright CLI: ```bash ./scripts/run-playright.sh --project chromium --headed ./scripts/run-playright.sh -g "stack viewport" ./scripts/run-playright.sh --workers 1 ./scripts/run-playright.sh --update-snapshots ``` Useful environment variables: - `PLAYWRIGHT_REUSE_EXISTING_SERVER=true|false`: control reuse of the configured local example server. - The wrapper sets `PLAYWRIGHT_FORCE_COMPAT`, `PLAYWRIGHT_FORCE_CPU_RENDERING`, `PLAYWRIGHT_HTML_OUTPUT_DIR`, and `PLAYWRIGHT_HTML_OPEN=never` internally. Examples: ```bash ./scripts/run-playright.sh ./scripts/run-playright.sh --compat ./scripts/run-playright.sh --project chromium --headed ./scripts/run-playright.sh -g "stack viewport" PLAYWRIGHT_REUSE_EXISTING_SERVER=true ./scripts/run-playright.sh --project chromium ./scripts/run-playright.sh --next ``` #### Updating Screenshot Baselines Playwright snapshot files are stored under `tests/screenshots///.png`, using the path template from `playwright.config.ts`. Normal runs compare against those committed screenshots. To rewrite them, pass Playwright's native snapshot flag through the wrapper: ```bash ./scripts/run-playright.sh --update-snapshots ./scripts/run-playright.sh --next --update-snapshots ./scripts/run-playright.sh --project chromium --update-snapshots ``` #### Serving the examples manually for development By default, Playwright builds the examples in `playwright.globalSetup.ts` when it needs to start its own local server, then the configured `webServer` serves `.static-examples` at `http://localhost:3333`. If you want to serve the examples manually during development, you can run the same command yourself and then tell Playwright to reuse the existing server: ```bash yarn run build-and-serve-static-examples PLAYWRIGHT_REUSE_EXISTING_SERVER=true ./scripts/run-playright.sh ``` #### Playwright VSCode Extension and Recording Tests If you are using VSCode, you can use the Playwright extension to help you write your tests. The extension provides a test runner and many great features such as picking a locator using your mouse, recording a new test, and more. You can install the extension by searching for `Playwright` in the extensions tab in VSCode or by visiting the [Playwright extension page](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright).
--- ## How to Contribute Source: https://cornerstonejs.org/docs/llm/contribute/pull-request.md #### How to Contribute #### Reporting Bugs If you find a bug, we strongly encourage you to report it to the project maintainers. You can do this by creating a new issue on the project's issue tracker, or by sending a pull request with a fix for the bug. It is always helpful to provide as much information as possible when reporting bugs, and it would help a lot if you could provide an example that demonstrates the bug. #### I would like to contribute code - how do I do this? Fork the repository, make your change and submit a pull request. Before submitting the pull request: - make sure that your changes are well tested and that you have updated the project's documentation. - make sure your tests (`yarn run test`) and build (`yarn run build`) are properly working locally on your machine and make sure they all pass #### Any guidance on submitting changes? While we do appreciate code contributions, triaging and integrating contributed code changes can be very time consuming. Please consider the following tips when working on your pull requests: - Functionality is appropriate for the repository. Consider posting on the forum if you are not sure. - Code quality is acceptable. We don't have coding standards defined, but make sure it passes ESLint and looks like the rest of the code in the repository. - Quality of design is acceptable. This is a bit subjective so you should consider posting on the forum for specific guidance. - The scope of the pull request is not too large. Please consider separate pull requests for each feature as big pull requests are very time consuming to understand. - We will provide feedback on your pull requests as soon as possible. Following the tips above will help ensure your changes are reviewed. #### My changes require updating dependencies in the package.json files - what is the process for doing this? In general you will typically not be updating the various `package.json` files. But for the case when you do, you will have to also update the various Cornerstone3D lock files and as such you will have to do both a `yarn` and `bun` `install` without the `--frozen-lockfile` flag. :::danger Updating the `package.json` files must be done with care so as to avoid incorporating vulnerable, third-party packages and/or versions. Please research the added packages and/or versions for vulnerabilities. Here is what you should do when adding new packages and/or versions prior to committing and pushing your code: 1. Do your due diligence researching the added packages and/or versions for vulnerabilities. 2. Update the `package.json` files. 3. Execute `yarn run install:update-lockfile`. This updates both the `yarn.lock` and the `bun.lock` files. 4. Execute `yarn run audit` for a last security check. This runs both `yarn audit` and `bun audit`. 5. Include both the `yarn.lock` and `bun.lock` files as part of your commit. If any of your research or auditing for vulnerabilities find HIGH risk vulnerabilities do NOT commit or push your changes! Low and moderate risk vulnerabilities are acceptable. ::: --- # Getting-started ## Installation Source: https://cornerstonejs.org/docs/llm/getting-started/installation.md #### Installation #### NPM You can install `Cornerstone3D`, `Cornerstone3DTools`, and `StreamingImageVolumeLoader` using [npm](https://www.npmjs.com/). You can install the latest version of the packages by running: ```bash npm install @cornerstonejs/core npm install @cornerstonejs/tools npm install @cornerstonejs/dicom-image-loader npm install @cornerstonejs/nifti-volume-loader #### To use the polymorphic segmentation converters you need to install the following packages as well npm install @icr/polyseg-wasm ``` #### YARN If you are using [Yarn](https://yarnpkg.com/), you can install the packages by running: ```bash yarn add @cornerstonejs/core yarn add @cornerstonejs/tools yarn add @cornerstonejs/dicom-image-loader yarn add @cornerstonejs/nifti-volume-loader #### To use the polymorphic segmentation converters you need to install the following packages as well yarn add @icr/polyseg-wasm ``` #### PNPM If you are using [PNPM](https://pnpm.io), you can install packages by running: If `pnpm` is not already available in your environment, enable Corepack first: ```bash corepack enable ``` ```bash pnpm install @cornerstonejs/core pnpm install @cornerstonejs/tools pnpm install @cornerstonejs/dicom-image-loader pnpm install @cornerstonejs/nifti-volume-loader #### To use the polymorphic segmentation converters you need to install the following packages as well pnpm install @icr/polyseg-wasm ``` --- ## Overview Source: https://cornerstonejs.org/docs/llm/getting-started/overview.md import Link from '@docusaurus/Link'; #### Overview `Cornerstone3D` is a lightweight Javascript library for visualization of medical images in modern web browsers that support the HTML5 canvas element. Using `@cornerstonejs/core` and its accompanying libraries such as `@cornerstonejs/tools`, you can achieve a wide range of imaging tasks.


#### Features #### Rendering Using the new `Cornerstone3D` rendering engine and its Stack and Volume viewports, you can: - Render all transfer syntaxes including various compressed formats such as JPEG2000, JPEG Lossless - Stream the slices of a volume and view them in real-time as they are being loaded - View the same volume in different orientations such as axial, sagittal, and coronal without having to re-load the entire volume again (minimum memory footprint) - View oblique slices in a volume - Render different blends of the same volume (e.g. MIP (maximum intensity projection) and average intensity projection) - Fuse and overlay multiple images such as PET/CT fusion - Render color images and render them as a volume - Fall back to CPU rendering when GPU rendering is not available - Change calibration of the images by modifying the metadata for the viewport (e.g. pixel spacing) #### Manipulation `Cornerstone3DTools` enables the following features: - Zoom in and out of the image using mouse bindings - Pan the image in any direction - Scroll through the image in any orientation even in oblique slices - Change the window level of the image ![](../assets/overview-manipulation.gif) #### Annotation `Cornerstone3DTools` also enables annotating images using tools. All annotations are rendered as SVG elements which ensures that they are displayed at the best possible quality in any monitor resolution. Annotations in `Cornerstone3DTools` are stored in the actual physical space of the image which lets you render/modify the same annotations in multiple viewports. In addition, you can: - Activate certain tools on certain viewports with ToolGroups (e.g., on scroll activate slice scrolling on CT Axial viewport but volume rotation for PT MIP viewport) - Measure distances between two points using the Length tool - Measure length and width using bidirectional line tools - Calculate statistics such as mean, standard deviation of a region of interest using Rectangle/Elliptical ROI Tool - Use crosshairs to find corresponding points in images of different viewports and navigate slices using reference lines - Assign different tools to be activated while holding a specific modifier key (e.g. shift, ctrl, alt) - Create your own custom tools ![](../assets/overview-annotation.gif) #### Segmentation `Cornerstone3D` supports rendering segmentations of images as labelmaps in all viewports including stack, volume, and 3D. You can: - Render segmentations as labelmaps in the viewports (e.g. segmentation of CT lung) - Convert the label maps to surfaces in the 3D viewport and apply the same color. - View segmentations in any orientation (e.g. axial, sagittal, coronal) even in oblique slices - Change labelmap configuration (e.g. color, opacity, outline rendering, outline thickness etc.) - Edit/draw a segment in 3D Axial, Sagittal, Coronal using scissor tools such as Rectangle, Ellipse scissors - Apply a certain threshold to a labelmap for the region of interest #### Synchronization `Cornerstone3D` supports synchronization between multiple viewports. Currently, there are two implemented synchronizers and we are working on more. - WindowLevel synchronizer: synchronizes the window level of the source and target viewports - Camera synchronizer: synchronizes the camera of the source and target viewports For Generic/Next viewport integrations, camera-style synchronization is modeled as `ViewState` plus viewport projection: tools can read portable presentation with `viewportProjection.getPresentation(...)` and apply the translated native state through `setViewState(...)`. #### About this documentation Our documentation can be broken down into the following sections: - [**Getting Started**](/docs/category/getting-started): covers the scope of the project, related libraries and other relevant information, and installation instructions - [**Tutorials**](/docs/category/tutorials): provides a series of tutorials for different tasks such as rendering, tools, segmentation - [**How-to-Guides**](/docs/category/how-to-guides): provides guides for more advanced tasks such as custom loaders, custom metadata providers - [**Concepts**](/docs/category/concepts): explains an in-depth look at various technical concepts that are used in the library - [**Contributing**](/docs/category/contributing/): explains how to contribute to the project and how to report bugs - [**Migration Guides**](/docs/migration-guides/2x/general): includes instructions for upgrading from legacy to new versions of the library, and also upgrading from 1.x to 2.x - [**FAQ**](/docs/faq): provides answers to frequently asked questions - [**Help**](/docs/help): provides information of how to get help with the library - [**Examples**](/docs/examples): Displays the live examples of the library - [**API Reference**](/docs/api/core): provides a detailed description of the API and how to use each function If a page is no longer up-to-date, you can author a PR to update it by modifying the files in `/packages/docs/docs/*.md`. Read more on how to contribute [here](../contribute/pull-request.md). --- ## Related Libraries Source: https://cornerstonejs.org/docs/llm/getting-started/related-libraries.md #### Related Libraries In this section we will explain various libraries that are related to `Cornerstone3D`. #### History Before explaining the libraries, we will first discuss the history of `Cornerstone3D`. Prior to `Cornerstone3D` we developed and maintained [`cornerstone-core`](https://github.com/cornerstonejs/cornerstone) and [`cornerstone-tools`](https://github.com/cornerstonejs/cornerstoneTools) since 2014. Since the significance of improvements in `Cornerstone3D` over `cornerstone-core` and `Cornerstone3DTools` over `cornerstone-tools` is much greater, in long term we will switch our focus to `Cornerstone3D` and provide adequate documentation for how to upgrade from legacy `cornerstone` to the new `Cornerstone3D`. In the meantime, we will continue to maintain the legacy `cornerstone` packages and take care of potential critical bugs. In addition to the `cornerstone-core` and `cornerstone-tools` packages, we have also maintained [`react-vtkjs-viewport`](https://github.com/OHIF/react-vtkjs-viewport) our first iteration to enable 3D rendering using [vtk-js](https://github.com/kitware/vtk-js) in React. `react-vtkjs-viewport` is currently being used in the current main OHIF Viewer for the MPR views. One of the main motivations that prompted the development of the `Cornerstone3D` was the desire to be able to decouple the rendering from the UI by React similar to `cornerstone-core`. In addition, `react-vtkjs-viewport`'s memory management was a major challenge for more complex scenarios such as a PET/CT fusion with 10 viewports. Similar to legacy cornerstone, we will shift our efforts from `react-vtkjs-viewport` to use the new `Cornerstone3D` and `Cornerstone3DTools` packages. #### Libraries #### vtk.js [`vtk-js`](https://github.com/kitware/vtk-js) is an open-source javascript library for 3D computer graphics, image processing and visualization. In the past, we have used `vtk-js` for rendering and interacting with 3D data in `react-vtkjs-viewport` library. `Cornerstone3D`'s Rendering Engine has been designed to use `vtk-js` for 3D rendering. `vtk-js` has annotation support using tools, but we have decided to use `Cornerstone3DTools` for this purpose, and only rely on `vtk-js` for the actual rendering. Our roadmap (not funded yet) includes enabling usage of `vtk-js` 3D annotation tools in `Cornerstone3D`. #### OHIF Viewer [Open Health Imaging Foundation (OHIF)](https://ohif.org/) image viewer is an open source image viewer that is being used in academic and commercial projects such as [The Cancer Imaging Archive (TCIA)](https://www.cancerimagingarchive.net/) and [NCI Imaging Data Commons](https://datacommons.cancer.gov/repository/imaging-data-commons). It is an extensible web imaging platform with zero footprint and installation required. Currently, OHIF 3.9 relies on the all the libraries in the `Cornerstone3D` monorepo for its image rendering and annotation features. --- ## Scope of Project Source: https://cornerstonejs.org/docs/llm/getting-started/scope.md #### Scope of Project #### Scope `Cornerstone3D` is a javascript library that enables 3D rendering of medical images using purely web standards. The library employs WebGL for GPU accelerated rendering whenever possible. `Cornerstone3DTools` is a peer library to `Cornerstone3D` and contains a number manipulation and annotations tools that are used to interact with the images. The `Cornerstone3D` scope **DOES NOT** encompass dealing with image/volume loading and metadata parsing. The `Cornerstone3D` scope **DOES** include image rendering and caching. Proper image loaders should be registered **TO** the cornerstone3D using `imageLoader.registerImageLoader` and `volumeLoader.registerVolumeLoader`. Examples of such image loaders are `wadors` loader using `cornerstoneDICOMImageLoader` for DICOM P10 instances over `dicomweb` and `wadouri` for the DICOM P10 instances over HTTP. In addition, `Cornerstone3D` has a metadata registration mechanism that allows metadata parsers to be registered **TO** the `Cornerstone3D` using `metaData.addProvider`. Using `cornerstoneDICOMImageLoader`, its image loaders and metadata providers self-register with the `Cornerstone3D`. You can always checkout the example helpers to see how an end-to-end example from metadata parsing to image loading and image rendering can be achieved. #### Typescript Since all libraries in the `Cornerstone3D` monorepo are written in Typescript, they provide a type-safe API. This means that you can use the library in a TypeScript environment and using type information, you can be assured that the parameters being passed to any method match what is expected. #### Browser Support `Cornerstone3D` uses the HTML5 canvas element and WebGL 2.0 GPU rendering to render images which is supported by all modern browsers. Our advanced volume rendering has recently been revamped to allow for better performance and memory management and without the requirement of using sharedArrayBuffer which previously was a requirement for rendering volumes. - Chrome > 68 - Firefox > 79 - Edge > 79 If you are using an older browser, or don't have any graphics cards, your device might not be able to render volumetric images with `Cornerstone3D`. However, you can still render stack images using the CPU fallback that we have implemented in `Cornerstone3D` for such scenarios. #### Monorepo hierarchy `Cornerstone3D` is a monorepo that contains the following packages: - `/packages/core`: The core library responsible for rendering images and volumesand caching. - `/packages/tools`: The tool library for manipulation, annotation and segmentation rendering and tools. - `/packages/dicom-image-loader`: The image loader for `wadors` and `wadouri` DICOM P10 instances over HTTP. - `/packages/nifti-volume-loader`: The image loader for NIfTI files. - `/packages/docs`: Documentation for all the packages including guides, examples, and API reference. --- ## React, Vue, Angular, etc. Source: https://cornerstonejs.org/docs/llm/getting-started/vue-angular-react-vite.md Here are some examples of how to use Cornerstone3D with React, Vue, Angular, and Vite-based frameworks. **Example repositories:** - [Cornerstone3D with Vite + React](https://github.com/cornerstonejs/vite-react-cornerstone3d) - [Cornerstone3D with Vite + Vue](https://github.com/cornerstonejs/vue-cornerstone3d) - [Cornerstone3D with Angular](https://github.com/cornerstonejs/angular-cornerstone3d) - [Community maintained project](https://github.com/yanqzsu/ng-cornerstone) - [Cornerstone3D with Next.js](https://github.com/cornerstonejs/nextjs-cornerstone3d) --- #### Setup and install #### Prerequisites - **Node.js** (e.g. 18+ or 20+ depending on the template) - **npm** or **yarn** #### Vue (Vite) 1. Clone or create a Vite + Vue project and install dependencies: ```bash npm install # or: yarn ``` 2. **Required setup:** - **Vite config:** Use `@originjs/vite-plugin-commonjs` for `dicom-parser`, set `optimizeDeps.exclude: ['@cornerstonejs/dicom-image-loader']`, `optimizeDeps.include: ['dicom-parser']`, and `worker: { format: 'es' }`. See [Vite basic setup](#basic-setup) below. - **Subpath:** For running under a subpath (e.g. `/subpath/`), set `base` from `process.env.BASE_PATH` and use scripts like `dev:subpath` / `build:subpath` that set `BASE_PATH=/subpath/`. The Vue template uses `cross-env` for this. - **Codec WASM:** Vite resolves the codec binaries on its own, so nothing is required here. Optionally set `init({ wasmBasePath })` to serve them from a location you choose, e.g. a CDN — see [Codec WASM location](#codec-wasm-location). 3. **How to run:** - **Dev (root):** `npm run dev` → open http://localhost:5173/ - **Build (root):** `npm run build` → output in `dist/` - **Preview (root):** `npm run preview` → open http://localhost:4173/ - **Dev (subpath):** `npm run dev:subpath` → open http://localhost:5173/subpath/ - **Build (subpath):** `npm run build:subpath` then `npm run preview:subpath` (or use `npm run dev:subpath` to test). #### Angular 1. Install dependencies (this runs **postinstall** scripts that set up the build): ```bash npm install ``` 2. **Required setup:** - **Postinstall / prebuild:** The project uses scripts to create Node stubs (`fs`/`path`) for the browser build and to bundle the DICOM image loader worker and copy codec WASM. These run on `npm install` and before `npm run build` (via `prebuild`). The **preview** script runs them before building so the production bundle has the worker and codecs. - **Serve:** In development, `@cornerstonejs/dicom-image-loader` is excluded from prebundle so the worker loads correctly. - **Codec WASM:** Required here, unlike the Vite templates — the `application` builder uses esbuild, which does not resolve the codecs' bare specifiers. Point the loader at the copied binaries with `init({ wasmBasePath })`; see [Codec WASM location](#codec-wasm-location). - **Assets:** Codec `.wasm` files are copied from `node_modules` into the build via `angular.json` assets; the worker is generated into `public/cs-dicom-loader/` (and that folder is typically gitignored). 3. **How to run:** - **Dev (root):** `npm start` or `npm run dev` → open http://localhost:4200/ - **Build (root):** `npm run build` → output in `dist/angular-vite-6/` - **Preview (root):** `npm run preview` → builds then serves at http://localhost:4201/ (use this if the dev server doesn’t load images correctly). - **Dev (subpath):** `npm run dev:subpath` → open http://localhost:4200/subpath/ - **Build (subpath):** `npm run build:subpath` → then run the preview script or serve `dist/angular-vite-6/browser` with the app under `/subpath/`. - **Preview (subpath):** `npm run preview:subpath` → builds for subpath then serves at http://localhost:4202/. For production, deploy the contents of `dist/angular-vite-6/browser` and serve it at `/` or at your subpath. #### React (Vite) 1. Install dependencies: ```bash npm install # or: yarn ``` 2. **Required setup:** - **Vite config:** Same as Vue: CommonJS plugin for `dicom-parser`, exclude `@cornerstonejs/dicom-image-loader` from `optimizeDeps`, include `dicom-parser`, and `worker: { format: 'es' }`. Optionally use a Cornerstone WASM plugin or `base` for subpath. - **Subpath:** Set `base: '/subpath/'` in `vite.config.ts` (or from env) for build/preview under a subpath. - **Codec WASM:** Vite resolves the codec binaries on its own, so nothing is required here. Optionally set `init({ wasmBasePath })` to serve them from a location you choose, e.g. a CDN — see [Codec WASM location](#codec-wasm-location). 3. **How to run:** - **Dev (root):** `npm run dev` → open http://localhost:5173/ - **Build:** `npm run build` → output in `dist/` - **Preview (root):** `npm run preview` → open http://localhost:4173/ - **Subpath:** Set `base: '/subpath/'` in config, then build and preview (or run dev with that base) and open the app at `http://localhost:5173/subpath/` or the preview URL with `/subpath/`. **Quick reference:** | Framework | Install | Dev (root) | Build | Preview / prod-like | | ------------ | ------------- | ------------- | --------------- | -------------------------------------- | | Vue (Vite) | `npm install` | `npm run dev` | `npm run build` | `npm run preview` | | Angular | `npm install` | `npm start` | `npm run build` | `npm run preview` (builds then serves) | | React (Vite) | `npm install` | `npm run dev` | `npm run build` | `npm run preview` | For subpath, use the `dev:subpath` / `build:subpath` / `preview:subpath` scripts where available (Vue, Angular) or set `base` in Vite config (React/Vue). --- #### Codec WASM location Each decoder locates its WASM binary with a bare `@cornerstonejs/codec-...` specifier inside `new URL(..., import.meta.url)`. Whether that needs any setup from you depends on the bundler: | Bundler | Behavior | | --------------------- | -------------------------------------------------------------------------------------------------------------- | | webpack 5 | Resolves the specifier through the package `exports` map and emits the binary as an asset. Nothing to do. | | Vite / Rollup (build) | Same: resolves and emits the binary (or inlines it, when under the asset inline limit). Nothing to do. | | esbuild | Does **not** resolve it. `new URL(...)` is treated as ordinary code, so the bare specifier survives the build. | Angular's `application` builder is esbuild-based, so Angular applications are the common case that needs configuration. In Vite, keep `@cornerstonejs/dicom-image-loader` out of dev-time dependency optimization (the `optimizeDeps.exclude` above), because prebundling runs esbuild — that is the same limitation seen from the dev server. When the specifier is not resolved, the request goes to a path that does not exist — usually answered by the SPA fallback, which surfaces as: ``` CompileError: WebAssembly.instantiate(): expected magic word 00 61 73 6d, found 3c 21 64 6f ``` (`3c 21 64 6f` is ` { // resolve fs for one of the dependencies config.resolve.fallback = { fs: false, }; return config; }, }; export default nextConfig; ``` #### Advanced Setup (PolySeg & Labelmap Interpolation) You might need to add ```js ``` #### Troubleshooting #### 1. Rollup Options By default, we don't include the `@icr/polyseg-wasm`, `itk-wasm`, and `@itk-wasm/morphological-contour-interpolation` libraries in our bundle to keep the size pretty small. Rollup **might** complain about these libraries, so you can add the following to the rollupOptions: ```js worker: { format: "es", rollupOptions: { external: ["@icr/polyseg-wasm"], }, }, ``` #### 2. Path Resolution Issues with @cornerstonejs/core If you encounter the error "No known conditions for "./types" specifier in "@cornerstonejs/core" package" during build (while development works fine), add the following alias to your Vite configuration: ```javascript resolve: { alias: { '@': fileURLToPath(new URL('./src', import.meta.url)), '@root': fileURLToPath(new URL('./', import.meta.url)), "@cornerstonejs/core": fileURLToPath(new URL('node_modules/@cornerstonejs/core/dist/esm', import.meta.url)), }, }, ``` #### 3. Tool Name Minification Issues If you experience issues with tool names being minified (e.g., LengthTool being registered as "FE"), you can prevent minification by adding: ```javascript build: { minify: false, } ``` :::note These solutions have been tested primarily on macOS but may also apply to other operating systems. If you're using Vuetify or other Vue frameworks, these configurations might need to be adjusted based on your specific setup. ::: #### 4. Webpack For webpack, simply install the cornerstone3D library and import it into your project. If you previously used `noParse: [/(codec)/],` to avoid parsing codecs in your webpack module, remove that line. The cornerstone3D library now includes the codecs as an ES module. Also since we are using wasm, you will need to add the following to your webpack configuration in the `module.rules` section: ```javascript { test: /\.wasm/, type: 'asset/resource', }, ``` #### 5. Svelte + Vite Similar to the configuration above, use the CommonJS plugin converting commonjs to esm. Otherwise, it will be pending at `await viewport.setStack(stack);`, the image will not be rendered. ```javascript import { defineConfig } from 'vite'; import { svelte } from '@sveltejs/vite-plugin-svelte'; import { viteCommonjs } from '@originjs/vite-plugin-commonjs'; export default defineConfig({ plugins: [svelte(), viteCommonjs()], optimizeDeps: { exclude: ['@cornerstonejs/dicom-image-loader'], include: ['dicom-parser'], }, }); ``` :::note Tip If you are using `sveltekit`, and config like `plugins: [ sveltekit(), viteCommonjs() ]`, `viteCommonjs()` may not work. Try replace `sveltekit` with `vite-plugin-svelte` and it will work. --- # How-to-guides ## Configuration Source: https://cornerstonejs.org/docs/llm/how-to-guides/configuration.md #### Configuration Cornerstone Core accepts configuration through its `init` function. Pass the configuration on the first call to `init`; subsequent calls return immediately after Cornerstone has been initialized. #### Logging Cornerstone writes its messages with [loglevel](https://github.com/pimterry/loglevel), through the same log root as dcmjs. Configure the logs with the loglevel interface, not with the `init` configuration. Thus one interface controls the logs of Cornerstone, of dcmjs, and of the other components of your application. ```ts import { logging } from '@cornerstonejs/utils'; logging.log.getLogger('cs3d.dicomImageLoader.wadouri').setLevel('info'); ``` The levels, from the most messages to the fewest messages, are: - `trace` - `debug` - `info` - `warn` - `error` - `silent` #### Logger names A logger name has this structure: ``` cs3d... ``` - `cs3d` is the root of all Cornerstone logs. - `` is the package, for example `core`, `tools`, `dicomImageLoader`, `niftiVolumeLoader`, `polymorphicSegmentation`, `labelmapInterpolation`, `adapters`, `metadata` or `ai`. - `` is the folder in the source of that package. It can have more than one part, and a file at the root of the package has no path. - `` is the file, or the area of the code when the messages come from more than one file. Examples of names: - `cs3d.core.RenderingEngine.StackViewport` - `cs3d.core.utilities.VoxelManager` - `cs3d.dicomImageLoader.wadouri` - `cs3d.adapters.Cornerstone3D.MeasurementReport` Two loggers are not below `cs3d`: `consistency.dicom` and `consistency.image`. These loggers are on the root of dcmjs, because dcmjs writes the same consistency messages. #### Set a level Logger names are independent strings; dots in a name do not create a parent and child relationship. Setting a named logger's level therefore affects only that exact logger. To find the loggers that exist, use `logging.log.getLoggers()`. ```ts import { logging } from '@cornerstonejs/utils'; // One exact logger logging.log.getLogger('cs3d.core.RenderingEngine').setLevel('debug'); // All existing loggers whose names start with an area prefix const prefix = 'cs3d.core.RenderingEngine'; Object.entries(logging.log.getLoggers()).forEach(([name, logger]) => { if (name === prefix || name.startsWith(`${prefix}.`)) { logger.setLevel('debug'); } }); // Update the root, then update existing loggers that do not have their own level logging.log.setLevel('warn'); logging.log.rebuild(); ``` You can set a level at any time, and the new level is immediately applicable. Named loggers created after a root level change inherit the new root level. Named loggers that already exist keep their inherited level until you call `logging.log.rebuild()`. Thus you can also give this control to your users, for example in a menu for support. #### Send the logs to a different location loglevel lets you replace the function that makes each log method. Use this to send the messages to a server, to a file, or to the interface of your application. Call `setLevel` after you replace the function, because the new methods are made at that time. ```ts import { logging } from '@cornerstonejs/utils'; const logger = logging.log.getLogger('cs3d.core.RenderingEngine'); const originalFactory = logger.methodFactory; logger.methodFactory = (methodName, level, loggerName) => { const originalMethod = originalFactory(methodName, level, loggerName); return (...args) => { originalMethod(...args); if (methodName === 'warn' || methodName === 'error') { myTelemetry.send({ logger: String(loggerName), methodName, args }); } }; }; logger.setLevel(logger.getLevel()); ``` --- ## Custom Image Loader Source: https://cornerstonejs.org/docs/llm/how-to-guides/custom-imageLoader.md #### Custom Image Loader In this how-to guide we will show you how to create a custom image loader. You should be familiar with the following core concepts: - [Image Loaders](../concepts/cornerstone-core/imageLoader.md) - [Image Objects](../concepts/cornerstone-core/images.md) - [Metadata Providers](../concepts/cornerstone-core/metadataProvider.md) #### Introduction Cornerstone **DOES NOT** deal with image loading. It delegates image loading to [Image Loaders](../concepts/cornerstone-core/imageLoader.md). Cornerstone team have developed commonly used image loaders (`CornerstoneDICOMImageLoader` for loading images from wado-compliant dicom servers using `wado-rs` or `wado-uri`, `CornerstoneWebImageLoader` to load web images such as PNG and JPEG and `cornerstone-nifti-image-loader` for loading NIFTI images). However, you might ask yourself: :::note How How can I build a custom image loader? ::: #### Implementation Let's implement an `imageLoader` that fetches pixel data using `XMLHttpRequest` and return an Image Load Object containing a Promise that resolves to a Cornerstone [`image`](../concepts/cornerstone-core/images.md). #### Step 1: Create an Image Loader Below, we create an `imageLoader` which accepts an `imageId` and returns an `imageLoadObject` as a Promise. ```js function loadImage(imageId) { // Parse the imageId and return a usable URL (logic omitted) const url = parseImageId(imageId); // Create a new Promise const promise = new Promise((resolve, reject) => { // Inside the Promise Constructor, make // the request for the image data const oReq = new XMLHttpRequest(); oReq.open('get', url, true); oReq.responseType = 'arraybuffer'; oReq.onreadystatechange = function (oEvent) { if (oReq.readyState === 4) { if (oReq.status == 200) { // Request succeeded, Create an image object (logic omitted) // This may require decoding the image into raw pixel data, determining // rows/cols, pixel spacing, etc. const image = createImageObject(oReq.response); // Return the image object by resolving the Promise resolve(image); } else { // An error occurred, return an object containing the error by // rejecting the Promise reject(new Error(oReq.statusText)); } } }; oReq.send(); }); // Return an object containing the Promise to cornerstone so it can setup callbacks to be // invoked asynchronously for the success/resolve and failure/reject scenarios. return { promise, }; } ``` #### Step 2: Ensure Image metadata is also available Our image loader returns an `imageLoadObject` containing pixel data and related information, but Cornerstone may also need [additional metadata](../concepts/cornerstone-core/metadataProvider.md) in order to display the image. See the [custom metadata provider](custom-metadata-provider.md) documentation for how to do this. #### Step 3: Registration of Image Loader After you implement your image loader, you need to register it with Cornerstone. First you need to decide which URL scheme your image loader supports. Let's say your image loader wants to support the `custom1` scheme, then any imageId that starts with `custom1://` will be handled by your image loader. ```js // registration cornerstone.imageLoader.registerImageLoader('custom1', loadImage); ``` #### Usage ```js // Images loaded as follows will be passed to our loadImage function: stackViewport.setStack(['custom1://example.com/image.dcm']); ```
Use Viewport API to load an image In previous versions of Cornerstone, you could use `loadImage` or `loadAndCacheImage` to load an image. However, in `Cornerstone3D`, this task can be achieved using `Viewports` APIs.
--- ## Custom Metadata Provider Source: https://cornerstonejs.org/docs/llm/how-to-guides/custom-metadata-provider.md #### Custom Metadata Provider In this how-to guide we will show you how to create a custom metadata provider. You should be familiar with the following core concepts: - [Image Loaders](../concepts/cornerstone-core/imageLoader.md) - [Image Objects](../concepts/cornerstone-core/images.md) - [Metadata Providers](../concepts/cornerstone-core/metadataProvider.md) #### Introduction Cornerstone **DOES NOT** deal with fetching of the metadata. It uses the registered metadata providers (in the order of priority) to call each providers passing the `imageId` and `type` of the metadata to be fetched. Usually, the metadata provider has a method to add parsed metadata to its cache. One question you might ask is: :::note How How can I build a custom metadata provider? ::: #### Implementation Through the following steps, we implement a custom metadata provider that stores the metadata for scaling factors of PT images. #### Step 1: Create an add method We need to store the metadata in a cache, and we need a method to add the metadata. ```js const scalingPerImageId = {}; function add(imageId, scalingMetaData) { const imageURI = csUtils.imageIdToImageURI(imageId); scalingPerImageId[imageURI] = scalingMetaData; } ```
imageId vs imageURI With the addition of `Volumes` in `Cornerstone3D`, and the caching optimizations that happen internally between `Volumes` and `Images` ([`imageLoader`](../concepts/streaming-image-volume/streaming.md#imageloader)) we should store the imageURI (instead of the `imageId`) inside the provider's cache, since the imageURI is unique for each image but can be retrieved with different loading schemes.
#### Step 2: Create a provider Next, a provider function is needed, to get the metadata for a specific imageId given the type of metadata. In this case, the provider only cares about the `scalingModule` type, and it will return the metadata for the `imageId` if it exists in the cache. ```js function get(type, imageId) { if (type === 'scalingModule') { const imageURI = csUtils.imageIdToImageURI(imageId); return scalingPerImageId[imageURI]; } } ``` #### Step 3: Register the provider Finally, we need to register the provider with cornerstone. ```js title="/src/myCustomProvider.js" const scalingPerImageId = {}; function add(imageId, scalingMetaData) { const imageURI = csUtils.imageIdToImageURI(imageId); scalingPerImageId[imageURI] = scalingMetaData; } function get(type, imageId) { if (type === 'scalingModule') { const imageURI = csUtils.imageIdToImageURI(imageId); return scalingPerImageId[imageURI]; } } export { add, get }; ``` ```js title="src/registerProvider.js" import myCustomProvider from './myCustomProvider'; const priority = 100; cornerstone.metaData.addProvider( myCustomProvider.get.bind(myCustomProvider), priority ); ``` #### Usage Example Now that the provider is registered, we can use it to fetch the metadata for an image. But first, let's assume during the image loading we fetch the metadata for the imageId and store it in the cache of the provider. Later, we can use the provider to fetch the metadata for the imageId and use it (e.g., to properly show SUV values for tools). ```js // Retrieve this metaData const imagePlaneModule = cornerstone.metaData.get( 'scalingModule', 'scheme://imageId' ); ``` --- ## Custom Tools Source: https://cornerstonejs.org/docs/llm/how-to-guides/custom-tools.md #### Custom Tools A Cornerstone Tool is any class that implements or extends the interface defined by the `BaseTool` or `AnnotationTool` abstract classes. Creating a custom tool is as simple as: ```js import csTools3d, { AnnotationTool, BaseTool } from '@Tools` class MyCustomTool extends BaseTool { // ... } csTools3d.addTool(MyCustomTool, { /* Tool Options */ }) ``` #### BaseTool A base tool has a name, configuration, options, strategies, bindings, and more. Base tools are often used to respond to user input and effect some change on the viewport (like its camera). Example `BaseTool`s include: - Pan - PetThreshold - StackScroll - StackScrollMouseWheel - WindowLevel - Zoom #### AnnotationTool An annotation tool often has "Annotations" that are tied to frame of reference. It has additional methods that allow tools to indicate they should handle/capture an interaction. This is most often used for "interactions near a handle" or "interactions near a rendered tool line". Annotation tools that are in the `Active` mode have an `addNewAnnotation` method that's called when a mouse event is not captured. This allows the active tool to create Annotations for the interaction. Example `AnnotationTool`s include: - Bidirectional - EllipticalROI - CircleROI - Length - Probe - RectangleROI - PlanarFreehandROI #### Next steps For next steps, you can: - [Check out the Usage documentation](#) - [Explore our example application's source code](#) --- ## Custom Volume Loading Order Source: https://cornerstonejs.org/docs/llm/how-to-guides/custom-volume-loading-order.md #### Custom Volume Loading Order In this how-to guide we will show you how to load a volume in a custom order. #### Introduction `Volumes` can be made from a set of 2D images, one question you might ask is: :::note How How can I re-order the image requests (top-down, bottom-up, etc.) in a volume loading process? ::: #### Implementation Let's re-order two volume loadings so that they load their slice together (instead of one volume after the other). To create a custom volume loading order, we need to get the `imageLoadRequests` from the volume objects and sort them in a custom order. #### Step 1: Create a Volume We create a volume similar to previous tutorials out of set of `imageIds` ```js const ptVolume = await volumeLoader.createAndCacheVolume(ptVolumeId, { imageIds: ptImageIds, }); const ctVolume = await volumeLoader.createAndCacheVolume(ctVolumeId, { imageIds: ctVolumeImageIds, }); ``` #### Step 2: Getting imageLoad requests Next, we need to get the imageLoad requests ```js const ctRequests = ctVolume.getImageLoadRequests(); const ptRequests = ptVolume.getImageLoadRequests(); ``` #### Step 3: Custom ordering of requests We use lodash helpers to merge the requests together in one after the other fashion. ```js import _ from 'lodash'; const ctPtRequests = _.flatten(_.zip(ctRequests, ptRequests)).filter( (el) => el ); ``` #### Step 4: Add requests back to imageLoadPoolManager We need to add back the requests to the `imageLoadPoolManager` (we need to take care of the values to be bound to the `callLoadImage` too). ```js ctPtRequests.forEach((request) => { const { callLoadImage, requestType, additionalDetails, priority, imageId, imageIdIndex, options, } = request; imageLoadPoolManager.addRequest( callLoadImage.bind(null, imageId, imageIdIndex, options), requestType, additionalDetails, priority ); }); ``` :::note Tip There is no need to call `volume.load` since this method basically does the same process as our steps 3 and 4. ::: #### Results ![customLoading](../assets/custom-loading.gif) --- # Migration-guides ## Legacy to Cornerstone3D 1.0 Source: https://cornerstonejs.org/docs/llm/migration-guides/legacy-to-3d.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### Legacy to 1.0 As we are moving to the `Cornerstone3D` library, we are introducing a new set of APIs that are not backwards compatible with the old `Cornerstone` library. In this page, we will provide a migration guide for users who are already using the old `Cornerstone` library. :::note Important Please note that this is a work in progress and we are still working on completing the migration guides. ::: #### init `Cornerstone` (legacy) didn't need to be initialized, but `CornerstoneTools` (legacy) should have been initialized. In `Cornerstone3D` both core and tools should be initialized before using the libraries. ```js cornerstoneTools.init(); ``` ```js // detects gpu and decides whether to use gpu rendering or cpu fallback cornerstone3D.init(); cornerstone3DTools.init(); ``` #### enabledElement Enabled elements in `Cornerstone3D` don’t exist in isolation as in Cornerstone. When setting a layout, elements are tied to a rendering engine as output targets. When this happens they are considered “enabled”. In `Cornerstone3D` we have two APIs for this: - `setViewports`: enables a list of viewports at once - `enableElement`: enables one viewport at a time ```js const element = document.getElementById('div-element'); cornerstone.enable(element); // Triggers ELEMENT_ENABLED event ``` ```js const element = document.getElementById("viewport-HTML-element"); const renderingEngine = new RenderingEngine(); // API1: set Viewports renderingEngine.setViewports([ { viewportId: "CTAxial", type: ViewportType.ORTHOGRAPHIC, element, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }, ]); // API2: Enable Element renderingEngine.enableElement({ viewportId: "CTAxial", type: ViewportType.ORTHOGRAPHIC, element, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }); // ELEMENT_ENABLED eventDetail includes: { element, viewportId, renderingEngineId, } ``` #### loadAndCacheImage In Cornerstone (legacy), you would load an image and cache it using the `loadAndCacheImage` API. However, in `Cornerstone3D` you should use the viewports API to load and cache images. ```js cornerstone.loadAndCacheImage(imageId).then((image) => { // Do things, e.g. display an image }); ``` ```js const viewport = renderingEngine.getViewport('CTViewport'); // one image in the stack await viewport.setStack([imageId]); // multiple imageIds await viewport.setStack( [imageId1, imageId2], 1 // frame 1 ); ``` #### displayImage This is a bit different in that you now set the data per viewport rather than per element, as in Cornerstone. When a viewport is later rendered, you are returned the viewport instance, which has helpers to access the HTML element, the renderer, etc. ```js cornerstone.displayImage(image, element); // Triggers cornerstone.events.IMAGE_RENDERED // with eventDetail as follows const eventDetail = { viewport: enabledElement.viewport, element, image, enabledElement, canvasContext: enabledElement.canvas.getContext('2d'), renderTimeInMs, }; ``` ```js // We gave the example for setting stack in the previous section on `loadAndCacheImage`, // here we give example for the volume // Define a set of imageIds as a volume. const ctVolume = await cornerstone3D.volumeLoader.createAndCacheVolume( volumeId, { imageIds: volumeImageIds} ) // Load the volume, the callback is called for each imageId ctVolume.load(callback) // Where eventDetail passed to the callback is (currently) of the form: // Success: { success: true, imageIdIndex, // The in-volume Z index imageId, // The imageId framesLoaded, // The total number of frames successfully loaded framesProcessed, // The total number of frames processed (successes + failures) numFrames, // The total number of frames in the volume. } // Failure: { success: false, imageId, imageIdIndex, framesLoaded, framesProcessed, numFrames, error, // The error given by the imageLoader } ``` #### updateImage We have effectively the same approach right now, but we have three different helpers that can be called to render: - All viewports associated with a rendering engine. - A single viewport. These are useful convenience helpers when using tools that may affect multiple viewports that all need to update (e.g. jump to a crosshair position on all three orthogonal MPR views). ```js cornerstone.updateImage(element, invalidated); ``` ```js // Updates every viewport in the rendering engine. renderingEngine.render() // Update a single viewport const myViewport = myScene.getViewport('myViewportId') myViewport.render() // on IMAGE_RENDERED event fired for all viewports: eventDetail: { viewport, } ``` #### disable The rendering engine controls when viewports are enabled/disabled, and will fire appropriate events as needed. ```js cornerstone.disable(element); // Triggers ELEMENT_DISABLED event ``` ```js renderingEngine.disableElement(element); // element disabled event will be fired for each canvas not retained. // OR //this will destroy all elements. renderingEngine.destroy(); // The ELEMENT_DISABLED event contains just a reference to the canvas element which is now disabled, and related IDs. eventDetail: { (viewportId, renderingEngineId, canvas); } ``` #### pageToPixel and pixelToCanvas We are no longer rendering a single image at a time. In `Cornerstone3D`, the viewport renders a specific plane in 3D space, determined by the camera parameters (e.g. focal point, frustum, clipping range). Data and annotations will be stored in 3D space ('world space', per frame of reference), and so in order to interact with, and render representations of annotations on the screen, you need to be able to convert between canvas space and world space. It should be noted that, in order to share tools between Stack and Volume viewports, we also render StackViewports in 3D space. So basically, they are 2D images positioned and oriented based on their metadata in space. ```js // Coordinate mapping functions cornerstone.pageToPixel(element, pageX, pageY); cornerstone.pixelToCanvas(element, { x, y }); ``` ```js const canvasCoord = viewport.canvasToWorld([xCanvas, yCanvas]); const worldCoord = viewport.worldToCanvas([xWorld, yWorld, zWorld]); ``` #### getPixels The `getPixels` approach is no longer valid in 3D, as you may be viewing the data at any (oblique) plane. Additionally, the viewport may be rendering a fusion with more than one volume (e.g., PET/CT) in it. The developer must now fetch the data array itself and use this data as necessary for their specific use case. ```js cornerstone.getPixels(element, x, y, width, height); ``` ```js const { dimensions, direction, spacing, origin, scalarData, imageData, metadata, } = viewport.getImageData(); /** * * You can grab the vtkImageData to get pixel information * * - `dimensions` - The x,y,z dimensions of the volume * - `spacing` - The x,y,z spacing of the volume * - `origin` - The x,y,z position of the center of the first voxel. * - `direction` - The row, column and normal direction cosines. * - `imageData` - The underlying vtkImageData object (The tenderable object used in the underlying vtk.js rendering library). * - `scalarData` - This a single TypedArray (e.g. Float32Array) which contains all of the voxel values for the volume. Through the VTK AP this could also be accessed using getScalars() from the vtkDataArray underlying the vtkImageData object. * */ ``` #### events The following table demonstrates some expected schema changes for events. The key differences are that: - Several IDs will function as lookup keys for core API methods (renderingEngineId, viewportId, volumeId). This is similar to the `enabledElement` property currently provided in custom events, and can be used to obtain all of the imaging data that is being visualized. - Snapshots of state at time of interaction return camera properties and coordinates in world space within the viewports's frame of reference.
CornerstoneTools CornerstoneTools3D Explanation for schema change
N/A renderingEngineId The Id of the rendering engine instance driving the viewport.
N/A viewportId The Id of the viewport itself.
```js viewport: { scale, translation: { x, y }, voi: { windowWidth, windowCenter, windowWidth, windowCenter}, invert, pixelReplication, rotation, hflip, vflip, modalityLUT, voiLUT, colormap, labelmap, displayedArea: { tlhc: { x, y }, brhc: { x, y }, rowPixelSpacing, columnPixelSpacing, presentationSizeMode: 'NONE' } } ```
```js camera: { (viewUp, viewPlaneNormal, position, focalPoint, orthogonalOrPerspective, viewAngle); } ```
The viewport previously described the state in 2D, and we need additional information to uniquely define 3D views. Horizontal and vertical flipping is no longer a change to the view, but rather a transform applied to the volume actor itself in the scene.
```js // Location in 2D within the image startPoints / lastPoints / currentPoints / deltaPoints: { Page, Image, Client, } ```
```js // Location in 3D in world space { (CanvasCoord, WorldCoord); } ```
The canvas coordinates define where on the 2D canvas the event occurred. We also give the projected world coordinate (3D) at the plane defined by the focal point and the camera normal.
--- ## 2x ### General Source: https://cornerstonejs.org/docs/llm/migration-guides/2x/1-general.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### General #### Video Guide Watch this video guide for a [visual walkthrough](https://www.youtube.com/embed/tkQiVLftpuI?si=HbFitXWowvlndI0i) of the migration process: #### Frameworks We have worked hard to enhance the developer experience when using Cornerstone3D with various frameworks like React, Vue, Angular, Vite, and Webpack. For more information, please refer to the [frameworks](../../getting-started/vue-angular-react-vite.md) page. You need to modify your Vite and Webpack configurations to correctly import the Cornerstone3D library. Check each framework's repository for more details. #### Removal of SharedArrayBuffer We have streamlined the process of loading volumes without sacrificing speed by eliminating the need for shared array buffers. This change resolves issues across various frameworks, where previously, specific security headers were required. Now, you can remove any previously set headers, which lowers the barrier for adopting Cornerstone 3D in frameworks that didn't support those headers. Shared array buffers are no longer necessary, and all related headers can be removed. You can remove `Cross-Origin-Opener-Policy` and `Cross-Origin-Embedder-Policy` from your custom headers if you don't need them in other aspects of your app. #### Typescript Version We have upgraded the typescript version from 4.6 to 5.5 in the 2.0 version of the cornerstone3D. This upgrade most likely don't require any changes in your codebase, but it is recommended to update the typescript version in your project to 5.5 to avoid any issues in the future.
Why? The upgrade to TypeScript 5.4 allows us to leverage the latest features and improvements offered by the TypeScript standard. You can read more about it here: https://devblogs.microsoft.com/typescript/announcing-typescript-5-5/
#### ECMAScript Target In Cornerstone3D version 1.x, we targeted ES5. With the release of version 2.0, we have updated our target to `ES2022`.
Why? It will result in a smaller bundle size and improved performance. There is a good chance that your setup already supports ES2022: https://compat-table.github.io/compat-table/es2016plus/
#### Remove of CJS, only ESM builds Starting with Cornerstone3D 2.x, we will no longer ship the CommonJS (CJS) and UMD builds of the library. You most likely won't need to make any changes to your codebase. If you are aliasing the cjs library in your bundler, you can remove it completely.
Why? Both Node.js and modern browsers now support ECMAScript Modules (ESM) by default.
:::note Tip If you must use CJS, for example, if you are using `dicom-image-loader` and `dicom-parser`, you need to use `vite-plugin-commonjs` to convert CommonJS to ESM. For more information, please refer to the [Frameworks](../../getting-started/vue-angular-react-vite.md) page. ::: #### Package Exports The Cornerstone libraries now utilize the `exports` field in their `package.json` files. This allows for more precise control over how modules are imported and ensures compatibility with different build systems. Below are examples of how to import modules from each package, along with explanations of the `exports` field configuration.
@cornerstonejs/adapters ```json { "exports": { ".": { "import": "./dist/esm/index.js", "types": "./dist/esm/index.d.ts" }, "./cornerstone": { "import": "./dist/esm/adapters/Cornerstone/index.js", "types": "./dist/esm/adapters/Cornerstone/index.d.ts" }, "./cornerstone/*": { "import": "./dist/esm/adapters/Cornerstone/*.js", "types": "./dist/esm/adapters/Cornerstone/*.d.ts" }, "./cornerstone3D": { "import": "./dist/esm/adapters/Cornerstone3D/index.js", "types": "./dist/esm/adapters/Cornerstone3D/index.d.ts" }, "./cornerstone3D/*": { "import": "./dist/esm/adapters/Cornerstone3D/*.js", "types": "./dist/esm/adapters/Cornerstone3D/*.d.ts" }, "./enums": { "import": "./dist/esm/adapters/enums/index.js", "types": "./dist/esm/adapters/enums/index.d.ts" } // ... other exports } } ``` **Import Examples:** ```js import * as cornerstoneAdapters from '@cornerstonejs/adapters'; // Imports the main entry point import * as cornerstoneAdapter from '@cornerstonejs/adapters/cornerstone'; // Imports the Cornerstone adapter import { someModule } from '@cornerstonejs/adapters/cornerstone/someModule'; // Imports a specific module from the Cornerstone adapter import * as cornerstone3DAdapter from '@cornerstonejs/adapters/cornerstone3D'; // Imports the Cornerstone3D adapter // ... other imports ```
@cornerstonejs/core ```json { "exports": { ".": { "import": "./dist/esm/index.js", "types": "./dist/esm/index.d.ts" }, "./utilities": { // Subpath export "import": "./dist/esm/utilities/index.js", "types": "./dist/esm/utilities/index.d.ts" }, "./utilities/*": { // Wildcard subpath export "import": "./dist/esm/utilities/*.js", "types": "./dist/esm/utilities/*.d.ts" } // ... other exports } } ``` **Import Examples:** ```js import * as cornerstoneCore from '@cornerstonejs/core'; // Imports the main entry point import * as utilities from '@cornerstonejs/core/utilities'; // Imports the utilities module import { someUtility } from '@cornerstonejs/core/utilities/someUtility'; // Imports a specific utility // ... other imports ```
@cornerstonejs/tools ```json { "exports": { ".": { "import": "./dist/esm/index.js", "types": "./dist/esm/index.d.ts" }, "./tools": { // Subpath export for tools "import": "./dist/esm/tools/index.js", "types": "./dist/esm/tools/index.d.ts" }, "./tools/*": { // Wildcard subpath export for tools "import": "./dist/esm/tools/*.js", "types": "./dist/esm/tools/*.d.ts" } // ... other exports } } ``` **Import Examples:** ```js import * as cornerstoneTools from '@cornerstonejs/tools'; // Imports the main entry point import * as tools from '@cornerstonejs/tools/tools'; // Imports the tools module import { someTool } from '@cornerstonejs/tools/tools/someTool'; // Imports a specific tool // ... other imports ```
@cornerstonejs/dicom-image-loader ```json { "exports": { ".": { "import": "./dist/esm/index.js", "types": "./dist/esm/index.d.ts" }, "./imageLoader": { // Subpath export for the image loader "import": "./dist/esm/imageLoader/index.js", "types": "./dist/esm/imageLoader/index.d.ts" } // ... other exports } } ``` **Import Examples:** ```js import * as dicomImageLoader from '@cornerstonejs/dicom-image-loader'; // Imports the main entry point import * as imageLoader from '@cornerstonejs/dicom-image-loader/imageLoader'; // Imports the imageLoader module specifically // ... other imports ```
#### cloneDeep The `structuredClone` function has replaced the previous method. You don't need to make any changes to your codebase that uses Cornerstone3D.
Why? Why to depend on a third-party library when we can use the native browser API?
--- #### --- ### @cornerstonejs/streaming-image-volume-loader Source: https://cornerstonejs.org/docs/llm/migration-guides/2x/2-streaming-loader.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### @cornerstonejs/streaming-image-volume-loader After years of development on Cornerstone3D, we recognized that volume loading should be treated as a first-class feature rather than a separate library. As a result, we have merged all functionality related to streaming image loading into the core library. 1. **Removal of Separate Library**: The `@cornerstonejs/streaming-image-volume-loader` package has been removed. 2. **Integration into Core**: All streaming image volume loading functionality is now part of the `@cornerstonejs/core` package. #### How to Migrate: If you were previously using `@cornerstonejs/streaming-image-volume-loader`, you'll need to update your imports and potentially adjust your code to use the new integrated volume loading API in `@cornerstonejs/core`. ```js import { cornerstoneStreamingImageVolumeLoader, cornerstoneStreamingDynamicImageVolumeLoader, StreamingImageVolume, StreamingDynamicImageVolume, helpers, Enums, } from '@cornerstonejs/streaming-image-volume-loader'; Enums.Events.DYNAMIC_VOLUME_TIME_POINT_INDEX_CHANGED; ``` ```js import { cornerstoneStreamingImageVolumeLoader, cornerstoneStreamingDynamicImageVolumeLoader, StreamingImageVolume, StreamingDynamicImageVolume, } from '@cornerstonejs/core'; import { getDynamicVolumeInfo } from '@cornerstonejs/core/utilities'; import { Enums } from '@cornerstonejs/core/enums'; Enums.Events.DYNAMIC_VOLUME_TIME_POINT_INDEX_CHANGED; ``` --- ### @cornerstonejs/core Source: https://cornerstonejs.org/docs/llm/migration-guides/2x/3-core.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### @cornerstonejs/core #### Initialization #### Removal of `detect-gpu` and `detectGPUConfig` Cornerstone3D 2.x has removed the dependency on `detect-gpu`. This change addresses issues reported by users working in environments where internet access is restricted, as `detect-gpu` relied on internet connectivity to determine GPU models. #### Key Changes: 1. **Default GPU Tier**: We now use a default GPU tier of 2 (medium tier). 2. **No Internet Dependency**: The library no longer requires internet access for GPU detection. 3. **Configurable GPU Tier**: You can still configure your own GPU tier if needed. #### How to Migrate: If you were previously relying on `detect-gpu` for GPU tier detection, you'll need to update your initialization code. Here's an example of how to initialize Cornerstone3D with a custom GPU tier: ```js cornerstone3D.init({ gpuTier: 3 }); ``` #### removal of `use16BitDataType` This flag requested 16-bit data type from the web worker. Now, we always use the native data type for cache storage and convert it for rendering when necessary. #### removal of `enableCacheOptimization` It is no longer needed since we automatically optimize cache for you. #### Volume Viewports Actor UID, ReferenceId, and VolumeId #### Previous Behavior When adding a volume to volume viewports, the logic used to determine the actor's UID was as follows: ```js const uid = actorUID || volumeId; volumeActors.push({ uid, actor, slabThickness, referenceId: volumeId, }); ``` In this setup, the actor UID and `referenceId` were both set to the `volumeId`. This was problematic because it created actors with identical UIDs, even when they should have been unique. Throughout the codebase, we relied on `actor.uid` to retrieve volumes from the cache, which added further confusion. #### Updated Behavior We’ve made the following changes to improve clarity and functionality. The actor UID is now distinct, using this logic: ```js const uid = actorUID || uuidv4(); volumeActors.push({ uid, actor, slabThickness, referencedId: volumeId, }); ``` #### Key Changes 1. **Unique Actor UID**: The actor UID is now always a unique identifier (`uuidv4()`), while the `referencedId` is set to the `volumeId`. If your code relied on `actor.uid` to retrieve volumes, you should now use `referencedId` or the new `viewport.getVolumeId()` method to get the `volumeId`—which is the preferred approach. 2. **Renaming `referenceId` to `referencedId`**: To improve clarity, `referenceId` has been renamed to `referencedId`. This change aligns with our library’s naming conventions, such as `referencedImageId` and `referencedVolumeId`. Since an actor can be derived from either a volume or an image, using the term `referencedId` provides a more accurate description of its role. These changes should make the logic easier to follow and prevent issues with duplicate UIDs. #### Migrations ```js const defaultActor = viewport.getDefaultActor(); const volumeId = defaultActor.uid; const volume = cache.getVolume(volumeId); ``` or ```js volumeId = viewport.getDefaultActor()?.uid; cache.getVolume(volumeId)?.metadata.Modality; ``` or ```js const { uid: volumeId } = viewport.getDefaultActor(); ``` ```js const volume = cache.getVolume(viewport.getVolumeId()); ``` #### Viewport APIs #### ImageDataMetaData ```js interface ImageDataMetaData { // ... other properties numComps: number; // ... other properties } ``` ```js export interface ImageDataMetaData { // ... other properties numberOfComponents: number; // ... other properties } ``` #### Reset Camera Previously, we had a `resetCamera` method that took positional arguments. Now it takes an object argument. ```js viewport.resetCamera(false, true, false); ``` ```js viewport.resetCamera({ resetZoom: true, resetPan: false, resetToCenter: false, }); ``` #### Rotation The `rotation` property has been removed from `getProperties` and `setProperties`, and has moved to `getViewPresentation` and `setViewPresentation` or `getCamera` and `setCamera`. ```js viewport.getProperties().rotation; viewport.setProperties({ rotation: 10 }); ``` ```js const { rotation } = viewport.getViewPresentation(); // or const { rotation } = viewport.getCamera(); viewport.setViewPresentation({ rotation: 10 }); // or viewport.setCamera({ rotation: 10 }); ```
Why? `rotation` is not a property of the viewport but rather a view prop. You can now access it through `getViewPresentation`.
#### getReferenceId `getReferenceId` is now `getViewReferenceId` ```js viewport.getReferenceId() -- > viewport.getViewReferenceId(); ```
Why? It is more accurate to use `getViewReferenceId` to reflect the actual function of the method since it returns view-specific information, and not about the actor reference.
#### New PixelData Model and VoxelManager The Cornerstone library has undergone significant changes in how it handles image volumes and texture management. These changes aim to improve performance, reduce memory usage, and provide more efficient data access, especially for large datasets. 1. Single Source of Truth - Previously: Data existed in both image cache and volume cache, leading to synchronization issues. - Now: Only one source of truth - the image cache. - Benefits: Improved syncing between stack and volume segmentations. 2. New Volume Creation Approach - Everything now loads as images. - Volume streaming is performed image by image. - Only images are cached in the image cache. - For volume rendering, data goes directly from image cache to GPU, bypassing CPU scalar data. - Benefits: Eliminated need for scalar data in CPU, reduced memory usage, improved performance. 3. VoxelManager for Tools - Acts as an intermediary between indexes and scalar data. - Provides mappers from IJK to indexes. - Retrieves information without creating scalar data. - Processes each image individually. - Benefits: Efficient handling of tools requiring pixel data in CPU. 4. Handling Non-Image Volumes - Volumes without images (e.g., NIFTI) are chopped and converted to stack format. - Makes non-image volumes compatible with the new image-based approach. 5. Optimized Caching Mechanism - Data stored in native format instead of always caching as float32. - On-the-fly conversion to required format when updating GPU textures. - Benefits: Reduced memory usage, eliminated unnecessary data type conversions. 6. Elimination of SharedArrayBuffer - Removed dependency on SharedArrayBuffer. - Each decoded image goes directly to the GPU 3D texture at the correct size and position. - Benefits: Reduced security constraints, simplified web worker implementation. **Results** - Streamlined data flow from image cache directly to GPU. - Improved memory usage and performance. - Enhanced compatibility with various volume formats. - Optimized overall system architecture for image and volume handling. - Simplified web worker implementation (ArrayBuffer is now sufficient). #### Introduction of VoxelManager A new `VoxelManager` class has been introduced to handle voxel data more efficiently. This change eliminates the need for allocating large scalar data arrays for volumes, instead relying on individual images and an adapter called VoxelManager. **Migration Steps:** 1. Replace direct scalar data access with `VoxelManager` methods: Instead of accessing `volume.getScalarData()`, use `volume.voxelManager` to interact with the data. 2. Scalar Data length: Use `voxelManager.getScalarDataLength()` instead of `scalarData.length`. 3. Scalar Data Manipulation: a. Use `getAtIndex(index)` and `setAtIndex(index, value)` for accessing and modifying voxel data. b. For 3D coordinates, use `getAtIJK(i, j, k)` and `setAtIJK(i, j, k, value)`. 4. Available VoxelManager Methods: - `getScalarData()`: Returns the entire scalar data array (only for IImage, not for volumes). - `getScalarDataLength()`: Returns the total number of voxels. - `getAtIndex(index)`: Gets the value at a specific index. - `setAtIndex(index, value)`: Sets the value at a specific index. - `getAtIJK(i, j, k)`: Gets the value at specific IJK coordinates. - `setAtIJK(i, j, k, value)`: Sets the value at specific IJK coordinates. - `getArrayOfModifiedSlices()`: Returns an array of modified slice indices. - `forEach(callback, options)`: Iterates over voxels with a callback function. - `getConstructor()`: Returns the constructor for the scalar data type. - `getBoundsIJK()`: Returns the bounds of the volume in IJK coordinates. - `toIndex(ijk)`: Converts IJK coordinates to a linear index. - `toIJK(index)`: Converts a linear index to IJK coordinates. 5. Handling modified slices: Use `voxelManager.getArrayOfModifiedSlices()` to get the list of modified slices. 6. Iterating over voxels: Use the `forEach` method for efficient iteration: ```javascript voxelManager.forEach( ({ value, index, pointIJK, pointLPS }) => { // Manipulate or process voxel data }, { boundsIJK: optionalBounds, imageData: optionalImageData, // for LPS calculations } ); ``` 7. Getting volume information: - Dimensions: `volume.dimensions` - Spacing: `volume.spacing` - Direction: `volume.direction` - Origin: `volume.origin` 8. For RGB data: If dealing with RGB data, the `getAtIndex` and `getAtIJK` methods will return an array `[r, g, b]`. 9. Performance considerations: - Use `getAtIndex` and `setAtIndex` for bulk operations when possible, as they're generally faster than `getAtIJK` and `setAtIJK`. - When iterating over a large portion of the volume, consider using `forEach` for optimized performance. 10. Dynamic Volumes: For 4D datasets, additional methods are available: - `setTimePoint(timePoint)`: Sets the current time point. - `getAtIndexAndTimePoint(index, timePoint)`: Gets a value for a specific index and time point. Example of migrating a simple volume processing function: ```javascript function processVolume(volume) { const scalarData = volume.getScalarData(); for (let i = 0; i < scalarData.length; i++) { if (scalarData[i] > 100) { scalarData[i] = 100; } } } ``` ```javascript function processVolume(volume) { const voxelManager = volume.voxelManager; const length = voxelManager.getScalarDataLength(); for (let i = 0; i < length; i++) { const value = voxelManager.getAtIndex(i); if (value > 100) { voxelManager.setAtIndex(i, 100); } } } ``` By following these expanded migration steps and utilizing the full capabilities of the VoxelManager, you can efficiently work with volume data while benefiting from the improved performance and reduced memory usage of the new system. **Migration Steps For Volumes (IImageVolume):** 1. When processing volume data, search your custom codebase for `getScalarData` or `scalarData`. Instead, use `voxelManager` to access the scalar data API. :::info If you can't use the atomic data API through `getAtIndex` and `getAtIJK`, you can fall back to `voxelManager.getCompleteScalarDataArray()` to rebuild the full scalar data array like cornerstone3D 1.0. However, this is not recommended due to performance and memory concerns. Use it only as a last resort. Also you can do `.setCompleteScalarDataArray` as well. ::: **Migration Steps For Stack Images (IImage):** 1. there is not much changed here for stack images, you can still use `image.getPixelData()` OR access the scalarData array from the `voxelManager` with `image.voxelManager.getScalarData()`. :::info ONLY For volumes, there is no direct `scalarData` array. Instead, use `voxelManager` to access the scalar data at index or ijk. Manipulation of scalar data for single images remains unchanged. ::: #### Image Volume Construction The construction of image volumes has been updated to use `VoxelManager` and new properties, eliminating the need for large scalar data arrays. :::info As mentioned, there is no scalarData array in the volume object, and imageIds is sufficient to describe the volume. ::: ```typescript const streamingImageVolume = new StreamingImageVolume({ volumeId, metadata, dimensions, spacing, origin, direction, scalarData, sizeInBytes, imageIds, }); ``` ```typescript const streamingImageVolume = new StreamingImageVolume({ volumeId, metadata, dimensions, spacing, origin, direction, imageIds, dataType, numberOfComponents, }); ``` **Migration Steps:** 1. Remove `scalarData` and `sizeInBytes` from the constructor parameters. 2. Add `dataType` and `numberOfComponents` to the constructor parameters. 3. The `VoxelManager` will be created internally based on these parameters. **Explanation:** This change reflects the shift from using large scalar data arrays to using the VoxelManager for data management. It allows for more efficient memory usage and better handling of streaming data. #### Accessing Volume Properties Some volume properties are now accessed differently due to the `VoxelManager` integration. The reason is we don't create the vtkScalarData fully for volume so you can't access it like before. ```typescript const numberOfComponents = imageData .getPointData() .getScalars() .getNumberOfComponents(); ``` ```typescript const { numberOfComponents } = imageData.get('numberOfComponents'); ``` **Migration Steps:** 1. Replace `getPointData().getScalars().getNumberOfComponents()` with `get('numberOfComponents')`. 2. Use the destructuring syntax to extract the `numberOfComponents` property. :::info These changes represent a significant update to the Cornerstone library's handling of image volumes and textures. The introduction of the VoxelManager and the elimination of large scalar data arrays for volumes provide several benefits: 1. Reduced memory usage: By relying on individual images instead of a large array buffer, memory usage is significantly reduced, especially for large datasets. 2. Improved performance: The VoxelManager allows for more efficient data access and manipulation, leading to better overall performance. 3. Better streaming support: The new approach is better suited for streaming large datasets, as it doesn't require loading the entire volume into memory at once. 4. More flexible data management: The VoxelManager provides a unified interface for accessing and modifying voxel data, regardless of the underlying data structure. Developers will need to update their code to use the new VoxelManager API and adjust how they interact with volume data and textures. While these changes may require significant updates to existing code, they provide a more efficient and flexible foundation for working with large medical imaging datasets. ::: We have applied this new design to both volume and stack viewports. #### Image Loader #### VolumeLoader The volume loading and caching functionality has undergone significant changes in version 2. The main updates include simplification of the API, removal of certain utility functions, and changes in the way volumes are created and cached. #### Changes in Volume Creation Functions The `createLocalVolume` function has been updated to take `volumeId` as the first parameter and options as the second parameter. ```typescript function createLocalVolume( options: LocalVolumeOptions, volumeId: string, preventCache = false ): IImageVolume { // ... } ``` ```typescript function createLocalVolume( volumeId: string, options = {} as LocalVolumeOptions ): IImageVolume { // ... } ``` **Migration Steps:** 1. Update all calls to `createLocalVolume` by moving the `volumeId` parameter to the first position. 2. Remove the `preventCache` parameter and handle caching separately if needed. #### Changes in Derived Volume Creation The `createAndCacheDerivedVolume` function now returns synchronously instead of returning a Promise. ```typescript async function createAndCacheDerivedVolume( referencedVolumeId: string, options: DerivedVolumeOptions ): Promise { // ... } ``` ```typescript function createAndCacheDerivedVolume( referencedVolumeId: string, options: DerivedVolumeOptions ): IImageVolume { // ... } ``` **Migration Steps:** 1. Remove `await` keywords when calling `createAndCacheDerivedVolume`. 2. Update any code that expects a Promise to handle the synchronous return value. #### Renamed Functions Some functions have been renamed for clarity: - `createAndCacheDerivedSegmentationVolume` is now `createAndCacheDerivedLabelmapVolume` - `createLocalSegmentationVolume` is now `createLocalLabelmapVolume` **Migration Steps:** 1. Update all calls to these functions with their new names. 2. Ensure that any code referencing these functions is updated accordingly. #### Target Buffer Type Migration The `targetBufferType` option has been replaced with a `targetBuffer` object throughout the library. This change affects multiple functions and interfaces. ```typescript interface DerivedImageOptions { targetBufferType?: PixelDataTypedArrayString; // ... } function createAndCacheDerivedImage( referencedImageId: string, options: DerivedImageOptions = { targetBufferType: 'Uint8Array', } ): Promise { // ... } function createAndCacheDerivedImages( referencedImageIds: Array, options: DerivedImageOptions & { targetBufferType?: PixelDataTypedArrayString; } = {} ): DerivedImages { // ... } ``` ```typescript interface DerivedImageOptions { targetBuffer?: { type: PixelDataTypedArrayString; }; // ... } function createAndCacheDerivedImage( referencedImageId: string, options: DerivedImageOptions = {} ): IImage { // ... } function createAndCacheDerivedImages( referencedImageIds: string[], options: DerivedImageOptions & { targetBuffer?: { type: PixelDataTypedArrayString; }; } = {} ): IImage[] { // ... } ``` **Migration Steps:** 1. Update all interfaces and function signatures that use `targetBufferType` to use `targetBuffer` instead. 2. Change all occurrences of `targetBufferType: 'SomeType'` to `targetBuffer: { type: 'SomeType' }`. 3. Update all function calls that previously used `targetBufferType` to use the new `targetBuffer` object structure. 4. Review and update any code that relies on the `targetBufferType` property, ensuring it now uses `targetBuffer.type`. #### Changes in `createAndCacheDerivedImage` Function The `createAndCacheDerivedImage` function now returns an `IImage` object directly instead of a Promise. ```typescript export function createAndCacheDerivedImage( referencedImageId: string, options: DerivedImageOptions = {}, preventCache = false ): Promise { // ... return imageLoadObject.promise; } ``` ```typescript export function createAndCacheDerivedImage( referencedImageId: string, options: DerivedImageOptions = {} ): IImage { // ... return localImage; } ``` **Migration Steps:** 1. Update any code that expects a Promise from `createAndCacheDerivedImage` to work with the directly returned `IImage` object. 2. Remove the `preventCache` parameter from function calls, as it's no longer used. #### Derived Image Creation The `createAndCacheDerivedImage` function has been updated to return an `IImage` object directly instead of a Promise. ```typescript function createAndCacheDerivedImage( referencedImageId: string, options: DerivedImageOptions = {} ): Promise { // ... } ``` ```typescript function createAndCacheDerivedImage( referencedImageId: string, options: DerivedImageOptions = {} ): IImage { // ... } ``` **Migration Steps:** 1. Remove any `await` or `.then()` calls when using `createAndCacheDerivedImage`. 2. Update error handling to catch synchronous errors instead of Promise rejections. #### Image Loading Options The `targetBufferType` option has been replaced with a `targetBuffer` object. ```typescript const options: DerivedImageOptions = { targetBufferType: 'Uint8Array', }; ``` ```typescript const options: DerivedImageOptions = { targetBuffer: { type: 'Uint8Array' }, }; ``` **Migration Steps:** 1. Replace `targetBufferType` with `targetBuffer` in all option objects. 2. Update the value to be an object with a `type` property. #### Segmentation Image Helpers The segmentation image helper functions have been renamed and updated. ```typescript function createAndCacheDerivedSegmentationImages( referencedImageIds: Array, options: DerivedImageOptions = { targetBufferType: 'Uint8Array', } ): DerivedImages { // ... } function createAndCacheDerivedSegmentationImage( referencedImageId: string, options: DerivedImageOptions = { targetBufferType: 'Uint8Array', } ): Promise { // ... } ``` ```typescript function createAndCacheDerivedLabelmapImages( referencedImageIds: string[], options = {} as DerivedImageOptions ): IImage[] { return createAndCacheDerivedImages(referencedImageIds, { ...options, targetBuffer: { type: 'Uint8Array' }, }); } function createAndCacheDerivedLabelmapImage( referencedImageId: string, options = {} as DerivedImageOptions ): IImage { return createAndCacheDerivedImage(referencedImageId, { ...options, targetBuffer: { type: 'Uint8Array' }, }); } ``` **Migration Steps:** 1. Rename `createAndCacheDerivedSegmentationImages` to `createAndCacheDerivedLabelmapImages`. 2. Rename `createAndCacheDerivedSegmentationImage` to `createAndCacheDerivedLabelmapImage`. 3. Update function calls to use the new names and parameter structure. 4. Remove any `await` or `.then()` calls when using `createAndCacheDerivedLabelmapImage`. #### Cache Class The `Cache` class has undergone significant changes in version 2. Here are the main updates and breaking changes: #### Removal of Volume-specific Cache Size The separate volume cache size has been removed, simplifying the cache management, since we only rely on the image cache solely. **Migration Steps**: 1. Remove any references to `_volumeCacheSize` if you had #### isCacheable Method Update The `isCacheable` method has been updated to consider shared cache keys. Which means since we have moved to the image cache only, we need to be careful on which images can be decached so we don't remove the volume that is still referenced by the view. #### New putImageSync and putVolumeSync Methods A new `putImageSync` method has been added to directly put an image into the cache synchronously. ```typescript // Method did not exist ``` ```typescript public putImageSync(imageId: string, image: IImage): void { // ... (validation code) ``` public putVolumeSync(volumeId: string, volume: IImageVolume): void { // ... (validation code) } **Migration Steps**: 1. Use the new `putImageSync` and `putVolumeSync` methods when you need to add an image or volume to the cache synchronously. #### Renaming and Nomenclature #### Enums #### Removal of SharedArrayBufferModes As we no longer use SharedArrayBuffer, this Enum has been removed. The following methods have also been removed from @cornerstonejs/core: - getShouldUseSharedArrayBuffer - setUseSharedArrayBuffer - resetUseSharedArrayBuffer #### ViewportType.WholeSlide -> ViewportType.WHOLE_SLIDE to match the rest of the library before ```js const viewportInput = { viewportId, type: ViewportType.WholeSlide, element, defaultOptions: { background: [0.2, 0, 0.2], }, }; renderingEngine.enableElement(viewportInput); ``` after ```js const viewportInput = { viewportId, type: ViewportType.WHOLE_SLIDE, element, defaultOptions: { background: [0.2, 0, 0.2], }, }; renderingEngine.enableElement(viewportInput); ``` #### Events and Event Details #### VOLUME_SCROLL_OUT_OF_BOUNDS -> VOLUME_VIEWPORT_SCROLL_OUT_OF_BOUNDS is now `VOLUME_VIEWPORT_SCROLL_OUT_OF_BOUNDS` #### STACK_VIEWPORT_NEW_STACK -> VIEWPORT_NEW_IMAGE_SET is now VIEWPORT_NEW_IMAGE_SET adn we will gradually bring all viewports to use this event instead in addition the event is now occurring on the element not the eventTarget ```js eventTarget.addEventListener(Events.VIEWPORT_NEW_IMAGE_SET, newStackHandler); // should be now element.addEventListener(Events.VIEWPORT_NEW_IMAGE_SET, newStackHandler); ```
Why? We made this change to maintain consistency, as all other events like VOLUME_NEW_IMAGE were occurring on the element. This modification makes more sense because when the viewport has a new stack, it should trigger an event on the viewport element itself.
#### CameraModifiedEventDetail Does not publish the `rotation` anymore, and it has moved to ICamera which is published in the event ```js type CameraModifiedEventDetail = { previousCamera: ICamera, camera: ICamera, element: HTMLDivElement, viewportId: string, renderingEngineId: string, }; ``` access the rotation from the camera object which previously was in the event detail root. #### ImageVolumeModifiedEventDetail The `imageVolume` is no longer available in the event detail. Instead, only the `volumeId` is displayed in the event details to maintain consistency with other library entries. This change ensures a uniform approach across all library content. If you need the imageVolume you can get it from the `cache.getVolume` method --- --- ### 4D or Dynamic Volume Source: https://cornerstonejs.org/docs/llm/migration-guides/2x/4-dynamic-volume.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### 4D Or Dynamic Volume We think this is important enough to have a section for itself #### imageIdsGroups is now imageIdGroups if you were using splitImageIdsBy4DTags to get the imageIdsGroups now you should expect the return object to have ImageIdGroups instead of ImageIdsGroups migration ```js const { imageIdsGroups } = splitImageIdsBy4DTags(imageIds); ``` should be ```js const { imageIdGroups } = splitImageIdsBy4DTags(imageIds); ``` #### StreamingDynamicImageVolume #### Constructor Changes The constructor signature has been updated to include `imageIdGroups` instead of separate `scalarData` arrays. ```typescript constructor( imageVolumeProperties: Types.ImageVolumeProps & { splittingTag: string }, streamingProperties: Types.IStreamingVolumeProperties ) { // ... } ``` ```typescript constructor( imageVolumeProperties: ImageVolumeProps & { splittingTag: string; imageIdGroups: string[][]; }, streamingProperties: IStreamingVolumeProperties ) { // ... } ``` **Migration Steps:** 1. Update the constructor call to include `imageIdGroups` instead of `scalarData`. 2. Remove any code that previously handled `scalarData` arrays. #### New Methods for ImageId Management Version 2 introduces new methods for managing image IDs: - `getCurrentTimePointImageIds()` - `flatImageIdIndexToTimePointIndex()` - `flatImageIdIndexToImageIdIndex()` **Migration Steps:** 1. Use `getCurrentTimePointImageIds()` to get image IDs for the current time point. 2. Utilize `flatImageIdIndexToTimePointIndex()` and `flatImageIdIndexToImageIdIndex()` for converting between flat indices and time point/image indices. #### Removal of getScalarData Method and Using VoxelManager for Dynamic Image Volumes The `getScalarData()` method has been removed in version 2 in favor of the new voxel Manager In version 2, the `StreamingDynamicImageVolume` class now uses a `VoxelManager` to handle time point data. This change provides more efficient memory management and easier access to voxel data across different time points. Here's how you can use the `VoxelManager` to access and manipulate data in your dynamic image volumes: #### Accessing Voxel Data To access voxel data for the current time point: ```typescript const voxelValue = volume.voxelManager.get(index); ``` To access voxel data for a specific time point: ```typescript const voxelValue = volume.voxelManager.getAtIndexAndTimePoint(index, timePoint); ``` #### Getting Scalar Data To get the complete scalar data array for the current time point: ```typescript const scalarData = volume.voxelManager.getCurrentTimePointScalarData(); ``` To get the scalar data for a specific time point: ```typescript const scalarData = volume.voxelManager.getTimePointScalarData(timePoint); ``` #### Getting Volume Information You can access various volume properties through the `VoxelManager`: ```typescript const scalarDataLength = volume.voxelManager.getScalarDataLength(); const dataType = volume.voxelManager.getConstructor(); const dataRange = volume.voxelManager.getRange(); const middleSliceData = volume.voxelManager.getMiddleSliceData(); ``` **Migration Steps:** 1. Replace direct access to `scalarData` arrays with calls to the appropriate `VoxelManager` methods. 2. Update any code that manually managed time points to use the `VoxelManager`'s time point-aware methods. 3. Use `getCurrentTimePointScalarData()` or `getTimePointScalarData(tp)` instead of the removed `getScalarData()` method. 4. If you need to perform operations across all time points, you can iterate through them using the `numTimePoints` property and the `getTimePointScalarData(tp)` method. By leveraging the `VoxelManager`, you can efficiently work with dynamic image volumes without manually managing multiple scalar data arrays. This approach provides better performance and memory usage, especially for large datasets with many time points. #### Exports Imports If you were previously using `@cornerstonejs/streaming-image-volume-loader`, you'll need to update your imports and potentially adjust your code to use the new integrated volume loading API in `@cornerstonejs/core`. ```js import { cornerstoneStreamingDynamicImageVolumeLoader, StreamingDynamicImageVolume, helpers, Enums, } from '@cornerstonejs/streaming-image-volume-loader'; Enums.Events.DYNAMIC_VOLUME_TIME_POINT_INDEX_CHANGED; ``` ```js import { cornerstoneStreamingDynamicImageVolumeLoader, StreamingDynamicImageVolume, } from '@cornerstonejs/core'; import { getDynamicVolumeInfo } from '@cornerstonejs/core/utilities'; import { Enums } from '@cornerstonejs/core/enums'; Enums.Events.DYNAMIC_VOLUME_TIME_POINT_INDEX_CHANGED; ``` #### getDataInTime The imageCoordinate option is now worldCoordinate, to better reflect that it's a world coordinate and not an image coordinate. ```typescript function getDataInTime( dynamicVolume: Types.IDynamicImageVolume, options: { frameNumbers?; maskVolumeId?; imageCoordinate?; } ): number[] | number[][]; ``` ```typescript function getDataInTime( dynamicVolume: Types.IDynamicImageVolume, options: { frameNumbers?; maskVolumeId?; worldCoordinate?; } ): number[] | number[][]; ``` #### Usage Example ```typescript const result = getDataInTime(dynamicVolume, { frameNumbers: [0, 1, 2], imageCoordinate: [100, 100, 100], }); ``` ```typescript const result = getDataInTime(dynamicVolume, { frameNumbers: [0, 1, 2], worldCoordinate: [100, 100, 100], }); ``` #### generateImageFromTimeData ```typescript function generateImageFromTimeData( dynamicVolume: Types.IDynamicImageVolume, operation: string, frameNumbers?: number[] ); ``` ```typescript function generateImageFromTimeData( dynamicVolume: Types.IDynamicImageVolume, operation: Enums.GenerateImageType, options: { frameNumbers?: number[]; } ): Float32Array; ``` #### Key Changes 1. `operation` now uses `Enums.GenerateImageType` enum. 2. Frame numbers are passed in an options object. 3. Function explicitly returns `Float32Array`. #### Usage Example ```typescript const result = generateImageFromTimeData(dynamicVolume, 'SUM', [0, 1, 2]); ``` ```typescript const result = generateImageFromTimeData( dynamicVolume, Enums.GenerateImageType.SUM, { frameNumbers: [0, 1, 2], } ); ``` #### Summary of Other Changes - New `updateVolumeFromTimeData` function added for in-place volume updates. - Both functions now use `voxelManager` for improved performance. - Enhanced error handling and standardized error messages. - Operations now use `Enums.GenerateImageType` for better type safety. --- ### @cornerstonejs/tools Source: https://cornerstonejs.org/docs/llm/migration-guides/2x/5-tools.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### @cornerstonejs/tools #### triggerAnnotationRenderForViewportIds Now only requires viewportIds and doesn't need renderingEngine anymore ```js triggerAnnotationRenderForViewportIds(renderingEngine, viewportIds) ---> triggerAnnotationRenderForViewportIds(viewportIds) ```
Why? Since there is one rendering engine per viewport, there is no need to pass the rendering engine as an argument.
#### Tools #### StackScrollMouseWheelTool -> StackScrollTool We've decoupled the Mouse Wheel from the tool itself, allowing it to be applied as a binding similar to other mouse bindings. This change offers several advantages: - It can be combined with other mouse bindings - It can be paired with keyboard bindings ```js cornerstoneTools.addTool(StackScrollMouseWheelTool); toolGroup.addTool(StackScrollMouseWheelTool.toolName); toolGroup.setToolActive(StackScrollMouseWheelTool.toolName); ``` ```js cornerstoneTools.addTool(StackScrollTool); toolGroup.addTool(StackScrollTool.toolName); toolGroup.setToolActive(StackScrollTool.toolName, { bindings: [ { mouseButton: MouseBindings.Wheel, }, ], }); ``` #### BaseTool The `getTargetVolumeId` method has been removed in favor of `getTargetId`, and `getTargetIdImage` has been renamed to `getTargetImageData` to make it more clear that it is an image data. #### Usage Example ```typescript const volumeId = this.getTargetVolumeId(viewport); const imageData = this.getTargetIdImage(targetId, renderingEngine); ``` ```typescript const imageData = this.getTargetImageData(targetId); ``` #### New Segmentation Model We have a new segmentation model that is more flexible and easier to use. #### Same Terminology, Different Architecture In Cornerstone3D version 2, we've made significant architectural changes to our segmentation model while maintaining familiar terminology. This redesign aims to provide a more flexible and intuitive approach to working with segmentations across different viewports. Here are the key changes and the reasons behind them: 1. **Viewport-Specific, Not Tool Group-Based**: - Old: Segmentations were tied to tool groups, which typically consist of multiple viewports. This created complications when users wanted to add segmentations to some viewports but not others within the same tool group. - New: Segmentations are now viewport-specific. Instead of adding or removing representations to a tool group, users can add them directly to viewports. This provides much finer control over what each viewport renders. - Why: We discovered that tying rendering to a tool group is not an effective approach. It often necessitated creating an extra tool group for a specific viewport to customize or prevent rendering. 2. **Simplified Identification of Segmentation Representations**: - Old: Required a unique segmentationRepresentationUID for identification. - New: Segmentation representations are identified by a combination of `segmentationId` and representation `type`. This allows each viewport to have different representations of the same segmentation. - Why: This simplification makes it easier to manage and reference segmentation representations across different viewports. 3. **Decoupling of Data and Visualization**: - Old: Segmentation rendering was tightly coupled with tool groups. - New: Segmentation is now treated purely as data, separate from the tools used to interact with it. - Why: While it's appropriate for tools to be bound to tool groups, viewport-specific functionalities like segmentation rendering should be the responsibility of individual viewports. This separation allows for more flexible rendering and interaction options across different viewports. 4. **Polymorphic Segmentation Support**: - The new architecture better supports the concept of polymorphic segmentations, where a single segmentation can have multiple representations (e.g., labelmap, contour, surface) that can be efficiently converted between each other. - Why: This flexibility allows for more efficient storage, analysis, and real-time visualization of segmentations. 5. **Consistent API Across Representation Types**: - The new API provides a unified way to work with different segmentation representations, making it easier to manage complex scenarios involving multiple viewports and representation types. - Why: This consistency simplifies development and reduces the likelihood of errors when working with different segmentation types. These architectural changes provide a more robust foundation for working with segmentations, especially in complex multi-viewport scenarios. The new approach has proven to be highly effective and opens up possibilities for future enhancements. While the core concepts remain similar, the way you interact with segmentations in your code will change significantly. This migration guide will walk you through these changes, providing before-and-after examples to help you update your existing codebase to the new architecture. #### Segmentation State The `Segmentation` type has been restructured to better organize segment information and representation data. Let's take a look at the changes before we talk about migration guides. ```typescript type Segmentation = { segmentationId: string; type: Enums.SegmentationRepresentations; label: string; activeSegmentIndex: number; segmentsLocked: Set; cachedStats: { [key: string]: number }; segmentLabels: { [key: string]: string }; representationData: SegmentationRepresentationData; }; ``` ```typescript type Segmentation = { segmentationId: string; label: string; segments: { [segmentIndex: number]: Segment; }; representationData: RepresentationsData; }; type Segment = { segmentIndex: number; label: string; locked: boolean; cachedStats: { [key: string]: unknown }; active: boolean; }; ``` The new segmentation state model offers a more organized data structure. Previously scattered information such as `cachedStats`, `segmentLabels`, and `activeSegmentIndex` has been consolidated under the `segments` property. This restructuring enhances clarity and efficiency. In the following sections, we'll discuss migration guides that will explain how to access and modify these properties within the new structure. This reorganization primarily affects the segmentation store level. #### Representation Data Key The `SegmentationRepresentations` enum has been updated to use title case instead of uppercase to make it match the rest of the Enums. ```typescript enum SegmentationRepresentations { Labelmap = 'LABELMAP', Contour = 'CONTOUR', Surface = 'SURFACE', } ``` ```typescript enum SegmentationRepresentations { Labelmap = 'Labelmap', Contour = 'Contour', Surface = 'Surface', } ``` This change affects how representation data is accessed: ```typescript const representationData = segmentation.representationData.SURFACE; const representationData = segmentation.representationData.LABELMAP; const representationData = segmentation.representationData.CONTOUR; ``` ```typescript const representationData = segmentation.representationData.Surface; const representationData = segmentation.representationData.Labelmap; const representationData = segmentation.representationData.Contour; ``` #### Segmentation Representation The representation structure has been simplified and is now viewport-specific. ```typescript type ToolGroupSpecificRepresentation = | ToolGroupSpecificLabelmapRepresentation | ToolGroupSpecificContourRepresentation; type ToolGroupSpecificRepresentationState = { segmentationRepresentationUID: string; segmentationId: string; type: Enums.SegmentationRepresentations; active: boolean; segmentsHidden: Set; colorLUTIndex: number; }; type SegmentationState = { toolGroups: { [key: string]: { segmentationRepresentations: ToolGroupSpecificRepresentations; config: SegmentationRepresentationConfig; }; }; }; ``` ```typescript type SegmentationRepresentation = | LabelmapRepresentation | ContourRepresentation | SurfaceRepresentation; type BaseSegmentationRepresentation = { colorLUTIndex: number; segmentationId: string; type: Enums.SegmentationRepresentations; visible: boolean; active: boolean; segments: { [segmentIndex: number]: { visible: boolean; }; }; }; type SegmentationState = { viewportSegRepresentations: { [viewportId: string]: Array; }; }; ``` Previously, the segmentation representation was tool group specific, which led to some issues. In the new structure, segmentation representation is viewport specific. It now consists of a segmentationId, a type, and various settings for that segmentation. As a result of this change, several functions have been removed or modified. Here's a summary of the changes: #### Removed Functions - `getDefaultSegmentationStateManager` - `getSegmentationRepresentations` - `getAllSegmentationRepresentations` - `getSegmentationIdRepresentations` - `findSegmentationRepresentationByUID` - `getToolGroupIdsWithSegmentation` - `getToolGroupSpecificConfig` - `setToolGroupSpecificConfig` - `getGlobalConfig` - `setGlobalConfig` - `setSegmentationRepresentationSpecificConfig` - `getSegmentationRepresentationSpecificConfig` - `getSegmentSpecificRepresentationConfig` - `setSegmentSpecificRepresentationConfig` - `getToolGroupIdFromSegmentationRepresentationUID` - `addSegmentationRepresentation` - `getSegmentationRepresentationByUID` #### New Functions - `addSegmentations(segmentationInputArray)` - `removeSegmentation(segmentationId)` - `getSegmentation(segmentationId)` - `getSegmentations()` - `getSegmentationRepresentation(viewportId, specifier)` - `getSegmentationRepresentations(viewportId, specifier)` - `removeSegmentationRepresentation(viewportId, specifier, immediate)` - `removeAllSegmentationRepresentations()` - `removeLabelmapRepresentation(viewportId, segmentationId, immediate)` - `removeContourRepresentation(viewportId, segmentationId, immediate)` - `removeSurfaceRepresentation(viewportId, segmentationId, immediate)` - `getViewportSegmentations(viewportId, type)` - `getViewportIdsWithSegmentation(segmentationId)` - `getCurrentLabelmapImageIdForViewport(viewportId, segmentationId)` - `updateLabelmapSegmentationImageReferences(segmentationId, imageIds)` - `getStackSegmentationImageIdsForViewport(viewportId, segmentationId)` - `destroy()` #### Removal of SegmentationDisplayTool There's no need to add the SegmentationDisplayTool to the toolGroup anymore. Before ```js toolGroup2.addTool(SegmentationDisplayTool.toolName); toolGroup1.setToolEnabled(SegmentationDisplayTool.toolName); ``` Now ```js // nothing ``` #### Stack Labelmaps To create a Stack Labelmap, you no longer need to manually create a reference between labelmap imageIds and viewport imageIds. We now handle this process automatically for you. This is a long Why ... The previous model required users to provide an imageIdReferenceMap, which linked labelmap imageIds to viewport imageIds. This approach presented several challenges when implementing advanced segmentation use cases: 1. Manual creation of the map was error-prone, particularly regarding the order of imageIds. 2. Once a segmentation was associated with specific viewport imageIds, rendering it elsewhere became problematic. For example: - Rendering a CT image stack segmentation on a single key image. - Rendering a CT image stack segmentation on a stack that includes both CT and other images. - Rendering a DX dual energy segmentation from energy 1 on energy 2. - Rendering a CT labelmap from a stack viewport on a PT labelmap in the same space. These scenarios highlight the limitations of the previous model. We've now transitioned to a system where users only need to provide imageIds. During rendering, we match the viewport's current imageId against the labelmap imageIds and render the segmentation if there's a match. This matching process occurs in the SegmentationStateManager, with the criterion being that the segmentation must be in the same plane as the referenced viewport. This new approach enables numerous additional use cases and offers greater flexibility in segmentation rendering. ```js segmentation.addSegmentations([ { segmentationId, representation: { type: csToolsEnums.SegmentationRepresentations.Labelmap, data: { imageIdReferenceMap: cornerstoneTools.utilities.segmentation.createImageIdReferenceMap( imageIds, segmentationImageIds ), }, }, }, ]); ``` ```js segmentation.addSegmentations([ { segmentationId, representation: { type: csToolsEnums.SegmentationRepresentations.Labelmap, data: { imageIds: segmentationImageIds, }, }, }, ]); ``` #### Adding Segmentations #### Function Signature Update The `addSegmentations` function now accepts an optional `suppressEvents` parameter. ```typescript function addSegmentations( segmentationInputArray: SegmentationPublicInput[] ): void; ``` ```typescript function addSegmentations( segmentationInputArray: SegmentationPublicInput[], suppressEvents?: boolean ): void; ``` **Migration Steps:** 1. Update any calls to `addSegmentations` to include the `suppressEvents` parameter if needed. 2. If you don't want to suppress events, you can omit the second parameter. #### SegmentationPublicInput Type Updates The `SegmentationPublicInput` type has been extended to include an optional `config` property. ```typescript type SegmentationPublicInput = { segmentationId: string; representation: { type: Enums.SegmentationRepresentations; data?: RepresentationData; }; }; ``` ```typescript type SegmentationPublicInput = { segmentationId: string; representation: { type: Enums.SegmentationRepresentations; data?: RepresentationData; }; config?: { segments?: { [segmentIndex: number]: Partial; }; label?: string; }; }; ``` **Migration Steps:** 1. Update any code that creates or manipulates `SegmentationPublicInput` objects to include the new `config` property if needed. 2. Replace specific segmentation data types with the generic `RepresentationData` type. #### Adding Segmentation Representations #### Viewport-Centric Approach The API now focuses on viewports instead of tool groups, providing more granular control over segmentation representations. ```typescript function addSegmentationRepresentations( toolGroupId: string, representationInputArray: RepresentationPublicInput[], toolGroupSpecificRepresentationConfig?: SegmentationRepresentationConfig ): Promise; ``` ```typescript function addSegmentationRepresentations( viewportId: string, segmentationInputArray: RepresentationPublicInput[] ); ``` **Migration Steps:** 1. Replace `toolGroupId` with `viewportId` in function calls. 2. Remove the `toolGroupSpecificRepresentationConfig` parameter. 3. Update any code that relies on the returned Promise of segmentation representation UIDs. #### RepresentationPublicInput Changes The `RepresentationPublicInput` type has been simplified and some properties have been renamed or removed. ```typescript type RepresentationPublicInput = { segmentationId: string; type: Enums.SegmentationRepresentations; options?: { segmentationRepresentationUID?: string; colorLUTOrIndex?: Types.ColorLUT | number; polySeg?: { enabled: boolean; options?: any; }; }; }; ``` ```typescript type RepresentationPublicInput = { segmentationId: string; type?: Enums.SegmentationRepresentations; config?: { colorLUTOrIndex?: Types.ColorLUT[] | number; }; }; ``` **Migration Steps:** 1. Remove the `options` property and move `colorLUTOrIndex` to the `config` object. 2. Remove `segmentationRepresentationUID` and `polySeg` properties if used, polySEG is default enabled. 3. Update the `colorLUTOrIndex` type to accept an array of `Types.ColorLUT` instead of a single value. #### New Representation-Specific Functions Version 2 introduces new functions for adding specific types of segmentation representations to viewports. ```typescript // No equivalent functions in version 1 ``` ```typescript function addContourRepresentationToViewport( viewportId: string, contourInputArray: RepresentationPublicInput[] ); function addLabelmapRepresentationToViewport( viewportId: string, labelmapInputArray: RepresentationPublicInput[] ); function addSurfaceRepresentationToViewport( viewportId: string, surfaceInputArray: RepresentationPublicInput[] ); ``` **Migration Steps:** 1. Replace generic `addSegmentationRepresentations` calls with the appropriate representation-specific function. 2. Update the input array to match the new `RepresentationPublicInput` type. 3. Remove any type-specific logic from your code, as it's now handled by these new functions. #### Multi-Viewport Functions Version 2 introduces new functions for adding segmentation representations to multiple viewports simultaneously. ```typescript // No equivalent functions in version 1 ``` ```typescript function addContourRepresentationToViewportMap(viewportInputMap: { [viewportId: string]: RepresentationPublicInput[]; }); function addLabelmapRepresentationToViewportMap(viewportInputMap: { [viewportId: string]: RepresentationPublicInput[]; }); function addSurfaceRepresentationToViewportMap(viewportInputMap: { [viewportId: string]: RepresentationPublicInput[]; }); ``` **Migration Steps:** 1. If you were previously adding representations to multiple tool groups, refactor your code to use these new multi-viewport functions. 2. Create a `viewportInputMap` object with viewport IDs as keys and arrays of `RepresentationPublicInput` as values. 3. Call the appropriate multi-viewport function based on the representation type. #### Events Since we moved from toolGroup to viewport, many events have been renamed to include `viewportId` instead of `toolGroupId`, and some event details have been changed to include `segmentationId` instead of `segmentationRepresentationUID` or toolGroupId #### Removal of ToolGroup Specific Events The `triggerSegmentationRepresentationModified` and `triggerSegmentationRepresentationRemoved` functions have been removed. Instead, the library now uses a more generalized approach for handling segmentation events. ```typescript function triggerSegmentationRepresentationModified( toolGroupId: string, segmentationRepresentationUID?: string ): void { // ... } function triggerSegmentationRepresentationRemoved( toolGroupId: string, segmentationRepresentationUID: string ): void { // ... } ``` ```typescript function triggerSegmentationRepresentationModified( viewportId: string, segmentationId: string, type?: SegmentationRepresentations ): void { // ... } function triggerSegmentationRepresentationRemoved( viewportId: string, segmentationId: string, type: SegmentationRepresentations ): void { // ... } ``` **Migration Steps:** 1. Replace `toolGroupId` with `viewportId` in function calls. 2. Replace `segmentationRepresentationUID` with `segmentationId`. 3. Add the `type` parameter to specify the segmentation representation type. #### Simplified Segmentation Modified Event The `triggerSegmentationModified` function has been simplified to always require a `segmentationId`. ```typescript function triggerSegmentationModified(segmentationId?: string): void { // ... } ``` ```typescript function triggerSegmentationModified(segmentationId: string): void { // ... } ``` **Migration Steps:** 1. Ensure that `segmentationId` is always provided when calling `triggerSegmentationModified`. 2. Remove any logic that handles the case where `segmentationId` is undefined. #### Updated Event Detail Types Several event detail types have been updated to reflect the changes in the segmentation system: ```typescript type SegmentationRepresentationModifiedEventDetail = { toolGroupId: string; segmentationRepresentationUID: string; }; type SegmentationRepresentationRemovedEventDetail = { toolGroupId: string; segmentationRepresentationUID: string; }; type SegmentationRenderedEventDetail = { viewportId: string; toolGroupId: string; }; ``` ```typescript type SegmentationRepresentationModifiedEventDetail = { segmentationId: string; type: string; viewportId: string; }; type SegmentationRepresentationRemovedEventDetail = { segmentationId: string; type: string; viewportId: string; }; type SegmentationRenderedEventDetail = { viewportId: string; segmentationId: string; type: string; }; ``` **Migration Steps:** 1. Update event listeners to use the new event detail types. 2. Replace `toolGroupId` with `viewportId` where applicable. 3. Use `segmentationId` instead of `segmentationRepresentationUID`. 4. Add handling for the new `type` field in event details. #### Segmentation Config/Style In Cornerstone3D version 2.x, we have significantly refactored the segmentation configuration APIs to provide a more flexible and unified approach for managing segmentation styles across different representations (Labelmap, Contour, Surface). The old APIs for getting and setting segmentation configurations have been replaced with new functions that utilize a specifier object to target specific segmentations, viewports, and segments. #### Removed Functions - `getGlobalConfig` - `setGlobalConfig` - `getGlobalRepresentationConfig` - `setGlobalRepresentationConfig` - `getToolGroupSpecificConfig` - `setToolGroupSpecificConfig` - `getSegmentSpecificConfig` - `setSegmentSpecificConfig` - `getSegmentationRepresentationSpecificConfig` - `setSegmentationRepresentationSpecificConfig` #### New Functions - `getStyle(specifier)` - `setStyle(specifier, style)` - `setRenderInactiveSegmentations(viewportId, renderInactiveSegmentations)` - `getRenderInactiveSegmentations(viewportId)` - `resetToGlobalStyle()` - `hasCustomStyle(specifier)` #### Getting Global Segmentation Config ```js // Get the global segmentation config const globalConfig = getGlobalConfig(); // Get global representation config for a specific representation type const labelmapConfig = getGlobalRepresentationConfig( SegmentationRepresentations.Labelmap ); ``` ```js // Get the global style for a specific representation type const labelmapStyle = getStyle({ type: SegmentationRepresentations.Labelmap }); ``` #### Setting Global Segmentation Config ```js // Set the global segmentation config setGlobalConfig(newGlobalConfig); // Set global representation config for a specific representation type setGlobalRepresentationConfig( SegmentationRepresentations.Labelmap, newLabelmapConfig ); ``` ```js // Set the global style for a specific representation type setStyle({ type: SegmentationRepresentations.Labelmap }, newLabelmapStyle); ``` #### Getting and Setting ToolGroup-Specific Config ToolGroup-specific configurations have been removed in favor of viewport-specific styles. The following will set the style for a specific viewport and specific segmentation. ```js // Get toolGroup-specific config const toolGroupConfig = getToolGroupSpecificConfig(toolGroupId); // Set toolGroup-specific config setToolGroupSpecificConfig(toolGroupId, newToolGroupConfig); ``` ```js // Set style for a specific viewport and segmentation representation setStyle( { viewportId: 'viewport1', segmentationId: 'segmentation1', type: SegmentationRepresentations.Labelmap, }, newLabelmapStyle ); // Get style for a specific viewport and segmentation representation const style = getStyle({ viewportId: 'viewport1', segmentationId: 'segmentation1', type: SegmentationRepresentations.Labelmap, }); ``` #### Getting and Setting Segmentation Representation-Specific Config In Cornerstone3D version 2.x, the functions for getting and setting segmentation representation-specific configurations have been replaced with a unified style management API. The old functions: `getSegmentationRepresentationSpecificConfig` `setSegmentationRepresentationSpecificConfig` are no longer available. Instead, you should use the getStyle and setStyle functions with a specifier object to target specific segmentations and representations. ```js // Get segmentation representation-specific config const representationConfig = getSegmentationRepresentationSpecificConfig( toolGroupId, segmentationRepresentationUID ); // Set segmentation representation-specific config setSegmentationRepresentationSpecificConfig( toolGroupId, segmentationRepresentationUID, { LABELMAP: { renderOutline: true, outlineWidth: 2, }, } ); ``` ```js // Get style for a specific segmentation representation const style = getStyle({ segmentationId: 'segmentation1', type: SegmentationRepresentations.Labelmap, }); // Set style for a specific segmentation representation in all viewports setStyle( { segmentationId: 'segmentation1', type: SegmentationRepresentations.Labelmap, }, { renderOutline: true, outlineWidth: 2, } ); ``` #### Getting and Setting Segment-Specific Config ```js // Get segment-specific config const segmentConfig = getSegmentSpecificConfig( toolGroupId, segmentationRepresentationUID, segmentIndex ); // Set segment-specific config setSegmentSpecificConfig( toolGroupId, segmentationRepresentationUID, segmentIndex, newSegmentConfig ); ``` ```js // Set style for a specific segment setStyle( { segmentationId: 'segmentation1', type: SegmentationRepresentations.Labelmap, segmentIndex: 1, }, newSegmentStyle ); // Get style for a specific segment const segmentStyle = getStyle({ segmentationId: 'segmentation1', type: SegmentationRepresentations.Labelmap, segmentIndex: 1, }); ``` #### Setting Render Inactive Segmentations The function to enable or disable rendering of inactive segmentations has been updated. **Before** This was part of the segmentation configuration: ```js setGlobalConfig({ renderInactiveSegmentations: true }); ``` **After** Use `setRenderInactiveSegmentations`: ```js // Set whether to render inactive segmentations in a viewport setRenderInactiveSegmentations(viewportId, true); // Get whether inactive segmentations are rendered in a viewport const renderInactive = getRenderInactiveSegmentations(viewportId); ``` #### Resetting to Global Style To reset all segmentation styles to the global style: ```js resetToGlobalStyle(); ``` #### Example Migration ```js import { getGlobalConfig, getGlobalRepresentationConfig, getToolGroupSpecificConfig, setGlobalConfig, setGlobalRepresentationConfig, setToolGroupSpecificConfig, setSegmentSpecificConfig, getSegmentSpecificConfig, setSegmentationRepresentationSpecificConfig, getSegmentationRepresentationSpecificConfig, } from './segmentationConfig'; // Get the global segmentation config const globalConfig = getGlobalConfig(); // Set global representation config setGlobalRepresentationConfig(SegmentationRepresentations.Labelmap, { renderOutline: true, outlineWidth: 2, }); // Set toolGroup-specific config setToolGroupSpecificConfig(toolGroupId, { representations: { LABELMAP: { renderOutline: false, }, }, }); // Set segment-specific config setSegmentSpecificConfig( toolGroupId, segmentationRepresentationUID, segmentIndex, { LABELMAP: { renderFill: false, }, } ); ``` ```js import { getStyle, setStyle, setRenderInactiveSegmentations, getRenderInactiveSegmentations, resetToGlobalStyle, hasCustomStyle, } from '@cornerstonejs/core'; // Get the global style for Labelmap representation const labelmapStyle = getStyle({ type: SegmentationRepresentations.Labelmap }); // Set the global style for Labelmap representation setStyle( { type: SegmentationRepresentations.Labelmap }, { renderOutline: true, outlineWidth: 2, } ); // Set style for a specific viewport and segmentation setStyle( { viewportId: 'viewport1', segmentationId: 'segmentation1', type: SegmentationRepresentations.Labelmap, }, { renderOutline: false, } ); // Set style for a specific segment setStyle( { segmentationId: 'segmentation1', type: SegmentationRepresentations.Labelmap, segmentIndex: segmentIndex, }, { renderFill: false, } ); // Set render inactive segmentations for a viewport setRenderInactiveSegmentations('viewport1', true); // Get render inactive segmentations setting for a viewport const renderInactive = getRenderInactiveSegmentations('viewport1'); // Reset all styles to global resetToGlobalStyle(); ``` --- #### Summary - **Unified Style Management**: The new `getStyle` and `setStyle` functions provide a unified way to manage segmentation styles across different levels—global, segmentation-specific, viewport-specific, and segment-specific. - **Specifier Object**: The `specifier` object allows you to target specific viewports, segmentations, and segments. - `type` is required - if `segmentationId` is provided, the style will be applied to the specific segmentation representation in all viewports - if `segmentationId` and `segmentIndex` are provided, the style will be applied to the specific segment of the specific segmentation representation - if `viewportId` is provided, the style will be applied to all segmentations in the specific viewport - if `viewportId`, `segmentationId`, and `segmentIndex` are provided, the style will be applied to the specific segment of the specific segmentation in the specific viewport - **Hierarchy of Styles**: The effective style is determined by a hierarchy that considers global styles, segmentation-specific styles, and viewport-specific styles. #### Active #### Viewport-based Operations The API now uses viewport IDs instead of tool group IDs for identifying the context of segmentation operations. ```typescript function getActiveSegmentationRepresentation(toolGroupId: string); function getActiveSegmentation(toolGroupId: string); function setActiveSegmentationRepresentation( toolGroupId: string, segmentationRepresentationUID: string ); ``` ```typescript function getActiveSegmentation(viewportId: string); function setActiveSegmentation( viewportId: string, segmentationId: string, suppressEvent: boolean = false ); ``` #### Migration Steps: 1. Replace all instances of `toolGroupId` with `viewportId` in function calls. 2. Update `getActiveSegmentationRepresentation` and `getActiveSegmentation` calls to use the new `getActiveSegmentation` function. 3. Replace `setActiveSegmentationRepresentation` calls with `setActiveSegmentation`, using the new parameter structure. #### Return Type Changes The return type of `getActiveSegmentation` has changed from an implicit `undefined` to an explicit `Segmentation` type. ```typescript function getActiveSegmentation(toolGroupId: string); ``` ```typescript function getActiveSegmentation(viewportId: string): Segmentation; ``` #### Migration Steps: 1. Replace all calls to `getActiveSegmentationRepresentation` with `getActiveSegmentation`. 2. Update any code that relied on the `ToolGroupSpecificRepresentation` type to work with the `Segmentation` type instead. These changes aim to simplify the API and make it more intuitive to use. By focusing on viewport-based operations and removing the distinction between segmentation representations and segmentations, the new API should be easier to work with while maintaining the core functionality of the library. #### Visibility #### Viewport-Centric Approach The API now focuses on viewports rather than tool groups, reflecting a shift in the library's architecture. ```typescript function setSegmentationVisibility( toolGroupId: string, segmentationRepresentationUID: string, visibility: boolean ): void { // ... } ``` ```typescript function setSegmentationRepresentationVisibility( viewportId: string, specifier: { segmentationId: string; type?: SegmentationRepresentations; }, visibility: boolean ): void { // ... } ``` **Migration Steps:** 1. Replace `toolGroupId` with `viewportId` in function calls. 2. Use a `specifier` object instead of `segmentationRepresentationUID`. 3. Include `segmentationId` in the `specifier` object. 4. Optionally specify the `type` of segmentation representation. #### Segmentation Representation Types Version 2 introduces the concept of segmentation representation types, allowing for more granular control over different representation styles. ```typescript function getSegmentationVisibility( toolGroupId: string, segmentationRepresentationUID: string ): boolean | undefined { // ... } ``` ```typescript function getSegmentationRepresentationVisibility( viewportId: string, specifier: { segmentationId: string; type: SegmentationRepresentations; } ): boolean | undefined { // ... } ``` **Migration Steps:** 1. Update function names from `getSegmentationVisibility` to `getSegmentationRepresentationVisibility`. 2. Replace `toolGroupId` with `viewportId`. 3. Use a `specifier` object with `segmentationId` and `type` instead of `segmentationRepresentationUID`. #### Segment-Level Visibility Control The API for controlling individual segment visibility has been updated to align with the new viewport-centric approach. ```typescript function setSegmentVisibility( toolGroupId: string, segmentationRepresentationUID: string, segmentIndex: number, visibility: boolean ): void { // ... } ``` ```typescript function setSegmentIndexVisibility( viewportId: string, specifier: { segmentationId: string; type?: SegmentationRepresentations; }, segmentIndex: number, visibility: boolean ): void { // ... } ``` **Migration Steps:** 1. Update function names from `setSegmentVisibility` to `setSegmentIndexVisibility`. 2. Replace `toolGroupId` with `viewportId`. 3. Use a `specifier` object with `segmentationId` and optional `type` instead of `segmentationRepresentationUID`. #### New Utility Functions Version 2 introduces new utility functions for managing segmentation visibility. ```typescript function getHiddenSegmentIndices( viewportId: string, specifier: { segmentationId: string; type: SegmentationRepresentations; } ): Set { // ... } ``` This new function allows you to retrieve a set of hidden segment indices for a specific segmentation representation. #### Removed Functions The following functions have been removed in version 2: - `setSegmentsVisibility` - `getSegmentVisibility` Replace usage of these functions with the new API methods described above.
Why? Since the visibility should be set on the representation, and segmentation is not the owner of the visibility, a segmentation can have two representations with different visibility on each viewport
#### Locking #### Retrieving Locked Segments The function to retrieve locked segments has been renamed and its implementation changed: ```typescript function getLockedSegments(segmentationId: string): number[] | []; ``` ```typescript function getLockedSegmentIndices(segmentationId: string): number[] | []; ``` **Migration Steps:** 1. Update all calls from `getLockedSegments` to `getLockedSegmentIndices`. 2. Be aware that the implementation now uses `Object.keys` and `filter` instead of converting a Set to an array. #### Color #### Viewport-Centric Approach The API has shifted from a tool group-based approach to a viewport-centric one. This change affects several function signatures and how segmentations are referenced. ```typescript function setColorLUT( toolGroupId: string, segmentationRepresentationUID: string, colorLUTIndex: number ): void { // ... } ``` ```typescript function setColorLUT( viewportId: string, segmentationId: string, colorLUTsIndex: number ): void { // ... } ``` **Migration Steps:** 1. Replace `toolGroupId` with `viewportId` in function calls. 2. Replace `segmentationRepresentationUID` with `segmentationId`. 3. Update any code that relies on tool group-based segmentation management to use viewport-based management instead. #### Color LUT Management The `addColorLUT` function now returns the index of the added color LUT and has an optional `colorLUTIndex` parameter. ```typescript function addColorLUT(colorLUT: Types.ColorLUT, colorLUTIndex: number): void { // ... } ``` ```typescript function addColorLUT(colorLUT: Types.ColorLUT, colorLUTIndex?: number): number { // ... } ``` **Migration Steps:** 1. Update calls to `addColorLUT` to handle the returned index if needed. 2. Make the `colorLUTIndex` parameter optional in function calls. #### Segment Color Retrieval and Setting The functions for getting and setting segment colors have been renamed and their signatures updated to align with the new viewport-centric approach. ```typescript function getColorForSegmentIndex( toolGroupId: string, segmentationRepresentationUID: string, segmentIndex: number ): Types.Color { // ... } function setColorForSegmentIndex( toolGroupId: string, segmentationRepresentationUID: string, segmentIndex: number, color: Types.Color ): void { // ... } ``` ```typescript function getSegmentIndexColor( viewportId: string, segmentationId: string, segmentIndex: number ): Types.Color { // ... } function setSegmentIndexColor( viewportId: string, segmentationId: string, segmentIndex: number, color: Types.Color ): void { // ... } ``` **Migration Steps:** 1. Rename `getColorForSegmentIndex` to `getSegmentIndexColor`. 2. Rename `setColorForSegmentIndex` to `setSegmentIndexColor`. 3. Update function calls to use `viewportId` instead of `toolGroupId`. 4. Replace `segmentationRepresentationUID` with `segmentationId` in function calls. #### Other Changes #### Renaming ```js getSegmentAtWorldPoint-- > getSegmentIndexAtWorldPoint; getSegmentAtLabelmapBorder-- > getSegmentIndexAtLabelmapBorder; ``` #### getToolGroupIdsWithSegmentation ```typescript function getToolGroupIdsWithSegmentation(segmentationId: string): string[]; ``` ```typescript function getViewportIdsWithSegmentation(segmentationId: string): string[]; ``` **Migration Steps:** 1. Replace `getToolGroupIdsWithSegmentation` with `getViewportIdsWithSegmentation`. #### Segmentation Representation Management The way segmentation representations are added, retrieved, and removed has changed significantly. ```typescript function addSegmentationRepresentation( toolGroupId: string, segmentationRepresentation: ToolGroupSpecificRepresentation, suppressEvents?: boolean ): void; function getSegmentationRepresentationByUID( toolGroupId: string, segmentationRepresentationUID: string ): ToolGroupSpecificRepresentation | undefined; function removeSegmentationRepresentation( toolGroupId: string, segmentationRepresentationUID: string ): void; ``` ```typescript function addSegmentationRepresentation( viewportId: string, segmentationRepresentation: SegmentationRepresentation, suppressEvents?: boolean ): void; function getSegmentationRepresentation( viewportId: string, specifier: { segmentationId: string; type: SegmentationRepresentations; } ): SegmentationRepresentation | undefined; function removeSegmentationRepresentation( viewportId: string, specifier: { segmentationId: string; type: SegmentationRepresentations; }, immediate?: boolean ): void; ``` **Migration Steps:** 1. Update all calls to `addSegmentationRepresentation` to use `viewportId` instead of `toolGroupId`. 2. Replace `getSegmentationRepresentationByUID` with `getSegmentationRepresentation`, using the new specifier object. 3. Update `removeSegmentationRepresentation` calls to use the new specifier object instead of `segmentationRepresentationUID`. #### PolySEG #### Import The PolySEG has been unbundled and placed in a separate external package. To use it, add the `peerImport` function to your `init` function for Cornerstone Core. ```js async function peerImport(moduleId) { if (moduleId === '@icr/polyseg-wasm') { return import('@icr/polyseg-wasm'); } } import { init } from '@cornerstonejs/core'; await init({ peerImport }); ``` #### Options You don't need to provide polyseg options for the segmentation representation. It will automatically use PolySeg if the specified representation is unavailable. ```js await segmentation.addSegmentationRepresentations(toolGroupId2, [ { segmentationId, type: csToolsEnums.SegmentationRepresentations.Labelmap, options: { polySeg: { enabled: true, }, }, }, ]); ``` ```js await segmentation.addSegmentationRepresentations(viewportId2, [ { segmentationId, type: csToolsEnums.SegmentationRepresentations.Labelmap, }, ]); ``` #### Actor UID for labelmaps The way the actorUID is generated has changed to use a combination of segmentationId and SegmentationRepresentations.Labelmap. ```js const volumeInputs: Types.IVolumeInput[] = [ { volumeId: labelMapData.volumeId, actorUID: segmentationRepresentationUID, visibility, blendMode: Enums.BlendModes.MAXIMUM_INTENSITY_BLEND, }, ]; ``` ```js const volumeInputs: Types.IVolumeInput[] = [ { volumeId, actorUID: `${segmentationId}-${SegmentationRepresentations.Labelmap}`, visibility, blendMode: Enums.BlendModes.MAXIMUM_INTENSITY_BLEND, }, ]; ``` We've updated the `actorUID` to `${segmentationId}-${SegmentationRepresentations.Labelmap}`. This change allows us to uniquely identify representations without relying on the `segmentationRepresentationUID`. For this mean, `getSegmentationActor` is added for you to get the actor for a given labelmap ```ts export function getSegmentationActor( viewportId: string, specifier: { segmentationId: string; type: SegmentationRepresentations; } ): Types.VolumeActor | Types.ImageActor | undefined; ``` #### New Utilities `clearSegmentValue` is added to clear a specific segment value in a segmentation, it will make the segment value to 0 ```js function clearSegmentValue( segmentationId: string, segmentIndex: number ) ``` #### Renaming and Nomenclature #### Types PointsManager is now IPointsManager migration ```js import { IPointsManager } from '@cornerstonejs/tools/types'; ``` #### Units #### getCalibratedLengthUnitsAndScale Signature It is highly unlikely that you were using this function directly, but if you were, here's the migration The return type of the function has changed slightly, with `units` and `areaUnits` renamed to `unit` and `areaUnit` respectively. ```typescript const getCalibratedLengthUnitsAndScale = (image, handles) => { // ... return { units, areaUnits, scale }; }; ``` ```typescript const getCalibratedLengthUnitsAndScale = (image, handles) => { // ... return { unit, areaUnit, scale }; }; ``` #### getModalityUnit -> getPixelValueUnits To make more sense
Why? There was too much inconsistency in the units used throughout the library. We had `unit`, `areaUnits`, `modalityUnit`, and various others. Now, we have consolidated these units. You need to update your codebase to reflect the new unit system if you are hydrating annotations for Cornerstone3D. In addition modalityUnit is now pixelValueUnits to reflect the correct term, since for a single modality there can be multiple pixel values (e.g, PT SUV, PT RAW, PT PROC)
#### BasicStatsCalculator the option `noPointsCollection` has been renamed to `storePointData` #### getSegmentAtWorldPoint -> getSegmentIndexAtWorldPoint #### getSegmentAtLabelmapBorder -> getSegmentIndexAtLabelmapBorder --- #### Others #### roundNumber The utility has been relocated from `@cornerstonejs/tools` utilities to `@cornerstonejs/core/utilities`. migration ```js import { roundNumber } from '@cornerstonejs/core/utilities'; ``` #### jumpToSlice The utility has been relocated from `@cornerstonejs/tools` utilities to `@cornerstonejs/core/utilities`. migration ```js import { jumpToSlice } from '@cornerstonejs/core/utilities'; ``` #### pointInShapeCallback #### 1. New Import Path The `pointInShapeCallback` function has been moved. Update your imports as follows: ```js import { pointInShapeCallback } from '@cornerstonejs/core/utilities'; ``` #### 2. Updated Usage The function signature has changed to use an options object for improved clarity and flexibility. Below is a guide to how the usage has changed. **Old Usage:** ```js const pointsInShape = pointInShapeCallback( imageData, shapeFnCriteria, (point) => { // callback logic for each point }, boundsIJK ); ``` **New Usage:** ```js const pointsInShape = pointInShapeCallback(imageData, { pointInShapeFn: shapeFnCriteria, callback: (point) => { // callback logic for each point }, boundsIJK: boundsIJK, returnPoints: true, // Optionally, to return the points inside the shape }); ``` #### Key Changes: - **Options Object**: Configuration parameters such as `pointInShapeFn`, `callback`, `boundsIJK`, and `returnPoints` are now passed through an options object. - **Return Points**: Use the `returnPoints` option to specify if you want to return the points within the shape, previously it was always returning the points. If you relied on returning points directly, make sure to include `storePointData: true` in the tool options when you active it --- ### @cornerstonejs/dicom-image-loader Source: https://cornerstonejs.org/docs/llm/migration-guides/2x/6-dicom-image-loader.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### @cornerstonejs/dicom-image-loader #### Initialization and Configuration **Before:** ```js cornerstoneDICOMImageLoader.external.cornerstone = cornerstone; cornerstoneDICOMImageLoader.external.dicomParser = dicomParser; cornerstoneDICOMImageLoader.configure({ useWebWorkers: true, decodeConfig: { convertFloatPixelDataToInt: false, use16BitDataType: preferSizeOverAccuracy || useNorm16Texture, }, }); // Additional configuration... cornerstoneDICOMImageLoader.webWorkerManager.initialize(config); ``` **After:** ```js cornerstoneDICOMImageLoader.init(); // optionally you can pass a config object to init cornerstoneDICOMImageLoader.init({ maxWebWorkers: 2, // }); ``` **Migration Guide:** 1. You should replace configure with `init` 2. You don't need to pass cornerstone and dicomParser anymore, we just use them internally and import them as dependencies 3. Remove `useWebWorkers` option as web workers are now always used. 4. Remove `decodeConfig` options as they are no longer applicable. 5. Remove separate `webWorkerManager.initialize` call as it's now handled internally. 6. Set `maxWebWorkers` in the configure options instead of a separate config object. 1. by default we set half of the available cores #### Removal of External Module The `externalModules` file has been removed. Any code relying on `cornerstone.external` should be updated to use direct imports or the new configuration method. We just treat the cornerstonejs/core and dicomparser as any other dependency and import them directly internally #### Webpack Configuration Remove the following Webpack rule if present in your configuration: ```json { test: /\.worker\.(mjs|js|ts)$/, use: [ { loader: 'worker-loader', }, ], }, ``` Web workers are now handled internally by the library. #### Always `Prescale` By default, Cornerstone3D always prescales images with the modality LUT (rescale slope and intercept). You probably don't need to make any changes to your codebase.
Why? The viewport previously made the decision to prescale, and all viewports followed this approach. However, we found prescaling bugs in some user-implemented custom image loaders. We have now fixed these issues by consistently applying prescaling.
#### Decoders Update `@cornerstonejs/dicomImageLoader` previously utilized the old API for web workers, which is now deprecated. It has transitioned to the new web worker API via our new internal wrapper over `comlink` package. This change enables more seamless interaction with web workers and facilitates compiling and bundling the web workers to match the ESM version of the library.
Why? To consolidate the web worker API using a new ES module format, which will enable new bundlers like `vite` to work seamlessly with the library.
So if you had custom logic in your webpack or other bundler you can remove the following rule ```json { test: /\.worker\.(mjs|js|ts)$/, use: [ { loader: 'worker-loader', }, ], }, ``` #### Removing support for non web worker decoders We have removed support for non-web worker decoders in the 2.0 version of the cornerstone3D. This change is to ensure that the library is more performant and to reduce the bundle size.
Why? We see no compelling reason to use non-worker decoders anymore. Web worker decoders offer superior performance and better compatibility with modern bundlers.
#### Removal of `minAfterScale` and `maxAfterScale` on `imageFrame` in favor of `smallestPixelValue` and `largestPixelValue`, previously they were 4 all used together and was making it hard to use the correct one. #### DICOM Image Loader ESM default We have changed the default export of the DICOM Image Loader to ESM in the 2.0 version of the cornerstone3D and correctly publish types This mean you don't need to have an alias for the dicom image loader anymore Probably in your webpack or other bundler you had this ```js alias: { '@cornerstonejs/dicom-image-loader': '@cornerstonejs/dicom-image-loader/dist/dynamic-import/cornerstoneDICOMImageLoader.min.js', }, ``` Now you can remove this alias and use the default import
Why? ESM is the future of javascript, and we want to ensure that the library is compatible with modern bundlers and tools.
--- --- ### @cornerstonejs/nifti-volume-loader Source: https://cornerstonejs.org/docs/llm/migration-guides/2x/7-nifti-volume-loader.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### `@cornerstonejs/nifti-image-volume-loader` After migrating to the new pixel data model for volumes, we have also updated the Nifti image volume loader to align with this model. This change brings the loader more in line with the Cornerstone3D API and the rest of the library. We now have a dedicated Nifti image loader (not a volume loader) for loading Nifti files, creating a more consistent API across all image loaders in the library. A significant improvement is the ability to use stack viewports for Nifti files. You no longer need volume viewports to render Nifti files (though you can still use volume viewports).
Why? The process now involves calling the Nifti URL and parsing the first bytes of the file (via stream decoding) to obtain metadata. We then create imageIds based on this metadata and use them to create the volume. This approach shifts from our previous volume-first method to an imageId-first approach, aligning with the rest of the Cornerstone3D API.
```js const niftiURL = 'https://ohif-assets-new.s3.us-east-1.amazonaws.com/nifti/MRHead.nii.gz'; const volumeId = 'nifti:' + niftiURL; const volume = await volumeLoader.createAndCacheVolume(volumeId); setVolumesForViewports( renderingEngine, [{ volumeId }], viewportInputArray.map((v) => v.viewportId) ); ``` ```js import { cornerstoneNiftiImageLoader, createNiftiImageIdsAndCacheMetadata, } from '@cornerstonejs/nifti-volume-loader'; const niftiURL = 'https://ohif-assets-new.s3.us-east-1.amazonaws.com/nifti/CTACardio.nii.gz'; // register the image loader for nifti files imageLoader.registerImageLoader('nifti', cornerstoneNiftiImageLoader); // similar to the rest of the cornerstone3D image loader const imageIds = await createNiftiImageIdsAndCacheMetadata({ url: niftiURL }); // For stack viewports viewport.setStack(imageIds); // for volume viewports const volume = await volumeLoader.createAndCacheVolume(volumeId, { imageIds, }); await volume.load(); setVolumesForViewports( renderingEngine, [{ volumeId }], viewportInputArray.map((v) => v.viewportId) ); ``` --- --- ### Developer Experience Source: https://cornerstonejs.org/docs/llm/migration-guides/2x/8-deverloper-experience.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### Developer Experience #### Dependency Cycles We have removed all dependency cycles in the library, ensuring it is now free of any such issues. To maintain this, we have added rules in our linters that will catch any dependency cycles in pull requests during continuous integration. Additionally, you can run `yarn run format-check` to ensure that the formatting is correct and to check for dependencies as well. #### Karma tests There has been a lot of work to clean up tests let's dive in #### Setup and Cleanup Before, we had scattered logic: ```js beforeEach(function () { csTools3d.init(); csTools3d.addTool(BidirectionalTool); cache.purgeCache(); this.DOMElements = []; this.stackToolGroup = ToolGroupManager.createToolGroup('stack'); this.stackToolGroup.addTool(BidirectionalTool.toolName, { configuration: { volumeId: volumeId }, }); this.stackToolGroup.setToolActive(BidirectionalTool.toolName, { bindings: [{ mouseButton: 1 }], }); this.renderingEngine = new RenderingEngine(renderingEngineId); imageLoader.registerImageLoader('fakeImageLoader', fakeImageLoader); volumeLoader.registerVolumeLoader('fakeVolumeLoader', fakeVolumeLoader); metaData.addProvider(fakeMetaDataProvider, 10000); }); afterEach(function () { csTools3d.destroy(); cache.purgeCache(); eventTarget.reset(); this.renderingEngine.destroy(); metaData.removeProvider(fakeMetaDataProvider); imageLoader.unregisterAllImageLoaders(); ToolGroupManager.destroyToolGroup('stack'); this.DOMElements.forEach((el) => { if (el.parentNode) { el.parentNode.removeChild(el); } }); }); ``` Now it's centralized: ```js beforeEach(function () { const testEnv = testUtils.setupTestEnvironment({ renderingEngineId, toolGroupIds: ['default'], viewportIds: [viewportId], tools: [BidirectionalTool], toolConfigurations: { [BidirectionalTool.toolName]: { configuration: { volumeId: volumeId }, }, }, toolActivations: { [BidirectionalTool.toolName]: { bindings: [{ mouseButton: 1 }], }, }, }); renderingEngine = testEnv.renderingEngine; toolGroup = testEnv.toolGroups['default']; }); afterEach(function () { testUtils.cleanupTestEnvironment({ renderingEngineId, toolGroupIds: ['default'], }); }); ```
Why? It was causing many issues with timeout and race conditions.
#### Viewport Creation We've centralized the previously repeated logic for viewport creation into one place. ```js const element = testUtils.createViewports(renderingEngine, { viewportId, viewportType: ViewportType.STACK, width: 512, height: 128, }); ``` #### Image Id Previously, for the fake image loader, you should have used: ```js const imageId1 = 'fakeImageLoader:imageURI_64_64_10_5_1_1_0'; ``` This string encoded various parameters. Now, it has been restructured into an object for better clarity: ```js const imageInfo1 = { loader: 'fakeImageLoader', name: 'imageURI', rows: 64, columns: 64, barStart: 32, barWidth: 5, xSpacing: 1, ySpacing: 1, sliceIndex: 0, }; const imageId1 = testUtils.encodeImageIdInfo(imageInfo1); ``` same exists for volumeId ```js const volumeId = testUtils.encodeVolumeIdInfo({ loader: 'fakeVolumeLoader', name: 'volumeURI', rows: 100, columns: 100, slices: 4, xSpacing: 1, ySpacing: 1, zSpacing: 1, }); ``` --- ## 3x ### PolySeg Source: https://cornerstonejs.org/docs/llm/migration-guides/3x/1-polyseg.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### Externalized PolySeg PolySeg has been moved from the `cornerstoneTools` package and is now a standalone package called @cornerstonejs/polymorphic-segmentation. #### Usage Now, it's not included in the `cornerstoneTools` package anymore. If you need to enable polymorphic conversions, you'll have to install it and initialize `cornerstoneTools` with it. ```js import * as polyseg from '@cornerstonejs/polymorphic-segmentation'; import { init } from '@cornerstonejs/tools'; init({ addons: { polyseg, }, }); ``` :::note This change was made because we weren't shipping the cornerstone tools with our `polyseg-wasm` dependencies. There were a few issues with bundlers complaining about the static assets included. Now, those who don't want to use it are fine, and those who do will need to install it and initialize `cornerstoneTools` themselves. ::: #### Exports We weren't exposing any functions from the `tools` directory. If you need something, import it from `@cornerstonejs/polymorphic-segmentation`. It exports the following: ```js import { canComputeRequestedRepresentation, // computes computeContourData, computeLabelmapData, computeSurfaceData, // updates updateSurfaceData, // init init, } from '@cornerstonejs/polymorphic-segmentation'; ``` #### computeAndAddContourRepresentation, computeAndAddLabelmapRepresentation, computeAndAddSurfaceRepresentation have been removed from the `tools` directory. If you happen to need them (unlikely), you'll have to build them yourself. ```js import { utilities } from '@cornerstonejs/tools'; import { computeLabelmapData } from '@cornerstonejs/polymorphic-segmentation'; const { computeAndAddRepresentation } = utilities.segmentation; // for labelmap const labelmapData = await computeAndAddRepresentation( segmentationId, Representations.Labelmap, () => computeLabelmapData(segmentationId, { viewport }), () => null ); // for surface import { computeSurfaceData, updateSurfaceData, } from '@cornerstonejs/polymorphic-segmentation'; const SurfaceData = await computeAndAddRepresentation( segmentationId, Representations.Surface, () => computeSurfaceData(segmentationId, { viewport }), () => updateSurfaceData(segmentationId, { viewport }) ); // same for contour import { computeContourData } from '@cornerstonejs/polymorphic-segmentation'; const contourData = await computeAndAddRepresentation( segmentationId, Representations.Contour, () => computeContourData(segmentationId, { viewport }), () => undefined ); ``` --- ### Labelmap Thresholding Tools Source: https://cornerstonejs.org/docs/llm/migration-guides/3x/2-threshold-tools.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### Key Changes: - The nested `strategySpecificConfiguration` object has been removed completely - Configuration properties have been moved to the root level of the configuration object - Threshold configuration has been restructured: - `threshold` array is now a `range` property inside a `threshold` object - Additional threshold properties (`isDynamic`, `dynamicRadius`) are part of the same object - `setBrushThresholdForToolGroup()` function signature has changed to accept a structured threshold object - Strategy-specific properties like `useCenterSegmentIndex` have been moved to the root configuration level - `activeStrategy` is now a standalone property in tool operations data, no longer inside a nested configuration #### Migration Steps: #### 1. Replace strategySpecificConfiguration with direct properties **Before:** ```diff - configuration: { - activeStrategy: 'THRESHOLD_INSIDE_SPHERE_WITH_ISLAND_REMOVAL', - strategySpecificConfiguration: { - THRESHOLD: { - threshold: [-150, -70], - // other threshold properties - }, - useCenterSegmentIndex: true, - }, - } ``` **After:** ```diff + configuration: { + activeStrategy: 'THRESHOLD_INSIDE_SPHERE_WITH_ISLAND_REMOVAL', + threshold: { + range: [-150, -70], + isDynamic: false, + // other threshold properties directly here + }, + useCenterSegmentIndex: true, + } ``` #### 2. Update threshold configuration structure **Before:** ```diff - strategySpecificConfiguration: { - THRESHOLD: { - threshold: [-150, -70], // Previous threshold array format - isDynamic: false, - dynamicRadius: 5 - } - } ``` **After:** ```diff + threshold: { + range: [-150, -70], // New 'range' property replaces 'threshold' + isDynamic: false, + dynamicRadius: 5 + } ``` #### 3. Update setBrushThresholdForToolGroup calls **Before:** ```diff - segmentationUtils.setBrushThresholdForToolGroup( - toolGroupId, - thresholdArgs.threshold, - thresholdArgs - ); ``` **After:** ```diff + segmentationUtils.setBrushThresholdForToolGroup( + toolGroupId, + fullThresholdArgs + ); ``` Note that `thresholdArgs` should now be an object with the structure: ```javascript { range: [min, max], // Previously 'threshold' isDynamic: boolean, dynamicRadius: number } ``` --- ### Labelmap Interpolation Source: https://cornerstonejs.org/docs/llm/migration-guides/3x/3-labelmap-interpolation.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### Not a composition but a utility Previously, interpolation was a brush composition, restricting its use to tools inheriting from a brush. However, interpolation should really be a utility anyone can use, even without a tool. Before, you had to use this workaround for interpolation: ```js addButtonToToolbar({ title: 'Run Overlapping Interpolation', onClick: () => { const toolGroup = ToolGroupManager.getToolGroup(toolGroupId); const activeName = toolGroup.getActivePrimaryMouseButtonTool(); const brush = toolGroup.getToolInstance(activeName); brush.interpolate?.(element1, { extendedConfig: false }); }, }); ``` Now it's as simple as this: ```js import * as labelmapInterpolation from '@cornerstonejs/labelmap-interpolation'; labelmapInterpolation.interpolate({ segmentationId, segmentIndex, }); ``` :::note We once again had to implement a workaround for `itk-wasm` as a dynamic dependency to prevent bundler problems in cornerstone3D 2.0. However, this caused numerous issues. Now, it's a separate, standalone utility package that doesn't need to be bundled with cornerstone3D. ::: #### Migration Remove the `labelmap` interpolation from your custom tools composition. Before: ```javascript const RECTANGLE_STRATEGY = new BrushStrategy( 'Rectangle', compositions.regionFill, compositions.setValue, initializeRectangle, compositions.determineSegmentIndex, compositions.preview, compositions.labelmapInterpolation ); ``` After: ```javascript const RECTANGLE_STRATEGY = new BrushStrategy( 'Rectangle', compositions.regionFill, compositions.setValue, initializeRectangle, compositions.determineSegmentIndex, compositions.preview ); ``` --- ### Segmentation Statistics API Source: https://cornerstonejs.org/docs/llm/migration-guides/3x/4-get-statistics.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### Key Changes: - Statistics calculation has been moved from brush tool methods to a dedicated utility function - Statistics are now calculated asynchronously using web workers - The function signature for getting statistics has changed completely - Progress events are now emitted during statistics calculation #### Migration Steps: #### 1. Replace tool-based statistics methods with the standalone utility **Before:** ```diff - const toolGroup = ToolGroupManager.getToolGroup(toolGroupId); - const activeName = toolGroup.getActivePrimaryMouseButtonTool(); - const brush = toolGroup.getToolInstance(activeName); - const stats = brush.getStatistics(viewport.element, { indices }); ``` **After:** ```diff + const stats = await segmentationUtils.getStatistics({ + segmentationId, + segmentIndices: indices, + viewportId: viewport.id, + }); ``` :::note ViewportId is needed since some statistics calculations are performed regarding the base image in the viewport. ::: --- ### Adapters API Source: https://cornerstonejs.org/docs/llm/migration-guides/3x/5-adapters.md import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; #### Key Changes: - MeasurementsReport has two maps instead of objects for setting the adapter classes, mapping the tool type to adapter class and the tracking id to adapter class. - A new register additional tracking id method exists to allow adding custom adapter methods. - Adapter implementations now have a base class to handle some of the definition. This allows calling into the base class to handle some of the definition such as the is tracking handling. - The MeasurementsReport class is now extensible to create a new class with completely different default handling. To do this, the two map attributes need to be redeclared, and the new instance registered for the handlers. - There is now an init method to create tracking identifiers and register a new handler. - The annotation changed event no longer requires the viewport id/rendering id - This change is done so that measurements can be updated when not visible - The measurement report no longer takes the image to/from world coords as this is provided as a method exported from `@cornerstonejs/core/utilities` - The adapters can hydrate world coordinates, eg for MPR reconstruction #### Migration Steps: #### 1. Replace MeasurementsReports.CORNERSTONE_TOOL_CLASSES_BY_UTILITY_TYPE **Before:** ```diff - const toolClass = MeasurementReports.CORNERSTONE_TOOL_CLASSES_BY_UTILITY_TYPE[toolType]; ``` **After:** ```diff - const toolClass = MeasurementReports.measurementAdapterByToolType.get(toolType); ``` #### 2. Replace Tool instance adapter registration which is identical to existing registration **Before:** ```diff - class MyNewToolAdapter { ... identical to eg Probe Adapter } ``` **After:** ```diff - const MyNewToolAdapter = Probe.initCopy('MyNewTool'); ``` #### 3. Replace old tool registration with registerTrackingIdentifier **Before:** ```diff - class OldToolAdapter { ... identical to eg Length v1.0 except has :v1.0 at end of tracking identifier } ``` **After:** ```diff - MeasurementReport.registerTrackingIdentifier(Length, `${Length.trackingIdentifierTextValue}:v1.0`); ``` #### 4. Remove image to/from world coords in use of MeasurementReport **Before:** ``` // Use cs3d adapters to generate toolState. let storedMeasurementByAnnotationType = MeasurementReport.generateToolState( datasetToUse, // NOTE: we need to pass in the imageIds to dcmjs since the we use them // for the imageToWorld transformation. The following assumes that the order // that measurements were added to the display set are the same order as // the measurementGroups in the instance. sopInstanceUIDToImageId, metaData, csUtilities.imageToWorldCoords ); ``` **After:** ``` // Use cs3d adapters to generate toolState. let storedMeasurementByAnnotationType = MeasurementReport.generateToolState( datasetToUse, // NOTE: we need to pass in the imageIds to dcmjs since the we use them // for the imageToWorld transformation. The following assumes that the order // that measurements were added to the display set are the same order as // the measurementGroups in the instance. sopInstanceUIDToImageId, metaData ); ``` --- ## 4x ### Camera Field of View Changes Source: https://cornerstonejs.org/docs/llm/migration-guides/4x/1-camera-fov.md #### Camera Field of View Changes #### What Changed In version 4.x, images now display edge-to-edge in viewports without the 10% padding that was present in 3.x. #### Visual Difference - **Before (3.x)**: Images had automatic padding around edges (configurable color for background) - **After (4.x)**: Images fill the entire viewport ![](../../assets/fov.png) #### How to Revert If you need the old behavior with padding, add this configuration during initialization: ```javascript import { init } from '@cornerstonejs/core'; init({ rendering: { useLegacyCameraFOV: true, }, }); ``` That's it. Your images will display with padding like they did in 3.x. #### Why We Changed This The new approach provides better screen utilization and more accurate full-width display, especially important for modern high-resolution displays and mobile devices. --- ### Dynamic Volume API Changes Source: https://cornerstonejs.org/docs/llm/migration-guides/4x/2-dynamic-volume-api.md #### Dynamic Volume API Changes #### What Changed In version 4.x, the deprecated timepoint-based API for dynamic volumes has been removed in favor of the dimension group-based API. #### Removed APIs The following deprecated properties and methods have been removed: #### IDynamicImageVolume Interface - `timePointIndex` getter/setter - `numTimePoints` property #### StreamingDynamicImageVolume Class - `timePointIndex` getter/setter - `numTimePoints` property - `getCurrentTimePointImageIds()` method - `flatImageIdIndexToTimePointIndex()` method - `isTimePointLoaded()` method - `checkTimePointCompletion()` method #### Events - `DYNAMIC_VOLUME_TIME_POINT_INDEX_CHANGED` - `DYNAMIC_VOLUME_TIME_POINT_LOADED` #### Migration Guide No migration is needed if you're already using the dimension group-based API. If you're still using the deprecated timepoint API, update your code as follows: #### Property Updates ```javascript // Before (3.x) volume.timePointIndex = 2; // Zero-based const index = volume.timePointIndex; const count = volume.numTimePoints; // After (4.x) volume.dimensionGroupNumber = 3; // One-based (2 + 1) const groupNumber = volume.dimensionGroupNumber; const count = volume.numDimensionGroups; ``` #### Method Updates ```javascript // Before (3.x) const imageIds = volume.getCurrentTimePointImageIds(); const tpIndex = volume.flatImageIdIndexToTimePointIndex(flatIndex); const isLoaded = volume.isTimePointLoaded(timePointIndex); // After (4.x) const imageIds = volume.getCurrentDimensionGroupImageIds(); const groupNumber = volume.flatImageIdIndexToDimensionGroupNumber(flatIndex); const isLoaded = volume.isDimensionGroupLoaded(groupNumber); ``` #### Event Updates ```javascript // Before (3.x) eventTarget.addEventListener( Events.DYNAMIC_VOLUME_TIME_POINT_INDEX_CHANGED, handler ); eventTarget.addEventListener(Events.DYNAMIC_VOLUME_TIME_POINT_LOADED, handler); // After (4.x) eventTarget.addEventListener( Events.DYNAMIC_VOLUME_DIMENSION_GROUP_CHANGED, handler ); eventTarget.addEventListener( Events.DYNAMIC_VOLUME_DIMENSION_GROUP_LOADED, handler ); ``` #### Important Notes - Dimension group numbers are **1-based** (starting from 1) - The old timePointIndex was **0-based** (starting from 0) - When converting, add 1 to timePointIndex to get dimensionGroupNumber #### Why We Changed This The dimension group terminology better reflects the actual data structure and aligns with DICOM standards, making the API more intuitive and consistent. --- ### Async beforeSend Callback Source: https://cornerstonejs.org/docs/llm/migration-guides/4x/3-async-beforesend.md #### Async beforeSend Callback #### What Changed In version 4.x, the `beforeSend` callback in dicomImageLoader's LoaderOptions has been changed from synchronous to asynchronous, now returning a Promise. #### API Change The `beforeSend` callback signature has been updated to support async operations: ```typescript // Before (3.x) beforeSend?: ( xhr: XMLHttpRequest, imageId: string, defaultHeaders: Record, params: LoaderXhrRequestParams ) => Record | void; // After (4.x) beforeSend?: ( xhr: XMLHttpRequest, imageId: string, defaultHeaders: Record, params: LoaderXhrRequestParams ) => Promise | void>; ``` #### Migration Guide #### Update Synchronous Callbacks If you have existing synchronous `beforeSend` callbacks, wrap the return value in a Promise: ```javascript import dicomImageLoader from '@cornerstonejs/dicom-image-loader'; // Before (3.x) - Synchronous dicomImageLoader.init({ beforeSend: function (xhr, imageId, defaultHeaders, params) { const headers = { Authorization: 'Bearer ' + getAuthToken(), 'Custom-Header': 'value', }; return headers; }, }); // After (4.x) - Async with Promise dicomImageLoader.init({ beforeSend: function (xhr, imageId, defaultHeaders, params) { return Promise.resolve({ Authorization: 'Bearer ' + getAuthToken(), 'Custom-Header': 'value', }); }, }); // Or use async/await syntax dicomImageLoader.init({ beforeSend: async function (xhr, imageId, defaultHeaders, params) { return { Authorization: 'Bearer ' + getAuthToken(), 'Custom-Header': 'value', }; }, }); ``` #### Leverage Async Capabilities Now you can perform asynchronous operations in `beforeSend`: ```javascript // Fetch auth token asynchronously dicomImageLoader.init({ beforeSend: async function (xhr, imageId, defaultHeaders, params) { // Can now make async calls const token = await fetchAuthToken(); const customHeaders = await getCustomHeaders(imageId); return { Authorization: 'Bearer ' + token, ...customHeaders, }; }, }); ``` #### Benefits - **Async Operations**: Fetch authentication tokens or headers from remote sources - **Token Refresh**: Automatically refresh expired tokens before requests - **Conditional Headers**: Dynamically determine headers based on async checks - **Better Integration**: Works seamlessly with modern async authentication flows #### Important Notes - The callback must now return a Promise, even for synchronous operations - Use `Promise.resolve()` for immediate values or `async/await` syntax - The XHR request will wait for the Promise to resolve before sending - Rejected promises will cause the image load to fail #### Why We Changed This Modern authentication workflows often require asynchronous operations (token refresh, OAuth flows, etc.). Making `beforeSend` async enables proper integration with these patterns without workarounds. --- ### Tool consistency of using label instead of text Source: https://cornerstonejs.org/docs/llm/migration-guides/4x/4-tool-consistency-text-label.md #### Tool consistency of using label instead of text #### What Changed In version 4.x, all of the tools now consistently use `annotation.data.label` instead of `annotation.data.text`. Previously, the arrow and label, and key image tools used a mixture of text and label leading to inconsistencies between the displayed values. To support better consistency between the shape of the annotation data created, the addNewAnnotation methods have been changed to call the createAnnotation method instead of each tool creating their own data. #### What you need to change? If you were previously using the `ArrowAnnotateTool`, `LabelTool` or `KeyImageTool` text field, you need to use the label field instead. It is additionally recommended for any annotation tools that you have defined outside CS3D to modify the addNewAnnotation method to call the `this.createAnnotation` method instead of creating your annotation data manually. This will help ensure consistency with any new changes to the basic shape of annotation data. #### Why We Changed This The inconsistency in label naming occasionally caused the wrong label to be used where people expected text to be set or label to be set and changed the wrong value. This allows treating all annotations the same way. Creating annotations consistently allows for updating all annotations when new fields are added or field values are modified. --- ### Node.js 20 Upgrade Source: https://cornerstonejs.org/docs/llm/migration-guides/4x/5-node-20-upgrade.md #### Node.js 20 Upgrade #### Overview Cornerstone3D 4.x requires Node.js 20 or higher. This is an upgrade from the previous requirement of Node.js 18. #### Changes Required #### Update Node Version Update your local development environment to use Node.js 20 or higher: ```bash #### Using nvm (Node Version Manager) nvm install 20 nvm use 20 #### Or install directly from nodejs.org ``` #### Update Package.json Update your `package.json` engines field: ```json { "engines": { "node": ">=20" } } ``` --- ### Rendering Engine Viewport Accessors Source: https://cornerstonejs.org/docs/llm/migration-guides/4x/6-rendering-engine-viewport-accessors.md #### Rendering Engine Viewport Accessors #### Overview `RenderingEngine.getStackViewport()`, `RenderingEngine.getStackViewports()`, and `RenderingEngine.getVolumeViewports()` have been removed. Use `getViewport()` or `getViewports()` and then filter by the behavior you need. This change matters for ViewportV2 because a `PlanarViewport` can expose stack-style or volume-style compatibility methods without being a `StackViewport` or `VolumeViewport`. #### Why This Changed The old accessors classified viewports by legacy concrete classes. That breaks down for V2 viewports, where the same viewport may support: - image-slice workflows such as `setStack()` - volume workflows such as `setVolumes()` - shared image queries such as `getCurrentImageId()` or `hasImageURI()` without actually being a legacy stack or volume viewport type. #### Migration #### `getStackViewport(viewportId)` Before: ```ts const viewport = renderingEngine.getStackViewport(viewportId); await viewport.setStack(imageIds); ``` After: ```ts import { utilities } from '@cornerstonejs/core'; const viewport = renderingEngine.getViewport(viewportId); if (!utilities.viewportSupportsStackCompatibility(viewport)) { throw new Error(`Viewport ${viewportId} does not implement setStack`); } await viewport.setStack(imageIds); ``` #### `getStackViewports()` Before: ```ts const stackViewports = renderingEngine.getStackViewports(); ``` After: ```ts import { utilities } from '@cornerstonejs/core'; const stackViewports = renderingEngine .getViewports() .filter(utilities.viewportSupportsStackCompatibility); ``` If you only need image-slice queries, use the narrower guard instead: ```ts const sliceViewports = renderingEngine .getViewports() .filter(utilities.viewportSupportsImageSlices); ``` #### `getVolumeViewports()` Before: ```ts const volumeViewports = renderingEngine.getVolumeViewports(); ``` After: Choose the guard that matches the operation you need: ```ts import { utilities } from '@cornerstonejs/core'; const volumeInputViewports = renderingEngine .getViewports() .filter(utilities.viewportSupportsVolumeCompatibility); const volumeActorViewports = renderingEngine .getViewports() .filter(utilities.viewportSupportsVolumeActors); const volumeURIViewports = renderingEngine .getViewports() .filter(utilities.viewportSupportsVolumeURI); ``` #### New Capability Guards Cornerstone3D now exposes capability-based helpers under `utilities`: - `viewportSupportsStackCompatibility` - `viewportSupportsImageSlices` - `viewportSupportsStackCalibration` - `viewportSupportsVolumeCompatibility` - `viewportSupportsVolumeActors` - `viewportSupportsVolumeId` - `viewportSupportsVolumeURI` These guards let your code depend on supported behavior instead of legacy viewport classes. --- ### 4.0 Migration Guides Source: https://cornerstonejs.org/docs/llm/migration-guides/4x/index.md import DocCardList from '@theme/DocCardList'; import {useCurrentSidebarCategory} from '@docusaurus/theme-common'; #### 4.0 Migration Guides Here you can find the migration guides for the 4.0 release. item.docId !== 'migration-guides/4x/index')}/> --- ## 5x ### 5.x Migration Reference Notes Source: https://cornerstonejs.org/docs/llm/migration-guides/5x/1-migration-notes.md #### 5.x Migration Reference Notes This page tracks smaller migration-impacting behavior changes that are useful as reference during 4.x -> 5.x upgrades. #### `disableScale` and `imageFrame.preScale` #### What Changed In 5.x, when `disableScale` is `true`, Cornerstone3D no longer sets `imageFrame.preScale` and preserves the original pixel min/max range (`minAfterScale = minBeforeScale`, `maxAfterScale = maxBeforeScale`). This is intentional for cases where scaling is identity (for example slope/intercept being 1/0). #### Why This Matters In 4.x, some workflows implicitly relied on `imageFrame.preScale` always being present. In 5.x, that object may be `undefined` when scaling is disabled. #### Migration Guidance - Treat `imageFrame.preScale` as optional and guard access accordingly. - If your downstream logic requires a pre-scale descriptor, create one in your application code when `disableScale` is enabled. - If you only need pixel statistics, use `minPixelValue`/`maxPixelValue` from the image frame values directly instead of assuming post-scale values. #### `instance` data object model in metadata modules #### What Changed In 5.x, this is primarily a documentation clarification rather than a new runtime behavior change: `instance` data should be understood as a single per-frame object that includes computed per-frame values merged into one object. This object can use inheritance to compose values from multiple metadata levels. Because of that, consumers should not assume all attributes are directly iterable/enumerable on the object itself. #### 4.x vs 5.x interpretation - **4.x:** this shape/behavior existed in practice, but was not clearly documented. - **5.x:** the same model is now explicitly documented so integrations can rely on the intended contract. #### Migration Guidance - Do not rely on object enumeration (`Object.keys`, `for...in`) to discover all available attributes on instance data. - Access known attributes explicitly, or use module utilities that understand the composed/inherited object structure. - When building instance data from naturalized metadata, prefer the `combineFramesInstance` utility so downstream modules receive the expected base object shape. #### SEG adapter: `createFromDICOMSegBuffer` deprecated in favor of `createFromDicomSegImageId` #### What Changed A new `adaptersSEG.Cornerstone3D.Segmentation.createFromDicomSegImageId` entry point has been added. Its second argument is a SEG instance `imageId` (with pixels sourced from the provided per-frame `imageId`s / decoder) — it does **not** accept a Part 10 `ArrayBuffer`, despite the older name implying a buffer. `createFromDICOMSegBuffer` is **not removed**. It remains exported as a deprecated alias that preserves its original 4.x contract (a Part 10 `ArrayBuffer` as the second argument) by delegating to `createLabelmapsFromDICOMBuffer`. Existing buffer-based callers continue to work unchanged; no major version bump is required to upgrade. New code should migrate to `createFromDicomSegImageId` (for the per-frame `imageId` path) or `createLabelmapsFromDICOMBuffer` (for the buffer path). ```ts // 4.x const results = await adaptersSEG.Cornerstone3D.Segmentation.createFromDICOMSegBuffer( referencedImageIds, arrayBuffer, // <-- ArrayBuffer { metadataProvider } ); // 5.x const results = await adaptersSEG.Cornerstone3D.Segmentation.createFromDicomSegImageId( referencedImageIds, segImageId, // <-- SEG instance imageId { metadataProvider, frameImageIds } ); ``` #### Why This Matters The new name exists because the per-frame `imageId` path changed the second argument contract entirely (`ArrayBuffer` -> `imageId`). Rather than silently repurpose the same-named function with an incompatible contract, the new behavior lives under the new name `createFromDicomSegImageId`. The original `createFromDICOMSegBuffer` is retained as a deprecated alias that keeps its old `ArrayBuffer` contract, so existing callers keep working without code changes and the upgrade does not require a major version bump. #### The `frameImageIds` option (optional) `frameImageIds` is **optional** and most integrations never need to set it. It is the list of loadable imageIds — one per SEG frame — that the adapter passes to the image loader to read pixel data. In other words, it is the set of frames the segmentation contains, exactly as produced when the segmentation object is loaded. It exists because of a change in how pixels are obtained: the old buffer-based path decoded the entire SEG from a single Part 10 `ArrayBuffer` held in memory, so individual frames never needed their own imageIds, whereas the new path loads each frame's pixels through the image loader and therefore needs one addressable imageId **per frame**. You only need to pass it for **data sources whose imageIds do not follow the DICOMweb (WADO-RS) or WADO-URI conventions.** When the SEG `imageId` uses a frame-addressing scheme the adapter recognizes, the per-frame list is derived automatically and `frameImageIds` can be omitted: - **WADO-RS / DICOMweb** — frames are separate resources (`.../frames/1`, `.../frames/2`, …), so the list is derived by substituting the frame number. - **WADO-URI** — frames are selected with a query parameter (`?frame=1`, `&frame=2`, …), so the list is derived by appending the frame query. For any other imageId form (custom schemes, blob/object URLs that are not WADO-URI, application-specific loaders, etc.) there is no general rule for turning a base `imageId` into per-frame imageIds, so the adapter cannot auto-generate the list. In those cases pass `frameImageIds` explicitly (or a `getFrameImageId(segImageId, frameNumber)` callback). If you omit it for an unrecognized multi-frame `imageId`, every frame falls back to the same base `imageId` and decodes identical pixels. ```ts // Single-frame SEG, WADO-RS, and WADO-URI imageIds: frameImageIds is not needed. const results = await adaptersSEG.Cornerstone3D.Segmentation.createFromDicomSegImageId( referencedImageIds, segImageId, { metadataProvider } ); // Non-WADO scheme only: provide the per-frame imageIds from loading the SEG. const results = await adaptersSEG.Cornerstone3D.Segmentation.createFromDicomSegImageId( referencedImageIds, segImageId, { metadataProvider, frameImageIds, // one loadable imageId per SEG frame } ); // Or supply a builder instead of the full list: // getFrameImageId: (segImageId, frameNumber) => `${segImageId}?frame=${frameNumber}` ``` #### Migration Guidance - If you load a SEG via per-frame `imageId`s (the OHIF / imageLoader path), switch the call to `createFromDicomSegImageId` and pass the SEG instance `imageId` as the second argument. - If you still have a Part 10 `ArrayBuffer`, use `createLabelmapsFromDICOMBuffer` (`(referencedImageIds, arrayBuffer, metadataProvider, options)`) or `generateToolState`, which retain the buffer-based entry point. - Existing `createFromDICOMSegBuffer(referencedImageIds, arrayBuffer, { metadataProvider })` calls keep working unchanged — the function is now a deprecated alias for the buffer path. Migrate at your own pace to `createLabelmapsFromDICOMBuffer`. #### ESM packaging and TypeScript `moduleResolution` #### What Changed The published `@cornerstonejs/*` packages now declare themselves as ESM (`"type": "module"`) and emit relative imports with explicit `.js` extensions in both the runtime `.js` files and the `.d.ts` declarations. This makes the packages resolve correctly under **native Node ESM** (server-side rendering, Node test runners, packaging linters, and Node 25+ which hard-fails on missing extensions), not just inside bundlers. #### Why This Matters - **Bundler consumers are unaffected.** webpack, Vite, Next, and similar tools resolve `./foo` and `./foo.js` identically, so applications such as OHIF require no changes. - **Native Node now works.** Importing a package on a Node code path no longer fails with `ERR_MODULE_NOT_FOUND` due to extensionless specifiers. - **CommonJS `require()` is not a supported package entry path.** Consume `@cornerstonejs/*` packages with ESM `import`, dynamic `import()`, or a bundler that resolves the ESM export map. #### Migration Guidance Use a modern TypeScript module resolution mode — `"bundler"`, `"node16"`, or `"nodenext"` — which is the default for current toolchains and understands the `.js`-extensioned imports inside the shipped `.d.ts` files. The legacy `moduleResolution: "node"` (a.k.a. `node10`) does **not** map a `.js` specifier in a declaration back to its `.d.ts`, and it ignores the package `exports` map entirely. On that setting some deep re-exported types may resolve as `any` or fail to resolve. This is a **type-resolution** concern only — runtime behavior is unaffected — but if you see missing types, switch to `"bundler"`/`"node16"`/`"nodenext"`. #### Viewport elements set `touch-action: none` #### What Changed The rendering engine now sets `touch-action: none` on every element it enables as a viewport, and restores the element's prior inline value when the viewport is disabled. Previously this was left to the application. #### Why This Matters Without `touch-action: none`, the browser claims viewport gestures before Cornerstone sees them: a one-finger drag scrolls the page instead of running the active tool, a two-finger pinch zooms the document rather than the image, and a double-tap triggers the browser's own zoom. Touch tools cannot work on an element the browser is still handling, which is why this is applied unconditionally rather than behind a configuration flag — there is no useful behavior to preserve on the other side of the switch. The visible consequence is that **dragging on a viewport no longer scrolls the page** on touch devices. Applications that relied on a viewport being a valid place to start a page scroll need to provide scrollable area around the viewport instead. Two notes on scope: - Only viewport elements are affected. The rest of your layout is untouched. - The value is applied inline, so it overrides a `touch-action` coming from a CSS class for the duration that the viewport is enabled. On disable the element's original inline value is restored, and any CSS-supplied value takes effect again. #### Migration Guidance - **Remove application-level workarounds.** If you set `touch-action: none` (or attached `preventDefault` touch listeners) on viewport elements to get touch tools working, that code is now redundant and can be deleted. - **Check your scroll affordances on small screens.** If a page relied on viewport drags to scroll, add padding, a scroll container, or a gutter outside the viewport elements so the page remains scrollable on a phone or tablet. --- ### Generic Viewport Source: https://cornerstonejs.org/docs/llm/migration-guides/5x/2-generic-viewport.md #### Generic Viewport Migration Guide Generic Viewport adds new viewport implementations and an optional compatibility mode for routing legacy viewport types through those implementations. Most application code that creates a viewport and then calls the standard data APIs still works through compatibility adapters. The code most likely to need changes is code that depends on concrete viewport classes, old rendering-engine accessors, generic `setDataIds()`, or raw `viewport.type` checks. #### How Generic Viewport Is Enabled You can use Generic Viewport directly by requesting a Generic viewport type: ```ts renderingEngine.enableElement({ viewportId, element, type: Enums.ViewportType.PLANAR_NEXT, }); ``` You can also opt legacy viewport creation into Next-backed compatibility adapters: ```ts import { init } from '@cornerstonejs/core'; init({ rendering: { useGenericViewport: true, }, }); ``` When `rendering.useGenericViewport` is true, legacy viewport requests are remapped internally: | Requested type | Runtime type | | --------------------------- | ------------------------------- | | `ViewportType.STACK` | `ViewportType.PLANAR_NEXT` | | `ViewportType.ORTHOGRAPHIC` | `ViewportType.PLANAR_NEXT` | | `ViewportType.VIDEO` | `ViewportType.VIDEO_NEXT` | | `ViewportType.ECG` | `ViewportType.ECG_NEXT` | | `ViewportType.WHOLE_SLIDE` | `ViewportType.WHOLE_SLIDE_NEXT` | | `ViewportType.VOLUME_3D` | `ViewportType.VOLUME_3D_NEXT` | Direct Generic viewport types use the new APIs. Remapped legacy viewport types use compatibility adapters that preserve legacy methods such as `setStack()`, `setVolumes()`, `setVideo()`, `setEcg()`, and `setWSI()` where applicable. These adapters are a temporary migration layer, not the long-term Next API surface, and their legacy helpers should be expected to be removed in a later breaking release. Keep those API families separate for a given viewport instance: use the legacy methods on compatibility viewports, or use Generic methods such as `setDisplaySets()` and `addDisplaySet()` on direct Generic viewports. Mixing legacy data mounting with direct Generic data mounting on the same viewport can leave legacy presentation defaults and Generic data state out of sync. #### Extending Viewport Types (New Pattern) #### When you actually need a new viewport type The built-in viewport types cover a fixed set of render paths: stack and volume image slices, 3D volumes, whole-slide tiles, video frames, and ECG waveforms. You only need to register a _new_ type when you want a viewport to draw something none of those render paths model. A good example is a **3D contour viewport for a digital twin**. Cornerstone can already render contour geometry — for example DICOM RT Structure Set contours — but only as a segmentation _overlay_ aligned to an image source view. A digital-twin view instead makes the contour geometry the **primary source data**: there is no underlying image, so the contour itself defines the view, including navigation and camera. No built-in source render path models contour geometry as the primary data, so a custom `Contour3D` viewport class owns its own data shape, render path, and view state while still participating in the rendering engine, projection service, and tooling like any other viewport. Rule of thumb: register a new type only for a genuinely new **data shape** or **render path**. If you can express what you need with an existing viewport's source/overlay bindings and presentation, do that instead. #### Registering the type Built-in and extension viewport type names live on **`Enums.ViewportTypes`**, a runtime constants map in the enums package (not the legacy `ViewportType` enum). - **Built-ins:** `Enums.ViewportTypes.STACK`, `Enums.ViewportTypes.PLANAR_NEXT`, etc. - **Extensions:** `registerViewportType({ name: 'Contour3D', ... })` then `Enums.ViewportTypes.Contour3D` - **Types:** augment `ViewportTypeConstants` (and `ViewportTypeRegistry` for the wire-value union) from the constants you export. `Enums.ViewportTypes` is typed from `ViewportTypeConstants`, so new keys pick up the correct literal types automatically. The deprecated `Enums.ViewportType` enum is unchanged at runtime and is **not** extended when you register new types. #### 1) Declare the name and type augmentation in one place Export the name and wire value as constants and derive the type augmentation from them. The literal strings then live in exactly one file and every other step imports the constants instead of retyping them. Because this file now carries runtime values it is a regular `.ts` module, not a `.d.ts` (a `.d.ts` is type-only and cannot emit the `export const`s). ```ts // my-extension/src/viewportTypes.ts import '@cornerstonejs/core'; // Single source of truth for this extension's viewport type. export const CONTOUR_3D_NAME = 'Contour3D'; export const CONTOUR_3D_TYPE = 'myOrg:contour3d'; declare module '@cornerstonejs/core' { interface ViewportTypeRegistry { [CONTOUR_3D_TYPE]: typeof CONTOUR_3D_TYPE; } interface ViewportTypeConstants { readonly [CONTOUR_3D_NAME]: typeof CONTOUR_3D_TYPE; } } ``` The computed keys are valid because both constants have string-literal types, so `Enums.ViewportTypes.Contour3D` and the `'myOrg:contour3d'` wire value are both derived from these two declarations. #### 2) Register the type at runtime Import the constants and pass them straight to `registerViewportType`. The call is typed against the step 1 augmentation: `name` is constrained to a declared key and `type` is pinned to that key's wire value, so a mismatched pair is a compile-time error. Importing `viewportTypes.ts` here also pulls in its `declare module` augmentation, so the new name is present on `Enums.ViewportTypes` wherever this module is loaded. ```ts import { registerViewportType } from '@cornerstonejs/core'; import { CONTOUR_3D_NAME, CONTOUR_3D_TYPE } from './viewportTypes'; registerViewportType({ name: CONTOUR_3D_NAME, type: CONTOUR_3D_TYPE, ViewportClass: Contour3DViewport, }); ``` After this runs, `Enums.ViewportTypes.Contour3D === 'myOrg:contour3d'`. #### 3) Enable elements using the registered type Reuse the same constant, or use the `Enums.ViewportTypes` accessor once registration has populated it: ```ts import { Enums } from '@cornerstonejs/core'; import { CONTOUR_3D_TYPE } from './viewportTypes'; renderingEngine.enableElement({ viewportId: 'digitalTwinViewport', element, type: CONTOUR_3D_TYPE, // or Enums.ViewportTypes.Contour3D }); ``` Notes: - Call `registerViewportType(...)` in your extension entry module **before** any `enableElement(...)` that uses `Enums.ViewportTypes.Contour3D`. - `declare module` only affects TypeScript; it does not register constructors. Runtime registration is required. - Use namespaced wire values (for example, `myOrg:contour3d`) to avoid collisions across extensions. - Import `CONTOUR_3D_TYPE`/`CONTOUR_3D_NAME` rather than retyping the literals. `Enums.ViewportTypes.Contour3D` is the enum-like ergonomic accessor and is equivalent to `CONTOUR_3D_TYPE` after registration. Code that branches on `viewport.type` should also account for the runtime type. Direct planar Generic viewports report `ViewportType.PLANAR_NEXT`; remapped stack and orthographic compatibility adapters still expose their requested legacy type while delegating to the planar Generic implementation internally. #### Removed Rendering Engine Accessors The following `RenderingEngine` methods have been removed: - `getStackViewport(viewportId)` - `getStackViewports()` - `getVolumeViewports()` These methods classified viewports by concrete legacy classes. That does not work reliably with Generic Viewport because a `PLANAR_NEXT` viewport can support stack-style and volume-style behavior without being an instance of `StackViewport` or `VolumeViewport`. Use `getViewport()` and capability guards instead: ```ts import { utilities } from '@cornerstonejs/core'; const viewport = renderingEngine.getViewport(viewportId); if (!utilities.viewportSupportsStackCompatibility(viewport)) { throw new Error(`Viewport ${viewportId} does not support setStack`); } await viewport.setStack(imageIds); ``` For viewport lists: ```ts const stackViewports = renderingEngine .getViewports() .filter(utilities.viewportSupportsStackCompatibility); const volumeViewports = renderingEngine .getViewports() .filter(utilities.viewportSupportsVolumeCompatibility); ``` Available capability guards include: - `viewportSupportsImageSlices` - `viewportSupportsStackCompatibility` - `viewportSupportsStackCalibration` - `viewportSupportsVolumeCompatibility` - `viewportSupportsVolumeActors` - `viewportSupportsVolumeId` - `viewportSupportsVolumeURI` #### Replace Class Checks With Capability Checks Code like this is fragile under Generic Viewport: ```ts if (viewport instanceof StackViewport) { await viewport.setStack(imageIds); } ``` Prefer checking for the behavior you need: ```ts if (utilities.viewportSupportsStackCompatibility(viewport)) { await viewport.setStack(imageIds); } ``` The same applies to `BaseVolumeViewport`, `VolumeViewport`, and `VolumeViewport3D` checks. Use volume capability guards when the code needs `setVolumes()`, actor access, volume-id checks, or volume-URI checks. #### Be Careful With `viewport.type` If `rendering.useGenericViewport` is enabled, a viewport requested as `ViewportType.STACK` or `ViewportType.ORTHOGRAPHIC` has runtime type `ViewportType.PLANAR_NEXT`. Before: ```ts if (viewport.type === Enums.ViewportType.STACK) { // stack-specific path } ``` After: ```ts if (utilities.viewportSupportsImageSlices(viewport)) { // image-slice path } ``` Use `viewport.type` when you truly need to know the runtime implementation. Use capability guards when you need to know what operations are supported. #### Generic `setDataIds()` Is Replaced The generic base `Viewport.setDataIds()` API has been replaced by the variadic `setDisplaySets()`. Direct Generic viewport code should register logical display set ids and then mount them: ```ts import { Enums, utilities, type PlanarViewport } from '@cornerstonejs/core'; const viewport = renderingEngine.getViewport(viewportId); const displaySetId = 'ct-stack'; utilities.genericViewportDisplaySetMetadataProvider.add(displaySetId, { kind: 'planar', imageIds, initialImageIdIndex: 0, }); await viewport.setDisplaySets({ displaySetId, options: { orientation: Enums.OrientationAxis.AXIAL, }, }); ``` For a volume-backed planar slice, include the `volumeId` in the registered display set: ```ts utilities.genericViewportDisplaySetMetadataProvider.add(displaySetId, { kind: 'planar', imageIds, initialImageIdIndex: Math.floor(imageIds.length / 2), volumeId, }); ``` If you are using a remapped legacy viewport type through `rendering.useGenericViewport`, prefer keeping the legacy method while migrating: ```ts await viewport.setStack(imageIds); await viewport.setVolumes([{ volumeId }]); ``` #### Direct Generic Viewports Use Display Set APIs These direct Generic viewport types should use `setDisplaySets()` or `addDisplaySet()`: - `ViewportType.PLANAR_NEXT` - `ViewportType.VIDEO_NEXT` - `ViewportType.ECG_NEXT` - `ViewportType.WHOLE_SLIDE_NEXT` - `ViewportType.VOLUME_3D_NEXT` Do not assume direct Generic viewports expose the legacy data-loading method names. For example, direct `PLANAR_NEXT` code should use `setDisplaySets()` instead of `setStack()` or `setVolumes()`. #### Presentation Is Split By Scope Generic Viewport separates viewport navigation from per-data appearance: - View presentation: pan, zoom or scale, rotation, flips, and display area. Direct Next viewports expose this through `viewportProjection`, not viewport instance methods. - Data presentation: VOI, opacity, colormap, blend mode, interpolation, and visibility for one mounted dataset. Before: ```ts viewport.setProperties({ voiRange, colormap, invert: true, }); ``` Direct Next API: ```ts viewport.setDisplaySetPresentation(displaySetId, { voiRange, colormap, invert: true, }); ``` Legacy compatibility adapters keep `setProperties()` and map those values to display set presentation internally for migration only. Because the adapters are temporary, code that can move directly to Next should use `setDisplaySetPresentation()` instead. #### Camera Compatibility Legacy adapters still expose `getCamera()` and `setCamera()`, but clean Next viewport code should use semantic APIs. Treat those adapter methods as temporary migration compatibility that should be expected to be removed in a later breaking release, not as a stable Next camera API. `ViewState` is the viewport source of truth. `setViewState()` and `updateViewState()` are the direct Next mutation paths. ```ts viewport.setViewState({ flipHorizontal: true, rotation: 90, }); viewport.updateViewState(({ rotation = 0 }) => ({ rotation: rotation + 30, })); const nextViewState = viewportProjection.withPresentation(viewport, { zoom: 1.5, pan: [40, -20], }); if (nextViewState) { viewport.setViewState(nextViewState); } ``` Read presentation through the projection service: ```ts const presentation = viewportProjection.getPresentation(viewport, { selector: { pan: true, zoom: true, rotation: true, }, }); ``` Direct Next viewports do not expose `getViewPresentation()` or `setViewPresentation()`. Legacy compatibility adapters may still expose those methods and delegate them through `viewportProjection.withPresentation(...)` followed by `setViewState(...)`. Those compatibility methods are temporary, should not be used in new Next code, and should be expected to be removed in a later breaking release. Before: ```ts viewport.setCamera({ focalPoint, position, }); ``` Now, for display navigation: ```ts const nextViewState = viewportProjection.withPresentation(viewport, { zoom: 2, }); if (nextViewState) { viewport.setViewState(nextViewState); } ``` For spatial navigation across viewports, use references: ```ts targetViewport.setViewReference(sourceViewport.getViewReference()); targetViewport.render(); ``` For planar compatibility adapters, position-only camera patches are not supported: ```ts viewport.setCamera({ position }); ``` Use `focalPoint`, `parallelScale`, `setViewState()`, `updateViewState()`, viewport projection, or view-reference APIs instead. Lower-level planar camera helpers are available for custom synchronizers and tooling that need to derive renderer cameras without going through a viewport. They are grouped under a `planarProjection` namespace export to signal that they sit a tier below the stable viewport API and may change before 3.0 stable: ```ts import { planarProjection } from '@cornerstonejs/core'; const sliceBasis = planarProjection.createImageSliceBasis({ image, canvasWidth, canvasHeight, }); const icamera = planarProjection.resolveICamera({ sliceBasis, camera: viewState, canvasWidth, canvasHeight, }); planarProjection.applyToRenderer({ renderer, activeSourceICamera: icamera }); ``` The namespace also exposes `derivePresentation` (canvas-space pan/zoom/rotation without the world-space focal-point step) and `createVolumeSliceBasis` (for volume-backed planar viewports). Treat these as helper APIs around the planar camera model rather than as the primary viewport control surface. #### Planar Camera State Differences Planar Generic viewports store zoom-to-point anchors as semantic view state. When a stored anchor is replayed on another slice, the anchor is projected onto the current slice plane. This keeps the camera on-plane, but it is not invertible across slice changes: cine or synchronization code that stores a camera on slice N, replays it on slice M, and later returns to slice N can see anchor drift. Use view references for spatial slice synchronization, and treat view presentation as display-only state. When both `viewState.displayArea.scaleMode` and `viewState.scaleMode` are set, the display-area scale mode wins. Set only one of those fields unless the display area is intentionally overriding the broader view-state scaling mode. `PlanarViewport.resetViewState({ resetPan, resetZoom })` resets pan, zoom, rotation, orientation, and flip presentation state by default. It does not reset the current slice. Pass `resetOrientation: false` or `resetFlip: false` to keep those fields. Legacy stack and volume viewports expose `resetCamera` through compatibility adapters, but that name is a temporary migration API that should be expected to be removed in a later breaking release. New Next code should call `resetViewState` on direct Next viewports and explicitly call `setImageIdIndex`, `setOrientation`, or `setViewState` for fields that should not follow the default reset. #### Event And Enabled Element Notes Some event and enabled-element fields are now optional because not every Next viewport has a frame of reference or a legacy camera snapshot at all times: - `CameraModifiedEventDetail.previousCamera` - `CameraModifiedEventDetail.element` - `CameraResetEventDetail.element` - `IEnabledElement.FrameOfReferenceUID` Guard those fields before using them. #### Migration Checklist Search your codebase for these patterns: ```sh rg "getStackViewport|getStackViewports|getVolumeViewports|setDataIds" rg "instanceof (StackViewport|VolumeViewport|BaseVolumeViewport|VolumeViewport3D)" rg "viewport\\.type === Enums\\.ViewportType\\.(STACK|ORTHOGRAPHIC|VIDEO|ECG|WHOLE_SLIDE|VOLUME_3D)" rg "setCamera\\(\\{\\s*position" ``` Then migrate in this order: 1. Replace removed rendering-engine accessors with `getViewport()` or `getViewports()` plus capability guards. 2. Replace concrete class checks with capability guards. 3. If enabling `rendering.useGenericViewport`, audit `viewport.type` checks for legacy types that now run as Next runtime types. 4. For direct Generic viewports, replace generic `setDataIds()` and legacy data loading calls with logical display set ids plus `setDisplaySets()`. 5. Move clean Next presentation code from `setProperties()` to `setDisplaySetPresentation(displaySetId, ...)`. 6. Replace durable camera-state storage with `ViewState`, `viewportProjection`, or view reference APIs. --- ### 5.0 Migration Guides Source: https://cornerstonejs.org/docs/llm/migration-guides/5x/index.md import DocCardList from '@theme/DocCardList'; import {useCurrentSidebarCategory} from '@docusaurus/theme-common'; #### 5.0 Migration Guides Here you can find migration notes for moving from Cornerstone3D 4.x to 5.x. #### Shared utilities (`@cornerstonejs/utils`) 5.x introduces `@cornerstonejs/utils`, a package for shared helpers (for example small math utilities and general-purpose logging) that are also surfaced from other Cornerstone3D packages during the transition. - **Optional for now:** you do not need to change your imports on day one. You can keep consuming the same helpers through their existing package entry points while those re-exports remain available. - **Direction of travel:** over time, **`@cornerstonejs/utils` is intended to be the only published home** for these shared utilities. New code and incremental refactors may prefer importing from `@cornerstonejs/utils` so you are aligned with where they will ultimately live. #### Metadata Module In 5.x, the metadata module is designed as a shared handling layer for viewer metadata concerns, so core behavior is implemented once and reused rather than replicated across DICOMweb-specific code, OHIF-specific flows, JSON ingestion paths, and other module-specific integrations (which often resulted in multiple implementations with differing bugs). #### Optional metadata module features in current CS3D The current CS3D version keeps existing metadata flows working, and also introduces optional features you can adopt incrementally: - Typed getters for metadata lookup. - addMetadata providers for adding information to caches/waiting for async results to be added. - clear metadata handling for removing specific changes and/or removing all cached data - Providers that register directly for a specific single metadata type, so resolution can short-circuit quickly. - Shared caches used across metadata providers and metadata types. - Shared cache usage for Part10, DICOMweb, imageId-derived values, and other cached metadata outputs. - Provision for new metadata types such as display set, series-level, and study-level results. #### Metadata provider caching updates In 5.x, metadata providers are first-class extension points in the retrieval pipeline. Instead of each caller manually transforming and writing metadata, providers can normalize source payloads, compose with other providers, and rely on shared cache behavior for consistent lookups. - Use add-path metadata ingestion when the data requires parameters/externally provided values. For example, the NATURALIZED data is computed from binary part 10 or DICMweb metadata format - Use `metaData.addMetaData(MetadataModules.NATURALIZED, imageId, { dicomwebJson })` for DICOMweb JSON payloads. - Use `metaData.addMetaData(MetadataModules.NATURALIZED, imageId, { part10Buffer })` for async Part10 ingestion (`ArrayBuffer`, `Uint8Array`, or resolver function). - The metadata layer now owns frame/base imageId mapping and derived cache invalidation; avoid direct frame propagation with `setCacheData`. #### New metadata handling (and backwards compatibility) - In 5.x, **NATURALIZED metadata is the base state for DICOM imaging data**. Other metadata modules (for example `INSTANCE` and derived module lookups) are expected to resolve from that canonical naturalized state rather than from source-specific conversion code. - Data sources should migrate to providing metadata through naturalized add handlers (`{ dicomwebJson }` and `{ part10Buffer }`) instead of performing custom source-local conversion to instance/natural objects. - New in 5.x: metadata ingestion is handled through add-path typed-provider requests, so callers can pass source data as options (`{ dicomwebJson }` or `{ part10Buffer }`) and let the provider chain naturalize/cache it. - Existing usage still works: if your app already resolves metadata through the legacy provider chain (`addProvider` / prior `metaData.get(...)` flow), that behavior remains supported while you migrate. - Recommended migration path: move NATURALIZED writes to `metaData.addMetaData(...)` calls and remove custom frame/base propagation logic from app code. - This migration is optional until you adopt the new metadata handler path; legacy flows remain supported during transition. - Why this matters: shared naturalization creates consistent behavior across DICOMweb, Part10, and other ingest paths, and helps eliminate recurring bugs caused by multiple, slightly different conversion implementations. #### Naturalized handlers `registerNaturalizedHandlers()` now registers NATURALIZED handlers as composable read and add provider chains: - **Base imageId query filter:** a shared `baseImageIdQueryFilter` can be plugged into typed provider chains and is registered for `NATURALIZED` at high priority so frame-specific imageIds resolve on canonical base imageId first. - **Synchronous naturalization handler (add path):** when callers provide `{ dicomwebJson }`, the handler naturalizes DICOMweb-style metadata into NATURALIZED output. - **Asynchronous Part10 handler (add path):** accepts `{ part10Buffer }` (`ArrayBuffer`, `Uint8Array`, or resolver function), resolves to NATURALIZED, and commits to shared cache. - **Cache interaction:** with base-image filtering ahead of cache providers, NATURALIZED cache keys remain canonical and downstream typed modules can rely on consistent lookups. Recommended usage: - `metaData.addMetaData(MetadataModules.NATURALIZED, imageId, { dicomwebJson })` for sync naturalization from DICOMweb metadata. - `metaData.addMetaData(MetadataModules.NATURALIZED, imageId, { part10Buffer })` for async naturalization from Part10 payloads. #### Standard cache behavior in 5.x The metadata cache in 5.x is a new shared layer that can be reused by different metadata providers and types. It centralizes cache population, in-flight de-duplication, and query-key consistency so provider implementations can focus on source-specific lookup logic. - Standard caches are now "read-through" caches: fetching metadata is expected to populate cache as a side effect of that fetch. - `metaData.get(type, imageId, options)` resolves providers, and cache providers store successful results for the `(type, imageId)` key. - Async lookups are de-duplicated in-flight, then committed to cache when resolved. - For most modules, avoid manual `setCacheData(...)` calls; prefer provider-based lookup + automatic caching. - Source metadata caches (NATURALIZED and ingestion inputs such as DICOMweb/Part10 handlers) should be keyed by base imageId, while derived per-frame caches are keyed by frame imageId. #### Adding a new cache type - Register a cache for your module/type by calling `addCacheForType('yourType')` during provider registration. - Then register one or more typed providers for that type; returned values are automatically cached under the query key. - Keep provider logic as source-of-truth retrieval; let the cache layer handle storage and re-use. #### Writable cache ingestion path - Prefer add-path ingestion (`metaData.addMetaData(...)`) over direct writable cache setters. - Register writable behavior with `addWritableCacheForType(type)` (currently intended for `NATURALIZED`) so add-path ingestion writes to shared cache consistently. - Use `addCacheForType(type, { secondaryOf: ... })` to register derived caches that should be invalidated when a base cache type changes. - This keeps write behavior centralized in providers while preserving typed-provider cache behavior for reads. #### ImageId mapping changes (old vs new) - Previous behavior: - Frame/base conversion knowledge was spread across call sites, so mappings were inconsistent and not guaranteed to be unique. - Source metadata was sometimes written per-frame instead of at a canonical base imageId. - Metadata 5.x behavior: - NATURALIZED/source metadata is canonicalized to base imageId only. - `INSTANCE` metadata is per-frame and indexed by the frame imageId (including frame selector). - A `FRAME_IMAGE_IDS` typed provider exposes frame-related imageIds generated from canonical base imageId + NATURALIZED metadata. - `FRAME_IMAGE_IDS` now resolves in this order: cache first, then NATURALIZED-backed generation. - If NATURALIZED is unavailable, `FRAME_IMAGE_IDS` resolves to `null`. - If NATURALIZED has no photometric interpretation, `FRAME_IMAGE_IDS` returns a `Set` containing only the base imageId. - If NATURALIZED defines `NumberOfFrames`, frame ids are generated for `1..NumberOfFrames` and include: - DICOMweb path form (`/instances/{sopUID}/frames/{frameNo}`) - Query-param form (`?frame={frameNo}` or `&frame={frameNo}`) - Frame imageId -> base imageId normalization is handled by two filters: one for `/frames/{frameNo}` and one for `[?&]frame={frameNo}`. - The reusable generator `generateFrameImageIdsFromNaturalized(baseImageId, naturalized)` is exported for non-metadata clients that need the same expansion behavior. - A cache provider sits in front of frame/base filters so normalized base lookups and frame imageId expansion are reused. - Migration guidance: - Keep `convertMultiframeImageIds(...)` for generating frame imageIds. - Store or fetch NATURALIZED/source metadata using canonical base imageId. - Resolve per-frame metadata via `INSTANCE`/derived modules using frame imageIds, and rely on provider filters for `frame<->base` normalization. item.docId !== 'migration-guides/5x/index')}/> --- # Tutorials ## Annotation Tools Source: https://cornerstonejs.org/docs/llm/tutorials/basic-annotation-tool.md #### Annotation Tools In this tutorial, you will learn how to use annotation tools to annotate. #### Preface In order to render a volume we need: - Initialize cornerstone and related libraries. - HTMLDivElements to render different orientation of the volume (e.g., one for Axial, one for Sagittal) - The path to the images (`imageId`s). #### Implementation **Initialize cornerstone and related libraries** ```js import { init as coreInit } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; import { init as cornerstoneToolsInit } from '@cornerstonejs/tools'; await coreInit(); await dicomImageLoaderInit(); await cornerstoneToolsInit(); ``` We have already stored images on a server for the purpose of this tutorial. First let's create two HTMLDivElements and style them to contain viewports. ```js const content = document.getElementById('content'); // element for axial view const element1 = document.createElement('div'); element1.style.width = '500px'; element1.style.height = '500px'; // element for sagittal view const element2 = document.createElement('div'); element2.style.width = '500px'; element2.style.height = '500px'; content.appendChild(element1); content.appendChild(element2); ``` Next, we need a `renderingEngine` ```js const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); ``` Loading a volume is possible by using the `volumeLoader` API. ```js // Define a volume in memory const volume = await volumeLoader.createAndCacheVolume(volumeId, { imageIds }); ``` We can then create a `viewport`s inside the renderingEngine by using the `setViewports` API. ```js const viewportId1 = 'CT_AXIAL'; const viewportId2 = 'CT_SAGITTAL'; const viewportInput = [ { viewportId: viewportId1, element: element1, type: ViewportType.ORTHOGRAPHIC, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }, { viewportId: viewportId2, element: element2, type: ViewportType.ORTHOGRAPHIC, defaultOptions: { orientation: Enums.OrientationAxis.SAGITTAL, }, }, ]; renderingEngine.setViewports(viewportInput); await volume.load(); ``` In order for us to use tools, add them inside `Cornerstone3DTools` internal state via the `addTool` API. ```js addTool(BidirectionalTool); ``` Next, create a ToolGroup and add the tools we want to use. ToolGroups makes it possible to share tools between multiple viewports, so we also need to let the ToolGroup know which viewports it should act on. ```js const toolGroupId = 'myToolGroup'; const toolGroup = ToolGroupManager.createToolGroup(toolGroupId); // Add tools to the ToolGroup toolGroup.addTool(BidirectionalTool.toolName); toolGroup.addViewport(viewportId1, renderingEngineId); toolGroup.addViewport(viewportId2, renderingEngineId); ``` :::note Tip Why do we add renderingEngineUID to the ToolGroup? Because viewportId is unique within each renderingEngine. ::: Next, set the Tool to be active, which means we also need to define a bindings for the tool (which mouse button makes it active). ```js // Set the toolGroup.setToolActive(BidirectionalTool.toolName, { bindings: [ { mouseButton: csToolsEnums.MouseBindings.Primary, // Left Click }, ], }); ``` Let's load the volume and set the viewports to render the volume. ```js setVolumesForViewports( renderingEngine, [ { volumeId, callback: ({ volumeActor }) => { // set the windowLevel after the volumeActor is created volumeActor .getProperty() .getRGBTransferFunction(0) .setMappingRange(-180, 220); }, }, ], [viewportId1, viewportId2] ); // Render the image renderingEngine.renderViewports([viewportId1, viewportId2]); ``` #### Final code
Final Code ```js import { init as coreInit, RenderingEngine, Enums, volumeLoader, setVolumesForViewports, } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; import { init as cornerstoneToolsInit, ToolGroupManager, WindowLevelTool, ZoomTool, Enums as csToolsEnums, addTool, BidirectionalTool, } from '@cornerstonejs/tools'; import { createImageIdsAndCacheMetaData } from '../../../../utils/demo/helpers'; const { ViewportType } = Enums; const content = document.getElementById('content'); // element for axial view const element1 = document.createElement('div'); element1.style.width = '500px'; element1.style.height = '500px'; // element for sagittal view const element2 = document.createElement('div'); element2.style.width = '500px'; element2.style.height = '500px'; content.appendChild(element1); content.appendChild(element2); // ============================= // /** * Runs the demo */ async function run() { await coreInit(); await dicomImageLoaderInit(); await cornerstoneToolsInit(); const imageIds = await createImageIdsAndCacheMetaData({ StudyInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.334240657131972136850343327463', SeriesInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.226151125820845824875394858561', wadoRsRoot: 'https://d14fa38qiwhyfd.cloudfront.net/dicomweb', }); // Instantiate a rendering engine const renderingEngineId = 'myRenderingEngine'; const volumeId = 'myVolume'; const renderingEngine = new RenderingEngine(renderingEngineId); const volume = await volumeLoader.createAndCacheVolume(volumeId, { imageIds, }); const viewportId1 = 'CT_AXIAL'; const viewportId2 = 'CT_SAGITTAL'; const viewportInput = [ { viewportId: viewportId1, element: element1, type: ViewportType.ORTHOGRAPHIC, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }, { viewportId: viewportId2, element: element2, type: ViewportType.ORTHOGRAPHIC, defaultOptions: { orientation: Enums.OrientationAxis.SAGITTAL, }, }, ]; renderingEngine.setViewports(viewportInput); await volume.load(); addTool(BidirectionalTool); const toolGroupId = 'myToolGroup'; const toolGroup = ToolGroupManager.createToolGroup(toolGroupId); // Add tools to the ToolGroup toolGroup.addTool(BidirectionalTool.toolName); toolGroup.addViewport(viewportId1, renderingEngineId); toolGroup.addViewport(viewportId2, renderingEngineId); toolGroup.setToolActive(BidirectionalTool.toolName, { bindings: [ { mouseButton: csToolsEnums.MouseBindings.Primary, // Left Click }, ], }); setVolumesForViewports( renderingEngine, [ { volumeId, callback: ({ volumeActor }) => { // set the windowLevel after the volumeActor is created volumeActor .getProperty() .getRGBTransferFunction(0) .setMappingRange(-180, 220); }, }, ], [viewportId1, viewportId2] ); // Render the image renderingEngine.renderViewports([viewportId1, viewportId2]); } run(); ```
You should be able to annotate images with the tools you added. ![](../assets/tutorial-annotation.png) #### Read more Learn more about: - [ToolGroup](../concepts/cornerstone-tools/toolGroups.md) - [Annotations](../concepts/cornerstone-tools/annotation/index.md) For advanced usage of Annotation tools, please visit Volume Annotation Tools example page. :::note Tip - Visit [Examples](examples.md#run-examples-locally) page to see how to run the examples locally. - Check how to debug examples in the [Debugging](examples.md#debugging) section. ::: --- ## Manipulation Tools Source: https://cornerstonejs.org/docs/llm/tutorials/basic-manipulation-tool.md #### Manipulation Tools In this tutorial, you will learn how to add a zoom manipulation tool. #### Preface In order to render a volume we need: - init the libraries - A HTMLDivElement to render the viewport - The path to the images (`imageId`s). #### Implementation **Initialize cornerstone and related libraries** ```js import { init as coreInit } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; import { init as cornerstoneToolsInit } from '@cornerstonejs/tools'; await coreInit(); await dicomImageLoaderInit(); await cornerstoneToolsInit(); ``` We have already stored images on a server for the purpose of this tutorial. First let's create a HTMLDivElements and style it. ```js const content = document.getElementById('content'); const element = document.createElement('div'); // Disable the default context menu element.oncontextmenu = (e) => e.preventDefault(); element.style.width = '500px'; element.style.height = '500px'; content.appendChild(element); ``` Next, we need a `renderingEngine` ```js const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); ``` We can use a StackViewport for this example. ```js const viewportId = 'CT_AXIAL_STACK'; const viewportInput = { viewportId, element, type: ViewportType.STACK, }; renderingEngine.enableElement(viewportInput); ``` RenderingEngine will handle creation of the viewports, and we can get the viewport object and set the images on it. ```js const viewport = renderingEngine.getViewport(viewportId); viewport.setStack(imageIds); viewport.render(); ``` In order for us to use manipulation tools, add them inside `Cornerstone3DTools` internal state via the `addTool` API. ```js addTool(ZoomTool); addTool(WindowLevelTool); ``` Next, create a ToolGroup and add the tools we want to use. ToolGroups makes it possible to share tools between multiple viewports, so we also need to let the ToolGroup know which viewports it should act on. ```js const toolGroupId = 'myToolGroup'; const toolGroup = ToolGroupManager.createToolGroup(toolGroupId); toolGroup.addTool(ZoomTool.toolName); toolGroup.addTool(WindowLevelTool.toolName); toolGroup.addViewport(viewportId, renderingEngineId); ``` :::note Tip Why do add renderingEngineUID to the ToolGroup? Because viewportId is unique within each renderingEngine. ::: Next, set the Tool to be active, which means we also need to define a bindings for the tool (which mouse button makes it active). ```js // Set the windowLevel tool to be active when the mouse left button is pressed toolGroup.setToolActive(WindowLevelTool.toolName, { bindings: [ { mouseButton: csToolsEnums.MouseBindings.Primary, // Left Click }, ], }); toolGroup.setToolActive(ZoomTool.toolName, { bindings: [ { mouseButton: csToolsEnums.MouseBindings.Secondary, // Right Click }, ], }); ``` #### Final Code
Final Code ```js import { init as coreInit, RenderingEngine, Enums } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; import { init as cornerstoneToolsInit, ToolGroupManager, WindowLevelTool, ZoomTool, Enums as csToolsEnums, addTool, } from '@cornerstonejs/tools'; import { createImageIdsAndCacheMetaData } from '../../../../utils/demo/helpers'; const { ViewportType } = Enums; const content = document.getElementById('content'); const element = document.createElement('div'); // Disable the default context menu element.oncontextmenu = (e) => e.preventDefault(); element.style.width = '500px'; element.style.height = '500px'; content.appendChild(element); // ============================= // /** * Runs the demo */ async function run() { await coreInit(); await dicomImageLoaderInit(); await cornerstoneToolsInit(); const imageIds = await createImageIdsAndCacheMetaData({ StudyInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.334240657131972136850343327463', SeriesInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.226151125820845824875394858561', wadoRsRoot: 'https://d14fa38qiwhyfd.cloudfront.net/dicomweb', }); // Instantiate a rendering engine const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); const viewportId = 'CT_AXIAL_STACK'; const viewportInput = { viewportId, element, type: ViewportType.STACK, }; renderingEngine.enableElement(viewportInput); const viewport = renderingEngine.getViewport(viewportId); viewport.setStack(imageIds); viewport.render(); const toolGroupId = 'myToolGroup'; const toolGroup = ToolGroupManager.createToolGroup(toolGroupId); addTool(ZoomTool); addTool(WindowLevelTool); toolGroup.addTool(ZoomTool.toolName); toolGroup.addTool(WindowLevelTool.toolName); toolGroup.addViewport(viewportId); toolGroup.setToolActive(WindowLevelTool.toolName, { bindings: [ { mouseButton: csToolsEnums.MouseBindings.Primary, // Left Click }, ], }); toolGroup.setToolActive(ZoomTool.toolName, { bindings: [ { mouseButton: csToolsEnums.MouseBindings.Secondary, // Right Click }, ], }); viewport.render(); } run(); ```
![](../assets/basic-manipulation-tool.png) #### Read more Learn more about: - [ToolGroup](../concepts/cornerstone-tools/toolGroups.md) - [Tools](../concepts/cornerstone-tools/tools.md) --- ## Segmentation Tools Source: https://cornerstonejs.org/docs/llm/tutorials/basic-segmentation-tools.md #### Segmentation Tools In this tutorial, you will learn how to use the segmentation tools to draw and edit segmentations. #### Preface In order to render a volume we need: - Initialize cornerstone and related libraries. - HTMLDivElements to render different orientation of the volume (e.g., one for Axial, one for Sagittal) - The path to the images (`imageId`s). #### Implementation **Initialize cornerstone and related libraries** ```js import { init as coreInit } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; import { init as cornerstoneToolsInit } from '@cornerstonejs/tools'; await coreInit(); await dicomImageLoaderInit(); await cornerstoneToolsInit(); ``` We have already stored images on a server for the purpose of this tutorial. First let's create three HTMLDivElements and style them to contain viewports for Axial, Sagittal, and Coronal views. ```js const content = document.getElementById('content'); const viewportGrid = document.createElement('div'); viewportGrid.style.display = 'flex'; viewportGrid.style.flexDirection = 'row'; // element for axial view const element1 = document.createElement('div'); element1.style.width = '500px'; element1.style.height = '500px'; // element for sagittal view const element2 = document.createElement('div'); element2.style.width = '500px'; element2.style.height = '500px'; // element for coronal view const element3 = document.createElement('div'); element3.style.width = '500px'; element3.style.height = '500px'; viewportGrid.appendChild(element1); viewportGrid.appendChild(element2); viewportGrid.appendChild(element3); content.appendChild(viewportGrid); ``` For the brush tool, add the `BrushTool`. Both these tools should be added to the `Cornerstone3D` via the `addTool` API and the `ToolGroup`: ```js addTool(BrushTool); ``` for the toolGroup: ```js const toolGroupId = 'CT_TOOLGROUP'; // Define tool groups to add the segmentation display tool to const toolGroup = ToolGroupManager.createToolGroup(toolGroupId); // Segmentation Tools toolGroup.addTool(BrushTool.toolName); ``` And for having the brush tool active as the left mouse button is pressed, set the `BrushTool` to be active: ```js toolGroup.setToolActive(BrushTool.toolName, { bindings: [{ mouseButton: csToolsEnums.MouseBindings.Primary }], }); ``` Next, we can deal with volume loading. First, let's load the actual CT volume we are intending to use for rendering. ```js const volumeName = 'CT_VOLUME_ID'; const volumeId = `${volumeName}`; // Define a volume in memory for CT const volume = await volumeLoader.createAndCacheVolume(volumeId, { imageIds, }); ``` We need another volume for segmentation (we don't want to modify the CT volume for segmentation). We can use the CT volume (`volumeId`) as a reference for metadata to create a new volume for segmentation. ```js const segmentationId = 'MY_SEGMENTATION_ID'; // Create a segmentation of the same resolution as the source data for the CT volume volumeLoader.createAndCacheDerivedLabelmapVolume(volumeId, { volumeId: segmentationId, }); ``` Then, add the created segmentation to the `Cornerstone3DTools` segmentation state. This is done via the `addSegmentation` API: ```js // Add the segmentations to state. As seen the labelmap data // which is the cached volumeId is provided to the state segmentation.addSegmentations([ { segmentationId, representation: { // The type of segmentation type: csToolsEnums.SegmentationRepresentations.Labelmap, // The actual segmentation data, in the case of labelmap this is a // reference to the source volume of the segmentation. data: { volumeId: segmentationId, }, }, }, ]); ``` :::note Important Creation and addition of a segmentation to the `Cornerstone3DTools` segmentation state does not render it on the viewports. `Cornerstone3DTools` have decoupled `Segmentation` from a `Segmentation Representation`. In short, the `Segmentation` has the necessary data for rendering different `Segmentation Representation`s such as `Labelmap`, `Contour` (not supported yet, see roadmap). So you can have multiple `representation` of a single `Segmentation`. Read more at the end of this tutorial. ::: Let's create a rendering engine and add viewports, and let the ToolGroup know about the viewports it is acting on: ```js // Instantiate a rendering engine const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); // Create the viewports const viewportId1 = 'CT_AXIAL'; const viewportId2 = 'CT_SAGITTAL'; const viewportId3 = 'CT_CORONAL'; const viewportInputArray = [ { viewportId: viewportId1, type: ViewportType.ORTHOGRAPHIC, element: element1, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }, { viewportId: viewportId2, type: ViewportType.ORTHOGRAPHIC, element: element2, defaultOptions: { orientation: Enums.OrientationAxis.SAGITTAL, }, }, { viewportId: viewportId3, type: ViewportType.ORTHOGRAPHIC, element: element3, defaultOptions: { orientation: Enums.OrientationAxis.CORONAL, }, }, ]; renderingEngine.setViewports(viewportInputArray); toolGroup.addViewport(viewportId1, renderingEngineId); toolGroup.addViewport(viewportId2, renderingEngineId); toolGroup.addViewport(viewportId3, renderingEngineId); ``` Let's set the volume to load and set it on the viewports ```js // Set the volume to load await volume.load(); // Set volumes on the viewports await setVolumesForViewports( renderingEngine, [ { volumeId, callback: ({ volumeActor }) => { // set the windowLevel after the volumeActor is created volumeActor .getProperty() .getRGBTransferFunction(0) .setMappingRange(-180, 220); }, }, ], [viewportId1, viewportId2, viewportId3] ); ``` Finally, we create a labelmap representation of the segmentation and add it to the toolGroup ```js await segmentation.addLabelmapRepresentationToViewportMap({ [viewportId1]: [ { segmentationId, type: csToolsEnums.SegmentationRepresentations.Labelmap, }, ], [viewportId2]: [ { segmentationId, type: csToolsEnums.SegmentationRepresentations.Labelmap, }, ], [viewportId3]: [ { segmentationId, type: csToolsEnums.SegmentationRepresentations.Labelmap, }, ], }); // Render the image renderingEngine.render(); ``` #### Final code
Final code ```js import { init as coreInit, RenderingEngine, Enums, volumeLoader, setVolumesForViewports, } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; import { init as cornerstoneToolsInit, ToolGroupManager, Enums as csToolsEnums, addTool, BidirectionalTool, BrushTool, segmentation, } from '@cornerstonejs/tools'; import { createImageIdsAndCacheMetaData } from '../../../../utils/demo/helpers'; const { ViewportType } = Enums; const content = document.getElementById('content'); const viewportGrid = document.createElement('div'); viewportGrid.style.display = 'flex'; viewportGrid.style.flexDirection = 'row'; // element for axial view const element1 = document.createElement('div'); element1.style.width = '500px'; element1.style.height = '500px'; // element for sagittal view const element2 = document.createElement('div'); element2.style.width = '500px'; element2.style.height = '500px'; // element for coronal view const element3 = document.createElement('div'); element3.style.width = '500px'; element3.style.height = '500px'; viewportGrid.appendChild(element1); viewportGrid.appendChild(element2); viewportGrid.appendChild(element3); content.appendChild(viewportGrid); // ============================= // /** * Runs the demo */ async function run() { await coreInit(); await dicomImageLoaderInit(); await cornerstoneToolsInit(); const imageIds = await createImageIdsAndCacheMetaData({ StudyInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.334240657131972136850343327463', SeriesInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.226151125820845824875394858561', wadoRsRoot: 'https://d14fa38qiwhyfd.cloudfront.net/dicomweb', }); // Instantiate a rendering engine const renderingEngineId = 'myRenderingEngine'; addTool(BrushTool); const toolGroupId = 'CT_TOOLGROUP'; // Define tool groups to add the segmentation display tool to const toolGroup = ToolGroupManager.createToolGroup(toolGroupId); // Segmentation Tools toolGroup.addTool(BrushTool.toolName); toolGroup.setToolActive(BrushTool.toolName, { bindings: [{ mouseButton: csToolsEnums.MouseBindings.Primary }], }); const volumeName = 'CT_VOLUME_ID'; const volumeId = `${volumeName}`; // Define a volume in memory for CT const volume = await volumeLoader.createAndCacheVolume(volumeId, { imageIds, }); const segmentationId = 'MY_SEGMENTATION_ID'; // Create a segmentation of the same resolution as the source data for the CT volume volumeLoader.createAndCacheDerivedLabelmapVolume(volumeId, { volumeId: segmentationId, }); segmentation.addSegmentations([ { segmentationId, representation: { // The type of segmentation type: csToolsEnums.SegmentationRepresentations.Labelmap, // The actual segmentation data, in the case of labelmap this is a // reference to the source volume of the segmentation. data: { volumeId: segmentationId, }, }, }, ]); // Create the viewports const viewportId1 = 'CT_AXIAL'; const viewportId2 = 'CT_SAGITTAL'; const viewportId3 = 'CT_CORONAL'; const viewportInputArray = [ { viewportId: viewportId1, type: ViewportType.ORTHOGRAPHIC, element: element1, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }, { viewportId: viewportId2, type: ViewportType.ORTHOGRAPHIC, element: element2, defaultOptions: { orientation: Enums.OrientationAxis.SAGITTAL, }, }, { viewportId: viewportId3, type: ViewportType.ORTHOGRAPHIC, element: element3, defaultOptions: { orientation: Enums.OrientationAxis.CORONAL, }, }, ]; const renderingEngine = new RenderingEngine(renderingEngineId); renderingEngine.setViewports(viewportInputArray); toolGroup.addViewport(viewportId1, renderingEngineId); toolGroup.addViewport(viewportId2, renderingEngineId); toolGroup.addViewport(viewportId3, renderingEngineId); // Set the volume to load await volume.load(); // Set volumes on the viewports await setVolumesForViewports( renderingEngine, [ { volumeId, callback: ({ volumeActor }) => { // set the windowLevel after the volumeActor is created volumeActor .getProperty() .getRGBTransferFunction(0) .setMappingRange(-180, 220); }, }, ], [viewportId1, viewportId2, viewportId3] ); await segmentation.addLabelmapRepresentationToViewportMap({ [viewportId1]: [ { segmentationId, type: csToolsEnums.SegmentationRepresentations.Labelmap, }, ], [viewportId2]: [ { segmentationId, type: csToolsEnums.SegmentationRepresentations.Labelmap, }, ], [viewportId3]: [ { segmentationId, type: csToolsEnums.SegmentationRepresentations.Labelmap, }, ], }); // Render the image renderingEngine.render(); } run(); ```
You should be able to draw segmentations with the brush tool ![](../assets/basic-segmentation-tools.png) #### Read more Learn more about: - [Segmentation](../concepts/cornerstone-tools/segmentation/index.md) - [SegmentationTools](../concepts/cornerstone-tools/segmentation/segmentation-tools.md) :::note Tip - Visit [Examples](examples.md#run-examples-locally) page to see how to run the examples locally. - Check how to debug examples in the [Debugging](examples.md#debugging) section. ::: --- ## Render Stack of Images Source: https://cornerstonejs.org/docs/llm/tutorials/basic-stack.md #### Render Stack of Images In this tutorial, you will learn how to render a stack of images. #### Preface In order to render a set of images we need: - run initializers for the libraries - an `element` (HTMLDivElement) to use as the container for the viewport - the path to the images (`imageId`s). #### Implementation We have already stored images on a server for the purpose of this tutorial. 1. Initialize the libraries ```js import { init as coreInit } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; await coreInit(); await dicomImageLoaderInit(); ``` 2. Create an HTML element and style it to look like a viewport. First let's create an HTML element and style it to look like a viewport. ```js const content = document.getElementById('content'); const element = document.createElement('div'); element.style.width = '500px'; element.style.height = '500px'; content.appendChild(element); ``` Next, we need a `renderingEngine` and a `viewport` to render the images. ```js const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); ``` We can then create a `viewport` inside the renderingEngine by using the `enableElement` API. Note that since we don't want to render a volume for the purpose of this tutorial, we specify the type of the viewport to be `Stack`. ```js const viewportId = 'CT_AXIAL_STACK'; const viewportInput = { viewportId, element, }; renderingEngine.enableElement(viewportInput); ``` RenderingEngine will handle creation of the viewports, and we can get the viewport object and set the images on it, and choose the index of the image to be displayed. :::info The imageIds that we use here are generated using the `createImageIdsAndCacheMetaData` function. ```js const imageIds = await createImageIdsAndCacheMetaData({ StudyInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.334240657131972136850343327463', SeriesInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.226151125820845824875394858561', wadoRsRoot: 'https://d14fa38qiwhyfd.cloudfront.net/dicomweb', }); ``` ::: ```js const viewport = renderingEngine.getViewport(viewportId); viewport.setStack(imageIds, 60); viewport.render(); ``` :::note Tip Since imageIds is an arrays of imageId, we can set which one to be displayed using the second argument of `setStack`. ::: #### Final code
Final code ```js import { RenderingEngine, Enums, init as coreInit } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; import { createImageIdsAndCacheMetaData } from '../../../../utils/demo/helpers'; const content = document.getElementById('content'); const element = document.createElement('div'); element.style.width = '500px'; element.style.height = '500px'; content.appendChild(element); // ============================= // /** * Runs the demo */ async function run() { await coreInit(); await dicomImageLoaderInit(); // Get Cornerstone imageIds and fetch metadata into RAM const imageIds = await createImageIdsAndCacheMetaData({ StudyInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.334240657131972136850343327463', SeriesInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.226151125820845824875394858561', wadoRsRoot: 'https://d14fa38qiwhyfd.cloudfront.net/dicomweb', }); const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); const viewportId = 'CT_AXIAL_STACK'; const viewportInput = { viewportId, element, type: Enums.ViewportType.STACK, }; renderingEngine.enableElement(viewportInput); const viewport = renderingEngine.getViewport(viewportId); viewport.setStack(imageIds, 60); viewport.render(); } run(); ```
You should see the following: ![](../assets/tutorial-basic-stack.png) #### Read more Learn more about: - [imageId](../concepts/cornerstone-core/imageId.md) - [rendering engine](../concepts/cornerstone-core/renderingEngine.md) - [viewport](../concepts/cornerstone-core/viewports.md) For advanced usage of Stack Viewport, please visit StackViewport API example page. :::note Tip - Visit [Examples](../examples.md) page to see how to run the examples locally. ::: --- ## Render Video Source: https://cornerstonejs.org/docs/llm/tutorials/basic-video.md #### Render Video In this tutorial, you will learn how to render a video. #### Preface In order to render a video we need: - Initialize cornerstone and related libraries. - an `element` (HTMLDivElement) to use as the container for the viewport - the URL to the video. - a server that will serve the video as MP4 using byte range requests - ideally, the video in 'fast start' format #### Implementation **Initialize cornerstone and related libraries** ```js import { init as coreInit } from '@cornerstonejs/core'; await coreInit(); ``` **Create an HTML element** We have already stored images on a server for the purpose of this tutorial. First let's create an HTML element and style it to look like a viewport. ```js const content = document.getElementById('content'); const element = document.createElement('div'); element.style.width = '500px'; element.style.height = '500px'; content.appendChild(element); ``` Next, we need a `renderingEngine` and a `viewport` to render the images. ```js const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); ``` We can then create a `viewport` inside the renderingEngine by using the `enableElement` API. Note that since we want to render a video, we have to specify the `ViewportType.VIDEO`. ```js const viewportId = 'CT_AXIAL_STACK'; const viewportInput = { viewportId, element, type: ViewportType.VIDEO, }; renderingEngine.enableElement(viewportInput); ``` RenderingEngine will handle creation of the viewports, and we can get the viewport object and set the video URL on it, and choose the index of the image to be displayed. ```js const viewport = renderingEngine.getViewport(viewportId); await viewport.setVideoURL( 'https://ohif-assets-new.s3.us-east-1.amazonaws.com/video/rendered.mp4' ); await viewport.play(); ``` :::note Tip For a compliant DICOMweb server, the video will be available on the rendered endpoint. It may require an accept header to force it to be served in MP4 format if it is in MPEG2. It may not support either the fast start encoding or the byte range format, absence of which will prevent seeking through large videos. Small videos will likely be buffered entirely, so they can still seek. For instance you can look at this example in OHIF which uses the rendered endpoint: `https://d33do7qe4w26qo.cloudfront.net/dicomweb/studies/2.25.96975534054447904995905761963464388233/series/2.25.15054212212536476297201250326674987992/instances/2.25.179478223177027022014772769075050874231/rendered` ::: #### Final code
Final code ```js import { init as coreInit, RenderingEngine, Enums } from '@cornerstonejs/core'; const { ViewportType } = Enums; const content = document.getElementById('content'); const element = document.createElement('div'); element.style.width = '500px'; element.style.height = '500px'; content.appendChild(element); // ============================= // /** * Runs the demo */ async function run() { await coreInit(); // Instantiate a rendering engine const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); const viewportId = 'CT_AXIAL_STACK'; const viewportInput = { viewportId, element, type: ViewportType.VIDEO, }; renderingEngine.enableElement(viewportInput); const viewport = renderingEngine.getViewport(viewportId); await viewport.setVideoURL( 'https://ohif-assets-new.s3.us-east-1.amazonaws.com/video/rendered.mp4' ); await viewport.play(); } run(); ```
:::note Tip - Visit [Examples](examples.md#run-examples-locally) page to see how to run the examples locally. - Check how to debug examples in the [Debugging](examples.md#debugging) section. ::: #### Video Annotations If the video viewport is instantiated with a setVideo call on an imageId with associated metadata, then it is possible to use annotations with the video viewport. These annotations will be shown on either a range of frames or a single frame, with some amount of time range allowed so that the annotation will actually be seen. The `AnnotationMultiSelect` class supports setting and retrieving time ranges on annotations. This is done by modifying the imageID in the `/frames/` section or the `frameNumber=` attribute. These become a range when the annotation applies to a range of values. The frame range is automatically set when created to the current range being played on the video when the video is playing, or the frame number currently being displayed when not playing. --- ## Render Volume Source: https://cornerstonejs.org/docs/llm/tutorials/basic-volume.md #### Render Volume In this tutorial, you will learn how to render a volume. #### Preface In order to render a volume we need: - Initialize cornerstone and related libraries. - HTMLDivElements to render different orientation of the volume (e.g., one for Axial, one for Sagittal) - The path to the images (`imageId`s). #### Implementation #### Step 1: Initialize cornerstone and related libraries ```js import { init as coreInit } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; await coreInit(); await dicomImageLoaderInit(); ``` We have already stored images on a server for the purpose of this tutorial. First let's create two HTMLDivElements and style them to contain viewports. ```js const content = document.getElementById('content'); const viewportGrid = document.createElement('div'); viewportGrid.style.display = 'flex'; viewportGrid.style.flexDirection = 'row'; // element for axial view const element1 = document.createElement('div'); element1.style.width = '500px'; element1.style.height = '500px'; // element for sagittal view const element2 = document.createElement('div'); element2.style.width = '500px'; element2.style.height = '500px'; viewportGrid.appendChild(element1); viewportGrid.appendChild(element2); content.appendChild(viewportGrid); ``` Next, we need a `renderingEngine` ```js const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); ``` Loading a volume is possible by using the `volumeLoader` API. ```js const volumeId = 'myVolume'; // Define a volume in memory const volume = await volumeLoader.createAndCacheVolume(volumeId, { imageIds }); ``` We can then create a `viewport`s inside the renderingEngine by using the `setViewports` API. ```js const viewportId1 = 'CT_AXIAL'; const viewportId2 = 'CT_SAGITTAL'; const viewportInput = [ { viewportId: viewportId1, element: element1, type: ViewportType.ORTHOGRAPHIC, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }, { viewportId: viewportId2, element: element2, type: ViewportType.ORTHOGRAPHIC, defaultOptions: { orientation: Enums.OrientationAxis.SAGITTAL, }, }, ]; renderingEngine.setViewports(viewportInput); ``` RenderingEngine will handle creation of the viewports. Next, we need to perform the `load` on the volume. :::note Important Defining a volume is not the same as loading it. ::: ```js // Set the volume to load volume.load(); ``` Finally, let the viewports know about the volume. ```js setVolumesForViewports( renderingEngine, [{ volumeId }], [viewportId1, viewportId2] ); // Render the image renderingEngine.renderViewports([viewportId1, viewportId2]); ``` #### Final code
Click to see the full code ```js import { init as coreInit, RenderingEngine, Enums, volumeLoader, setVolumesForViewports, } from '@cornerstonejs/core'; import { init as dicomImageLoaderInit } from '@cornerstonejs/dicom-image-loader'; import { createImageIdsAndCacheMetaData } from '../../../../utils/demo/helpers'; const { ViewportType } = Enums; const content = document.getElementById('content'); const viewportGrid = document.createElement('div'); viewportGrid.style.display = 'flex'; viewportGrid.style.flexDirection = 'row'; // element for axial view const element1 = document.createElement('div'); element1.style.width = '500px'; element1.style.height = '500px'; // element for sagittal view const element2 = document.createElement('div'); element2.style.width = '500px'; element2.style.height = '500px'; viewportGrid.appendChild(element1); viewportGrid.appendChild(element2); content.appendChild(viewportGrid); // ============================= // async function run() { await coreInit(); await dicomImageLoaderInit(); // Get Cornerstone imageIds and fetch metadata into RAM const imageIds = await createImageIdsAndCacheMetaData({ StudyInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.334240657131972136850343327463', SeriesInstanceUID: '1.3.6.1.4.1.14519.5.2.1.7009.2403.226151125820845824875394858561', wadoRsRoot: 'https://d14fa38qiwhyfd.cloudfront.net/dicomweb', }); // Instantiate a rendering engine const renderingEngineId = 'myRenderingEngine'; const renderingEngine = new RenderingEngine(renderingEngineId); const volumeId = 'myVolume'; // Define a volume in memory const volume = await volumeLoader.createAndCacheVolume(volumeId, { imageIds, }); const viewportId1 = 'CT_AXIAL'; const viewportId2 = 'CT_SAGITTAL'; const viewportInput = [ { viewportId: viewportId1, element: element1, type: ViewportType.ORTHOGRAPHIC, defaultOptions: { orientation: Enums.OrientationAxis.AXIAL, }, }, { viewportId: viewportId2, element: element2, type: ViewportType.ORTHOGRAPHIC, defaultOptions: { orientation: Enums.OrientationAxis.SAGITTAL, }, }, ]; renderingEngine.setViewports(viewportInput); volume.load(); setVolumesForViewports( renderingEngine, [{ volumeId }], [viewportId1, viewportId2] ); } run(); ```
You should be able to see:
![](../assets/tutorial-basic-volume-1.png)
#### Read more Learn more about: - [volumes](../concepts/cornerstone-core/volumes.md) - [rendering engine](../concepts/cornerstone-core/renderingEngine.md) - [viewport](../concepts/cornerstone-core/viewports.md) For advanced usage of Volume Viewport, please visit VolumeViewport API example page. :::note Tip - Visit [Examples](../examples.md) page to see how to run the examples locally. ::: --- ## Examples Source: https://cornerstonejs.org/docs/llm/tutorials/examples.md import Link from '@docusaurus/Link'; #### Examples We have already written plenty number of examples that you can access [here](/docs/examples). When you click on an example you will be taken to its example page. You can interact with each example and see how it works.
Click here to open examples page
#### Source Code and Debugging If you are interested in looking into the source code for each example, we have added a link to the source code when you open the chrome developer tools. You can see the following video to see how to do this. In summary, after opening the chrome developer tools, click on the `console` and click on the `index.ts` that is shown in the console. You can put breakpoints at any line of the code and investigate the variables and functions that are being called.
#### Run Examples Locally You can also run each example locally. It should be noted that `Cornerstone3D` is a monorepo and contains three packages (`core`, `tools`, `streaming-image-volume`). Examples for each of these packages are included in the `examples` directory inside each package. You can run each example by using its name as an argument to the `example` script. For instance, It should be noted that the example name is not case sensitive, and even it can suggest the name of the example you are looking for if you make a typo. ```bash 1. Clone the repository 2. `yarn install --frozen-lockfile` 3. `yarn run example petct` \// this should be run from the root of the repository ``` :::note Important Use the root of the repository as the working directory when running the example. Previously, you had to run the example in each package directory. This is no longer the case. ::: :::danger In general run `yarn install` with the `--frozen-lockfile` flag to help avoid supply chain attacks by enforcing reproducible dependencies. That is, if the `yarn.lock` file is clean and does NOT reference compromised packages, then no compromised packages should land on your machine by using this flag. ::: --- ## Introduction Source: https://cornerstonejs.org/docs/llm/tutorials/intro.md #### Introduction The purpose of this introduction is to give a proper overview of the components that tutorials _rely on_ in order to work properly. Tutorials are learning-oriented and is a great place for you to start trying out various features of our libraries, and we don't want you to get distracted or confused by the implementation details; therefore, we have isolated the learning part of the tutorials (without all the other necessary implementation details) so that you can focus on learning. :::note Info Tutorials are wholly learning-oriented, and specifically, they are oriented towards _learning how_ rather than _learning that_. ([Documentation Philosophy for Cornerstone3D](https://documentation.divio.com/)) ::: #### Running a Tutorial Locally We have included a `tutorial` example in the repo, which you can find at `packages/tools/examples/tutorial/index.ts`. This file contains all the necessary setup code (explained above) for running a tutorial locally. When you open the file, you will see a dedicated place for you to copy and paste and insert the code from the tutorial. So, this way, you don't have to worry about the setup code, and you can focus on the tutorial itself. How to run it? ```bash #### from the root of the library yarn install --frozen-lockfile #### run the tutorial example yarn run example tutorial ``` :::danger In general run `yarn install` with the `--frozen-lockfile` flag to help avoid supply chain attacks by enforcing reproducible dependencies. That is, if the `yarn.lock` file is clean and does NOT reference compromised packages, then no compromised packages should land on your machine by using this flag. ::: Then open a new tab in your browser and navigate to `http://localhost:3000/`. 🎉 Happy Learning 🎉 #### Curious Learner For curious learners, here are some components that are used (behind the scene) for each tutorial. #### Image Loaders `Cornerstone3D` does not deal with loading images. As we will learn later, `Cornerstone3D` also is capable of rendering `Volumes` in any orientation too. Therefore, proper image and volume loaders should be registered with `Cornerstone3D` so that it can work as intended. Examples of such loaders are - imageLoader: `cornerstoneDICOMImageLoader` - volumeLoader: `cornerstoneStreamingImageVolumeLoader` #### Metadata Providers In order for `Cornerstone3D` to properly show the properties of an image such as voi, suv values, etc., it needs metadata (in addition to the image data itself). Therefore, proper metadata providers should be registered with `Cornerstone3D` so that it can work as intended. Examples of such providers are #### Library Initialization Both `Cornerstone3D` and `Cornerstone3DTools` need to be initialized by calling `.init()` methods. ---