Figure 1: Two contributions are combined for the visualisation. Parameters for the dimensions of the transfer function (data range, colour, and opacity) are defined independently.
Computer Graphics & Geometry
Andreas H. Konig
Institute of Computer Graphics Vienna University of Technology
e-mail: koenig@cg.tuwien.ac.at
Eduard M. Groller
Institute of Computer Graphics Vienna University of Technology
e-mail: meister@cg.tuwien.ac.at
A new user-interface paradigm for the specification of transfer functions is presented. The specification is usually a difficult task as mapping information for a number of different domains (data range, colour, opacity, etc.) has to be defined. In the presented approach, the definition of the mapping information can be realised independently for each property domain. A set of specification tools is provided for each domain, enabling users with different levels of experience or demanding time restrictions to choose an appropriate approach for their needs. Real-time feedback during the manipulation of parameters has been proven to be crucial to the specification. An interactive direct-volume-rendering display is realised by utilising dedicated hardware acceleration.
Keywords: Volume Visualisation, Transfer Function Specification, VolumePro ray-casting system
When employing direct-volume-rendering for the visualisation of volumetric data sets, typically a transfer function is used to classify the data. Data values are mapped into optical properties like colour or transparency information. During the rendering procedure the transfer function is evaluated for specific data values in order to gather the contribution of certain voxels or sampled locations to the resulting visualisation.
In general a transfer function 1/2 is defined as a mapping from a cartesian product of scalar fields F to a cartesian product of optical properties O:
t
: F1´F2´...´Fn ® O1´O2´...´OmUp to now the values of the dimensions n and m had to be kept small due to the overwhelming specification effort by the user. Typically a transfer function maps density values (n = 1) to opacity and colour properties (m = 2). All other parameters which are necessary for shading the rendered voxels are determined by a more or less sophisticated illumination model [8]. Transfer functions of higher complexity take usually the magnitude of the gradient within the volume-data set into account (n = 2) [6]. Even for these restricted cases (n; m ¹ 2) the effort for specifying the transfer function is not to be neglected in daily clinical practice. Especially medical doctors with little experience in computers or the mathematical background of volume rendering are usually not able to handle complex paradigms for specifying high-dimensional functions needed for the visualisation.
Several approaches to provide a more user-friendly methodology for the specification of transfer functions have been proposed. Some methods analyse input data [2, 5] or output images [1] to provide the user with an initial transfer-functions setup. When intuitive interaction mechanisms are missing, it is still hard for the user to fine-tune these proposed transfer functions to his needs. Other approaches generate an initial set of transfer functions, which is evolved for some generations by a genetic algorithms scheme [3]. It was also proposed to provide the user with an arranged display of previews, covering as much as possible of the entire domain of the transfer-function specification-parameters [7]. This approach proved to be very useful, as the user is just required to choose an appropriate region in the space of transfer function dimensions in order to gain an initial set of parameters for fine-tuning. The major drawback of this approach was the amount of pre-processing time needed for rendering the large number of previews covering the en-tire transfer function domain. With the rise of rendering hardware dedicated to generating direct-volume-rendering displays of large data sets with interactive frame rates [9], this problem can be over-come. In this paper a strategy similar to this approach is presented. With the possibility of high speed rendering, even more properties in the possible space of transfer function input dimensions can be explored. The goal of the developed approach is to provide medical doctors with a system, which can be used intuitively. Nevertheless the possibilities for the specification of transfer functions must not be restricted.
The following section describes the approach developed by the authors. Then the tools for the specification of contributions from the domains of data-value, colour, and opacity are described. The results section discusses first feedback from medical doctors using the proposed system. Eventually conclusions and topics for future work are stated.
2 Advanced transfer function specification
The specification of transfer functions usually is a difficult task. Appropriate settings have to be found in several search domains of sometimes infinite resolution. Trying to find a useful transfer function by moving a single sample through this high-dimensional search space is not possible. The approach presented in this paper tries to define the specification step by step. If it is possible, the search-space domains are treated separately. Appropriate settings are found independently for these domains. The domains used as examples for the demonstration of the presented approach are:
The presented approach is easily extendable by using different or additional specification domains, like gradient magnitude, principal curvature magnitude [4], etc.
For each specification domain, different user interaction approaches have been developed. A different approach will be chosen by the user, depending on the experience of the user, a-priori knowledge about the investigated volume data set, the dedication to fine-tuning, the need for efficient generation of the transfer function in time-critical fields of application, and the suitability of predefined settings.
The basic concept of our approach is shown in figure 1. Contributions to the visualisation are composed by choosing a certain number of intervals in the domain of the data values ("peaks"), selecting colours for these ranges and have them combined with specified opacities into a single transfer function. If i contributions are to be used, then the transfer function f = {fo ; fc} mapping data values d resolves to:
fo(d) = S pi(d) * Oi
fc(d) = ci if pi(d) ¹ 0 else black
where fo(d) is the opacity definition of the transfer function to be specified, fc(d) the colour definition of the transfer function, pi(d) is the contribution of a certain region of the data domain, oi the opacity combination values for the contributions, and ci are the colours for single contributions. When peaks pi(d) are overlapping, colours ci have to be interpolated. An appropriate colour space (like HLS) has to be used for the interpolation.
The core component of our specification scheme is a visualisation display capable of rendering images of the investigated volume data set with real-time frame rates (see figure 2). Utilising the VP500 hardware ray-casting system [9], this rendering performance can be achieved. Whenever the user is specifying a contribution from the different domains, the influence of the changes made to the 3D rendering are displayed in this window. Note, that not only the final transfer function will be used for the display, but also transfer functions composed by the partially defined contributions from the domains (like certain data only, coloured data regions only, and finally the composed transfer function). Small preview images rendering the volume data set using the intermediate (or final) transfer-function components are generated. Clicking one of these preview images, the user will activate the transfer function used for the preview. The rendering window will display the volume-data set classified with this transfer function, allowing the user to do further investigations by rotating and zooming the view. Using this approach, every manipulation of the transfer function by the user will result in immediate feedback of the changes in the 3D-rendering display (unlike other approaches, which try to give real-time feedback by classifying and colouring 2D slices).
Figure 1: Two contributions are combined for the visualisation. Parameters for the dimensions of the transfer function (data range, colour, and opacity) are defined independently.
The first step in the composition of a visualisation is the specification of certain data ranges pi(d), which shall contribute to the visualisation (e.g. skin, bone, etc). In the current prototype-like implementation, the contributions are specified in the domain of the voxel values of the volume data set. Nevertheless, any scalar dimension of the data could be used for the selection, e.g. gradient, principal curvature magnitude, etc.
Figure 2: Real-time feedback for all modifications by the user is provided by a rendering window, using VolumePro technology for the calculation of the direct volume rendered image.
The contributing regions of the data domain are called peaks (see figure 3). The main properties of a peak are its location (peak center c) and the range of its contribution (peak width w). The contribution characteristic of specific data values within the range of a peak is defined by the shape of the peak. The following shapes have been proven to be useful:
where c is the center of the trapezoid, w is the width (including the slopes), and s is the width of the trapezoids slopes. There are two variation of the trapezoid-shaped peak, which have to be taken special care of: whereas the tent (s = w/2) is useful to extract information which is represented in a very narrow region within the data volume, the box shape (s = 0) should be avoided, as it usually produces artefacts due to the discontinuities at the vanishing slopes. Figure 3 gives a comparison.
Figure 3: Different peak shapes (from top to bottom): trapezoid, tent, box, and ramp.
Several ways for specifying a peak have been included in the presented method. Depending on the task to be solved as well as on the level of experience of the user, a different approach may be chosen.
The central part of the user interface for this step is a bar displaying the range of data values (see figure 4). A grey-level ramp is displayed in this bar in order to give the user simple visual feedback which region of the data domain he is investigating.
When transfer function contributions (peaks) are displayed in the user interface, the numerical properties of a peak (center and width) are shown by plotted markers on this range bar, giving the user a hint, where the contribution he is investigating is located in the data domain.
By default the entire range of data values is displayed. Along with the bar, the limits of the currently active data range are given numerically (e.g. for a data set with a signal resolution of 12 bit the number 0 would be displayed at the left end of the bar, whereas 4095 would mark the right-most extent of the bar). When the user wants to "zoom" into the data range (in order to get a new set of suggestions around an interesting region) the limits of the displayed range can be changed easily by activating a button and clicking the mouse into the spot of the bar, where the new limit should be. An-other button would be used to "reset" the limits of the bar and display the entire range of data values again.
A histogram of the data set is painted into the range bar. Peaks in this histogram give hints where certain materials are to be found in the data domain.
There are a number of possibilities to specify a peak. Depending on the a-priori knowledge about the data set, the experience of the user or the special demands of the visualisation task, a different approach can be chosen. The following techniques have been proven to be useful:
Whichever approach is chosen by the user, the goal of this specification step is one selected contribution from the range of data values for every feature, which has to be distinguished in the final visualisation. Therefore this "data selection" step will be repeated by the user until the set of objects to be visualised is complete (and represented by thumbnail images in the user interface). The following step (described in the next section) yields similar selections from the domain of colour values.
The next step in the specification process deals with the assignment of colour information c i to the ranges selected in the first step (see figure 5). For each peak defined in the first step, colours can be assigned independently. Colour assignment is investigated for one "active" peak at time, which is marked with a red frame in the user interface.
The main part of the user interface for colour specification is a bar representing the hue and saturation domain of possible colours. Whenever a preview image is displayed in this user interface, it is connected to this colour domain bar by a thin line, marking the colour parameters for the colour used to create the rendering. This enables the user to associate hue and saturation information with the presented colour representation quickly. For the sake of convenience, the leftmost part of the colour domain bar does not represent hue, but a grey-level ramp of entirely de-saturated colours with varying luminosity. Using this approach the usually three-dimensional colour space of a HLS-colour system has been reduced to a two-dimensional specification scheme. As the quick specification of transfer function colours usually does not need other luminosity than an average value (0.5 is in the middle of full and no intensity in the HLS-colour system), this approach seems to be sufficient. If the user wants to specify colours with other luminosities, other approaches can be used (see below!).
Figure 4:Specification of contributions from the range of data values.
Again a number of approaches are presented to the user:
The goal of this specification step is the assignment of colour properties for every data-range peak selected in the first step. The final step will deal with the appropriate combination of these "coloured peaks" in the opacity domain.
The final step in the specification process deals with the combination of the colour-assigned data-range peaks prepared in the first steps into a single visualisation. Different opacities o i are assigned to the single peaks, when they are combined into a single transfer function, which is used on the volume data set. The specification of these opacities o i is the contribution of this step. Again a user-friendly interface with real-time rendering feed-back and a number of different approaches for the specification is provided (see figure 6).
Figure 6: User interface for the combination of "coloured peaks" by assigning opacities.
Figure 5: Specification of contributions from the range of colours.
The user usually modifies the opacities until the visualisation rendered with the final transfer function fits his needs. Whenever set-tings defined in the first two steps seem not to be appropriate, the user would go back to data range or colour specification and do some modifications to the definitions there. Changes will of course be reflected in the definitions done in later steps.
The concepts presented in this paper have been implemented in a prototype software system. A standard PC system (500MHz, 512MB RAM) was used as the hardware platform. It is equipped with a Real Time Visualisation VP500 VolumePro board in order to provide interactive direct volume rendering performance. The prototype was developed using MS Visual C++ 6.0 running under Windows NT 4.0 SP5.
Figure 7 shows the entire user interface of the system. This screen shot has been taken after a user has defined three different contributions: skin, vessels (with contrast medium), and bone. These contributions have been assigned the colours green, red, and white. The user has chosen the opacity value of 24% for the skin, 40% for the vessels, and 100% for the bone. The volume-data set rendered with the final transfer function is shown in the lower left hand corner.
Other images created with the presented system can be viewed in higher resolution at the web page http://www.cg.tuwien.ac.at/ATFSpec
Figure 7: Specification of a transfer function with three contributions
A new user interface paradigm for the specification of transfer functions in the field of medical volume visualisation has been presented. The specification task is simplified by realising the definition of mapping parameters for each search domain independently. A prototype software implementation (supporting data range, colour, and opacity as search domains) has been used to prove the usefulness of the presented concepts. Several specification tools for each search domain were presented to a group of medical doctors. De-pending on the users experience and special demands of the visualisation, different tools can be used. VolumePro technology has been used to provide the direct volume rendering performance crucial for interactive feedback to the users when modifying the transfer function. Topics for future work include the investigation of the usefulness of additional search domains (gradient magnitude, principal curvature magnitude, shading coefficients, etc).
The work presented in this publication has been funded by the Vis Med project. Vis Med is supported by Tiani Med-graph, Vienna, http://www.tiani.com , and the Forschungsf orderungsfonds ur die gewerbliche Wirtschaft, Austria, http://www.telecom.at/fff . Please refer to http://www.vismed at for further information on this project. The data sets used to create the images in this publication are courtesy of customers of Tiani Medgraph GesmbH, Vienna. Please check http://www.tiani.comfor detailed information on Tiani Medgraph.
[1] Shiaofen Fang, Tom Biddlecom, and Mihran Tuceryan. Image-based transfer function design for data exploration in volume visualisation. In Proceedings of IEEE Visualisation, pages 319-326, 1998.
[2] Issei Fujishiro, Taeko Azuma, and Yuriko Takeshima. Au-tomating transfer function design for comprehensible volume rendering based on 3D field topology analysis. In Proceedings of IEEE Visualisation, pages 467-470, 1999.
[3] Taosong He, Lichan Hong, Arie Kaufman, and Hanspeter Pfis-ter. Generation of transfer functions with stochastic search techniques. In Proceedings of IEEE Visualisation, pages 227- 234, 1996.
[4] Jiri Hladuvka, Andreas K? onig, and Eduard Gr? oller. Curvature-based transfer functions for direct volume rendering. Accepted for Spring Conference on Computer Graphics (SCCG) 2000, 2000.
[5] Gordon Kindlmann and James W. Durkin. Semi-automatic generation of transfer functions for direct volume rendering. In Proceedings of IEEE Volume Visualisation, pages 79-86, 1998.
[6] Marc Levoy. Display of surfaces from volume data. IEEE Computer Graphics and Applications, 8(3):29-37, May 1988.
[7] J. Marks, B. Andalman, P. A. Beardsley, W. Freeman, S. Gib-son, J. Hodgins, T. Kang, B. Mirtich, H. Pfister, W. Ruml, K. Ryall, J. Seims, and S. Shieber. Design galleries: A gen-eral approach to setting parameters for computer graphics and animation. In Proceedings of ACM SIGGRAPH, pages 389- 400, 1997.
[8] Nelson Max. Optical models for direct volume rendering. IEEE Transactions on Visualisation and Computer Graphics, 1(2):99-108, June 1995. ISSN 1077-2626.
[9] H. Pfistner, J. Hardenbergh, J. Knittel, H. Lauer, and L. Seiler. The VolumePro real time ray-casting system. Proceedings of ACM SIGGRAPH, pages 251-260, 1999.
Computer Graphics & Geometry