Contents

• 1. Introduction
• 2. Previous Work
• 3. Measurement
• 4. Data Preparation
• 5. Color Editing
• 6. Synthesis and Rendering
• 7. Results
• 8. Conclusions and Future Work
• References

1 Introduction

Nowadays image based measurements of the Bidirectional Reflectance Distribution Function (BRDF) and the Bidirectional Texture Function (BTF) are commonplace. But still the measurement devices are different. Most image based BRDF measurement devices use curved material samples to obtain as many data samples per image as possible in contrast to BTF measurement devices that need flat samples. A common assumption when measuring BTFs is orthographic projection and directional lighting resulting in the same incident and outgoing angles for all Apparent BRDFs (ABRDF) of the BTF. But since the cameras in such devices as well as the light sources have finite distance compared to the size of the material samples, these angles vary across the area of the sample. This enables the extraction of BRDF samples from the BTF measurements and fitting of analytical models to this data samples.

One interesting class of materials with stochastic spatial varying reflectance are modern metallic car paints. They show both significant angular and spatial variation of reflectance and in some cases angular dependent color shifts. We present a modified version of the Cook-Torrance BRDF model that is able to reproduce all relevant effects of modern car paint, which are:

• surface gloss,
• spatially varying glitter and depth effect,
• color shift (flip-flop),

Furthermore the model allows for simple changes of the car paints color which enables a designer to visualize a paint in different colors using only one measurement.

2 Previous Work

BRDF measurements
In computer graphics literature there is a rich branch about measuring the BRDF for photo-realistic rendering. The use of measured BRDFs in graphics was introduced by Ward  [ 13] and the proposed measure-and-fit approach (parameters of an analytic reflectance model are fitted to the measured data) is still widely used. In recent work Ngan et al.  [10] qualitatively compared several popular BRDF models. They used a database of about 100 isotropic BRDFs captured by Matusik et al.  [8] which also contains several car paint samples and concluded that physical based models like the Cook-Torrance model  [1] are better suited for these kind of materials than empirical models like the Ward  [ 13] or Lafortune  [5] models. Furthermore, they exemplified that more than one lobe is needed for these kind of layered materials.

BTF measurements
The Bidirectional Texture Function was introduced by Dana et al. in  [2]. They presented their measurement setup consisting of a video camera, a halogen lamp and a robot arm to orient the material sample. This device captured 205 images per material sample. They published their BTF database (CUReT) consisting of 61 different measured samples from many classes of natural materials. Since then a lot of work was done in the area of BTF measurement. Most devices use a gonioreflectometer setup with CCD-based cameras. For an overview we refer to the State-of-the-Art Report from Müller et al.  [9].

Car paint measurements
The findings of Ngan et al.  [10] had some influence on the work of Guenther et al.  [4] who built an BRDF acquisition setup similar to Marschner et al.  [7] in order to obtain sampled car paint BRDFs. They fitted a Cook-Torrance  [1] BRDF model with 3 lobes to the data which can be directly used for rendering. To simulate the spatially varying appearance of sparkles in metallic paints they added the sparkle simulation from Ershov et al.  [3].

Another work on measurement has been done by Sung et al.  [12]. They determine individual flake orientations in the car paint by using a confocal laser scanning microscope and gain a distribution function from this measurements. Additionaly they capture the angular dependent reflectance using a goniometer like setup and build a model for the reflectance based on their measurements.

Our work is an extension of the car paint rendering method from Rump et al. [15]. We use their hybrid approach by enabling color editing on the paint.

The rest of the paper is organized as follows: in the next Section we introduce the BTF measurement procedure we used for appearance capture and explain necessary calibration steps. The section 4 introduces our model for car paint and describes in detail how to efficiently extract BRDF samples from BTF data and how to fit a BRDF model to these datapoints. Section 5 describes our car paint editing tool and section 6 the rendering method. We close the paper with results and conclusions.

3 Measurement

We propose to capture the appearance of spatially varying car paints using BTF measurements. Generally any of the standard devices reviewed in  [9] can be used here. We used a device equipped with 151 digital cameras in order to reduce measurement times and the number of moving parts (see Figure 2). A key feature of the setup is that the built in flashlights are utilized as light sources. Using the 151 cameras with their 151 built-in flashlights we capture 22,801 high-dynamic-range (HDR) images per material which corresponds to typical BTF dataset sizes.

In contrast to pure BTF measurement, where directional lighting and orthographic projection is assumed to be the ideal case, we rely on the fact that in reality image sensor and light source are typically relatively close to the sample. As a result the correct in- and outgoing angles slightly vary across the planar sample. We exploit these variations in order to increase the angular sampling density of the homogeneous BRDF part of the paint. Therefore, we require a very accurate geometrical calibration of the setup. In order to match the real paint color as close as possible we also rely on accurate radiometric calibration. In the following subsections we describe the calibration techniques we used to achieve these goals.

3.1 Geometrical Calibration

For the analytical model fitting procedure described in Section 4.1 we need an as accurate as possible calibration of the intrinsic and extrinsic parameters of each camera. Since the light sources are a fix part of the cameras, no special calibration procedure is needed for them. For the cameras, we only have to determine position and orientation.

Our calibration object exhibits well-known features which can be automatically detected and identified in an image of each camera. We decided on using a planar object with a rectangular grid of 11x11=121 LEDs as features. The grid was chosen as large as possible to increase stability but still small enough that it can be completely seen by all cameras. The number of 11x11 LEDs was chosen to have as many features for the calibration as possible but still small enough that even for the grazing viewing angles neighbouring features may be clearly distinguished. Compared to typical passive calibration targets the use of LEDs as features when observed in an otherwise dark room makes the detection extremely robust and stable even for grazing viewing angles. We use Zhang’s camera calibration algorithm  [14] to solve for the distinct extrinsic and the common intrinsic parameters and optimize for the invididual intrinsic parameters afterwards

3.2 Dynamic Range

In general, the dynamic range of a reflection on car paint can be as high as the dynamic range of the illumination. Hence the reflection properties have to be measured over a very large dynamic range. We realized this by varying the radiation quantity of the light sources (Flash Quantity) and the sensitivity of the CCD sensors (ISO speed). By combining four measurement loops with the settings (FQ2;ISO50), (FQ1;ISO50), (FQ0;ISO50) and (FQ0;ISO400) we achieved a high dynamic range BTF representation of max. 136dB, corresponding to an RGB representation with 23bit per channel.

3.3 Radiometric Calibration

To achieve a consistent color reproduction from all cameras, for each RGB channel of every camera, a response curve was measured taking the unique characteristics of every CCD sensor into account. Slight changes in the characteristics of the sensors are compensated by a white balancing utilizing four reflectance standard targets from 2.3% to 96.9% reflectivity that are positioned near the paint sample.

4 Data Preparation

4.1 Modeling

In this section we explain our model for metallic car paints. The basic idea is to split the reflectance in three parts:

• A homogeneous intensity part modelled by a lobe based BRDF model
• An angular dependent color table to model the color flop effects of pearlescent paints
• A spatial varying part using standard BTF techniques

As suggested by Ngan et al.  [10] we use the Cook-Torrance BRDF model  [1] with multiple lobes to represent the homogeneous BRDF part of the car paint assuming that the paint is not anisotropic. Anisotropy of many car paints is at an order of just a few percent making this assumption resonable. The formula is

(1)

Here ω_i,ω_o are the incoming and outgoing directions. k_d is the diffuse intensity, k_si, m_i and F_0,i are the per-lobe specular coefficient, the distribution exponent and the Fresnel parameter respectively. K is the number of lobes, D is Beckmann’s microfacet distribution. Both the microfacet distribution D and the geometric attenuation term G are from  [1]. For the Fresnel term the approximation of Schlick is used. X denotes the whole set of parameters.

To model the color shift of the car paint the BRDF is split up into an intensity part described by the Cook-Torrance model and a spectral view- and light-dependent part. In the case of spectral measurements the intensity would be

(2)

with normalization of the visible wavelength band to [0,1]. Having a discrete sampling of this spectrum containing N regular samples, this simplifies to

(3)

Since our measurements are in RGB we have N = 3 and end up with the following formula:

(4)

This enables dividing color and intensity of the homogeneous BRDF part:

(5)

The BRDF model part is fit only at the intensity I and we store the scaled colors in a 4D table χ. Since the color-shifts are low-frequent compared to the gloss reflection a coarse sampling of the colors suffices (we use the angular resolution of the BTF measurement). Since this table contains the color information from the paint it is possible to recolor the material by simple substitution of the table. Compressing the table using for example piecewise polynomial basis functions like B-splines should work well and is left for future work.

To obtain the color table χ we simply average the rectified images of the BTF datasets and store the resulting color together with the corresponding incident and outgoing direction in the table. During evaluation linear interpolation of the data points is used.

The complete BRDF model for the homogeneous part of the car paint is now as follows:

   (6)

4.2 BRDF Sample extraction

In this chapter we describe our algorithm to extract BRDF data samples from the BTF measurement data.

We rely on the fact that the cameras of the measurement device have about 64cm distance from a 10x10cm planar material sample. This induces an angular variation of about 12^∘ from one corner of the sample to the opposite corner. In addition to the angle variations induced by the camera placements on the hemispherical gantry this gives a very dense sampling of the materials BRDF.

The basic idea of the data sample extraction is to apply an adaptive box filter to the rectified BTF images to filter out the spatial variation introduced by the flakes. This is simply done by choosing a box size and averaging over image patches of this size. The exact light- and view directions are calculated for the center of the patch based on the calibration data from the calibration step described in Section 3.1. The patch average and the two directions are one sample for the BRDF.

We noticed during our experiments that fitting of the whole BRDF model to these samples at once is numerically unstable (what was also reported by Günther et al.  [4]). The optimizer fails to fit the gloss lobe because there are too few samples of this lobe in the whole sampleset. To solve this problem, we extract two different sets of samples from the BTF data. The first set _d represents the diffuse and glitter part of the BRDF while the second set _s contains data for the gloss lobe.

To extract the diffuse/glitter sampleset _d from the BTF the following steps are performed:

1. Select BTF images out of the total set of images with a good distribution over the view and the light hemispheres.
2. Based on θ_H a patch size for every image is adaptively chosen: . (θ_H denotes the angle between halfway vector and normal vector, θ_H,max is the greatest accepted value for θ_H of a sample.)
3. Take the average over the patches of the selected size and store it together with the directions obtained from the calibrated light and camera positions. This corresponds to the application of an adaptive box filter to the data.

Using an adaptive patch size based on θ_H leads to a better representation of the glitter lobes, since they have the greatest angular variation for small θ_H.

The second sampleset _s is chosen only from images which contain the mirror direction and are very accurately calibrated.The quality of the calibration is measured by comparing the position of the highlight in an image with the reflection position calculated from the calibrated camera and light positions. To find the highlight a list of the n pixels of maximum intensity in the image is built. Then a floodfill with a certain tolerance factor from the pixel of maximum intensity is performed. After the fill an area threshold is checked to test whether the highlight or only a bright flake sparkle was found. If the filled area is too small the next pixel in the list is checked the same way until the highlight is found. The size of the filter for this second sampleset must be chosen carefully since it must be as small as possible in order to preserve as much information about the shape of the gloss lobe as possible but big enough to filter out the noise from the flakes. From our experience a patch size of 4x4 pixels (compared to an image size of about 700x700 pixels for the BTF measurement) lead to the best results since it is large enough to filter out some of the noise induced by the flakes but is still small enough to capture enough samples in the small angular area that is of interest for the gloss lobe.

4.3 The fitting procedure

Having extracted enough samples an optimization process is started to find the parameter set X that leads to the smallest error when comparing the data samples and the BRDF model evaluated with the same angles. Thus the optimization process minimizes an error function well suited for this problem by iterativly changing the BRDF parameters X.

We use an error function that is similar to the one from Günther et al.  [4]:

(7)

Here is one of the sets of data samples from the previous step, M is the Cook-Torrance BRDF model with parameters X. A small positive constant ε prevents division by 0, since for some parameters X the BRDF model may be 0. P is a plausibility function for the parameters X and depends on the BRDF model:

(8)

With a big constant K.

We implemented a quasi-newton-minimizer using the SR-1 update method for the hessian matrix. To overcome problems with local minima of the error function we start the fitting process from several start values and take the best result.

The following pseudo-code describes the whole BRDF fitting algorithm. Parameter sets X are divided into X_s = (k_l,m_l,F_0l) for the last lobe modeling the surface gloss and X_d for the rest of the BRDF. Parameter sets with subscript 0 are start values.

In the first part J resp. K start values X₀ are tried to gain parameters for X_d,X_s. The best result from X_d is already taken for the fitting of X_s. In the second part the parameters are corrected by turns to compensate for the error induced by the separate fit process.

4.4 Separation of BRDF and BTF part

To separate the homogenous BRDF part from the spatial varying BTF part we simply evaluate the BRDF for every pixel of every rectified BTF image and make a simple subtraction in RGB space. The light and view directions for the pixels can be calculated from the camera calibration data from Section 3. The resulting difference images form a new BTF which we simply call :

(9)

Figure 3 shows one images off such a difference BTF. As it can be seen the base color of the paint is no longer part of the BTF. It does only contain colorful effects from the coated flakes. This enables the replacement of the paints color by modifying the color table χ only.

5 Color Editing

Since the color information of the car paint resides in our color table χ we are able to offer very simple editing of the paint. We created a tool that enables the user to define an affine 2D transformation that maps the original colors in the table to new ones. This new table can then directly be used for rendering. Figure 4 shows a screenshot of the tool in use. We display all colors in the color table by a small black cross onto a color triangle. Unicolor paints thus show up as a concentrated spot of crosses (jittered due to small inaccuracies of the measurement). However, pearlescent paints show angular dependent color shifts and therefore complex figures in the color triangle. Their color shift behaviour can be changed by applying the affine transformation.

The transformation T is defined as solution of the following equation:

   (10)

6 Synthesis and Rendering

As explained in Section 4.1 we need to put together the BTF and the BRDF part of our car paint at rendering time. This is our final spatial varying BRDF:

   (11)

This model in its raw form is not suitable for direct rendering because simple tiling of the otherwise randomly distributed flakes would be clearly visible and disturbing. Furthermore the size of the uncompressed data is too large for effective rendering. Furthermore, the uncompressed size of the BTF part for a 128x128 pixel patch with the full set of 151x151 directions would be more than 2 GB which is far too large for effective rendering. The second problem can easily be alleviated by employing BTF compression algorithms. Currently, we compress the flake data BTF using the per-view factorization method from  [11] with a number of 20-25 PCA components which reduces the storage requirements to about 500 MB (including MIP-map levels).

To deal with the tiling problem we propose simple copying of random patches (of size about 16x16 texels).

In order to avoid aliasing artifacts for farther viewing distances we prefilter the BTF part in the spatial domain using a simple box filter and use MIP-mapping during rendering.

For rendering within a global illumination framework we use importance sampling of the BRDF part and the environment map to generate appropriate outgoing directions. Note that for metallic paint it is important to sample the whole BRDF and not only the gloss lobe because the paint layer typically collects significant amounts of light also from off-specular directions. We use the method from  [6] to sample both the BRDF and the environment map and use a balance heuristic for the combination of both.

In order to convert the resulting images from HDR to displayable or printable LDR we use the measured response curve of one camera from the measurement setup instead of a tonemapping operator.

7 Results

We applied the data preparation and fitting process described in Section  4 to 3 different car paints which are depicted in Figure 6. More rendering results can be seen in Figure 1 . We used two different environment-maps here, the first is the well known Uffizi environment from Paul Debevec and the second was captured by ourselves and is shown in Figure 5. It is particular since it contains direct sunlight. The synthesis and rendering stage described in Section 6 was implemented as a MentalRay-shader. Furthermore we recolored the car paints and recombined the BRDF and BTF parts to create new paints. Results can be seen in Figure 7.

Figure 6: The original car paints rendered in two different environments. Top row: Uffizi environment. Bottom row: Main entrance environment

8 Conclusions and Future Work

We presented a novel measurement and rendering framework especially designed for metallic car paint and based on BTF measurements. The measurements capture important effects like depth impression and color-shifts which are preserved in our hybrid analytical and image-based model. The highly specular parts of the material are represented with a BRDF model whose parameters are fitted to the data using a novel BTF resampling procedure. Rendering is performed using BRDF and BTF rendering techniques. Further dividing the BRDF model into an intensity and a color part enables for simple editing of the car paints color.

In future work we plan to investigate more sophisticated BTF compression techniques. Concerning predictive rendering under complex illumination, spectral measurement and rendering is an important issue worth of additional research. Furthermore, we want to integrate our model within a PRT rendering framework in order to achieve real-time rendering with global illumination effects.

References