Abstract

This article is devoted to real time processing and analysis of experimental data to obtain good looking, medically and biomechanical validated human avatar motion data. Common approach for obtaining high quality and in the same time good looking results usually involves both medical professionals and artist work. Artists, using common tools, can process motion data in a way of conserving data visual quality, but nullifying motion medical value in the same time. The approach suggested in this article allows semiautomatic creation of such precise multipurpose data suitable for medical, biomechanical and visual applications via combining different kind of data available from various sources.

For such wide range of application, a lot of parameters, like actual joint angles, translations and limbs positions, beside the visual quality, will become important. Presented approach tries to address all these problems at once.

Keywords: biomechanics, experimental data processing, motion capture, electrogoniomtry.

1. Introduction

Generally, motion capture data are gathered using motion capture system, which can be based upon different principles. Most of motion capture systems are camera based systems, where fixed set of cameras (three or more) synchronously register motion of markers fixed on the object. Such approach itself generates several inaccuracies. The markers which are being registered by camera are not firmly fixed on human avatar and thus its positions slightly drift relatively to the human avatar parts markers are attached on. The work [1] describes these inaccuracies in details. The second source of inaccuracy is errors originated from markers position determination process itself. The reason is image based technology used in motion capture system. Unfortunately, as long as we have to deal with live human motion, we cannot significantly increase the quality of motion capture data using the same technology approach alone. Though some steps can be still made:

· Slow motion is often gives better results as fast one.

· Markers installed on scrawny actor often give better result then on stout one through markers vibration elimination. It is possible to scale the result motion afterwards to adopt for other type of human with respect of both age and sex.

Data, which can be used to increase accuracy of motion capture data in order of relative “usefulness” for proposed method is:

· 6-DOF electrogoniometry: discrete kinematics data (GONIO)

· computer-aided tomography

These two data sources, especially 6-DOF electrogoniometry, generally produce data, which have limited use in direct medical application, but it can be applied to enhance visual and scientific quality of motion capture data.

In the developed technique all these three data sources are interactively combined together in order to produce validated data about human motion. The produced data are supposed to be medically, visually and scientifically correct and utterly conform to all types of measurement as much as possible. The additional measurements can be platform reaction force and/or acceleration measured by additional accelerometers fixed on live actor body.

The calculated data were delivered to the end-user completely or partly, depending on user choice and gives user possibility of verifying the quality of both processed and original data simultaneously via both 3D rendering view and graphical representation of data. Additional functions of described approach software implementation, include, but not limited to: virtual environment models visualization to enhance 3D rendering view representation, visualization of anatomical frames (AF) and anatomical landmarks (AL) in order to enhance user work efficiency and data representation clearness.

2. Motion capture data

Motion capture data are produced by motion capture systems like Elite and stored in PGD or C3D data format[2]. The exact process of motion capture is based on stereophotogrammetric measurement. Finding out human movement in 3-D space requires determination of instantaneous position and orientation of systems of axes, which should be considered to be immoveable relatively to the bone segments under analysis. To achieve this aim, stereophotogrammetric measurement systems are used. Clusters of an adequate number (equal or greater than three) of active or passive markers are positioned either directly or by means of some kind of fixture on the skin surface of the body segments of interest. During the physical exercise performed by the subject under analysis, the laboratory coordinates of these markers are reconstructed by the stereophotogrammetric system. Subsequently, the instantaneous position and orientation of the coordinate frame associated to each cluster are estimated and associated to the corresponding underlying bone. Thus traditional motion analysis (GAIT) allows collecting motion data related to several joint segments simultaneously. GAIT, to a first approximation, deals with reconstruction of human body segment pose, from an adequate number of markers for each segment: joint kinematics comes afterward, with several additional issues and assumptions. The main problem with GAIT is less accuracy when applied in vivo because of skin artifacts[1]. Accuracy of both position and orientation measurements are up to 40 mm and 30° respectively. Enhancement of GAIT data with medical imaging and GONIO data is described below.

3. 6-DOF electrogoniometry

Gathering additional data about human avatar motion is possible via different methods. Several methods for kinematics tracking are possible, including stereometric, electromagnetic, both flexible goniometric and electromechanical linkage systems with one, two, three or six degrees of-freedom (DOFs) and techniques based on medical imaging. In this work a 6 Revolute Instrumented Spatial 19 Linkage (6R-ISL) and a three-dimensional digitizer (3DD) were used simultaneously to collect both static and continuous poses of unconstrained or constrained motions for every joint. Validation was performed using a calibrated ball-and-socket joint. A parametrical model of the 6R-ISL (i.e. Virtual Goniometer or VG, see Figure 1) was designed using standard multibody system geometry.

First application of 6R-ISL system to measure human motion in 3D space was first described in 1972 by Kinzel et al. Sommer[3] and Miller developed a 6R-ISL for the wrist. Grood and Suntay tested and used a 6R-ISL to describe the kinematics of the human knee joint. In 1992, Kirstukas et al. [4] proposed a method to improve the design of a 6R-ISL in a desired range of motion using computer graphics and numerical methods. Liu and Panjabi [5] used a linear and non-linear numerical calibration procedure for each of the potentiometers of their ISL and obtained higher accuracy with low-cost potentiometers (0.5% of independent linearity). Salvia et al. [6]designed a small 6R-ISL to study in vivo wrist circumduction and the resulting pivot point. The use of a 6 DOFs electrogoniometer or digitizer for registration of continuous and discrete kinematics data, respectively, has reinforced the interest in the improvement of the accuracy of these devices which now is very precise and reaches up to 0.01 radians.

Fig. 1 6R-ISL electrogoniometer.

4. Computer-aided tomography

Computer-aided tomography allows gathering morphological data which can be used later in visualization of human avatar moment and enhancement of GAIT data. The Elscint Spiral Twin Flash equipment was used for computer-aided tomography data measurement. It allows parallel double beam scanning of samples with maximum length of scanning of 1000mm, depth of layer of 0.5mm and diameter of scanning beam from 180 to 500mm.

Most of tomography’s measurements were performed on biological material taken from fresh frozen cadavers. The cadavers were defreezed before measurement take place. Parts to be scanned were separated from corpse before initial freezing. Defreeze was performed 48 hours before scanning. Soft tissues are not removed.

In the following table, example values of parameters, which were used during data collection in our work, are presented. We have obtained good results, with these values of parameters:

Pitch	1.5D
Slice Thickness	Epiphysis (joint level): 2.7 mm Diaphysis: 5.0 mm
Slice Increment	Epiphysis: From 0.7 mm to 1.0 mm Diaphysis: From 3.0 mm to 4.0 mm
Image matrix	512² or 768²
Scan diameter	520 mm or 430 mm

If the specimen length is greater than 1000 mm, which is the maximal scanning length allowed by the abovementioned system, then data collection is performed in two steps and an alignment of both datasets is performed later, using the reference plates inserted into the bones. Step 1 includes the area from above the iliac crests to the upper part of the tibial bone diaphysis, while step 2 includes the area from the distal part of the femoral diaphysis to the whole foot for lower limb for example[7].

The result of such scanning is a 3D bone image represented on Fig. 2.

Fig. 2. Right femur bone of male specimen.

5. Registration process

The process of combining all abovementioned sources of data into one validated, medically and scientifically correct data stream is called registration process.

a. GAIT data preprocessing

Before dealing with GAIT data the preliminary processing is performed. Numerical smoothing and fitting of the original motion data were performed using wavelet transformation and cubic smoothing spline[8]. Wavelet transformation allowed removing numerical trembling in GAIT data. The first 6 entries were left after Fast Fourier Transform (FFT) transformation over 32 point range (The number of taken points have to be divisible by power of two to have FFT algorithm working). Spline smoothing allowed getting distribution estimation of both first and second derivatives from both original GAIT and registered GONIO. Such filtering was necessary to obtain smooth behavior for the all available degrees-of-freedom. Smoothing parameters for spline approximation were determined from the curves of joint flexion acceleration. Afterwards, the same parameters were applied for smoothing of the remaining DOFs. For example, there are five independent degrees of freedom (DOF) for human lower limbs. Both hips were assumed to be pure ball-and-socket joints with the three rotational DOFs measured from GAIT data. Rotations in the three anatomical planes (i.e., flexion/extension - Fle/Ext, abduction/adduction – Add/Abd, internal/external rotation – Int/Ext) for the hip, and Fle/Ext for the knee and the ankle joints were taken from GAIT as the five independent DOFs of each limb. The three translations at the hip joint (i.e, anterior/posterior – Ant/Post, superior/inferior – Sup/Inf, medio/lateral – Med/Lat) were assumed to be zero, and the other two rotations plus the three translations at the knee and the ankle joints were taken from GONIO-based passive motion. In other words, the five time-histories for each limb from GAIT were used to synchronize in vitro joint by in vivo full limb kinematics. Though other DOFs were assumed unreliable for registration process, these DOFs will be included later in the final model after the primary registration took place.

b. GAIT data scaling

It real life the situation of having the same GAIT and GONIO data simultaneously from the same subject is almost impossible since GAIT data gathered in vivo, whereas GONIO data collected in vitro. It introduces the necessity to match one data source to another. There are two possibilities of such matching: scaling of GONIO data to GAIT data and scaling of GAIT data to GONIO. Since GONIO data in general are more accurate the GAIT data were scaled.

The scaling procedure for lower limbs processing previously located ALs is based on center of femoral heads (RFH, LFH), centers of the posterior edge of calcaneus (RCA, LCA). ALs positions determination is described in [9].

A set of vectors and for both volunteer (L) and cadaver (C) were defined. Scaling factor for each limb is given by:

Scale factor of both sides was averaged to give the final factor, which was used to scale each frame of the GAIT pelvic motion data using the following relation:

where n is the frame index, is the total number of frames, and PE is the origin of the pelvis as defined at the middle point between the two antero-superior iliac spines.

Similar scaling is applied independently on each frame of the foot (F) on the GAIT data. Both right and left feet were scaled independently using and respectively to obtain the relative translation vector in Eq. 1. Simultaneously, for each frame n, the relative orientation vectors (FPE means Foot relative to Pelvis), expressed according to the OVP [10] convention, were recalculated by:

(1)

where indicates the determination of the relative attitude vector (i.e., OVP) from absolute attitude vectors (i.e., both and ).

Equation Eq.(1) is directly related to GAIT data transformation, and will be mentioned later in comparing registered and original kinematics. This equation also allows building a cost function for parametrical adjustment of the registered data for further optimization.

c. GAIT data registration

The final step of simple registration is to find correct frame in GONIO motion for each frame in GAIT motion and correct GAIT motion upon it.

We have to find two nearest frames for each GAIT data frame in GONIO data. It this case the interpolation between found GONIO data frames is performed. If it is not possible to find two these frames the GONIO data is not suitable for registration of this GAIT data. The process of finding correspondence between frames is called synchronization.

After synchronization is achieved the final position of bones is found.

The current pose (i.e. both orientation and position) of each limb segment was then calculated from equation Eq.2.

(2)

where is the joint index (A = ankle, K = knee in case of lower limb), i is the limb side (R = right, L = left), k is the DOF index, is independent variable, is interpolations functions and is Flexion/Extension angle

The relative pelvis and feet pose was evaluated by:

(3)

The final result of this approach is sensitive to the motion data accuracy obtained for the root of the limb hierarchy. The pelvic bone plays the root role in most cases.

6. Results

The approach described was implemented in a SMART (Skeleton Motion Analysis and Registration Tool) project.

a. SMART structure

The SMART is written on C/C++ program language and utilizes object model of program structure Fig. 3. It consists of five major parts which cover the individual aspect of goal to satisfy which program was developed:

· Computational module. The main part of programs that performs all general calculations over GAIT, GONIO and medical imaging data. It stores and processes all data in computational friendly format.

· Program menu and user interface which communicates with user. It is implemented as independent part of the program to simplify porting SMART to different from PC and MS Windows ® platforms and operation systems respectively.

· File formats reading and export modules allow reading different file formats and converting them into internal format of data representation to supply to the computational and rendering module.

· Registration part. It compliments main computational part and performs registration as data modification process over computational module data.

· Engine rendering utilize DirectX API [12]to achieve efficient 3D rendering performance. SMART partially embeds engine architecture and thus utilize it fully to achieve maximum performance available

Fig. 3 SMART structure

b. Quality of registration

The quality of registration is plausible in terms of visual application even in large joint zooming. Even in maximum zooming when one joint covers entire graphics display there are no large visual jittering and bones interpenetrations. The joint movement remains plausible and corresponding the current medical knowledge about human joint in both visual and numerical representations.

The applied graphs show the joint behavior deference before Fig. 4 and after Fig. 5 registration process took place.

Fig. 4. Standard dependences of internal/external rotation (light gray) and adduction/abduction (dark gray) from knee flexion (range [–140°,0°]). Cycling motion, unregistered case.

Fig. 5. Standard dependences of internal/external rotation (light gray) and adduction/abduction (dark gray) from knee flexion (range [–140°,0°]). Cycling motion registered case.

The Fig. 5 corresponds the well known dependencies found in the literature [12]. The visual quality of image is again shown in both variants: unregistered and registered. The unregistered motion is taken as is it has been received from the motion capture system. The smoothing gives some improvement to Fig. 4, but not very much and distortion presented on the graph remains intact.

Fig. 6. Knee flexion at -52°. Unregistered variant. Patella in hard linked to tibia. Cycling motion.

Fig. 7. Knee flexion at -52°. Unregistered variant. Patella in hard linked to tibia. Cycling motion.

These images show how registration process affects raw GAIT data and enhance entire motion in joint level.

c. Regressions tables

Regression tables for different joints are depending on flexion range of independent coordinate. For example, for ankle joint in range [-26.0°, 23.0°].

We have only one independent flexion and regression tables for other coordinates can be expressed as fifth order polynomial with following coefficients:

	a5	a4	a3	a2	a1	a0
Add/Abd °	9.37x10-8	2.14x10-6	-2.50x10-5	-1.44x10-3	-8.04x10-2	18.596
Int/Ext °	-1.88x10-7	-1.89x10-6	1.06x10-4	2.52x10-3	4.90x10-3	24.731
Ant/Post trans mm	1.12x10-8	-8.56x10-7	2.19x10-5	-5.57x10-4	9.93x10-2	-4.8615
Sup/Inf trans mm	4.09x10-8	5.71x10-7	-1.84x10-5	9.95x10-4	-8.34x10-2	-4.7855
Med/Lat trans mm	-8.15x10-8	-1.89x10-6	1.43x10-5	2.03x10-3	-7.27x10-2	2.7820

d. Synchronous motion

The motion quality is maintained not only on joint but on level of entire motion. The motion of deep bob and walking over obstacle is presented.

Fig. 8. Lower limbs position up-down and left-right at 0°, -14°, -30°, -74°, -110°, -140° angles of knee flexion which corresponds frames 530, 410, 370, 350, 320, 220 accordingly.

The corresponding graphs to this motion are presented below:

Fig. 9. The graphs of three main flexions (in frontal plane) in OVP convention in up-down order in marked point: Left thigh, left foot, left shank.

Fig. 10. The graphs of six additional flexions in OVP convention in up-down order in marked point: Left shank X, left thigh Y, left shank Y, left thigh X, left foot X, left foot Y.

Please note ranges on Fig. 9 and Fig. 10. The main motion can be easily selected among other three motions.

The dislocation graphs are presented on Fig. 11.

Fig. 11. The graphs of six translations in OVP convention in up-down order in marked point: Left foot Y, left shank Z, left foot X, left foot Z, left shank Y, left shank X.

Translations are decent only in the knee and can achieve up to 30 mm in the point of deep bob, while in other joints they can be freely neglected.

The same graphs for the obstacle stepping motion:

Fig. 12. Obstacle stepping motion. Frames 0, 70, 140, 220, 290, 320.in left right and up-down order.

Fig. 13. Obstacle stepping motion. The graphs of three main flexions (in frontal plane) in OVP convention in up-down order in marked point: left thigh, left shank, left thigh.

Fig. 14. The graphs of six additional flexions in OVP convention in up-down order in marked point: Left thigh Y, left shank Y, left shank X, left thigh X, left foot X, left foot Y

Fig. 15. The graphs of six translations in OVP convention in up-down order in marked point: Left foot X, left foot Z, left shank Z, left foot Y, left shank X, left shank Y

e. Live SMART work

The SMART software have user friendly interface allowing even not a mathematics and computer since professional work with, though it is still required from user to understand what is going on to ensure successful registration process.

This video shows how SMART can be used as viewer for PGD files: Viewer for unregistered motion.

Simple registration process with preloaded registration parameters: Registration (real time)

Visual evaluation of registration quality: Viewing of the just registered motion.

f. Performance achievements

The SMART software is currently able to perform animated (30 fps) visualization of GAIT data using high-resolution bones with total up to 300.000 triangles in the bone surface models totally on Pentium IV 1.6 GHz, GeForce 4 Ti, thanks to state-of-the-art render optimization technologies. Such high resolution is generally excessive for educational and research purposes, but still demanded by clinical professionals. With the progress of computer hardware, it is expected it will be possible to shift this limit above. Nevertheless, the software allows visualizing virtually infinite number of triangles at once, but in this case it will be achieved at the expense of interactivity and user friendliness.

The speed registration process itself depends on the number of frames in GAIT data and registration parameters. The default settings allow decent quality registration in most cases. Using default setting the speed of registration is about 120 frames per second for lower limbs skeleton consisting of 7 bones.

7. Conclusion

The SMART was made in tight collaboration with ULB, where target users audience is evaluating it. Several revision of software has been made already and now it in the testing stage. End user has an option whether to set all options at once and perform simple non-interactive registration or take a control over registration process and perform registration interactively. Both methods produces plausible results, and have pro an contra. First one is faster and suteable for registrations a lot of animation data sequences with the same parameters, while second one is the best to find out best registration parameters on the “try and see” basis.

The future goals of this project are placing of additional features such as movie recording and implementing the full two steps advances registration. This registration is supposed to be fully interactive and will allow user to adjust registration parameters not only between registrations steps , but directly during registration process depending on how the process of registration is going and see the results of changes in the special preview mode, since advanced registration with high degree of accuracy is generally slow and time consuming process especially with a long kinematics sequences. Another direction of improvements is generalization of project to support and define when needed anatomical landmarks for entire human skeleton. The integration into Multimod architecture framework is planned.

References

[1] Cappozzo A., Catani F., Della Croce U., Leardini A. (1995) Position and orientation of bones during movement: anatomical frame definition and determination. Clin. Biomech, 10, pp. 171-178.

[2] Cappozzo,A. and Della Crose, U. The PGD Lexicon. CAMARC II Internal Report 15 May 1994.

[3] Kinzel G.L., Hall A-S.J., Hillberry BM. Measurement of the total motion between two body segments. I. Analytical development. J Biomech 1972;5(1):93–105.

[4] Kirstukas S.J., Lewis J.L., Erdman A.G. 6R instrumented spatial linkages for anatomical joint motion measurement—part 2: calibration. J Biomech Eng 1992;114(1):101–10.

[5] Liu W, Panjabi M.M. On improving the accuracy of instrumented spatial linkage system. J Biomech 1996;29(10):1383–5.

[6] Van Sint Jan S, Salvia P, Hilal I, Sholukha VA, Rooze M, Clapworthy G., “Registration of 6-DOF electrogoniometry and CTmedical imaging for 3D joint modeling”, J Biomech 2002;35(11):1475-84.

[7] Viceconty M., Van Sint Jan S, Hilal I, Cholukha V. A, Salvia P, Technical Report on Data Collection Procedure. VAKHUM Project Internal Report, D3.2. From <http://www.ulb.ac.be/project/vakhum/public_report/public_report.htm> Sep 2002.

[8] De Boor,C. A Practical Guide to Splines, 346p. Springer Verlag., 2001

[9] Benedetti M.G., Catani F., Leardini A., Cappozzo A. Anatomical Landmark Definition and Identification. CAMARC II Internal Report; 15 March 1994

[10] Woltring H.J, 3-D attitude representation of human joints: a standardization proposal. J Biomech 27, 1399-1414, 1994.

[11] Kelly Dempski, “Real-Time Rendering Tricks and Techniques in DirectX”, Premier Press; Book and CD edition (March 2, 2002)

[12] Wilson D.R., Feikes J.D.,", Zavatsky A.B., O'Connor J.J. The components of passive knee movement are coupled to flexion angle. J Biomech 33, 465-473, 2000.