A Novel Interface Paradigm for Supporting CAD Environments

 

 

Kuo-Cheng Wu

Future Workspaces Research Centre, University of Salford, UK

K.C.Wu@pgt.salford.ac.uk

 

Terrence Fernando

Future Workspaces Research Centre, University of Salford, UK

T.Fernando@salford.ac.uk

 


Contents

 


Abstract

 This paper presents the design and implementation of an interactive 2D sketching and 3D sculpting paradigm to support creative designs.  This work builds on advances in a large number of related technologies such as multi-screen displays, constraint-based assembly modelling, deformable modelling and state-of-the-art multi-sensory interfaces.  This paper presents the current design and implementation features of the proposed design paradigm.

Keywords: Freeform modelling, constraint modelling, user interaction, design workspace, computer-aided design.

 

1. Introduction

Current CAD environments offer non-intuitive interfaces for supporting designer's creativity, especially when the designer is attempting to define innovative and complex products.  The problem is caused by the mismatch in mapping between the user's mental model and the user interfaces, hindering creativity and increasing the lead time.  The user's mental model is mainly affected by the nature of interaction with the physical system, coupled with the person's knowledge and understanding of the problem. As suggested by Norman [1], if the interface of the physical system can reduce the distance of semantic (deciding what to do) and articulatory distance (how to do it), that is, the user actions can be expressed as higher-level languages and the meaning of that expression can directly be interpreted into a physical form, then the system could be easier to learn and use.

Sketching and sculpting are two design activities, which have direct-mapping between the user's intent and the user's action.  During the design process, sketching is the ultimate user interface analogy for "paper and pencil" metaphor [2].  The vast majority of designers prefer use of traditional sketches to support the creative process.  Moreover, by incorporating "rules" or "constraints" such as relations between model entities in the sketching activity, designers can not only draw an object, but also facilitate the process of making the object [3].  However, it has been technically difficult from both the hardware and the software perspective to provide such a user interface [4, 5].  Another technique which provides a direct-mapping between the user's intent and the user's action for designing a complete product is 3D sculpting.  This allows designers to directly manipulate the model without worrying about the underlying mathematical representations, and to create complex surfaces and curves intuitively and efficiently.  This approach is ideal for the design of complex mechanical products that require the rapid development of innovative surface and curve features for applications such as industrial design, aircraft skin, automotive styling and product design.

Furthermore, the perception of the design plays an important role in supporting creativity and assessing the design to make sure the product is fulfilling the set requirements.  Different types of displays are now available in the market to assess and interact with the evolving model in a different way.  Touch screens and stereoscopic displays allow the designers to visualise and interact with the design.  It is possible that future design stations could use a combination of such screens to support the designer's thinking process and to enhance productivity.

The aim of this research is to develop a novel design paradigm which supports creativity and productivity within the design process.  This work aims to contribute to the development of future CAD systems and to go beyond current simple user-menu based CAD systems.  The key objectives of this research are: to develop an intuitive modelling framework closer to the designer's thinking process and explore the use of 2D sketching and deformable modelling; to develop a novel interface paradigm which combines variety of appropriate technologies such as direct interaction, sketching, sculpting, constraint-based geometric modelling, assembly simulation, multi-screen display environments with gesture interaction and optical infra-red tracking.

2. Related work

This section states the theory of human-computer interaction and reviews technological advances in the fields of user interaction techniques and display technology. 

2.1. Human computer interaction

Norman [1] proposed a theory of user interaction for the computer system.  He stated two different gulfs that need to be bridged by the system designer between the user's goal and the physical system (Figure 1).  The gulf of execution is the difference between the intentions of the users and what the system allows them to do.  On the other hand, the gulf of evaluation is the amount of effort a person must exert to interpret the physical state of the system and assess how well the expectations and intentions have been met.  As argued by Norman [1], two different interface languages (semantic directness and articulatory directness) can be used to describe the distances between both gulfs of execution and evaluation.  Semantic directness describes the relationships between user's intentions and meanings of expressions, while articulatory directness describes the relationships between the meanings of expressions and their physical forms.  Thus, providing suitable user actions which can match the user intentions (semantic directness) and proper input devices for those user actions (articulatory directness) would bridge the gulf of execution.  Similarly, having a feedback from the system which can be compared with the intentions (semantic directness) through output devices (articulatory directness) can reduce the distance of the gulf of evaluation.

Fig. 1. Gulfs of execution and evaluation

2.2. User interaction techniques

User interfaces are used to transfer the human behaviour into the computer system.  Ideal user interfaces would allow users to design and interact within the computer-generated environment as naturally as in the real world.  The user interaction techniques based on interfaces can be categorised into GUI-based system, tool-based system and gestural-based interaction [6].

In GUI-based systems, users deploy menus and other widgets such as buttons to interact with the environment.  In virtual environments, 3D GUIs have been implemented and utilised in many applications such as medical [7] and maintenance simulation [8, 9].  The tool-based interaction involves physical devices which allow users to interactively manipulate the virtual world.  Several researchers [10-12] have adopted a pen-based tool for 2D sketching.  Michalik [13] presented a constraint-based sketching system for 3D curves and surfaces.  On the other hand, gestural-based interaction can capture movements to support design operations within virtual environments.  O'Hagan used finger and hand tracking in place of a mouse to select and manipulate data [14].  In Matsumiya's work [15], he allowed the user to deform a primitive object using a glove device.  Currently magnetic, acoustic and optical tracking are used to support gesture interaction.  However, in comparison with magnetic and acoustic tracking, optical tracking provides greater accuracy and lightweight wireless tracking.  Furthermore, optical tracking has the ability to track a larger number of devices in parallel [16].

In GUI-based interaction, the semantic distance is greater since users need to execute less-obvious task sequence to define the user intentions.  It can also be limited by having obstacles for execution and evaluation.  For example, the sketching proposed by Michalik [13] requires an auxiliary surface which can restrict the user's visibility for evaluation purpose, and the glove's wire can also limit the execution in Matsumiya's work [15] . 

2.3. Display technology

The visualisation helps designers to perceive the design in order to evaluate and produce a better result.  Several visualisation devices are available both in research and commercial fields.  2D visualisation is mainly based on desktop workstation and has recently evolved into more compact and touched-enabled devices. 

For 3D visualisation, there is a wider range of displays from desktop workstation to immersive displays.  Auto-stereoscopic displays are a desktop visualisation technology which allows the user to view in 3D without wearing any special glasses [17].  On the other hand, CAVE environments, which provide stereoscopic views using shutter glasses, enables the user to be immersed within a 3D environment and experience the evolving design [9].  Each of these visualisation devices has its own advantages.  Most of the previous research work mainly provides a single display dedicated for all modelling and navigation purposes.  For engineering design process, there is no single visualisation which can satisfy all the requirements.  In order to gain a comprehensive visual feedback the authors believe that a combination of appropriate visualisation displays should be adopted for promoting creativity.

3. The novel product design modelling system

The proposed freeform modelling environment for innovative product design integrates advanced user interaction techniques, state-of-the-art multi-sensory devices and a multi-display environment.  This section presents the workspace design environment, user interaction techniques, underlying system architecture and the software processes.

3.1. CAD Workspace design environment

The CAD workspace, discussed in this paper, is being developed to explore the following research hypothesis:

Figure 2 shows the initial prototype, which has been constructed to evaluate the above research hypothesis.  Several touch-enabled displays with a pen device allow the user to work on the screen itself providing better hand-eye coordination.  At present, one display is used for 2D sketching and the second one is used for selecting sculpting tools and other attributes such as colours.  The sketching interface allows the designer to draw the 2D model profiles.

Fig. 2. CAD design workspace

                 

(Left) Sketching interface; (Right) Gesturing interface.

 

The TRACE system, from BARCO, is used for 3D stereo visualization of the 3D Models.  A Vicon optical tracking system, attached on the 3D display, is used to track the designers hand to provide direct sculpting facilities on the stereoscopic objects.  This sculpting interface provides the user with a library of virtual sculpting tools to perform direct sculpting operations on the 3D model.

3.2. User interactions

The user interaction is the most critical concern in the design process.  Hence, this research adopts two high-level metaphors (sketching and sculpting) to reduce the semantic distance and support the designers' creativity.

Sketching interface paradigm

The sketching interface allows the designer to deploy the "pen and paper" metaphor on a horizontal touch display.  This allows the user to exploit their hand-eye-coordination for intuitive sketching actions to create 2D technical shapes such as points, lines, arcs, circles and splines (Figure 3a and 3b).  During the creation and manipulation of those strokes, the system also provides constraint mechanisms to create complete geometric shapes.  These constraints permit various forms of distance and angle dimensions between the geometries that it manages.  Logical constraints such as parallel, perpendicular, tangent, coincident and symmetric are also supported (Figure 3c and 3d).  Once the satisfactory 2D profile has been constructed, it can be converted into a 3D solid model through modelling operations such as spinning, sweeping, lofting and extrusion and displayed in the 3D scene. 

Fig. 3. (a, b) Freehand sketch; (c) constraint between sketches; (d) dynamic updating

Sculpting interface paradigm

The sculpting interface allows the designer to sculpt innovative shapes through gesturing and sculpting tools.  In gestural sculpting, the user can directly manipulate the model with hands and fingers, whilst tool sculpting allows the user to perform feature-based modelling with physical tools.

The gesture types are categorised into static and dynamic gestures (Table 1).  The user performs static gestures to select and deselect objects.  Selecting an object is carried out by pinching the forefinger with the thumb.  In this approach both fingers have reflective-markers attached for optical tracking.  Separating these two fingers would release the selected object.  Dynamic gesture is regarded as continuous movement of static gestures for doing model transformation and deformation.  Two dynamic gestures for deformation, pushing/pulling and stretching/shrinking, are available in the sculpting interface.  The pulling gesture allows the user to pull a certain area or point on the 3D model (Figure 4.a).  The point and area manipulators for pushing operation are defined through the finger intersection with the selected model.  The dynamic stretching gesture requires the user to use two hands simultaneously to control two manipulators (point or curve) to stretch or shrink the model (Figure 4.b and 4.c). 

Table 1. Gesture classification

 

Type

Operations

Static gesture

Pinch

Select/deselect objects.

Dynamic gesture

Transformation

Translate/rotate/scale objects.

Push/pull

Push/pull a point or area.

Stretch/shrink

Bi-manual control of manipulators (points /curves)

        

Fig. 4. (a) Pull/push operation; (b) Initial state for stretching; (c) Stretching the model by transforming the middle curve manipulator

A physical tool is used for performing a variety of feature modelling operations.  The user can select and attach any available deformable function from the virtual toolbox to the physical tool.  The virtual toolbox with 3D icons is a pallet containing pre-defined functions.  An examples of feature modelling with tools is the cut operation.  The cut tool is a knife-like object which allows the user to section the model (Figure 5).  Other potential tools are Boolean and blend tools.  Boolean operation allows the user to unite, intersect and subtract the model with other primitive models.  For blending faces of the model, the user can use cylindrical tool to connect faces smoothly, and the degree of blending depends on the radius of the virtual tool.

Fig. 5. Cutting tool

In situations where several parts are presented for an assembly task, the user can use their finger to select constraints such as parallelism, tangency, alignment and concentricity from 3D GUI buttons for mating parts.  The parts can also be constrained automatically during the transformation in run time.  Once the parts within the assembly are constrained, the user can use dynamic gesture to perform the physical simulation in real-time. 

The input devices for sketching and sculpting provide the physical and direct forms of modelling, which are related to their corresponding meanings of expressions.  This greatly enhances the articulatory directness and, thus, can bridge the gulf of execution.

3.3. System architecture

The system architecture is comprised of the user layer, the interface hardware layer and the software layer (Figure 6).  The user layer represents two user actions: sketching and sculpting.  The interface hardware layer supports two input devices (sketching pen, finger/hand gestures and tools), and two output devices (3D active stereo display and touch-flat display).  In the software layer, seven software modules are presented for supporting specific modelling purposes. 

The process flow can be grouped into two interactions within these three layers.  For sketch-based user interaction, the designer performs action with the sketching pen input device.  This sketch data is transmitted to the sketching manager to be processed and the result is displayed on the touch-flat display.  For sculpting interaction, the designer operates with his/her finger or tools which are captured by the optical tracking cameras.  The sculpting manager in turn analyses the input data, and outputs the results to the 3D active stereoscopic display.  In addition to the sculpting and sketching mangers, which are responsible for receiving user-input and presenting the GUI design, five other managers are available to provide various services:

 



Fig. 6. System architecture

Deformation manager

The deformation manager provides sculpting functions for the sculpting manager.  The functionality of the deformation manager component is provided by the Spatial Deformable Modeller [18].  It adopts an energy-based optimisation strategy [19] to deform precise B-spline NURBS curves and surfaces which allows users to modify the shape by applying loads and constraints.  Loads pull or push deformable models with a force to approximate locations while constraints control the exact shape of a deformable model at the locations specified.  Loads and constraints have parameters that affect either global or local shape of the deformable model in a predictable manner.  The degree of continuity available for the surfaces are control position, tangency, and curvature at points and along curves. These points and curves can lie in the interior of surfaces or along surface boundaries.  In this research, parameters of constraints and load are associated with finger/hand and tool devices, therefore, the model can be deformed and updated repeatedly to create an interactive sculpting environment.

2D-Constraint manager

It allows users to perform sketching rapidly and accurately while maintaining the consistency of the geometric models.  The constraints supported by the 2D constraint manager are: dimensions for managing the distance and angle between geometries; logical constraints such as parallel, perpendicular, tangent, coincident and symmetric.  To modify a model, the user simply specifies a change to the rules, such as a modified value for a dimension. The 2D-constraint manager then re-calculates the locations of all the geometries affected by the new dimension value, ensuring that their new locations are consistent with the previously applied dimensions and constraints.  This manager adopts the 2D DCM library from UGS D-CUBED Ltd [20].

Coordination manager

This manager acts as a coordinator among the sculpting manager, sketching manager, assembly manager and the geometry manager.  The coordination manager is invoked when performing the assembly simulation, validation checks and model conversion between sculpting and sketching operations.  For example, when the sketching manager requires geometry operations (spinning, sweeping, lofting and extrusion) for converting a 2D profile into 3D or when the sculpting manager requires feature-modelling (sectioning, boolean, blending, hollowing and offsetting), the coordination manager would be responsible for transferring the request to the geometry manager.  In the case of performing assembly simulation, the coordination manager would activate the assembly manager for supporting the task.

Geometry and assembly managers

The geometry manager encapsulates the low level modelling operations and provides geometric services through the coordination manager to create geometrically valid objects.  In addition to those operations listed above, the geometry manager also provides model enquiry and model creation facilities for both the gesturing and the sketching managers through the coordination manager.  The geometry manager has been implemented by using the UGS Parasolid Geometry Kernel [21] based on exact boundary representation.  The assembly manager facilitates building the relationships among multi-parts.  A part can be connected to another part with parallel, tangent or concentric constraints.

3.4. System process

The system process introduces the communication aspect among those managers in the software layer (Figure 7).  When the user performs either a sketching or a sculpting task, various managers will respond to each other according to the user action. 

Fig. 7. System process

For the sketching interaction, the sketching manager informs the coordination manager for initialization.  With the 2D-constraint manager, required constraints can be assigned to the 2D profile model and allows the user to carry out the sketching task.  When the sketching is completed, the coordination manager updates the final model in both the sculpting and the geometry managers with a validation check.

For the sculpting interaction, the sculpting manager informs the coordination and deformation managers for initialization.  According to the task, the coordination manager can send the command to other managers (deformation, geometry and assembly) for updating.  When the sculpting is completed, the coordination manager updates the final model in both the sketching and the geometry managers with a validation check.

4. Conclusions and future work

This research explored the use of 2D sketching and 3D sculpting to implement a highly interactive design paradigm, which combines direct interaction techniques, sketching and sculpting tools, assembly simulation and multi-screen display environment with optical tracking.

The definition of tools to support design intent during product development is a key issue for the next generation of CAD systems.  These design tools will not replace the designer, but will merely exist to enhance the designer's abilities.  This evolution of design tools will push many designers to create their own distinctive styles, and to market those styles more directly.  The current implementation of the system doesn't support feature modelling to perform deformation tasks.  This limitation prohibits the capability of creating new functionality at run time.  With the incorporation of assembly simulation, designers are able to design a complete product and analyse the requirement for functional aspects.  In additional, the benefit of a multi-display environment could support the designer assessing and evaluating the product efficiently. 

Future work required further development to complete the entire functionality of the system.  This work then needs to be evaluated using complete case studies and end users.  In particularly, incorporating user-defined feature-based deformation, designers are free to define, add, delete and modify the customised sculpting features according to their domain task rather then providing a large repertoire of features covering every possible application.  Moreover, a well-defined distributed and collaborative design environment could potentially allow designers to work collaboratively, discuss design opinions, present creativity and assess design from variety of design perspective

.

References

1.         Norman, S.a.D., S., User Centered System Design: New Perpectives on Human-Computer Interaction. 1986, London: LEA.

2.         Deisinger, J., et al. Towards Immersive Modelling - Challenges and Recommendatins: A Workshop Analysing the Needs of Designers. in ISATA 2000. 2000. Dublin, Ireland.

3.         Lawson, B., How designers think - The design process demystified. 1997: Architecture Press.

4.         Plimmer, B. and M. Applerley. From Sketch to Form on a Large Interactive Display Surface. in 14th Annual Conference of the NACCQ. 2001. Napier, New Zealand.

5.         Tovey, M., S. Porter, and R. Newman. Investigating automotive sketches as a source of geometry for 3D models. in The 4th International Symposium On Tools And Methods Of Competitive Engineering. 2002. Napier, New Zealand.

6.         Flasar, J., 3D Interaction in Virtual Environment. 2000.

7.         Stephenson, P., et al. Studydesk: semi-immersive volumetric data analysis. in The 1st international conference on Computer graphics and interactive techniques. 2003. Melbourne, Australia.

8.         Marcelino, L., N. Murray, and T. Fernando, A Constraint Manager to Support Virtual Maintainability. Computers and Graphics, 2003. 27: p. 19-26.

9.         Marcelino, L., N. Murray, and T. Fernando. A User Interface for Virtual Maintainability in Immersive Environments. in HCI International conference. 2003. Crete, Greece.

10.       L. Eggli, B. D. Brderlin, and G. Elber. Sketching as a solid modeling tool. in The Third Symposium on Solid Modeling and Applications. 1995.

11.       Grimstead, I.J. and R.R. Martin. Creating solid models from single 2D sketches. in The third ACM symposium on Solid modeling and applications. 1995. Salt Lake City, Utah, United States.

12.       Tolba, O., J. Dorsey, and L. McMillan. Sketching with projective 2D strokes. in The 12th Annual ACM Symposium on User Interface Software and Technology. 1999. Asheville, North Carolina.

13.       Michalik, P., Kim, D. and Bruderlin, B. Sketch- and constraint-based design of B-spline surfaces. in ACM Symposium on Solid Modeling and Applications. 2002.

14.       R. O'Hagan and A. Zelinsky. Finger Track - A Robust and Real-Time Gesture Interface. in Advanced Topics in Artificial Intelligence, Tenth Australian Joint Conference on Artificial Intelligence (AI'97). 2000. Perth, Australia.

15.       Matsumiya, M., H. Takemura, and N. Yokoya. An immersive modeling system for 3D free-form design using implicit surfaces. in The ACM symposium on Virtual reality software and technology. 2000. Seoul, Korea.

16.       Murray, N., J.Y. Goulermas, and T. Fernando. Visual Tracking for a Virtual Environment. in HCI International conference. 2003. Crete, Greece.

17.       Inition ltd., http://www.inition.co.uk/inition/product_stereovis_seereal_d4d.htm.

18.       Spatial Ltd. http://www.spatial.com/components/deformable/.

19.       Celniker, G. and W. Welch. Linear constraints for deformable non-uniform B-spline surfaces. in 1992 symposium on Interactive 3D graphics. 1992. Cambridge, MA USA.

20.       UGS D-CUBED. http://www.ugs.com/products/open/d-cubed/.

21.       UGS Parasolid.  http://www.ugs.com/products/open/parasolid/.