In this paper, we present a theoretical study on automated understanding of the design drawing. This can lead to design support through the natural interface of sketching. In earlier work, 24 plan-based conventions of depiction have been identified, such as grid, zone, axial system, contour, and element vocabulary. These are termed graphic units. Graphic units form a good basis for recognition of drawings as they combine shape with meaning. We present some of the theoretical questions that have to be resolved before an implementation can be made. The contribution of this paper is: (i) identification of domain knowledge which is necessary for recognition; (ii) outlining combined strategy of multi-agent systems and online recognition; (iii) functional structure for agents and their organisation to converge on sketch recognition.