SmartCam: Devices for Embedded Intelligent Cameras (EES.5411)

Many networked embedded systems combine sensing with cameras with processing to achieve certaincommunication, measurement or control goals. Video Camcorders, web cameras, video-phones are exam-ples of products where the combination of image sensing, digital storage and transmission is penetrating the mass electronics market. Other applications can be found in inspection, surveillance and robotic applica-tions; more recently they are also found in wireless intelligent agents such as AIBOs [1]. In (wireless) em-bedded applications, the vision sensor can e.g. be used to directly drive motion control electronics or wire-less links. Most of these applications have real-time demands and require huge processing capacities.

Vision sensors are traditionally of the CCD type, however, more recently CMOS realizations become more interesting, especially in the low-cost high volume application market [2]. They have several advan-tages, like low operating voltage, low power, and possible integration with processing logic. Examples of this integration have been realised in the form a so-called smart sensors; they have an on-chip SIMD data processing array controlled by an off-chip controller. Smart sensors can execute low-level image processing routines as soon as one or more image lines are converted; they do not have to wait for the whole image. High level image processing like feature extraction and object processing is performed with a separate pow-erful off-chip processor.

Current solutions have several problems. It is totally unclear what the right architectural parameters are for a given application domain. There are many parameters, like pixel array size, pixel properties, num-ber of AD-converter units, accuracy, number of pixels per SIMD processor element, processor element functionality, etc. Also the functionality of the external processor and its connectivity with the smart sensor has to be determined. Furthermore, intuitive mappings of algorithms on architecture components are used, after the architecture has been determined. It will be clear that this is far from optimal. Finally we foresee a further integration, making a combination of on-chip vision sensor, pixel processing, control, and feature and object processing possible. The result is a low-cost one-chip Smart Camera (so-called SmartCam) solution. This means that research is needed to explore the new architectural opportunities and consequences.

The SmartCam project investigates these new opportunities and contributes to a better and more quantitatively guided design trajectory. In particular, we will investigate the impact of current applications, define relevant architectural parameters and develop an architectural template, enhance and integrate our existing application mapping environments for SIMD and ILP (Instruction-Level Parallel) processors, and perform two case studies. We will base the research and development on our existing knowledge in smart sensor type of applications [4][5][6], and our processor design experience, both in the area of SIMD proc-essing [8][7] and ILP Processors [10][16][17][26]. Furthermore, we will cooperate with several industrial partners; Philips NatLab contributes through its Xetal project, Philips CFT+EMT through its Component Placement Machines project and RoboCup project. TNO-FEL through its iCAM project, In3D through its 3D-HP Measurement Project. Results will be applied in these projects.

For results, see http://www.ph.tn.tudelft.nl/~wcaarls/smartcam/

Two companies are involved in this project.