Abstract:   We present a new approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions. The approach exploits a sophisticated generative model of the human face originally developed for realistic facial animation. The face model, which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physics-based synthetic facial tissue and a set of anatomically motivated facial muscle actuators. We consider the estimation of dynamic facial muscle contractions from video sequences of expressive human faces. We develop an estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images. The technique estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions.

Abstract:   We present a new approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions. The approach exploits a sophisticated generative model of the human face originally developed for realistic facial animation. The face model, which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physics-based synthetic facial tissue and a set of anatomically motivated facial muscle actuators. We consider the estimation of dynamic facial muscle contractions from video sequences of expressive human faces. We develop an estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images. The technique estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions.

Abstract:   We present a new approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions. The approach exploits a sophisticated generative model of the human face originally developed for realistic facial animation. The face model, which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physics-based synthetic facial tissue and a set of anatomically motivated facial muscle actuators. We consider the estimation of dynamic facial muscle contractions from video sequences of expressive human faces. We develop an estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images. The technique estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions.

Abstract:   We present a new approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions. The approach exploits a sophisticated generative model of the human face originally developed for realistic facial animation. The face model, which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physics-based synthetic facial tissue and a set of anatomically motivated facial muscle actuators. We consider the estimation of dynamic facial muscle contractions from video sequences of expressive human faces. We develop an estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images. The technique estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions.

Abstract:   We present a new approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions. The approach exploits a sophisticated generative model of the human face originally developed for realistic facial animation. The face model, which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physics-based synthetic facial tissue and a set of anatomically motivated facial muscle actuators. We consider the estimation of dynamic facial muscle contractions from video sequences of expressive human faces. We develop an estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images. The technique estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions.

Abstract:   We present a new approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions. The approach exploits a sophisticated generative model of the human face originally developed for realistic facial animation. The face model, which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physics-based synthetic facial tissue and a set of anatomically motivated facial muscle actuators. We consider the estimation of dynamic facial muscle contractions from video sequences of expressive human faces. We develop an estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images. The technique estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions.

Abstract:   Model-based image coding applied to interpersonal communication achieves very low bit-rate image transmission. To accomplish it, accurate three-dimensional (3-D) motion estimation of a speaker is necessary. A new method of 3-D motion estimation is presented, consisting of two steps. In the first, facial contours and feature points of a speaker are extracted using filtering and Snake algorithms. Five feature points on a speaker's facial image are tracked between consecutive picture frames, which gives 2-D motion vectors of the feature points. Then, in the second step, the 3-D motion of a speaker's head is estimated using a three-layered neural network model, after training with many possible motion patterns of the human head using an existing 3-D general shape model. Experimental results show that our method not only achieves good results but is also more robust than existing methods, even when the motion of an object is rather large or complicated. Accurately estimated 3-D motion parameters can realise image transmission at a very low bit rate.

Abstract:   Many problems in computational vision (including stereo correspondence, motion analysis and surface reconstruction) can be solved effectively using a constrained optimization approach, where smoothness is the common constraint. Moreover, these problems can be cast in a variational form that minimizes an energy functional. Unfortunately, standard optimization techniques tend to find only local energy minima. Coarse to fine scale space tracking (where energy minima at reduced resolution are found and successively tracked to higher resolution) has been demonstrated to find solutions of practical value. For smoothness-constrained optimization problems, we show that scale space tracking can be implicitly implemented by appropriately adjusting the smoothness constraint. A useful physical model for controlled smoothness (deformable sheets) provides a natural framework for scale space tracking and addressing many vision problems that can be solved by appealing to a smoothness constraint. Deformable sheets are characterized by a global energy functional, and the smoothness constraint is represented by a linear internal energy term. In analogy to physical sheets, the model sheets are deformed by problem specific external forces and, in turn, impose smoothness on the applied forces. We have related deformable sheet smoothness properties to Gaussian blurring (the common expression of scale) and used this relationship to unify the concepts of scale and smoothness. In our formulation, the smoothness/scale state is controlled by a single parameter in the deformable sheet model. This single parameter control of scale makes it possible to perform scale space tracking by solving the differential equation that describes the trajectory of energy minima through scale space. Further, it permits adaptive scale step size selection based on the local properties of scale space, which allows for much larger steps than would be possible with the conservative step size required by nonadaptive techniques. We show that this process is characterized by a sparse linear system and prove that the associated matrix is positive definite and, consequently, nonsingular. Our analysis also provides for the determination of scale-dependent parameters, which is useful for efficient multiresolution processing. We have applied the deformable sheet model described to different problems in computational vision using real imagery with encouraging results, which are presented here.

Abstract:   The representation of shape in machine vision is reviewed with emphasis on the most common types of representation and recent developments. Both planar shape and solid shape are examined with connections and generalizations drawn wherever possible. Particular emphasis is placed on the importance of invariant descriptions and on the representation of shape classes.

Abstract:   In this paper, we present a new method for describing the shape of structures in grey-scale images, which is known as the intensity axis of symmetry (IAS). We describe the spatial and intensity variations of the image simultaneously rather than by the usual two-step process of 1) using intensity properties of the image to segment an image into regions and 2) describing the spatial shape of these regions. The result is an image shape description that is useful for a number of computer vision applications. Our method for computing this image shape description relies on minimizing an active surface functional that provides coherence in both the spatial and intensity dimensions while deforming into an axis of symmetry. Shape- based image segmentation is possible by identifying image regions associated with individual components of the IAS. The resulting image regions have geometric coherence and correspond well to visually meaningful objects in medical images.

Abstract:   Echocardiography has been widely used as a real-time non- invasive clinical tool to diagnose cardiac functions. Due to the poor quality and inherent ambiguity in echocardiograms, it is difficult to detect the myocardial boundaries of the left ventricle. Many existing methods are semi-automatic and detect cardial boundaries by serial computation which is too slow to be practical in real applications. In this paper, a new method for detecting the endocardial boundary by using a Hopfield neural network is proposed. Taking advantage of parallel computation and energy convergence capability in the Hopfield network, this method is faster and more stable for the detection of the endocardial border. Moreover, neither manual operations nor a priori assumptions are needed in this method. Experiments on several LV echocardiograms and clinical validation have shown the effectiveness of our method in these patient studies.

Abstract:   This article presents: (i) a multiscale representation of grey- level shape called the scale-space primal sketch, which makes explicit both features in scale-space and the relations between structures at different scales, (ii) a methodology for extracting significant blob-like image structures from this representation, and (iii) applications to edge detection, histogram analysis, and junction classification demonstrating how the proposed method can be used for guiding later-stage visual processes. The representation gives a qualitative description of image structure, which allows for detection of stable scales and associated regions of interest in a solely bottom-up data-driven way. In other words, it generates coarse segmentation cues, and can hence be seen as preceding further processing, which can then be properly tuned. It is argued that once such information is available, many other processing tasks can become much simpler. Experiments on real imagery demonstrate that the proposed theory gives intuitive results.

Abstract:   The use of energy-minimizing curves, known as ''snakes'' to extract features of interest in images has been introduced by Kass, Witkin and Terzopoulos [23]. A balloon model was introduced in [12] as a way to generalize and solve some of the problems encountered with the original method. A 3-D generalization of the balloon model as a 3-D deformable surface, which evolves in 3-D images, is presented. It is deformed under the action of internal and external forces attracting the surface toward detected edgels by means of an attraction potential. We also show properties of energy- minimizing surfaces concerning their relationship with 3-D edge points. To solve the minimization problem for a surface, two simplified approaches are shown first, defining a 3-D surface as a series of 2-D planar curves. Then, after comparing finite- element method and finite-difference method in the 2-D problem, we solve the 3-D model using the finite-element method yielding greater stability and faster convergence. This model is applied for segmenting magnetic resonance images.

Abstract:   Active contour models (snakes) are commonly used for locating the boundary of an object in computer vision applications. The minimisation procedure is the key problem to solve in the technique of active contour models. In this paper, a minimisation method for an active contour model using Hopfield networks is proposed. Due to its network structure, it lends itself admirably to parallel implementation and is potentially faster than conventional methods. In addition, it retains the stability of the snake model and the possibility for inclusion of hard constraints. Experimental results are given to demonstrate the feasibility of the proposed method in applications of industrial pattern recognition and medical image processing.

Abstract:   An interactive computer system has been developed for evaluating cytologic features derived directly from a digital scan of breast fine needle aspirate slides. The system uses computer vision techniques to analyze cell nuclei and classifies them using an inductive method based on linear programming. A digital scan of selected areas of the aspirate slide is done by a trained observer, while the analysis of the digitized image is done by an untrained observer. When trained and tested on 119 breast fine needle aspirates (68 benign and 51 malignant) using leave-one-out testing, 90% correctness was achieved. These results indicate that the method is accurate (good intraobserver and interobserver reproducibility) and that an untrained operator can obtain diagnostic results comparable to those achieved visually by experienced observers.

Abstract:   Most object recognition systems can only model objects composed of rigid pieces whose appearance depends only on lighting and viewpoint. Many real world objects, however, have variable appearances because they are flexible and/or have a variable number of parts. These objects cannot be easily modeled using current techniques. We propose the use of a knowledge representation called the VAPOR (Variable APpearance Object Representation) model to represent objects with these kinds of variable appearances. The VAPOR model is an idealization of the object; all instances of the model in an image are variations from the ideal appearance. The variations are evaluated by the description length of the data given the model, i.e., the number of information-theoretic bits needed to represent the model and the deviations of the data from the ideal appearance. The shortest length model is chosen as the best description. We demonstrate how the VAPOR model performs in a simple domain of circles and polygons and in the complex domain of finding cloverleaf interchanges in aerial images of roads.

Abstract:   In many applications of image analysis, simply connected objects are to be located in noisy images. During the last 5-6 years active contour models have become popular for finding the contours of such objects. Connected to these models are iterative algorithms for finding the minimizing energy curves making the curves behave dynamically through the iterations. These approaches do however have several disadvantages. The numerical algorithms that are in use constraint the models that can be used. Furthermore, in many cases only local minima can be achieved. In this paper, we discuss a method for curve detection based on a fully Bayesian approach. A model for image contours which allows the number of nodes on the contours to vary is introduced. Iterative algorithms based on stochastic sampling is constructed, which make it possible to simulate samples from the posterior distribution, making estimates and uncertainty measures of specific quantities available. Further, simulated annealing schemes making the curve move dynamically towards the global minimum energy configuration are presented. In theory, no restrictions on the models are made. In practice, however, computational aspects must be taken into consideration when choosing the models. Much more general models than the one used for active contours may however be applied. The approach is applied to ultrasound images of the left ventricle and to Magnetic Resonance images of the human brain, and show promising results.

Abstract:   Registering volumes that have been deformed with respect to each other involves recovery of the deformation. A 3-D elastic matching algorithm has been developed to use surface information for registering volumes. Surface extraction is performed in two steps: extraction of contours in 2-D image planes using active contours, and forming triangular patch surface models from the stack of 2-D contours. One volume is modeled as being deformed with respect to another goal volume. Correspondences between surfaces in the two image volumes are used to warp the deformed volume towards its goal. This process of contour extraction, surface formation and matching, and warping is repeated a number of times, with decreasing image volume stiffness. As the iterations continue the stretched volume is refined towards its goal volume. Registration examples of deformed volumes are presented.

Abstract:   This paper introduces a representation scheme for image sequences using nonuniform samples embedded in a deformable mesh structure. It describes a sequence by nodal positions and colors in a starting frame, followed by nodal displacements in the following frames. The nodal points in the mesh are more densely distributed in regions containing interesting features such as edges and corners; and are dynamically updated to follow the same features in successive frames. They are determined automatically by maximizing feature (e.g, gradient) magnitudes at nodal points, while minimizing interpolation errors within individual elements, and matching errors between corresponding elements. In order to avoid the mesh elements becoming overly deformed, a penalty term is also incorporated, which measures the irregularity of the mesh structure. The notions of shape functions and master elements commonly used in the finite element method have been applied to simplify the numerical calculation of the energy functions and their gradients. The proposed representation is motivated by the active contour or snake model proposed by Kass, Witkin, and Terzopoulos. The current representation retains the salient merit of the original model as a feature tracker based on local and collective information, while facilitating more accurate image interpolation and prediction. Our computer simulations have shown that the proposed scheme can successfully track facial feature movements in head-and-shoulder type of sequences, and more generally, interframe changes that can be modeled as elastic deformation. The treatment for the starting frame also constitutes an efficient representation of arbitrary still images.

Abstract:   A new approach is described for reconstructing coronary arteries from two sequences of projection images. The estimation of motion is performed on three-dimensional line segments (or centrelines), and is based on a 'prediction-projection-optimization' loop. The method copes with time varying properties, deformations and superpositions of vessels. Experiments using simulated and real data have been carried out, and the results found to be robust over a full cycle of a human heart. Local and global kinetic features can then be derived to obtain a greater insight on the cardiac functional state

Abstract:   Food material shape is often closely related to its qualify. Due to the demands of high quality, automated food shape inspection has become an important need for the food industry. Currently, accuracy and speed are two major problems for food shape inspection with computer vision. Therefore, in this study, a fast and accurate computer-vision based feature extraction and classification system was developed. In the feature extraction stage, a statistical model based feature extractor (SMB) and a multi-index active model-based (MAM) feature extractor were developed to improve the accuracy of classifications. In the classification stage, first the back-propagation neural network was applied as a multi-index classifier. Then, to speed up training, some minimum indeterminate zone (MIZ) classifiers were developed. Corn kernels, almonds, and animal-shaped crackers were used to rest the above techniques. The results showed that accuracy and speed were greatly improved when the MAM feature extractor was used in conjunction with the MIZ classifier.

Abstract:   Active contours, known as snakes, have found wide applications since their first introduction in 1987 by Kass et al. (Int. J. Comput. Vision 1, 321-331). However, one problem with the current models is that the performance depends on proper internal parameters and initial contour position, which, unfortunately, cannot be determined a priori. It is usually a hard job to tune internal parameters and initial contour position. The problem comes from the fact that the internal normal force at each point of contour is also a function of contour shape. To solve this problem, we propose to compensate for this internal normal force so as to make it independent of shape. As a result, the new model works robustly with no necessity to fine-tune internal parameters, and can converge to high curvature points like corners.

Abstract:   Neuroscientists have studied the relationship between nerve cell morphology and function for over a century. To pursue these studies, they need accurate three-dimensional models of nerve cells that facilitate detailed anatomical measurement and the identification of internal structures. Although serial transmission electron microscopy has been a source of such models since the mid 1960s, model reconstruction and analysis remain very time consuming. We have developed a new approach to reconstructing and visualizing 3D nerve cell models from serial microscopy. An interactive system exploits recent computer graphics and computer vision techniques to significantly reduce the time required to build such models. The key ingredients of the system are a digital ''blink comparator'' for section registration, ''snakes,'' or active deformable contours, for semiautomated cell segmentation, and voxel-based techniques for 3D reconstruction and visualization of complex cell volumes with internal structures.

Abstract:   This paper presents a method, based on the Filtered Backprojection technique (FBP), to extract directly the boundaries of X-ray images, without previous image reconstruction. We preprocess the raw data in order to compute directly the reconstructed values of the gradient or of the Laplacian at any location in the plane (defined with real coordinates). The reconstructed value of the gradient and of the Laplacian correspond to the exact mathematical definition of the differentials of the image. For noisy data, we propose also to use an extension of existing FBP techniques, adapted to the computation of the gradient and of the Laplacian. Finally, we show how to use the corresponding operators to perform the segmentation of a slice, without image reconstruction. Images of the reconstructed gradient, Laplacian, and segmented objects are presented.

Abstract:   A model for fusing the output of multiple segmentation modules is presented. The model is based on the particle system approach to modeling dynamic objects from computer graphics. The model also has built-in capabilities to extract regions, thin the edge image, remove ''twigs,'' and close gaps in the contours. The model functions both as an effective data fusion technique and as a model of an important human visual process. (C) 1994 Academic Press, Inc.

Abstract:   A new method to make an automatic initial estimation of the position of the middle of the left ventricular (LV) myocardial wall (LV myocardial midwall) in emission tomograms is presented. This method eliminates the manual interaction still required by other, more accurate LV delineation algorithms, and which consists of indicating the LV long axis and/or the LV extremities. A well-known algorithm from the world of neural networks, Kohonen's self-organizing maps, was adapted to use general shapes and to behave well for data with large background noise.

Abstract:   The variational method has been introduced by Kass et al. (1987) in the field of object contour modeling, as an alternative to the more traditional edge detection-edge thinning-edge sorting sequence. Since the method is based on a pre-processing of the image to yield an edge map, it shares the limitations of the edge detectors it uses. In this paper, we propose a modified variational scheme for contour modeling, which uses no edge detection step, but local computations instead-only around contour neighborhoods-as well as an ''anticipating'' strategy that enhances the modeling activity of deformable contour curves. Many of the concepts used were originally introduced to study the local structure of discontinuity, in a theoretical and formal statement by Leclerc & Zucker (1987), but never in a practical situation such as this one. The first part of the paper introduces a region-based energy criterion for active contours, and gives an examination of its implications, as compared to the gradient edge map energy of snakes. Then, a simplified optimization scheme is presented, accounting for internal and external energy in separate steps. This leads to a complete treatment, which is described in the last sections of the paper (4 and 5). The optimization technique used here is mostly heuristic, and is thus presented without a formal proof, but is believed to fill a gap between snakes and other useful image representations, such as split-and-merge regions or mixed line-labels image fields.

Abstract:   This paper presents an integrated approach to recovering the superquadric primitive from stereo images. While the depth data obtained from stereo matching algorithms are always sparse and noisy, to extract an object from the scene and obtain a smoothed depth map of the object, occluding contour detection and surface reconstruction are incorporated into the recovery process of superquadrics. The algorithm combines the recovery processes of occluding contour, surface and volumetric models in a cooperative and synergetic manner. The performance of the algorithm is demonstrated with two examples using real images.

Abstract:   Background Myocardial tissue tagging with the use of magnetic resonance imaging allows noninvasive regional analysis of heart wall motion and deformation. However, any evaluation of the effect of disease or treatment requires a baseline reference of normal values and variation. We studied the two-dimensional motion of material points imaged within the left ventricular wall using spatial modulation of magnetization (SPAMM) in 12 normal human volunteers. Methods and Results Five parallel short-axis and five parallel long-axis slices were acquired at five times during systole. SPAMM tags were generated at end diastole using a 7-mm grid. Intersection point data were analyzed for displacement, rotation, and torsion, and triangles of points were analyzed for local rotation and principal strains. Short-axis displacement was the least in the septum for all longitudinal levels (P<.001). Torsion about the long axis was uniform circumferentially because of the motion of the centroids used to reference the rotation. In the long-axis images, the base displaced longitudinally toward the apex, with the posterior wall moving farther than the anterior wall (13.4+/-2.2 versus 9.7+/-1.8 mm, P<.001) in this direction. The largest principal strain (maximum lengthening) was approximately radially oriented in both views. In the short-axis images, the minimum principal strain (maximum shortening) increased in magnitude toward the apex (P<.001) with little circumferential variation, except at midventricle, where the anterior wall showed greater contraction than the posterior wall (-0.21+/-0.03 versus -0.19+/-0.02, P<.02). Conclusions Consistent regional variations in deformation are seen in the normal human heart, Displacement and maximum shortening strains are well characterized with two-dimensional magnetic resonance tagging; however, higher-resolution images will be required to study transmural variations.

Abstract:   Most segmentation algorithms are composed of several procedures: split and merge, small region elimination, boundary smoothing,..., each depending on several parameters. The introduction of an energy to minimize leads to a drastic reduction of these parameters. The authors prove that the most simple segmentation tool, the ''region merging'' algorithm, made according to the simplest energy, is enough to compute a local energy minimum belonging to a compact class and to achieve the job of most of the tools mentioned above. The authors explain why ''merging'' in a variational framework leads to a fast multiscale, multichannel algorithm, with a pyramidal structure. The obtained algorithm is O(n ln n), where n is the number of pixels of the picture. This fast algorithm is applied to make grey level and texture segmentation and experimental results are shown.

Abstract:   A multiresolution approach to curve extraction in images is described. Based on a piecewise linear representation of curves, the scheme combines an efficient method of extracting line segments with a grouping process to identify curve traces. The line segments correspond to linear features defined at appropriate spatial resolutions within a quadtree structure and are extracted using a hierarchical decision process based on frequency domain properties. Implementation is achieved through the use of the multiresolution Fourier transform, a linear transform providing spatially localized estimates of the frequency spectrum over multiple scales. The scheme is simple to implement and computationally inexpensive, and results of experiments performed on natural images demonstrate that its performance compares favorably with that of existing methods. (C) 1994 Academic Press, Inc.

Abstract:   A method is described for extracting lineal features from an image using extended local information to provide robustness and sensitivity. The method utilizes both gradient magnitude and direction information, and incorporates explicit lineal and end-stop terms. These terms are combined nonlinearly to produce an energy landscape in which local minima correspond to lineal features called sticks that can be represented as line segments. A hill climbing (stick-growing) process is used to find these minima. The method is compared to two others, and found to have improved gap-crossing characteristics.

Abstract:   We propose an optimization approach to the estimation of a simple closed curve describing the boundary of an object represented in an image. The problem arises in a variety of applications, such as template matching schemes for medical image registration. A regularized optimization formulation with an objective function that measures the normalized image contrast between the inside and outside of a boundary is proposed. Numerical methods are developed to implement the approach, and a set of simulation studies are carried out to quantify statistical performance characteristics. One set of simulations models emission computed tomography (ECT) images; a second set considers images with a locally coherent noise pattern. In both cases, the error characteristics are found to be quite encouraging. The approach is highly automated, which offers some practical advantages over currently used technologies in the medical imaging field.

Abstract:   In this paper, we present a new approach for the automatic tracking of SPAMM (Spatial Modulation of Magnetization) grid in cardiac MR images and consequent estimation of deformation parameters. The tracking is utilized to extract grid points from MR images and to establish correspondences between grid points in images taken at consecutive frames. These correspondences are used with a thin plate spline model to establish a mapping from one image to the next. This mapping is then used for motion and deformation estimation. Spatio-temporal tracking of SPAMM grid is achieved by using snakes-active contour models with an associated energy functional. We present a minimizing strategy which is suitable for tracking the SPAMM grid. By continuously minimizing their energy functionals, the snakes lock on to and follow the in-slice motion and deformation of the SPAMM grid. The proposed algorithm was tested with excellent results on 123 images (three data sets each a multiple slice 2D, 16 phase Cine study, three data sets each a multiple slice 2D, 13 phase Cine study and three data sets each a multiple slice 2D, 12 phase Cine study).

Abstract:   Point displacement constraints constitute an attractive technique for interactive design of smooth curves, surfaces, and volumes. The user defines an arbitrary number of ''control points'' on the object and specifies their desired spatial location, while the system computes the object's degrees of freedom so that the constraints are satisfied. A constraint-based interface gives a feeling of direct manipulation of the object. In this article we introduce soft constraints, constraints which do not have to be met exactly. The softness of each constraint serves as a nonisotropic, local shape parameter enabling the user to explore the space of objects conforming to the constraints. Additionally, there is a global shape parameter which determines the amount of similarity of the designed object to a rest shape, or equivalently, the rigidity of the rest shape. We present an algorithm termed probabilistic point constraints (PPC) for implementing soft constraints. The PPC algorithm views constraints as stochastic measurements of the state of a static system. The softness of a constraint is derived from the covariance of the ''measurement.'' The resulting system of probabilistic equations is solved using the Kalman filter, a powerful estimation tool in the theory of stochastic systems. We also describe a user interface using direct-manipulation devices for specifying and visualizing covariances in 2D and 3D. The algorithm is suitable for any object represented as a parametric blend of control points, including most spline representations. The covariance of a constraint provides a continuous transition from exact interpolation to controlled approximation of the constraint. The algorithm involves only linear operations and allows real-time interactive direct manipulation of curves and surfaces on current workstations.

Abstract:   Image warping, often referred to as ''rubber sheeting,'' represents the deformation of a domain image space into a range image space. In this paper, a technique which extends the definition of a rubber-sheet transformation to allow a polygonal region to be warped into one or more subsets of itself, where the subsets may be multiply connected, is described. To do this, it constructs a set of ''slits'' in the domain image, which correspond to discontinuities and concavities in the range image, using a technique based on generalized Voronoi diagrams. The concept of medial axis is extended to describe inner and outer medial contours of a polygon. Polygonal regions are decomposed into annular subregions, and path homotopies are introduced to describe the annular subregions. These constructions motivate the definition of a ladder, which guides the construction of grid point pairs necessary to effect the warp itself. (C) 1994 Academic Press, Inc.

Abstract:   Image warping, often referred to as ''rubber sheeting,'' represents the deformation of a domain image space into a range image space. In this paper, a technique which extends the definition of a rubber-sheet transformation to allow a polygonal region to be warped into one or more subsets of itself, where the subsets may be multiply connected, is described. To do this, it constructs a set of ''slits'' in the domain image, which correspond to discontinuities and concavities in the range image, using a technique based on generalized Voronoi diagrams. The concept of medial axis is extended to describe inner and outer medial contours of a polygon. Polygonal regions are decomposed into annular subregions, and path homotopies are introduced to describe the annular subregions. These constructions motivate the definition of a ladder, which guides the construction of grid point pairs necessary to effect the warp itself. (C) 1994 Academic Press, Inc.

Abstract:   Reliable derivatives or digital images have always been hard to obtain, especially (but not only) at high orders. We analyze the sources of errors in traditional filters, such as derivatives or the Gaussian, that are used for differentiation. We then study a class of filters which is much more suitable for our purpose, namely filters that preserve polynomials up to a given order. We show that the errors in differentiation can be corrected using these filters. We derive a condition for the validity domain of these filters, involving some characteristics of the filter and of the shape. Our experiments show a very good performance for smooth functions.

Abstract:   Background In hypertrophic cardiomyopathy, ejection fraction is normal or increased, and force-length relations are reduced. However, three-dimensional (3D) motion and deformation in vivo have not been assessed in this condition. We have reconstructed the 3D motion of the left ventricle (LV) during systole in 7 patients with hypertrophic cardiomyopathy (HCM) and 12 normal volunteers by use of magnetic resonance tagging. Methods and Results Transmural tagging stripes were automatically tracked to subpixel resolution with an active contour model. A 3D finite-element model was used to interpolate displacement information between short- and long-axis slices and register data on a regional basis. Displacement and strain data were averaged into septal, posterior, lateral, and anterior regions at basal, midventricular, and apical levels. Radial motion (toward the central long axis) decreased slightly in patients with HCM, whereas longitudinal displacement (parallel to the long axis) of the base toward the apex was markedly reduced: 7.5 +/- 2.5 mm (SD) versus 12.5 +/- 2.0 mm, P<.001. Circumferential and longitudinal shortening were both reduced in the septum (P<.01 at all levels). The principal strain associated with 3D maximal contraction was slightly depressed in many regions, significantly in the basal septum (-0.18 +/- 0.05 versus -0.22 +/- 0.02, P<.05) walls. In contrast, LV torsion (twist of the apex about the long axis relative to the base) was greater in HCM patients (19.9 +/- 2.4 degrees versus 14.6 +/- 2.7 degrees, P<.01). Conclusions HCM patients had reduced 3D myocardial shortening on a regional basis; however, LV torsion was increased.

Abstract:   This paper considers the approach to dynamic image processing, which is one of the important problems in the future medical image processing. The tracking of the object and the analysis of the motion are discussed for the dynamic images of a nonrigid object with smooth shape, motion and deformation, which is the case in most medical images. This approach is based on an active contour model defined by an energy function in terms of both intra- and interframe constraints for the contour of the object. The contour of the target object is extracted and tracked by minimizing the energy function using multiscale dynamic programming and the motion is analyzed. The dynamic programming in multiscale proposed in this paper is to adjust the search neighborhood of the dynamic programming according to the scale. The coarse or fine neighborhood is defined for the coarse and fine scales, respectively, and the energy is minimized starting from the coarse scale and shifting to the fine scale. By this scheme, a large motion an deformation of the object can be handled. The proposed motion tracking method has been applied successfully to the dynamic image in the ''behavioral analysis of a slug aiming at the analysis of the neural mechanism of learning and memory formation in slugs,'' as well as to dynamic echocardiographic images. In the first application, the positive maximum of the curvature along the contour is extracted in the motion analysis as a characteristic point invariant to the deformation of the object. Then the shift of that point is traced. By this approach, the rough motion of the object can be estimated.

Abstract:   This paper proposes a robust method for tracking an object contour in a sequence of images. In this method, both object extraction and tracking problems can be solved simultaneously. Furthermore, it is applicable to the tracking of arbitrary shapes since it does not need a priori knowledge about the object shapes. In the contour tracking, energy-minimizing elastic contour models are utilized, which is newly presented in this paper. The contour tracking is formulated as an optimization problem to find the position that minimizes both the elastic energy of its model and the potential energy derived from the edge potential image that includes a target object contour. We also present an algorithm which efficiently solves energy minimization problems within a dynamic programming framework. The algorithm enables us to obtain optimal solution even when the variables to be optimized are not ordered. We show the validity and usefulness of the proposed method with some experimental results.

Abstract:   Our goal is to reconstruct both the shape and reflectance properties of surfaces from multiple images. We argue that an object-centered representation is most appropriate for this purpose because it naturally accommodates multiple sources of data, multiple images (including motion sequences of a rigid object), and self-occlusions. We then present a specific object-centered reconstruction method and its implementation. The method begins with an initial estimate of surface shape provided, for example, by triangulating the result of conventional stereo. The surface shape and reflectance properties are then iteratively adjusted to minimize an objective function that combines information from multiple input images. The objective function is a weighted sum of stereo, shading, and smoothness components, where the weight varies over the surface. For example, the stereo component is weighted more strongly where the surface projects onto highly textured areas in the images, and less strongly otherwise. Thus, each component has its greatest influence where its accuracy is likely to be greatest. Experimental results on both synthetic and real images are presented.

Abstract:   This paper presents a new method that combines a medial axis and implicit surfaces in order to reconstruct a 3D solid from on unstructured set of points scattered on the object's surface. The representation produced is based on iso-surfaces generated by skeletons, and is a particularly compact way of defining a smooth free-form solid. The method is based on the minimisation of an energy representing a ''distance'' between the set of data points and the iso-surface, resembling previous reserach(19). Initialisation, however, is more robust and efficient since there is computation of the medial axis of the set of points. Instead of subdividing existing skeletons in order to refine the object's surface, a new reconstruction algorithm progressively selects skeleton-points from the precomputed medial axis using an heuristic principle based on a ''local energy'' criterion. This drastically speeds up the reconstruction process. Moreover, using the medial axis allows reconstruction of objects with complex topology and geometry, like objects that have holes and branches or that are composed of several connected components. This process is fully automatic. The method has been successfully applied to both synthetic and real data.

Abstract:   Modeling a curve through minimizing its energy yields an overall smooth curve. A common way to model shape features is to perform the minimization subject to a number of interpolation constraints. This way of modeling is attractive because the designer is not bothered with the precise representation of the curve (e.g, control points). However, local shape specification by means of interpolation constraints is very limited. On the other hand, local deformation by repositioning control points is powerful but very laborious, and destroys the minimal energy property. In this paper, deform operators are introduced for 3D curve modeling that have built-in energy terms that have an intuitive effect. These operators allow local shape modification and do justice to the energy minimization way of modeling.

Abstract:   We present a graphically interactive procedure which is used to register a digital anatomic brain atlas with the tomographic patient volume. Patient structures to be segmented are outlined by local elastic deformation of corresponding objects from the anatomy model. This is performed in voxel space using a cost minimization procedure. The anatomic knowledge acquired in this manner is stored in a patient specific volume dataset and guides a raycaster with respect to the localization of object surfaces in order to control the result of the deformation process. Thus objects, which so far could not have been segmented appropriately or only after tedious manual editing efforts, become accessible by physicians. We present several results demonstrating the high quality and practicality of the method.

Abstract:   One approach to the detection of curves at subpixel accuracy involves the reconstruction of such features from subpixel edge data points. A new technique is presented for reconstructing and segmenting curves with subpixel accuracy using deformable models. A curve is represented as a set of interconnected Hermite splines forming a snake generated from the subpixel edge information that minimises the global energy functional integral over the set. While previous work on the minimisation was mostly based on the Euler-Lagrange transformation, the authors use the finite element method to solve the energy minimisation equation. The advantages of this approach over the Euler-Lagrange transformation approach are that the method is straightforward, leads to positive m-diagonal symmetric matrices, and has the ability to cope with irregular geometries such as junctions and corners. The energy functional integral solved using this method can also be used to segment the features by searching for the location of the maxima of the first derivative of the energy over the elementary curve set.

Abstract:   To provide an overview of the methods and clinical applications of three dimensional (3D) medical imaging in the oncologic patient. Methods and Materials: We briefly outline the techniques currently used to create 3D medical images with an emphasis on their strengths and shortcomings as they relate to oncologic imaging and radiation therapy planning, We then discuss some of the most important and promising oncologic applications of 3D imaging and suggest likely future directions in this rapidly developing field. Results: Since the first application of 3D techniques to medical data over a decade ago, 3D medical images have evolved from relatively crude representations of musculoskeletal abnormalities to detailed and accurate representations of a variety of soft tissue, vascular, and oncologic pathology. The rapid development of both computer hardware and software coupled with the application of 3D techniques to a variety of imaging modalities have expanded the clinical applications of this technology dramatically. Conclusions: 3D medical images are clinically practical tools for oncologic evaluation and effective radiation therapy planning.

Abstract:   The main aim of this work is the development of a vision-based road detection system fast enough to cope with the difficult real-time constraints imposed by moving vehicle applications. The hardware platform, a special-purpose massively parallel system, has been chosen to minimize system production and operational costs. This paper presents a novel approach to expectation-driven low-level image segmentation, which can be mapped naturally onto mesh-connected massively parallel SIMD architectures capable of handling hierarchical data structures. The input image is assumed to contain a distorted version of a given template; a multiresolution stretching process is used to reshape the original template in accordance with the acquired image content, minimizing a potential function. The distorted template is the process output.

Abstract:   Tissue tagging using magnetic resonance (MR) imaging has enabled quantitative noninvasive analysis of motion and deformation in vivo. One method for MR tissue tagging is Spatial Modulation of Magnetization (SPAMM), Manual detection and tracking of tissue tags by visual inspection remains a time-consuming and tedious process. We have developed an interactively guided semi-automated method of detecting and tracking tag intersections in cardiac MR images, A template matching approach combined with a novel adaptation of active contour modeling permits rapid analysis of MR images. We have validated our technique using MR SPAMM images of a silicone gel phantom with controlled deformations. Average discrepancy between theoretically predicted and semi-automatically selected tag intersections was 0.30 mm +/- 0.17 [mean +/- SD, NS (P < 0.05)]. Cardiac SPAMM images of normal volunteers and diseased patients also have been evaluated using our technique.

Abstract:   Point distribution models allow a compact description of an object's shape to be found from a set of example images. In previous work by the first author, a method of incorporating grey level information into a PDM was developed. This paper investigates fitting such a composite model to image data consisting of a set of images of a pig viewed from above. Model fitting is achieved by optimizing an objective function consisting of two components, one that measures the degree of grey level correspondence between the model and the data, and the other that measures how well the boundary of the model fits the data. The shape of the objective function as the model parameters are varied is investigated, and an optimization strategy developed. The strategy is used to find a pig in a number of images with backgrounds of increasing complexity. The strategy performs well with both an uncluttered and a realistic background. The performance with a simulated noisy background is not so good when the boundary component is included in the objective function. This is a result of the boundary component being more sensitive to noise in the image. In this case, it is better to optimize with the grey level component alone. A problem is identified when the grey level distribution changes significantly as the pig moves under the light source. It is suggested that this could be overcome by including variations in grey level distribution as modes in the model.

Abstract:   Active contour models (''snakes'') are a powerful tool for deformable object tracking in moving images. But the existing snake models are not well-adapted for tracking corners and objects on a complex background. In this paper, we present a novel active contour model, the ''Adjustable Polygons'', which is a set of active segments that can fit any object shape (including corners). A new energy based on textural characteristics of objects is also proposed, in order to resolve conflict situations while tracking objects on multiple contour background.

Abstract:   A new active contour model for finding and mapping the outer cortex in brain images is developed, A cross-section of the brain cortex is modeled as a ribbon, and a constant speed mapping of its spine is sought. A variational formulation, an associated force balance condition, and a numerical approach are proposed to achieve this goal, The primary difference between this formulation and that of snakes is in the specification of the external force acting on the active contour. A study of the uniqueness and fidelity of solutions is made through convexity and frequency domain analyses, and a criterion for selection of the regularization coefficient is developed. Examples demonstrating the performance of this method on simulated and real data are provided.

Abstract:   To obtain a three-dimensional reconstruction of the hippocampus from a volumetric MRI head study, it is necessary to separate that structure not only from the surrounding white matter, but also from contiguous areas of gray matter-the amygdala and cerebral cortex. At present it is necessary for a physician to manually segment the hippocampus on each slice of the volume to obtain such a reconstruction. This process is time consuming, and is subject to inter- and intra-operator variation as well as large discontinuities between slices. We propose a novel technique, making use of a combination of gray scale and edge-detection algorithms and some a priori knowledge, by which a computer may make an unsupervised identification of a given structure through a series of contiguous images. This technique is applicable even if the structure includes so-called false contours or missing contours. Applications include three-dimensional reconstruction of difficult-to-segment regions of the brain, and volumetric measurements of structures from series of two-dimensional images.

Abstract:   Fine needle aspiration (FNA) accuracy is limited by, among other factors, the subjective interpretation of the aspirate. We have increased breast FNA accuracy by coupling digital image analysis methods with machine learning techniques. Additionally, our mathematical approach captures nuclear features (''grade'') that are prognostically more accurate than are estimates based on tumor size and lymph node status. An interactive computer system evaluates, diagnoses and determines prognosis based on nuclear features derived directly from a digital scan of FNA slides. A consecutive series of 569 patients provided the data for the diagnostic study. A 166-patient subset provided the data for the prognostic study. An additional 75 consecutive, new patients provided samples to test the diagnostic system. The projected prospective accuracy of the diagnostic system was estimated to be 97% by 10-fold cross-validation, and the actual accuracy on 75 new samples runs 100%. The projected prospective accuracy of the prognostic system was estimated to be 86% by leave-one-out testing.

Abstract:   We describe some of the current challenges in developing and validating computer vision algorithms for manufacturing applications. We focus on the general theme of template-based processing, where geometric templates provide a basis for local feature analysis, registration and recognition (via constraint-based modelling) and model adaptation using statistical methods. We describe recent successful applications of template-based techniques in the areas of manufacturing part inspection and process understanding and monitoring. We also examine the question 'Why are there so few computer vision applications in manufacturing?' We suggest that two of the major bottlenecks remain speed of algorithm development and how to validate algorithm performance with a limited data set. Finally, we identify some of what we see as emerging and future potential application areas of computer vision methods in manufacturing, where the current trend is to provide tools for continuous product improvement rather than (final) product inspection, and 3D measurement capabilities.

Abstract:   The goal of the described work is to model visual articulation movements of prototypic speakers with respect to custom-made text A language-wide extension of the motion model leads to a visible speech synthesis and further more to an artificial computer trainer for speechreading. The developed model is based on a set of specific video key-pictures and the interpolation of interim pictures. The key-picture selection is realized by a fuzzy-c-means classification algorithm.

Abstract:   The automated analysis of 3D medical images can improve both diagnosis and therapy significantly. This automation raises a number of new fascinating research problems in the fields of computer vision, graphics and robotics. In this paper, I propose a list of such problems after a review of the current major 3D imaging modalities, and a description of the related medical needs. I then present some of the past and current work done in our research group EPIDAURE* at INRIA, on the following topics: segmentation of 3D images; 3D shape modelling; 3D rigid and nonrigid registration; 3D motion analysis; and 3D simulation of therapy. Most topics are discussed in a synthetic manner, and illustrated by results. Rigid matching is treated more thoroughly as an illustration of a transfer from computer vision towards 3D image processing. The later topics are illustrated by preliminary results, and a number of promising research tracks are suggested.

Abstract:   We present a model-based method for object identification in images of natural scenes. It has successfully been implemented for the classification of cars based on their rear view. In a first step, characteristic features such as lines and corners are detected within the image. Generic models of object-classes, described by the same set of features, are stored in a database. Each model represents a whole class of objects (e.g., passenger cars, vans, big trucks). In a preprocessing stage, the most probable object is selected by means of a corner-feature based Hough transform. This transformation also suggests the position and scale of the object in the image. Guided by similarity measures, the model is then aligned with image features using a matching algorithm based on the elastic net technique [1]. During this iterative process, the model is allowed to undergo changes in scale, position and certain deformations. Deformations are kept within limits such that one model can fit to all objects belonging to the same class, but not to objects of other classes. In each iteration step, quantities to assess the matching process are obtained.

Abstract:   A two-stage algorithm for extraction of the ventricular chambers (endocardial surfaces) in flow-enhanced magnetic resonance images is described, In the first stage, the approximate locations and sizes of the endocardial surfaces are determined by intensity thresholding. In the second stage, points on each approximated surface are repositioned to nearest locally maximum gradient magnitude points and a generalized cylinder is fitted to them, Examples of ventricular chambers in cine MR images determined by this algorithm are presented.

Abstract:   When interpreting a dynamic and uncertain world it is important to have a high-level vision component that can guide the reasoning of the whole vision system. This guidance is provided by an attentional mechanism that exploits knowledge of the specific problem being solved. Here we survey work relevant to the development of such an attentional mechanism, using surveillance as an application domain to tie together issues of spatial representation, events, behaviour, control and planning. The paper culminates in a brief description of HIVIS-WATCHER a program that makes use of all these areas.

Abstract:   Modal matching is a new method for establishing correspondences and computing canonical descriptions, The method is based on the idea of describing objects in terms of generalized symmetries, as defined by each object's eigenmodes. The resulting modal description is used for object recognition and categorization, where shape similarities are expressed as the amounts of modal deformation energy needed to align the two objects, In general, modes provide a global-to-local ordering of shape deformation and thus allow for selecting which types of deformations are used in object alignment and comparison, In contrast to previous techniques, which required correspondence to be computed with an initial or prototype shape, modal matching utilizes a new type of finite element formulation that allows for an object's eigenmodes to be computed directly from available image information, This improved formulation provides greater generality and accuracy, and is applicable to data of any dimensionality, Correspondence results with 2D contour and point feature data are shown, and recognition experiments with 2D images of hand tools and airplanes are described.

Abstract:   Edge detection is formulated as a curve fitting problem. First, high-gradient pixels are grouped into elongated regions and then a curve is fitted to each. The curve fitting method used in this work does not require solving a system of equations, and therefore is fast. Examples of edge detection by curve fitting on synthetic and real images are presented, and results obtained are compared with those determined by the Laplacian of Gaussian operator.

Abstract:   Objective: To use digital image analysis and machine learning to (1) improve breast mass diagnosis based on fine-needle aspirates and (2) improve breast cancer prognostic estimations. Design: An interactive computer system evaluates, diagnoses, and determines prognosis based on cytologic features derived from a digital scan of fine-needle aspirate slides. Setting: The University of Wisconsin (Madison) Departments of Computer Science and Surgery and the University of Wisconsin Hospital and Clinics. Patients: Five hundred sixty-nine consecutive patients (212 with cancer and 357 with benign masses) provided the data for the diagnostic algorithm, and an additional 118 (31 with malignant masses and 87 with benign masses) consecutive, new patients tested the algorithm. One hundred ninety of these patients with invasive cancer and without distant metastases were used for prognosis. Interventions: Surgical biopsy specimens were taken from all cancers and some benign masses. The remaining cytologically benign masses were followed up for a year and surgical biopsy specimens were taken if they changed in size or character. Patients with cancer received standard treatment. Outcome Measures: Cross validation was used to project the accuracy of the diagnostic algorithm and to determine the importance of prognostic features. In addition, the mean errors were calculated between the actual times of distant disease occurrence and the times predicted using various prognostic features. Statistical analyses were also done. Results The predicted diagnostic accuracy was 97% and the actual diagnostic accuracy on 118 new samples was 100%. Tumor size and lymph node status were weak prognosticators compared with nuclear features, in particular those measuring nuclear size. Compared with the actual time for recurrence, the mean error of predicted times for recurrence with the nuclear features was 17.9 months and was 20.1 months with tumor size and lymph node status (P=.11). Conclusion: Computer technology will improve breast fine-needle aspirate accuracy and prognostic estimations.

Abstract:   This paper presents a new method to segment brain parenchyma and cerebrospinal fluid spaces automatically in routine axial spin echo multispectral MR images. The algorithm simultaneously incorporates information about anatomical boundaries (shape) and tissue signature (grey scale) using a priori knowledge. The head and brain are divided into four regions and seven different tissue types. Each tissue type c is modeled by a multivariate Gaussian distribution N(mu(c), Sigma(c)). Each region is associated with a finite mixture density corresponding to its constituent tissue types, Initial estimates of tissue parameters {mu(c), Sigma(c)}(c=1,...,7) are obtained from L-means clustering of a single slice used for training. The first algorithmic step uses the EM-algorithm for adjusting the initial tissue parameter estimates to the MR data of new patients, The second step uses a recently developed model of dynamic contours to detect three simply closed nonintersecting curves in the plane, constituting the arachnoid/dura mater boundary of the brain, the border between the subarachnoid space and brain parenchyma, and the inner border of the parenchyma toward the lateral ventricles, The model, which is formulated by energy functions in a Bayesian framework, incorporates a priori knowledge, smoothness constraints, and updated tissue type parameters, Satisfactory maximum a posteriori probability estimates of the closed contour curves defined by the model were found using simulated annealing.

Abstract:   Point distribution models (PDMs) allow a compact description of an object's shape to be found from a set of example images. In previous work by the second author a method of incorporating grey level information into a PDM was developed. Further work investigated fitting such a composite model to image data consisting of a set of images of a pig viewed from above. This paper describes work on images containing more than one pig. A technique for initialising the model is used which searches the image for ridges in the grey level landscape. These generally lie along the backbone of the animal and provide a good starting point for automatic fitting. By minimising an objective function which measures the difference in grey level and the error in boundary correspondence, an accurate fit of model to data is obtained. Ridge detection initialises the model to within +/-12 pixels of the object. Strict limits on the boundaries of the search space constrain the minimisation process allowing convergence to the true minimum. The resulting fit is good even for objects which are partially obscured. Poor final values of the objective function allow the detection of erroneous results.

Abstract:   In this paper, we attack the figure-ground discrimination problem from a combinatorial optimization perspective. In general, the solutions proposed in the past solved this problem only partially: either the mathematical model encoding the figure-ground problem was too simple or the optimization methods that were used were not efficient enough or they could not guarantee to find the global minimum of the cost function describing the figure-ground model. The method that we devised and which is described in this paper is tailored around the following contributions. First, we suggest a mathematical model encoding the figure-ground discrimination problem that makes explicit a definition of shape (or figure) based on cocircularity, smoothness, proximity, and contrast. This model consists of building a cost function on the basis of image element interactions. Moreover, this cost function fits the constraints of an interacting spin system, which in turn is a well suited physical model to solve hard combinatorial optimization problems. Second, we suggest a combinatorial optimization method for solving the figure-ground problem, namely mean field annealing which combines the mean field approximation and annealing. Mean field annealing may well be viewed as a deterministic approximation of stochastic methods such as simulated annealing. We describe in detail the theoretical bases of this method, derive a computational model, and provide a practical algorithm. Finally, some experimental results are shown for both synthetic and real images.

Abstract:   This paper describes several new image understanding methods based on parallel operation. There are several constraint satisfaction approaches using an energy minimization. We show how we reconstruct three-dimensional surfaces from contours without elevation data and sparse points of known elevation data using this approach. We also introduce Active Net using this approach, and apply this model to segmentation and binocular stereo matching. We experimented with these energy minimization approaches to solve the problems of early and intermediate levels of computer vision, and show some of the results of our recent research.

Abstract:   We propose an efficient method for tracking several objects moving through a sequence of monocular images against a non-uniform background. Each object entering the scene is intercepted by an active contour model which locks on it as long as it moves in the scene. The procedure does not necessitate an interactive initialization. Some results are presented in case of real traffic scenes.

Abstract:   In this paper, we present a system for detection and tracking of tubular molecules in images. The automatic detection and characterization of the shape, location, and motion of these molecules can enable new laboratory protocols in several scientific disciplines. The uniqueness of the proposed system is twofold: At the macro level, the novelty of the system lies in the integration of object localization and tracking using geometric properties; at the micro level, in the use of high and low level constraints to model the detection and tracking subsystem. The underlying philosophy for object detection is to extract perceptually significant features from the pixel level image, and then use these high level cues to refine the precise boundaries. In the case of tubular molecules, the perceptually significant features are antiparallel line segments or, equivalently, their axis of symmetries. The axis of symmetry infers a coarse description of the object in terms of a bounding polygon. The polygon then provides the necessary boundary condition for the refine ment process, which is based on dynamic programming. For tracking the object in a time sequence of images, the refined contour is then projected onto each consecutive frame.

Abstract:   A constrained variational curve is a curve that minimizes some energy functional under certain interpolation constraints. Modeling curves using constrained variational principles is attractive, because the designer is not bothered with the precise representation of the curve (e.g. control points). Until now, the modeling of variational curves is mainly done by means of constraints. If such a curve of least energy is deformed locally (e.g, by moving its control points) the concept of energy minimization is lost. In this paper we introduce deform operators with built-in energy terms. We have tested our ideas in a prototype system for modeling uniform B-spline curves. Through the use of widgets, the user can interactively modify the range of influence and other properties of the operators. Experiments show that these operators offer a very intuitive way of modeling.

Abstract:   This paper reports on current developments in the area of model-based video coding, a technique which shows promise of achieving very, large bit-rate reductions for moving images. After an introduction and historical review, advances are summarized in several areas, among them improved 3D tracking of the human head and of facial expressions, the use of muscle-driven model animation with skin synthesis, techniques for luminance compensation, and switched coders. Bit rates ranging from 64 kb/s down to about 1 kb/s have been obtained using head-and-shoulder video sequences. Problems with model-based methods are identified and future developments in both CBR and VER transmission discussed.

Abstract:   A method for the segmentation of complex, three-dimensional objects using hierarchical active surface templates is presented. The templates consist of one or more active surface models which are specified according to a priori knowledge about the expected shape and location of the desired object. This allows complex objects to be naturally modeled as collections of simple subparts which are geometrically constrained. The template is adaptively deformed by the three-dimensional image data in which it is initialized such that the template boundaries are brought into correspondence with the assumed image object. An external energy field is developed based on a vector distance transform such that the surfaces are deformed according to object shape. The method is demonstrated by the segmentation of the human brain from three-dimensional magnetic resonance images of the head given an a priori model of a normal brain.

Abstract:   Correspondence establishment has been a difficult problem in machine vision. In this paper, we present an optimization technique for the task. The geometric constraints to the solution are formulated as forces, which are combined to provide clue for mapping between two sets of points such that the geometric constraints are best satisfied. The strong point of this method is that it is easy to integrate several sources of information to obtain a solution while keeping the decision simple, and does not suffer from the uncontrollable flexibility as in active contour models. We illustrate the method with the problem of establishing correspondence between parallel curves.

Abstract:   We undertake to develop a general theory of two-dimensional shape by elucidating several principles which any such theory should meet. The principles are organized around two basic intuitions: first, if a boundary were changed only slightly, then, in general, its shape would change only slightly. This leads us to propose an operational theory of shape based on incremental contour deformations. The second intuition is that not all contours are shapes, but rather only those that can enclose ''physical'' material. A theory of contour deformation is derived from these principles, based on abstract conservation principles and Hamilton-Jacobi theory. These principles are based on the work of Sethian (1985a, c), the Osher-Sethian (1988), level set formulation the classical shock theory of Lax (1971; 1973), as well as curve evolution theory for a curve evolving as a function of the curvature and the relation to geometric smoothing of Gage-Hamilton-Grayson (1986; 1989). The result is a characterization of the computational elements of shape: deformations, parts, bends, and seeds, which show where to place the components of a shape. The theory unifies many of the diverse aspects of shapes, and leads to a space of shapes (the reaction/diffusion space), which places shapes within a neighborhood of ''similar'' ones. Such similarity relationships underlie descriptions suitable for recognition.

Abstract:   Visual assessments of nuclear grade are subjective yet still prognostically important. Now, computer-based analytical techniques can objectively and accurately measure size, shape and texture features, which constitute nuclear grade. The cell samples used in this study were obtained by fine needle aspiration (FNA) during the diagnosis of 187 consecutive patients with invasive breast cancer. Regions of FNA preparations to be analyzed were digitized and displayed on a computer monitor. Nuclei to be analyzed were roughly outlined by an operator using a mouse. Next, the computer generated a ''snake'' that precisely enclosed each designated nucleus. Ten nuclear features were then calculated for each nucleus based on these snakes. These results were analyzed statistically and by an inductive machine learning technique that we developed and call ''recurrence surface approximation'' (RSA). Both the statistical and RSA machine learning analyses demonstrated that computer-derived nuclear features are prognostically move important than are the classic prognostic features, tumor size and lymph node status.

Abstract:   It is generally agreed that individual visual cues are fallible and often ambiguous. This has generated a lot of interest in design of integrated vision systems which are expected to give a reliable performance in practical situations. The design of such systems is challenging since each vision module works under a different and possibly conflicting set of assumptions. We have proposed and implemented a multiresolution system which integrates perceptual organization (grouping), segmentation, stereo, shape from shading, and line labeling modules. We demonstrate the efficacy of our approach using images of several different realistic scenes. The output of the integrated system is shown to be insensitive to the constraints imposed by the individual modules. The numerical accuracy of the recovered depth is assessed in case of synthetically generated data. Finally, we have qualitatively evaluated our approach by reconstructing geons from the depth data obtained from the integrated system. These results indicate that integrated vision systems are likely to produce better reconstruction of the input scene than the individual modules.

Abstract:   Magnetic resonance tissue tagging allows noninvasive in vivo measurement of soft tissue deformation, Planes of magnetic saturation are created, orthogonal to the imaging plane, which form dark lines (stripes) in the image, We describe a method for tracking stripe motion in the image plane, and show how this information can be incorporated into a finite element model of the underlying deformation, Human heart data were acquired from several imaging planes in different orientations and were combined using a deformable model of the left ventricle wall, Each tracked stripe point provided information on displacement orthogonal to the original tagging plane, i.e., a one-dimensional (1-D) constraint on the motion, Three-dimensional (3-D) motion and deformation was then reconstructed by fitting the model to the data constraints by linear least squares, The average root mean squared (rms) error between tracked stripe points and predicted model locations was 0.47 mm (n = 3100 points). In order to validate this method and quantify the errors involved, we applied it to images of a silicone gel phantom subjected to a known, well-controlled, 3-D deformation. The finite element strains obtained were compared to an analytic model of the deformation known to be accurate in the central axial plane of the phantom, The average rms errors were 6% in both the reconstructed shear strains and 16% in the reconstructed radial normal strain.

Abstract:   This paper proposes a new approach for inputting handwritten Distribution Facility Drawings (DFD) and their maps into a computer automatically by using the Facility Management Database (FMD). Our recognition method makes use of external information for drawing/map recognition. It identifies each electric-pole symbol and support cable symbol on drawings simply by consulting the FMD. Other symbols such as transformers and electric wires can be placed on drawings automatically. In this positioning of graphic symbols, we present an automatic adjustment method of a symbol's position on the latest digital maps. When a contradiction is unsolved due to an inconsistency between the content of the DFD and the FMD, the system requests a manual feedback from the operator. Furthermore, it uses the distribution network of the DFD to recognize the street lines on the maps which aren't computerized. This can drastically reduce the cost for computerizing drawings and maps.

Abstract:   We present in this paper a new curve and surface implicit model. This implicit model is based on hyperquadrics and allows a local and global control of the shape and a wide variety of allowable shapes. We define a hybrid hyperquadric model by introducing implicitly some local properties on a global shape model. The advantage of our model is that it describes global and local properties through a unique implicit equation, yielding a representation of the shape by means of its parameters, independently of the chosen numerical resolution. The data fitting is obtained through the minimization of energy, modeling the attraction to data independently of the implicit description of the shape, After studying the geometry of hyperquadrics and how the shape deforms when we modify slightly its implicit equation, we are able to define an algorithm for automatic refining of the fit by adding an adequate term to the implicit representation, This geometric approach malt:es possible an efficient description of the data points and an automatic tuning of the fit according to the desired accuracy. (C) 1996 Academic Press, Inc.

Abstract:   This paper presents a new edge detection scheme that detects two-dimensional (2-D) edges by a curve-segment-based detection functional guided by the zero-crossing contours of the Laplacian-of-Gaussian (LOG) to approach the true edge locations. The detection functional is shown to be optimal in terms of signal-to-noise ratio (SNR) and edge localization accuracy; it also preserves the nice scaling property held uniquely by the LOG in scale space.

Abstract:   A unified framework for 3-D shape reconstruction allows us to combine image-based and geometry-based information sources. The image information is akin to stereo and shape-from-shading, while the geometric information may be provided in the form of 3-D points, 3-D features, or 2-D silhouettes. A formal integration framework is critical in recovering complicated surfaces because the information from a single source is often insufficient to provide a unique answer. Our approach to shape recovery is to deform a generic object-centered 3-D representation of the surface so as to minimize an objective function, This objective function is a weighted sum of the contributions of the various information sources. We describe these various terms individually, our weighting scheme, and our optimization method, Finally, we present results on a number of difficult images of real scenes for which a single source of information would have proved insufficient. (C) 1996 Academic Press, Inc.

Abstract:   The goal of this paper is to offer a structured synopsis of the problems in image motion computation and analysis, and of the methods proposed, exposing the underlying models and supporting assumptions. A sufficient number of pointers to the literature will be given, concentrating mostly on recent contributions. Emphasis will be on the detection, measurement and segmentation of image motion. Tracking, and deformable motion isssues will be also addressed. Finally, a number of related questions which could require more investigations will be presented.

Abstract:   Analysis of brain images often requires accurate localization of cortical convolutions. Although magnetic resonance (MR) brain images offer sufficient resolution for identifying convolutions in theory, the nature of tomographic imaging prevents clear definition of convolutions in individual slices, Existing methods for solving this problem rely on heuristic adaptation of brain atlases created from a small number of individuals, These methods do not usually provide high accuracy because of large biological variations among individuals. We propose to localize convolutions by linking realistic visualizations of the cortical surface with the original image volume. We have developed a system so that a user can quickly localize key convolutions in several visualizations of an entire brain surface, Because of the links between the visualizations and the original volume, these convolutions are simultaneously localized in the original image slices, In the process of our development, we have implemented a fast and easy method for visualizing cortical surfaces in MR images, thereby making our scheme usable in practical applications.

Abstract:   We have devised, implemented, and tested a fast, spatially accurate technique for calculating the high-dimensional deformation held relating the brain anatomies of an arbitrary pair of subjects, The resulting three-dimensional (3-D) deformation map can be used to quantify anatomic differences between subjects or within the same subject over time and to transfer functional information between subjects or integrate that information on a single anatomic template. The new procedure is based on developmental processes responsible for variations in normal human anatomy and is applicable to 3-D brain images in general, regardless of modality, Hybrid surface models known as Chen surfaces (based on superquadrics and spherical harmonics) are used to efficiently initialize 3-D active surfaces, and these then extract from both scans the developmentally fundamental surfaces of the ventricles and cortex. The construction of extremely complex surface deformation maps on the internal cortex is made easier by budding a generic surface structure to model it, Connected systems of parametric meshes model several deep sulci whose trajectories represent critical functional boundaries, These sulci are sufficiently extended inside the brain to reflect subtle and distributed variations in neuroanatomy between subjects. The algorithm then calculates the high-dimensional volumetric warp (typically with 384(2) x 256 x 3 approximate to 0.1 billion degrees of freedom) deforming one 3-D scan into structural correspondence with the other. Integral distortion functions are used to extend the deformation held required to elastically transform nested surfaces to their counterparts in the target scan. The algorithm's accuracy is tested, by warping 3-D magnetic resonance imaging (MRI) volumes from normal subjects and Alzheimer's patients, and by warping full-color 1024(3) digital cryosection volumes of the human head onto MRI volumes, Applications are discussed, including the transfer of multisubject 3-D functional, vascular, and histologic maps onto a single anatomic template; the mapping of 3-D brain atlases onto the scans of new subjects; and the rapid detection, quantification, and mapping of local shape changes in 3-D medical images in disease and during normal or abnormal growth and development.

Abstract:   We present a new mathematical formulation of some curve and surface reconstruction algorithms by the introduction of auxiliary variables. For deformable models and templates, the extraction of a shape is obtained through the minimization of an energy composed of an internal regularization term (not necessary in the case of parametric models) and an external attraction potential. Two-step iterative algorithms have been often used where, at each iteration, the model is first locally deformed according to the potential data attraction and then globally smoothed (or fitted in the parametric case). We show how these approaches can be interpreted as the introduction of auxiliary variables and the minimization of a two-variables energy. The first variable corresponds to the original model we are looking for, while the second variable represents an auxiliary shape close to the first one. This permits to transform an implicit data constraint defined by a non convex potential into an explicit convex reconstruction problem. This approach is much simpler since each iteration is composed of two simple to solve steps. Our formulation permits a more precise setting of parameters in the iterative scheme to ensure convergence to a minimum. We show some mathematical properties and results on this new auxiliary problem, in particular when the potential is a function of the distance to the closest feature point. We then illustrate our approach for some deformable models and templates.

Abstract:   Shape modeling is an important constituent of computer vision as well as computer graphics research. Shape models aid the tasks of object representation and recognition. This paper presents a new approach to shape modeling which retains some of the attractive features of existing methods, and overcomes some of their limitations. Our technique can be applied to model arbitrarily complex shapes, which include shapes with significant protrusions, and to situations where no a priori assumption about the object's topology is made. A single instance of our model, when presented with an image having more than one object of interest, has the ability to split freely to represent each object. This method is based on the ideas developed by Osher and Sethian to model propagating solid/liquid interfaces with curvature-dependent speeds. The interface (front) is a closed, nonintersecting, hypersurface flowing along its gradient field with constant speed or a speed that depends on the curvature. It is moved by solving a ''Hamilton-Jacobi'' type equation written for a function in which the interface is a particular level set. A speed term synthesized from the image is used to stop the interface in the vicinity of object boundaries. The resulting equation of motion is solved by employing entropy-satisfying upwind finite difference schemes. We also introduce a new algorithm for rapid advancement of the front using what we call a narrow-band update scheme. The efficacy of the scheme is demonstrated with numerical experiments on low contrast medical images.

Abstract:   A method is described for automatically tracking spatial modulation of magnetization tag lines on gated cardiac images. The method differs from previously reported methods in that it uses Fourier based spatial frequency and phase information to separately track horizontal and vertical tag lines. Use of global information from the frequency spectrum of an entire set of tag lines was hypothesized to result in a robust algorithm with decreased sensitivity to noise. The method was validated in several ways: first, actual tagged cardiac images at end diastole were deformed known amounts, and the algorithm's predictions compared to the known deformations. Second, tagged, gated images of the thigh muscle (assumed to have similar signal to noise characteristics as cardiac images, but to not deform with time) were created. Again the algorithmic predictions could be compared to the known (zero magnitude) deformations and to thigh images which had been artificially deformed. Finally, actual cardiac tagged images were acquired, and comparisons made between manual, visual, determinations of tag line locations, and those predicted by the algorithm. At 0.5 T, the mean bias of the method was <0.34 mm even at large deformations and at late (noisy) times. The standard deviation of the method, estimated from the tagged thigh images, was <0.7 mm even at late times. The method may be expected to have even lower error at higher field strengths.

Abstract:   A new method for the application of active contours to biomedical images is described. The new approach, which involves extensive modification of the internal energy function acid a different method of minimising the energy functional, yields rapid, excellent fits to MR images.

Abstract:   Rationale and Objectives. We designed an image processing technique to automatically measur