PostDoc 

My main research project deals with the 2D-3D pose estimation problem. I have already dealt with this problem during my thesis, but there were lots of un-answered questions which I want (to start) to answer during my PostDoc.

The 2D-3D pose estimation problem is visualized in the left image: I assume the knowledge of a 3D object model and observe it in an image of a calibrated camera. The aim is to estimate the rigid motion (containing a 3D rotation and 3D translation) wich leads to a best fit between the object model and the image data. I am basically interested in: Developing a pose estimation algorithm which deals with 3D free-form surface models (e.g. the tea-pot). Extending surface models to kinematic chains (to model human body movements). Developing pose estimation algorithms for articulated objects modeled by free-form surface patches (2nd year PostDoc).

This page summarize research results during the first year of my PostDoc

Modeling free-form surfaces

I assume a 2-parametric surface and interpret the three 2D-functions along the x, y, and z-axes as three 2D signals. I apply a 2D-Fourier transform on these signals and thus am able to define a low-pass description of free-form surface models:

Pose estimation of rigid objects

The left images show results obtained from my algorithms. I implemented an algorithm for silhouette based pose estimation. This means as input image data I assume only the boundary of the object in the image as shown left. I am able to deal wih noisy image data, as the shadows under the car or the hand grasping the tea pot. Furthermore, it is possible to handle aspect changes of the object very efficiently and the object on the turntable is a nice example that I am able to deal even with 360 degree rotations of the object. The processing time on a Linux 2GHz machine varies between 100 and 300 ms for each frame(!). Have a look at the publications for a detailed description.

Modeling human motion

My example object model consists of 2 free-form surface patches: One for the torso and one for the arms. I then added 10 joints (5 on each arm) and implemented an openGL-viewer for the object. The joints are modeled in terms of twists as elements of se(3) which realize screw motions in SE(3). Special twists are a very nice representation for modeling joints on a manipulator or articulated object. The aim is to estimate the joint angles and the relative position and orientation of the object with respect to the image data.

Pose estimation of human models

The images show pose results during a longer image sequence. The original image is shown in the upper left frame. The lower left one shows the pose result by transforming and projecting the surface model. The two images on the right visualize the pose results in a virtual environment to verify the pose of the developed algorithm.

Future work (2nd year PostDoc)

• Introduce a multi-view scenario to gain more accurate results for higher order kinematic chains.
• Extend the surface with morphing approaches for more realistic models leading to better pose results.
• Compare the silhouette based pose results with a marker based approach (in cooperation with the GAIT-Lab Auckland).