A novel algorithm for detecting and tracking hand gestures in real time is proposed which uses an on-line adaptive learning approach to fit the hand skin color distribution of each individual user in various environments. The on-line adaptive skin-color l