Fall Detection

Surveys of caregivers and family members of the elderly have shown that knowing when an elderly person has fallen is one of their primary concerns. This is the case not only due to the obvious, immediate medical attention that a fallen elder may require, but also because frequent falling and instability can be a sign of more serious ailments. Two popular types of commercial products exist that are worn by the person and can alert a third party if the person falls. The first is a device that is worn around the neck and has a button that can be pressed to put the person in verbal contact with a dispatcher, who can alert the appropriate authority. The major drawback to this device is, of course, that the fallen person must be conscious enough to press the button after the fall. A second device is also worn around the neck, but relies on a built-in accelerometer to automatically detect a fall. While this is preferable to the first device, in both cases the person must remember to wear the device in order for it to be effective. We are developing a fall detection and alert system that uses video cameras and advanced computer vision and machine learning techniques to avoid both of the major problems with these commercial devices. Our system involves a three-phase process. First we use video cameras to track the person as they move about their home. Then at each frame of video, we analyze the person’s motion to determine whether a fall has occurred. If a fall occurs, the system attempts to initiate a video phone call with one of several previously determined emergency contacts.

Tracking

We will now describe each of the three phases in more detail, starting with the tracking. Our system requires that any location where a fall could occur be in view of at least one video camera. These cameras are generally mounted high on a wall; a small- to medium-sized room may only require one or two cameras, whereas a larger room might need three or more. The actual number also depends on the layout and amount of clutter in the room. The preferred setup is to have every location in a given room be in view of at least two cameras, which allows us to determine the exact location of a person moving around in the room via triangulation. Tracking is done independently and in real-time on the video from each camera. We use a background subtraction procedure to detect objects in the "foreground" (i.e. moving people) in each frame. This gives us information about the size and position of a person in a single frame. By looking at several frames in succession, we can also determine the trajectory of the moving person.

Figure 1. Left: A frame of video containing a moving person. Right: The same frame with the detected moving person colored black.

Figure 2. Trajectories for many people automatically extracted from video. Color indicates the general direction of the movement.

If a person is visible simultaneously in more than one camera, we can triangulate and determine their exact 3D coordinates. While only 2D information is currently used to perform the actual fall detection, the 3D location is useful information that can be provided to someone being alerted of a fall.

Fall Detection

The fall detection procedure begins with the information we get from tracking. This can include 2D or 3D position, velocity, and the ratio of height to width (a.k.a. the "aspect ratio") of any person being tracked. We can use this data to build dynamic statistical models, known as hidden Markov models, of activities that a person might perform, such as walking, sitting down, and falling. Then as we track a person, we can evaluate how similar their current activity is to ones we have seen in the past. If the current activity is more likely to be a fall than anything else, an alert will be triggered. We can also choose to use regression models, which can be as simple as saying that we consider any instance where the width of the person is greater than the height (i.e. aspect ratio < 1) to be a fall.

Figure 3: Left: an upright person with aspect ratio > 1. Right: a fallen person with aspect ratio < 1.

Once a fall has been detected, the system initiates an alert procedure that will be described next.

Fall Alert

Once a fall has been detected, the system initiates an alert procedure. The computer closest to the fall will verbally announce that it has detected a fall and will explain that the alert can be cancelled by pressing a button within a short window of time, generally less than a minute. Figure 4 shows the window that will be displayed.

Figure 4.