The Roche lobe is the region of space around a star in a binary system within which orbiting material is gravitationally bound to that star. If the star expands past its Roche lobe, then the material outside of the lobe will fall into the other star. It is approximately tear-drop shaped, with the apex of the tear-drop pointing towards the other star (and the apex is at the Lagrange L1 point of the system).
As such the Roche lobe is one of two volumes of space in the system. These volumes are bounded by a particular surface of equal potential energy. The potential energy is calculated in a frame of reference that corotates with the binary system. Because this frame of reference is a non-inertial frame, the gravitational potentials due to the masses of each of the two stellar nuclei (which vary inversely with distance from the center of each star) must be supplemented by a pseudo-potential corresponding to centrifugal force. This pseudo-potential is proportional to the square of the perpendicular distance from the axis of rotation of the system.
Close to each stellar center the equipotential surfaces are approximately spherical and concentric with the nearer star. Far from the stellar system, the equipotentials are approximately ellipsoidal and elongated parallel to the axis joining the stellar centers. A critical equipotential intersects itself at the center of mass of the system. It is this equipotential which defines the Roche lobes.
Where matter moves relative to the corotating frame it will be acted upon by a coriolis force. This is not derivable from the Roche lobe model as the coriolis force is a non-conservative force (i.e. not representable by a scalar potential).
See also Roche limit