In future Ambient Intelligence environments we assume intelligence embedded in the environment and its virtual, sometimes visualized agents (virtual humans). These environments support the human inhabitants or visitors in their activities and interactions by perceiving them through their sensors. In this paper we look at our research on bodily and gestural interaction with environments equipped with simple sensors, application-dependent intelligence, and an embodied virtual agent employed in the display of reactive and pro-active activity. The virtual humans we discuss play roles such as dance partner, conductor or trainer. All of them require the perception and the generation of bodily activity and other display of nonverbal communication. The role of affect and persuasion in these ambient entertainment environments is touched upon.