A diary is generally considered to be a book in which one keeps a regular record of events and experiences that have some personal significance. As such, it provides a useful means to privately express inner thoughts or to reflect on daily experiences, helping in either case to put them in perspective. Taking conventional diary keeping as our starting point, we have designed and built a digital diary, named Affective Diary, with which users can scribble their notes, but that also allows for bodily memorabilia to be recorded from body sensors and mobile media to be collected from users’ mobile phones. A premise that underlies the presented work is one that views our bodily experiences as integral to how we come to interpret and thus make sense of the world. We present our investigations into this design space in three related lines of inquiry: (1) a theoretical grounding for affect and bodily experiences; (2) a user-centred design process, arriving at the Affective Diary system; and (3) an exploratory end-user study of the Affective Diary with 4 users during several weeks of use. Through these three inquiries, our overall aim has been to explore the potential of a system that interleaves the physical and cultural features of our embodied experiences and to further examine what mediaspecific qualities such a design might incorporate. Concerning the media-specific qualities, the key appears to be to find a suitable balance where a system does not dictate what should be interpreted and, at the same time, lends itself to enabling the user to participate in the interpretive act. In the exploratory end-user study users, for the most part, were able to identify with the body memorabilia and together with the mobile data, it enabled them to remember and reflect on their past. Two of our subjects went even further and found patterns in their own bodily reactions that caused them to learn something about themselves and even attempt to alter their own behaviours.