Marwa

This project demonstrates an approach to connecting technology with creative practice with further exploration among the topics covered – facial recognition, pix2pix, stylegan, posenet, style-transfer, charrnn, sound classification, neural networks.

Presence PoseML takes a closer look at the ways in which PoseNet can be used to detect the posture of a user in real time, and provide alerts to better increase productivity, presence, and attentiveness in professional settings. Non-verbal communication influences the way other people may judge us and how one presents themselves, some will argue, determines their degree of engagement. Presence PoseML allows users to appear more awake and promotes wellbeing during meetings through subtle reminders.

CONCEPT

The conceptual inspiration behind this project was developed through creative and theoretical research in areas that have previously been successfully implemented in machine learning practices. AI and ML is widely used for bringing several enhancements to meeting room experiences, such as active participant framing, virtual backgrounds, filtering out sounds from pets, cars, paper shuffling, as well as facial recognition enabling meeting systems to recognise users as they enter a room for faster meeting starts.

A primary differentiator among vendors is the addition of AI-powered capabilities to their endpoints and software and further advancements show that this increases productivity gains by making meetings more intuitive, optimising use of meeting space, improving in-meeting experience, and ensuring the follow-on action items are accurately captured and distributed to meeting participants.

The scope for greater developments in adapting ML and AI for ease and accessibility in professional software is constantly being reworked, particularly as our individual dependence on online tools and technologies has been redefined over the last years.

We can clearly envision the power of pose estimation by considering its application in automatically tracking human movement. From virtual sports coaches and AI-powered personal trainers to tracking movements on factory floors to ensure worker safety, pose estimation has the potential to create a new wave of automated tools designed to measure the precision of human movement.

During my research, I was curious to find how wellbeing could be showcased within day-to-day workspaces. I developed my idea on using machine learning to identify your position while working or on conference calls, feeding back alerts on how to better position yourself for optimal posture health. By leveraging the advantage of AI-sourced health to facilitate remote healthcare services, in-home rehabilitation offers diverse healthcare services to individual households. This covers a range of activities, from assessing the patient’s functional ability in his/her environment to specific supervised treatment or exercises.

TECHNICAL IMPLEMENTATION

Presence PoseML uses Pose.Net, developed by Google’s TensorFlow and ml5.js, a JavaScript library for Machine Learning on the browser. The training for this project was done using Teachable Machine by gathering four pose classes: Neutral, Hunched, Leaning, and Offscreen. I chose this method of training as it allows you to classify multiple poses and simply creates a model for implementation.

I also owe a large part of my technical application to Daniel Shiffman’s Pose Estimation machine learning playlist in aiding my coding process and successfully adding my trained model to the training in p5.js.

REFLECTION & FUTURE DEVELOPMENT

To conclude, my project went successfully, and it is a working model of my initial idea. I faced some errors when adjusting the rate in which audio sounds were triggered on detection of a certain pose, however I managed to eventually rectify this issue. I also set out to collect data for regression where this technology could return information on the percentage of time during a video conference users held certain positions (N, H, L, O) which would then encourage you to take a neutral position with closer attentiveness and be mindful about your physical wellbeing.

I envision that a technology like Presence PoseML could be used as an add-on feature for Zoom, Teams, Google Meets applications, or built-in to a software of its own to promote professional wellness. For future development, I believe there is room for many more additional features and capabilities for this project. To list a few, Presence PoseML could include some animations for its UI on how to position yourself in the best posture, and in the optimal upright position that puts less pressure on your spine. The technology could also include more specific prompts according to the duration you have been sitting, for example, to ensure that the user stretches or readjusts his/her position every 20 minutes.

To add, Presence PoseML could be used in future development for e-learning and improving engagement in virtual learning settings. It may be difficult for tutors to monitor whether students are presenting themselves attentively and focused on the class materials, whereas Presence PoseML will be able to provide confidence levels to students donning neutral, attentive positions.