Vitor Fortes Rey

• Human activity recognition (HAR) using wearable sensors has benefited much less from recent advances in Deep Learning than fields such as computer vision and natural language processing. This is, to a large extent, due to the lack of large scale (as compared to computer vision) repositories of labelled training data for sensor based HAR tasks. While more and more people are wearing sensor enabled devices (e.g. fitness trackers) and often uploading to respective platforms this data cannot be labelled using crowdsourcing services such as Amazon Mechanical Turk (this is how most ImageNet images were labelled), by leveraging captions, or known topics of image collections. By contrast, labelling sensor data is much more difficult, as interpreting raw sensor data can be challenging even for experts. Recently, several approaches have been proposed to improve HAR by leveraging unlabelled data, cross-modality information and even online repositories. This dissertation investigates many of those approaches to bridge the label gap Those include producing more labelled IMU data using YouTube videos by applying pose estimation, but also techniques aimed at improving HAR representations. In this direction several scenarios were explored. For example, learning better representations for target sensors by leveraging sensors (or modalitites) that exist only at training time or learning user-independent representations using generative adversarial neural networks. I also explored adding new sensors to existing systems by leveraging unlabelled data that includes both sensors. Finally, regarding unlabelled data I also explored self-supervised learning techniques, evaluating different training procedures.