Objective 1
Human-In-The-Loop (HITL) Distributed Dynamic learning
description
Objective 1 concentrates on TRUMAN’s AI and ML techniques and is the starting point as it defines the characteristics and specifications of these techniques.
- “human- in-the-loop (HITL)”, because the human plays an important role in the improvement of AI systems;
- distributed because data are usually generated in a distributed manner as in the case of federated learning scenarios, as a function of where the service provider and consumer are located;
- “dynamic” because data continuously evolve in time.
It is crucial to monitor the process by which previous data are managed and exploited for analysing the data generated during the current time slot via methods like time-aware learning. Expand to read more.
By HITL, we mean techniques where there is a continual bidirectional human-machine interaction and where the human can spontaneously provide information. These would include different scenarios such as tagging a photo, reporting information about the human’s current activities, or the human and the machine engaging in a discussion because the machine thinks that the human has provided wrong information, e.g. mislabeling.
Contrarily, such an exchange could also occur when the human needs to fix the machine’s reasoning that leads to a wrong conclusion. The HITL approach can be adopted in all four main ML phases (i.e., data collection, data pre-processing, model training, and model execution).
The enforcement of this HITL approach motivates our choice of using Knowledge Graphs (KGs) as a more abstract and knowledge-level representation of tabular or Internet of Things (IoT) data. Indeed, KGs enforce the move from variables (or at most tabular data) to entities, from features to data properties describing entities and to object properties describing relations and actions among entities.
In turn, LLMs, optionally fed by KGs, provide an even further level of abstraction. This becomes key where the end-user is not an expert such that entities, their properties and relations are explained to the user in natural language, also making explicit a huge amount of common- sense knowledge that they embed.
AMBITIONS
Ambition 1.1. We will study how to adopt the HITL approach at different levels of abstraction, from IoT sensors and features to data and object properties as embedded in KGs, where KGs will facilitate the user in the reuse of past data as well as data generated by third parties up to the interaction.
The proper level of data abstraction will be tuned to the specific task and application scenario with the possibility of using two representation models (e.g., features and KGs), mutually reinforcing themselves in a meaningful interaction with the user. We will develop Personal KGs (PKGs) encoding the user’s subjective perspective on the world, and we will integrate this into KGs to encompass a multi-user perspective.
This will require the development of Temporal KGs (TKGs) and Spatio-Temporal KGs whenever there is a need to integrate IoT data. It is worth noting that Knowledge Graphs (KG) operate on two levels: the schema KG (SKG) and the data KG (DKG). The reliability of KG-based reasoning heavily depends on the quality of the SKG. If the SKG is flawed, any reasoning or explanations derived from it—whether concerning entities, time, or relationships—will be inaccurate, regardless of the data’s correctness. Current literature reveals a lack of theoretical and principled approaches in this area. To address this, our project will leverage techniques from information science. This approach will not only support a multi-dimensional representation of knowledge (known as facets) but also provide the flexibility needed to incorporate and update new facts and knowledge within the KG.
Ambition 1.2. We will analyse different steps, from local to global, by which data are first generated and then used to train and use ML models.
The ultimate goal is to integrate robustness solutions and evaluate their impact on the performance and accuracy of the AI systems and hence the trade-off between their trustworthiness and accuracy. We will concentrate on two main techniques, that is, how to distribute in time and space the data generation and preparation, and federated learning for what concerns the model training and usage.
Ambition 1.3. We will develop a robust and adaptable federated continual learning framework capable of effectively addressing spatial-temporal catastrophic forgetting in resource-constrained environments.
The framework will be designed to enhance communication efficiency, handle model heterogeneity, and explore alternative sharing modes. Through rigorous validation in diverse real-world scenarios, including healthcare, IoT, and finance, we will demonstrate the practical effectiveness of our approach. One fundamental focus will be on applying federated learning across individuals, for instance, with the goal of doing cross-individual activity recognition. A second fundamental focus will be on federated learning and temporal adaptation in the design of this framework.
Objective 2
Trustworthiness through robustness
description
The performance of AI technologies relies on access to large datasets of good quality and on the training of an accurate model. Hence, AI systems’ dependence on large data makes them vulnerable to adversarial attacks that can manipulate inputs or model parameters in order to tamper with the training process. Such attacks may be hard to detect because the manipulated model is designed to exhibit adversarial behaviour on inputs which usually are only known to the attacker. The literature features various proposals to mitigate poisoning attacks. Nevertheless, these solutions usually work with neural networks and sometimes require considerable and computationally heavy modification to existing algorithms.
AMBITIONS
Ambition 2.1. We will first investigate and design new adversarial attacks against the TRUMAN AI systems where the learning is distributed among multiple parties, is continual, and relies on the use of graph structures.
We will further develop mitigation solutions to counter these newly proposed attacks and make sure that they protect against state-of-the-art attacks as well.
Ambition 2.2. The goal is to study and identify new privacy attacks or attack implementations and further design/develop customised defense strategies, both based on differential privacy mechanisms as well as other empirical defenses, while continuously addressing the trade-off between robustness-efficiency and model quality.
Moreover, TRUMAN aims to clean the training data used for LLMs to ensure that personal and sensitive information is removed or replaced with synthetic data that maintains the same format. Where successfully applied this will become novel and highly relevant enabler for KI solutions by solving an essential data protection concern while retaining the data’s value for KI training.
Ambition 2.3. TRUMAN will first review both the attacks and the existing mitigation techniques and study their suitability and performance in the context of the project’s AI systems.
We will also explore mitigation techniques based on differentially private mechanisms. While some studies have shown that the adoption of differentially private mechanisms results in a negative correlation between the privacy budget and fairness violation, recent works show that when applied carefully, this trade-off can be addressed or at least bounded. We will take advantage of these studies to design and develop customised solutions for the newly developed AI models.
Objective 3
Trustworthiness through explainability and usability
description
While the system-focused approaches mentioned above are essential for mitigating bias and enhancing robustness, they are insufficient for human-centered AI systems. Rather, they must be accompanied by techniques that enhance the explainability (of system behaviours) from the user’s point of view. Explainability of system security, including privacy measures for achieving robustness, is essential for a multitude of reasons, from enabling the user to verify the system’s behaviour and the stated security measures to learning from the system and as a means for achieving compliance with the EU AI Act 3 regarding transparency and human oversight requirements for high-risk AI systems.
AMBITIONS
Ambition 3. We will conduct user studies for investigating suitable combinations for functional and structural explanations for the provided security/privacy/robustness protection that are trustworthy, understandable and meaningful and relevant for different types of users (e.g., end-users, system admins, decision makers).
Furthermore, this task will develop user interface prototypes providing such usable explanations, which will be evaluated with user tests at KU and OUC (focus groups and user tests) in at least two iteration cycles involving test users with diverse backgrounds regarding their gender, age, education, and culture.
Objective 4
Integration and demonstration of TRUMAN technologies
description
The ultimate goal of TRUMAN is to develop trustworthy solutions for AI that are ready for real-world integration. The designed solutions will, therefore, be integrated and deployed into real-world use cases. In order to explore a large number of challenges raised by different AI methods involving different data structures, algorithms and actors, the project will consider four use cases, each of which will focus on a different AI method (continual learning, KGs, or LLMs) and a particular phase of the AI life cycle (data collection, data pre-processing, model training, model execution).
AMBITIONS
Ambition 4. TRUMAN will integrate and demonstrate the developed solutions with four use cases:
- the collection of data of various types for market and scientific research on household lifestyle,
- the development of phishing website detectors that will make use of deep neural networks that are collectively trained (meant to assemble ground truth), deployed over several machines collecting privacy-sensitive information, and executing the AI model locally; and validated by humans to improve both accuracy and explainability,
- the training of a model detecting fraudulent transaction carried out by bank employees where the technology will make use of graph-structured data; and, finally,
- an LLM-based chatbot that will engage interactions with patients suffering from Parkinson’s disease and based on these interactions, offer advanced functionalities disease management coaching.
Objective 5
Generic methodology for trustworthy AI
description
Within the existing literature, the initial steps towards trustworthy AI systems have been rather bottom-up, driven by the specific type of data and/or the specific AI/ML technique and/or the specific domain or application scenario. For example, various solutions already exist to address one difficulty, e.g., the lack of fairness or transparency, given a specific AI technique. In parallel to addressing the problems of security, privacy, and fairness of a given AI technology, individually, we will also follow a top-bottom and holistic approach and develop a generic methodology and a road map to build trustworthy AI technologies. Indeed, the ultimate goal is to take a more principled approach towards building trustworthy AI systems.
AMBITIONS
Ambition 5. We will analyse the problem along three dimensions, namely with respect to:
- the different AI phases (data collection, preparation, model training, and model execution),
- the actual AI techniques which can exhibit different robustness issues at different phases; and,
- the underlying data which can be used at different levels of abstraction, depending on the specific application (for example KGs are very useful in corporate applications while LLMs are successful for applications involving human-machine interactions).
Objective 6
Promote broad awareness / adoption of TRUMAN findings and solutions
description
Throughout TRUMAN’s duration, its research topics and results addressing the innovation challenge of the call will continuously be publicised and promoted to raise awareness about the requirements of Trustworthy AI covering all the aspects of robustness (security, privacy, and fairness).
AMBITIONS