ABSTRACT

1 INTRODUCTION

Emotion artificial intelligence technology, which can recognise and react to human emotions using information such as voice, facial expressions, and physiological signs, is a prevailing yet controversial invention as AI developments continue to transform sectors.¹ Although Emotion AI presents revolutionary prospects in healthcare, customer service, education, and other fields, it also poses serious questions around privacy, autonomy, and the ethical handling of sensitive emotional data. Traditional data protection paradigms are challenged by the capacity to collect and analyse such highly sensitive data, necessitating a strong and proactive reaction from companies, researchers, and regulators.²

This paper examines Privacy-by-Design (PbD) as a fundamental approach to addressing these ethical and legal issues. Stakeholders can guarantee adherence to data protection frameworks, including the General Data Protection Regulation (EU GDPR)³ of the EU and comparable international regulations, as well as gain public trust in these cutting-edge

technologies by incorporating privacy considerations into the development lifecycle of Emotion AI systems. The paper explores methods for implementing PbD principles into practice, while foreseeing future difficulties in striking a balance between human rights protection and technological progress.

By doing so, this paper aims to answer the following question: How can PbD principles be embedded into the development lifecycle of Emotion AI systems to foster robust data protection and compliance with privacy regulations? Therefore, this study offers a comprehensive understanding of how Emotion AI development can be aligned with contemporary compliance standards and ethical practices, ensuring a co-existence between innovation and privacy.

2 METHODOLOGY

This study was conducted as a conceptual legal analysis, aimed at understanding the intersection between the principle of PbD and the evolving use of Emotion AI within contemporary data protection frameworks. Although this study does not rely on empirical data collection, it employed a comprehensive review of primary and secondary legal materials. Primary sources included instruments, mainly the EU GDPR, along with the EU AI Act, ECtHR, and CJEU jurisprudence, as well as the Data Protection Authorities’ Guidance. Secondary sources comprised mainly scholarly articles, books, and other relevant publications on the subject. The analysis proceeded in three stages. The paper presents a clarification of the conceptual foundations of PbD, drawing on its seven principles as articulated by Cavoukian. It then explores the implementation of the PbD in the Emotion AI setting. Finally, the paper was then followed by an examination of the compliance strategies proposed in existing frameworks, identifying practical enforcement challenges and gaps in regulatory guidance. By synthesising across legal systems and policy discourses, this conceptual analysis aimed not only to illustrate how PbD has been operationalised in the context of Emotion AI but also to point out where enforcement mechanisms fall short in practice. 

3 UNDERSTANDING EMOTION AI AND ITS PRIVACY AND ETHICAL CONCERNS

3.1. Defining Emotion AI

Combining artificial intelligence with the intricate realm of human emotions, Emotion AI forms an intriguing link between technology and psychology. This field has increasingly evolved since Rosalind Picard's groundbreaking research in 1995,⁴ and currently includes a variety of technologies that are able to recognise, understand, and react to human emotional

states. To develop systems that can successfully interpret human emotional states, this complex field integrates machine learning, computer vision, and psychological insights.⁵

These systems analyse several signs at once, including subtle bodily movements, variations in voice patterns, minor changes in facial expressions, and quantifiable physiological reactions, such as alterations in skin conductance and heart rhythm fluctuations.⁶ Through this inclusive approach, abstract emotional experiences can be converted into concrete, analysable data points that computers can process systematically.⁷

Emotion AI's real-world uses have grown significantly in a variety of industries. Measuring consumer responses to goods and services allows for more individualised shopping experiences in retail.⁸ These technologies are used by medical professionals to track patients' emotional states throughout therapy and to spot early signs of mental health issues.⁹ Emotion AI also plays a role in security applications, analysing crowd behaviour and detecting threats.¹⁰

The pervasive integration of Emotion AI highlights its profound impact on virtually every facet of human life. In fact, virtual assistants can now identify emotional undertones in speech patterns and modify their responses accordingly.¹¹ To improve comfort and safety, modern cars can now be equipped with technologies that monitor drivers' emotional states.¹² Smartphones increasingly feature advanced emotion-detection capabilities through their cameras and microphones.¹³ Additionally, there has been a notable acceleration in the institutional adoption of Emotion AI: educational institutions use it to monitor students'

emotional health and level of engagement during class activities,¹⁴ while healthcare facilities employ it to monitor the psychological moods and recovery status of their patients.¹⁵ In the same vein, corporate environments utilise these technologies to gauge employee engagement and workplace satisfaction.¹⁶

The extensive use of Emotion AI across diverse fields highlights both its possible advantages and the necessity of carefully weighing its effects on individual autonomy and privacy.

3.2. The Rise of Privacy and Ethical Concerns with Emotion AI

Although developers emphasise the importance of anonymity and collective emotional analysis, the collection of emotional data still poses serious privacy issues. While there might exist other factors, this section examines the issues and the implications associated with the collection of emotional data.

3.2.1. Bias and Discrimination

One of the most critical issues in AI is the ethical implications of bias and discrimination in Emotion AI systems. The training data these systems rely on has a fundamental impact on their performance. When this data includes pre-existing societal biases or prejudices, the emerging AI systems unavoidably reflect—and may even reinforce—these discriminatory patterns.¹⁷

When they arise, such biases can significantly impact different social and demographic groups. Emotion AI systems, in particular, may unfairly impact groups based solely on how the AI interprets their emotional responses.¹⁸ This could then result in difficult situations where the technological tools intended to improve human connection end up being used in a discriminatory way.

Beyond racial biases, this issue includes cultural variations in emotional expression. For instance, Emotion AI systems that are usually trained on data from dominant cultural

groups may struggle to accurately identify and interpret emotional responses from other cultures. In other words, when Emotion AI is deployed across various cultural contexts, its underlying cultural bias may lead to misinterpretations and inaccurate assessments.¹⁹

3.2.2. Transparency and Explainability

To build trust and accountability in human-machine interactions, Emotion AI systems must be transparent and understandable. Indeed, trust is a necessary condition for the effective deployment and uptake of Emotion AI technologies rather than just a desirable attribute.²⁰ In the absence of a solid foundation of trust, the most complex and powerful Emotion AI systems might fail to accomplish their purposes.

Therefore, maintaining and preserving trust requires Emotion AI systems to function with a high level of transparency in a number of crucial areas. This includes being transparent about how the system functions, how it processes information, and—above all—how it manages sensitive emotional data. Emotion AI users need a thorough awareness of the complete data lifecycle, from the initial phase of emotional reactions to the storage processes and the final uses of this data.²¹

However, recent studies demonstrate that many Emotion AI systems in use today function as "black boxes," with internal procedures that are frequently hidden from both users and observers.²² This lack of transparency presents serious challenges. Consumers struggle to trust these systems' outputs or evaluate their reliability when they cannot comprehend how decisions are made. Furthermore, this opacity makes it more difficult to detect and resolve potential issues with the systems, such as algorithmic biases or systematic errors.²³

This transparency issue has implications that go beyond user confidence. It affects the wider accountability of Emotion AI systems and raises significant questions about their responsible creation and application. Clear, intelligible, and transparent systems are

becoming increasingly necessary as these technologies permeate more facets of our lives, ensuring they genuinely serve the interests of both their users and society as a whole.²⁴

3.2.3. Consent and Data Security

A fundamental element of the ethical collection and use of personal information— especially emotional information—is informed consent. In the current digital environment, where Emotion AI systems are growing more common, this basic requirement has grown more complex. The alarming issue, however, is the lack of awareness among people regarding their emotions, which are being monitored and examined, let alone the possible consequences of such monitoring.²⁵ This lack of awareness is particularly concerning in situations where Emotion AI tools are used covertly or when people feel pressured to give their consent, such as a condition of employment or to access specific services.²⁶

3.2.4. The Risk of Manipulation

There are serious ethical issues regarding Emotion AI's ability to assess and predict human emotions, particularly in relation to manipulation. The capacity of this technology to interpret physiological signs, speech patterns, facial expressions, and behavioural data opens up possibilities for impacting human behaviour.²⁷ Businesses, for instance, could employ Emotion AI in advertising to develop highly targeted advertisements that take advantage of people's emotional weaknesses.²⁸ An algorithm might, for example, detect when a person feels lonely or insecure and display advertisements intended to capitalise on these feelings. This could lead to impulsive purchasing decisions or even dependence on particular goods or services.²⁹

4 DATA PROTECTION FRAMEWORKS RELEVANT TO EMOTION AI

Existing data protection standards, along with newly emerging AI-specific laws and regulations, have created a complex web of compliance requirements, making the legal environment governing Emotion AI more comprehensive. Considering that the EU has the world's most comprehensive and stringent data protection and AI standards, serving as a benchmark, this section focuses primarily on the EU GDPR and the EU Artificial Intelligence Act—both of which serve as key instruments governing data protection and Emotion AI.

4.1. Emotion AI Through the Lenses of the GDPR

The GDPR serves as the cornerstone legislation governing personal data processing within the European Union. Its jurisdiction transcends geographical boundaries, extending to any entity, regardless of location, that monitors the behaviour of individuals in the EU.

Article 9(1) of the GDPR expressly prohibits the “processing of personal data revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, and the processing of genetic data, biometric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person’s sex life or sexual orientation”. Although emotional data is not explicitly listed in Article 9(1), there is a broad consensus that it qualifies as biometric data when it meets the personal data criteria outlined in Article 4(1) of the GDPR.³⁰

As a result, companies using Emotion AI systems must carefully adhere to the GDPR's core requirements of purpose limitation, data minimisation, and storage limitation. This means restricting data collection to the most critical components, providing a clear explanation for the acquisition of emotional data, and retaining such data only for as long as necessary. The regulation also requires enterprises to disclose their use of Emotion AI in a transparent and understandable manner, while ensuring that data subjects have accessible means to manage their personal data.

To ensure regulatory compliance, organisations must also put in place thorough measures that combine organisational and technical safeguards. These include modern encryption techniques, strictly regulated access controls, and thorough documentation of all data processing operations, which should be part of these safeguards. Additionally, companies must conduct comprehensive Data Protection Impact Assessments to identify and mitigate potential risks before implementing large-scale Emotion AI systems that handle biometric data. By guaranteeing that privacy considerations are incorporated into the system's design from the beginning, these assessments act as essential preventive measures. In addition to demonstrating a commitment to compliance, adopting stringent security measures helps protect private emotional information from unethical or illegal use.

4.2. Emotion AI Systems Under the EU AI Act

In 2024, the EU put forward an AI regulation,³¹ marking the initial steps towards regulating AI and transforming the legal assessment of AI systems. This regulation uses a risk-based approach, categorising AI systems into four levels—unacceptable, high, limited, and minimal risk—based on the risk they pose to people and society.³² Depending on its classification, every category of AI system is subject to specific rules and restrictions.

This thoughtful approach reflects a sophisticated understanding of the diverse implications that different AI technologies may have. The level of regulatory monitoring is directly correlated with the possible harm associated with each AI application under the hierarchical oversight framework established by the AI Act. The regulation thus establishes distinct lines between permissible and impermissible AI systems.

According to Article 5 (1)(f) of the AI Act, “the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons” is strictly prohibited. This rationale for this prohibition is further detailed in Recital 44, which lists several serious concerns regarding emotion identification systems. These concerns include its intrusive nature, lack of specificity and generalisability, power imbalance between those using it and those affected, its limited reliability, and the potential for discriminatory outcomes.³³

Furthermore, Annexe III of the AI Act classifies emotion recognition systems as high-risk AI systems, subjecting them to stringent regulatory requirements. As a result, those who implement such systems must adhere to Article 50(3) of the AI Act, which stipulates that individuals who are exposed to these technologies must be made aware of how they work and that any personal information they handle must be processed in full compliance with the requirements of the GDPR.³⁴

4.3. European Case Law and Regulatory Guidance on Emotion AI

The use of biometric and emotion-inference technologies has been severely limited by European jurisprudence and supervisory guidelines, which stress the principles of necessity, proportionality and periodic review. The Court of Justice of the European Union (CJEU) has frequently ruled that national schemes authorising the systematic or blanket collection of biometric and genetic data violate the EU data-protection frameworks unless strictly limited and justified.

This principle was clearly articulated in the V.S. case, where the CJEU held that compulsory, systematic collection of biometric data for police records must meet a “strictly necessary” requirement in pursuit of specific, legitimate objectives.³⁵ Another landmark decision, Digital Rights Ireland Ltd v Minister, declared the EU Data Retention Directive invalid on the grounds that mass and indiscriminate retention of communications metadata without adequate protection is a severe interference with rights to data privacy.³⁶ According to this case, any automated or intrusive processing of personal data, especially sensitive or behavioural data, must adhere to stringent requirements for necessity, proportionality, transparency, and legal justification. Emotion AI, insofar as it infers inner emotional states, implicates several of those concerns.

In parallel, the ECtHR also considered intrusive biometric surveillance as a matter falling under Article 8 (private life). In Glukhin v. Russia, the ECtHR ruled that the use of facialrecognition technology by authorities was seen as “highly intrusive”, highlighting that the use poses serious proportionality and clarity issues in domestic law.³⁷ Earlier, Gaughran v. the United Kingdom highlighted the dangers of stigmatisation and life-course harm and criticised schemes that permitted the indefinite retention of biometric data (photographs and fingerprints) as being incompatible with Convention provisions.³⁸

Supervisory authorities have converted these principles into operational guidance. The European Data Protection Board's guidelines on facial recognition and law enforcement processing require stringent purpose limitation, data minimisation, and human oversight safeguards before any biometric processing. They also call for customised necessity

assessments rather than general authorisations.³⁹ Similarly, the European Data Protection Supervisor identified face emotion recognition as especially risky, pointing out the strong potential for discriminatory consequences and scientific ambiguity over accuracy.⁴⁰

At the National DPA level, the UK Information Commissioner’s Office (ICO) has frequently cautioned about the immaturity, bias, and intrusiveness of emotionrecognition technology. These concerns have been incorporated into the ICO's AI strategy and biometrics guidance.⁴¹ In a similar vein, France's CNIL⁴² and Spain's AEPD⁴³ published white papers and technical dispatches addressing automatic processing (speech, face, and neurodata) and underlining the concerns where inferred mental states meet with "special categories" of data. Regulators emphasise DPIAs, necessity, transparency, and purpose limitation. They also frequently caution against covert or workplace applications that lack robust protections.

5 COMPLIANCE STRATEGIES FOR PRIVACY-BY-DESIGN IN EMOTION AI

PbD emphasises the importance of incorporating privacy considerations throughout the development lifecycle to reduce risks and boost user confidence.⁴⁴ The present section offers a comprehensive insight into the underlying core principles of PbD and explores their implementation approaches in Emotion AI.

5.1. Privacy-by-Design Core Principles

PbD is a forward-thinking framework for safeguarding personal data, embedding privacy considerations into the design specifications of technologies, systems, and processes at the outset of personal data processing, rather than as an afterthought. This concept is further clarified by Cavoukian, who outlines seven fundamental principles that serve as a valuable reference for organisations seeking to comply with data protection regulations.⁴⁵ These are:

• Proactive, not Reactive: This approach anticipates and prevents privacy issues before they occur, rather than addressing them after violations have happened. It helps organisations maintain trust and avoid costly remediation efforts.
• Privacy as the Default Setting: Systems should safeguard user privacy automatically, without requiring user intervention. Every business procedure or IT system should automatically, by default, protect personal data.
• Privacy Embedded into Design: The system's architecture and design incorporate privacy protection from the start, rather than adding it after. This guarantees that privacy becomes a core feature.
• Full Functionality – Positive-Sum, not Zero-Sum: PbD seeks to accommodate all legitimate interests and objectives in a mutually beneficial "win-win" manner, rather than relying on an outdated zero-sum approach that requires needless trade-offs.
• End-to-end Security: Stringent procedures must be applied throughout the entire data lifecycle. This guarantees that all the data is securely collected, used, retained, and deleted at the end of the process.
• Visibility and Transparency: All components and operations should remain visible and transparent to both users and providers, fostering trust and accountability.
• Respect for User Privacy: Individuals’ interests are crucial, requiring robust privacy defaults, and user-friendly options, as well as proper notice.

In recent years, the concept of PbD has gained widespread acceptance and has been formally incorporated into major data protection frameworks such as the GDPR. The principle is emphasised in Recital 78 of the GDPR, which states that:

“When developing, designing, selecting, and using applications, services, and products that involve personal data processing, developers and manufacturers should consider the right to data protection, adhering to the state of the art, and ensuring that controllers and processors fulfil their obligations.”⁴⁶

Article 25(1) of the GDPR provides further guidance on implementing PbD. It requires controllers to take organisational and technical steps to implement data protection principles, such as data minimisation, and to incorporate the necessary safeguards to meet regulatory requirements and uphold individuals' rights. These measures should take into account various factors, including technological advancements, associated costs, and the nature, scope, and potential risks involved in data processing activities.

5.2. Implementing Privacy-by-Design in Emotion AI

PbD implementation in Emotion AI requires a thorough policy that covers every component of the technology, from data collection and processing to storage and utilisation.

5.2.1. Data Minimisation and Purpose Limitation

Fundamental PbD principles of purpose limitation and data minimisation necessitate that companies collect just the data necessary for a specific, legitimate purpose and use it only for that purpose. Therefore, applying PbD principles to emotional data poses substantial real-world challenges from an enforcement standpoint. Organisations must operationalise data minimisation and purpose limitation across intricate technical systems and business processes while navigating data protection regulations.

In practice, organisations must keep thorough records demonstrating their assessment of the importance of each emotional data point gathered. For instance, a mental health app would need to explain why specific emotional indicators are necessary for delivering therapeutic services, while omitting information that might be relevant but not essential for the service. This documentation becomes essential when Data Protection Authorities conduct regulatory audits and investigations.

The challenge of purpose limitation is reflected in data access controls and system architecture. Organisations must implement technical safeguards to prevent the use of emotional data for secondary purposes, such as product development or marketing, unless this is stated explicitly in the original consent. Addressing this challenge often requires sophisticated data governance frameworks and regular compliance audits. For instance, a workplace wellness program collecting emotional data must ensure strict separation between health-related processing and performance management systems to maintain compliance and protect individuals’ privacy.

5.2.2. Transparency and Consent

Implementing informed consent for emotional data within EU frameworks poses several practical challenges. Organisations should set up clear, accessible consent mechanisms that effectively clarify the intricacies of emotional AI systems, while ensuring users are not overwhelmed by excessive consent requests. Information on an organisation's data collection

and processing procedures, including the type of emotional data it collects, its intended uses, and the security measures in place, should be easily comprehensible and available.⁴⁷

The technical implementation of data subject rights raises significant challenges, particularly for emotional data that may be integrated into multiple systems or used for training AI models. For organisations to track emotional data throughout their infrastructure, fulfil access requests, and implement the right to be forgotten, they require strong data mapping and management systems. This becomes especially complex when emotional data is derived from multiple sources or combined with other personal data.

As emotional AI technologies evolve, organisations need to maintain ongoing transparency. For this, they are required to regularly update their documentation and communication channels. This includes incorporating practical processes for consent withdrawal at any time and the ability to view and request changes to their data.⁴⁸

5.2.3. Security and Data Protection

Adopting thorough data security procedures is another essential component for protecting sensitive information. These precautions should ensure that this private information is protected from exposure, misuse, and unauthorised access.⁴⁹ Indeed, ways to implement such precautions include using advanced encryption methods, establishing strict access controls, and conducting systematic audits of data protection frameworks to identify weaknesses.⁵⁰ By prioritising these initiatives, organisations may improve accountability, lower risks, and maintain ethical principles while managing Emotion AI systems as well as protecting people's rights and privacy.

5.2.4. Accountability and Algorithmic Transparency

To ensure that Emotion AI systems function in a way that avoids discrimination, maintain fairness and stop bias, algorithmic accountability and transparency are crucial.⁵¹ Thus, to make these algorithms as transparent as possible, organisations should provide a thorough, understandable explanation of the procedures and methods that underpin their operation.

Transparency challenges derive from the complexity of AI systems, especially with Emotion AI's subjective nature. Organisations must address this by providing clear documentation and illustrative insights about how algorithms avoid bias, handle emotional input, and justify outputs, without overloading stakeholders with technical jargon.

Promoting accountability in Emotion AI systems requires not only transparency but also clarity on decision-making procedures. Companies should specify exactly how particular decisions are made using the algorithm, including the inputs and the justification for the results. By doing this, companies promote confidence among stakeholders such as regulators, users, and the public, in addition to enabling improved oversight.⁵² Therefore, implementing such procedures strengthens the integrity and social responsibility of the Emotion AI systems.

5.2.5. User-Centric Design

User-centric design prioritises individuals’ needs, preferences, and rights in the development and deployment of Emotion AI systems.⁵³ This approach necessitates continuous user engagement throughout the design processes to ensure that technologies align with human-centric values and respect user experiences. Practical enforcement involves integrating user feedback loops into the system development process. This could be done by facilitating participatory workshops or usability testing to enhance system responsiveness and meet user expectations. Designers must balance innovation with compliance, ensuring that emotional data is handled ethically, securely, and lawfully.

Furthermore, user-centric design empowers individuals with greater control over their emotional data. This includes enabling users to manage their data in ways that have personal significance, providing access to their emotional data, and offering simple methods to correct errors or delete the data if they choose. By prioritising consumers' autonomy and privacy first, this approach promotes both fairness and increases users' trust in the technology.

5.2.6. Impact Assessments

For businesses using Emotion AI systems, Data Protection Impact Assessments (DPIAs) are essential systematic methods for assessing and mitigating potential privacy issues prior to their deployment.⁵⁴ These assessments include a thorough examination of the

methods used to collect, process, store, and use personal and emotional data while taking into account the particular sensitivities related to Emotion AI technology. A comprehensive DPIA typically considers various factors, including the necessity and appropriateness of data processing, potential threats to people's rights and freedoms, and the effectiveness of existing protections.

DPIAs can be used by organisations to identify certain vulnerabilities, including dangers of emotional manipulation, potential bias or discrimination in emotional analysis, and illegal access to emotional data. Based on these results, organisations can create focused mitigation strategies, like enhanced data encryption, stringent access controls, explicit data retention guidelines, and transparent and honest communication with users regarding the processing of their emotional data. DPIAs are a dynamic tool for upholding ethical norms and privacy compliance, and regular revisions are crucial as Emotion AI technology develops and new privacy risks appear.

6 CONCLUSIONS

Incorporating PbD principles into the development of Emotion AI is not only required by law but also essential to the long-term, ethical development of this technology. As this paper demonstrates, the sensitive nature of emotional data necessitates a thorough, proactive approach to privacy protection that goes beyond mere compliance with current regulations. Implementing PbD in Emotion AI systems requires careful consideration of numerous factors, including data minimisation, transparency, consent, and robust security mechanisms. Therefore, organisations must strike a balance between the innovation of Emotion AI and the fundamental right to privacy to prevent technological innovation from compromising individual autonomy and trust.

The development and implementation of Emotion AI systems will likely be influenced by future regulatory changes, especially with the implementation of the EU AI Act and the ongoing impact of the GDPR. Businesses will be better positioned to adapt to these shifting demands while upholding public trust if they adopt PbD principles early in their development process.

Finally, organisations can develop solutions that are not just secure and compliant but also ethical and with long-lasting effects by integrating privacy concerns into the foundation of emotion AI systems. To ensure that the advancement of Emotion AI aligns with social values and expectations while encouraging innovation in this rapidly evolving sector, it will be crucial to conduct ongoing research, collaborate with stakeholders, and regularly evaluate privacy standards.

FOOTNOTES

REFERENCES

AUTHORS INFORMATION

ACKNOWLEDGEMENTS

RIGHTS AND PERMISSIONS

EDITORS

ABOUT THIS ARTICLE

DETAILS FOR PUBLICATION

AI DISCLOSURE STATEMENT

АНОТАЦІЯ