Data are emotions. Emotions are data.
Two seemingly distant concepts—data and emotions—have surprisingly much in common. There is a subtle link between them, namely design, or more precisely the sensitivity and imagination of designers, who incorporate into their toolkit the knowledge and skills of Data Science and Artificial Intelligence specialists for the benefit of people and their needs.
We all use data on a daily basis. This happens when we use a mobile phone, check directions to a particular destination or look for a recipe for tomato soup. We often do not even realize that in such situations we use and produce data, as well as interact with solutions based on artificial intelligence. In practice, we leave many digital traces.
We all experience emotions every day. What is characteristic of emotions is that they come and go and differ in intensity. They may be positive or negative. Sometimes it is the joy of meeting friends, sometimes it is anger when a car breaks down on the way to an important meeting or shame after telling a little lie. There are emotions that we recognize and are aware of, and those which are experienced unconsciously.
For years, designers have been using methods that allow them to take into account the emotional needs of users, treating them as equally important as functional needs. The profession of a designer emerged with the industrial revolution and has been changing ever since with the technological progress. Entering the digital world was a natural step for designers. Today, the development of data analysis opens another fascinating chapter for designers.
"People crave data more often than they drink, more often than they eat, more often than they make love to each other. They check, look for or obtain data more often. Data have become a necessity. After breathing, and even before drinking water, data consumption is the most frequent activity" says Prof. Ryszard Tadeusiewicz (AGH University of Science and Technology, Krakow).
Since this is the case, designers cannot remain indifferent. Data enable us to look at the reality from a different perspective. They enable us to pose research questions, and data processing brings cognitive relief: "(…) we are not even aware how much choosing, selecting, searching, how much effort and time we are spared by this hidden data analysis."—says Monika Książek, Data Science Expert at Play. There is a great potential in data. Data are becoming the subject of a heated debate among designers. One of the most renowned curators of design exhibitions, Paola Antonella from the Museum of Modern Art (MoMA) in New York, highlighted the need to bring data closer to human beings, to humanize new areas of technology. She thinks that designers will play an important role here, as due to the nature of their work, they are aware of the importance of both the context and human beings, and see that technology is developing faster than common knowledge about it. In 2019 the Barbican Gallery in London hosted the exhibition "AI: More than Human" which challenged common perceptions of artificial intelligence and gave real-life examples of how new technologies affect everyday life.
Designers have a great opportunity to combine the world of data and the world of emotions, so that each of us can reap the best benefits from both of them.
About data and emotions
Data are becoming an inherent element of our culture. We are the source of data, we consume data, we benefit from them, and sometimes we are the victims of data. How to live in a world based on data?
Collections of texts and numbers are commodities equivalent to crude oil or gold. In the digital era, the amount of data constantly multiplies, and as it is processed, a new reality arises. Machines manage the analysis of large differentiated pools of data much better than humans, using methods and techniques of artificial intelligence.
This situation raises extreme emotions—from fear to admiration. Some feel the anxiety related to generating, consuming, and sharing their data. Others are becoming techno-optimists. Designers are already drawing our attention to the emotions which accompany data. We present a spectrum of attitudes that can be adopted towards data, from avoiding them completely to working with them. Moreover, we would like to open a discussion on the stereotypes related to the perception of data. We also encourage designers to go beyond standard practices of working with data, such as visualization, and to look at data as a new design material for their projects.
Data are emotions
Data are important elements of our reality. On the surface, they seem to be dull and impassive. In practice, they are able to evoke extreme emotions and create a strong aesthetic and intellectual experience.
Analysts have unique intuition when dealing with data. According to Michał Witkowski, Lead Game Economy Designer at Rage Quit Games, "(…) analysts provide people with both data and emotions." How should data be presented to us, touched or felt to trigger these emotions? Visualization is one of possible approaches to finding meaning and understanding data—it is not the final solution but a means to an end. Figures take on an intermediate form, which enables calculations, but communication requires transformation of data into visualizations or other forms of narrative.
Data are often perceived as objective entities, as opposed to emotions experienced by people. By making decisions based on data we can change the world as well as optimize our lives, improve health and well-being, and even build self-awareness.
"Emotions are born at the interface between the recipient and data. Emotions are never embedded in the data themselves." Prof. Ryszard Tadeusiewicz, AGH University of Science and Technology in Krakow
Number
project of experiencing data
2020
The Polish dictionary defines data as facts that can be relied upon in argumentation; information; chances of success; (sic!) or any information processed by a computer". In the era of mass computerization, the last definition seems to be the most appropriate. The first association with data are often tables or charts, i.e. information expressed numerically.
Such an interpretation allows us to treat numbers with reserve, as machine-made products, and dissociate their meaning from real life. Even the most tragic events in news services are reduced to simple numbers. The numbers are objective, raw, they do not tell a story. They need to be contextualized to be comprehensible.
In this project, article headlines ware collected, all of which contain number 20. It demonstrates that a single number, presented in different contexts, can evoke diverse emotions. Of course, each viewer will not experience emotions of the same kind or intensity. The emotions evoked by these short texts, as well as the intensity of the experience, depend solely on the viewer's personal experience and sensitivity. The context outlined by the words will differ slightly for every viewer. Number 20 will enjoy a range of interpretations.
Why am I doing this
interactive data visualization
2015
Reflecting on your own behavior and substance abuse may help you to handle the problem better. The aim of the Why am I doing this project is to address the problem and build an awareness of one's own needs, as well as to provide data on the phenomena related to addiction.
Addictions and related data arouse strong emotions. The visualization takes the form of a circle with black and red circles of different diameters inside. There are connections between them. The perimeter of the main circle lists addictive substances and behaviors, e.g. cigarettes, drugs, alcohol, but also sugar, pornography, shopping and food. Moving clockwise along this perimeter we find out which of these substances and behaviors are the most common. Black inner circles represent unwanted experiences that we want to eliminate through the use of addictive substances or behaviors, e.g. stress, boredom, loneliness. These are the causes of addictions. Red inner circles represent positive experiences we hope to achieve thanks to addictive substances or behaviors, e.g. pleasure, peace, satisfaction. This is what addicts want to achieve. The diameters of the circles correspond to the frequency of individual experiences. We can see networks of connections between the words on the perimeter and the emotions inside the main circle. The width of the connections between a substance or behavior and a particular experience represents the relative frequency of the experience in relation to the substance.
Workplace Safety Wearables
beacons
2020
In the face of the pandemic, many solutions have been developed to stop the spread of the coronavirus. This is particularly useful for jobs that have to be done despite the threat. If one of the employees contracts the disease, the app sends information to those who have been in contact with them.
Workplace Safety Wearable is based on micro-location technology and is dedicated for companies and workplaces. Specially prepared wearables precisely measure the position of a person or object in the workplace. The interactions and the total time of potential exposure to the virus are anonymously recorded in the database. This creates a network of points of contact as potential areas for infection. If one of the employees becomes ill, the relevant data—the patient's interactions, including the time and names of the exposed persons—are decoded from the database. The workers at risk are alerted to an epidemiological hazard through their device. As a result, the further spread of the coronavirus within an organization can be effectively blocked. The premises are then quarantined, thus eliminating the possibility of the virus spreading outside. The Workplace Safety platform is installed on the client server. The manager who operates it has access to general statistics, such as the number of employees involved, total exposure time between them (anonymously), the rate of compliance of employees with health care recommendations, the number of currently infected and immunized employees. Access to names is only granted after a real threat has occurred. employees may opt out from the system at any time.
Accessories for the Paranoid
the series of four parasitic objects
2020
Assuming that data are the oil of the 21st century, each of us owns a small ground treasure—a resource that is being discreetly mined by the biggest companies in the world. As users of modern services and products, we have long become habituated to trade-offs in which "free" services are offered in exchange for our personal data. The IoT has introduced a new kind of objects into our lives, whose functioning greatly depends on collecting such information: products that are able to surveil the users, have the ability to learn from their observations and then make their own decisions without further human interference. With the comfort of automation also comes a subtle danger of our data being misused against our own interest.
If attempts to restrict the flow of our personal data consequently restricted our access to the said services and products as well... would we have any other option but to obey and share? Accessories for the Paranoid is a series of four objects designed to regain control over our personal data. As our physical environment reads, collects and stores an increasing amount of user information, this series of parasitic objects are designed to produce fake data. Through blurring our digital profiles, our true data identities get hidden behind a veil of fictive information.
The WiFi Impressionist
interactive installation
2019
The WiFi Impressionist is a digital artist device that draws on the tradition of impressionism. In real time it paints electromagnetic landscapes from abstract and non-corporeal WiFi signals accessible to the human eye.
Urban space is filled with thousands of wireless signals from mobile phones, computers and network infrastructure. These are clouds of information and data that we can neither see nor touch. The WiFi Impressionist is a digital artist device. It draws on the tradition of impressionism and tries to capture the radio spectrum, which, like light, is part of the electromagnetic spectrum. It then translates the collected data into visualizations that are accessible to the human eye. In real time it paints electromagnetic landscapes from abstract and non-corporeal WiFi signals. So far, it has created visualizations for Amsterdam, Rotterdam and Arnhem in the Netherlands. The WiFi Impressionist's works depict clusters of intersecting signals that spin around us. It was inspired by the painting of the English artist William Turner (1775-1851), known for creating romantic landscapes.
Magic Lining
dress made using e-textile
2017
The outside of clothes is exposed to external factors and the eyes of other people. The development of technology gives the inner, very intimate and hidden side of clothing a new meaning.
Clothing affects our behavior in different social contexts. Its outer side is exposed to external factors and the eyes of others. However, there is also the inner, very intimate and hidden side of every piece of clothing. It is only accessible and felt by the wearer and is in constant contact with the body. With the advent of intelligent textiles, also known as e-textiles, a new role of clothing is being shaped. Magic Lining examines the relationship between e-textile vibrations and the emotional state of the wearer. It draws attention to the need to understand how different stimuli affect our everyday life and how they can be used, triggering new emotions and reactions in the wearer, influencing how the wearer feels, how they perceive the outside world and how they perceive themselves.
e-Residency
e-service
2014
Access to highly advanced technological solutions makes data an indispensable part of everyday life, necessary for the smooth functioning of the society. They are also an element of the country-wide strategy.
Estonia is the first country to offer e-Residency, a government-issued digital identity and status that provides access to Estonia’s advanced e-governance services and trusted, transparent business environment. Estonian e-Residency provides digital entrepreneurs with the freedom to establish and manage an EU-based company completely online and from anywhere in the world. So far, more than 70,000 people from 165+ countries have applied for e-Residency, establishing over 13,000 Estonian companies. The program is a government-wide initiative lead by the e-Residency team, the Police and Border Guard Board and a number of government ministries that are dedicated to sharing Estonia's digital infrastructure and capabilities with the world. A digital ID card provides access to e-services, but it is not a valid form of physical identification and cannot be used as a travel document, it does not confer citizenship, tax residency, physical residency or right of entry to Estonia or the European Union.
Emotions are data
Emotions are the essence of humanity and a fascinating material that can also be transferred into data and then processed.
How do we recognize emotions? How do we know if someone is experiencing joy or sadness, if they are scared or surprised? We learn this by living among people and experiencing different emotional states ourselves. Machines can record these states because people send infinite amount of information through facial expressions, posture, gestures, breathing speed, voice tone and heart rate. Ways of expressing emotions are culturally conditioned and thus they differ across the world. They also depend on gender, personality and temperament. All this information can and does become a valuable material, which is then recorded and analyzed.
Designers work on numerous solutions that are based on emotions. In presented projects humans are the source of data. Emotions are turn into data and treat them as a material for further analysis and creative processing.
#
the project of experiencing data
2020
Many phenomena and limitations impact how do we experience, interpret, and communicate data. In the # experiment, we present the spectrum of these dependencies.
Emotions in are an important and respected object of psychological research. As they are naturally subjective, any access to them is strongly limited—they can only be experienced, interpreted and communicated by the subject experiencing an emotion. This is a rather serious limitation to all psychological research, and it hinders the interpretation of data about emotions.
First, there is the epistemological processing of emotions by the subjects themselves: just because you experience an emotion, it does not mean that you know what it is, where it directs you, how to experience or deal with it. This phenomenon was demonstrated in the famous 1974 study by Dutton and Aron, which shows that people interpret strong sensations caused by fear as sexual arousal. It is actually the same mechanism that makes fist driving attractive to the members of the opposite sex.
Second, there is the problem with the nature of the human language. An objective description of emotions is limited by the language we speak, which narrows down the possibilities of examining the emotions actually experienced. Communicating emotions is difficult; personal feelings are difficult to express or explain to others, who do not have access to our subjective experiences. Thus, imperfect communication makes any study of emotions difficult. As if that were not enough, according to Sapir-Whorf hypothesis, language facilitates recognition and interpretation of the named emotions through labels, but at the same time it hinders recognition of those unnamed. This means that the language itself, combined with the culture of a particular social group, affects the emotions communicated by people. In this way, it subtly shapes the emotional experience. This phenomenon, as well as the limitations entailed by this research method, are explored in the experiment #.
Dear Data
postcards
2015
Drawing data enables us to connect with ourselves at a deeper level. Sharing such drawings – developing a friendship.
Dear Data is analog data drawing project consisted of 104 postcards. Each week for one year, Lupi and Posavec gathered information on chosen aspects of their lives, e.g., moments of expressing dissatisfaction or annoyance about a particular thing, or positive feelings they were experiencing during the past seven days. They each visualized the collected personal data on postcards and sent them to each other across the Atlantic – Lupi from New York, Stefanie from London.
On the front of the postcard, there is a unique representation of weekly data, on the other side, detailed explanation of how to read the drawing. Dear Data documents a process of developing a friendship through data visualization. Instead of using data to become more efficient, Lupi and Posavec argue we can use data to become more human and to connect with ourselves and others at a deeper level.
Now project is in the permanent collection of the Museum of Modern Art’s in New York. The presented set of postcards represents Week 7: A week of complaints.
Collector of Emotions
web application
2019
People feed Internet systems with their emotions, and this emotional content is turned into data. Systems transform emotions into data because they have been pre-programmed to do so by people, who have also defined the purpose of these processes.
Collector of Emotions is a web app that tracks and stores the full spectrum of human emotions. The data from facial expression readings are translated into different colors, in contrast with the numerical approach used in regular emotion tracking systems. The concept of a tracking system that collects a wide range of non-standardized emotions was formulated by a designer. Then, the system was developed by programmers: a built-in computer webcam records facial expressions and facial checkpoints. Next, the data are processed and presented as a visual record of emotions, including their degree and depth. The application encourages us to rethink allowing Internet systems mining the data concerning our emotions. These systems are primarily designed to bring profit, not broaden human experience. The designer poses the question: how do you feel when a system deconstructs, analyzes and tracks your emotions?
The application is available HERE.
Manifesting the Look of Love
the service of materializing the Look of Love
2014
Love gazes are data that convey beauty. You just have to capture and save them, freezing in time a unique and elusive moment.
The look of love is one of the most personal, palpable, and sought after human sensations. Manifesting the Look of Love is a proposed design service that creates highly personalized objects designed by this intimate form of communication between two lovers. The service allows couples to mark significant intervals in their relationship with the creation of a completely unique artifact, directly formed by how they see one another. The gaze of each couple is mapped using custom eye-tracking technology — it captures the gazes of those in love and turns them into a set of data, i.e. hundreds of thousands of points, informing where each person's eyesight was directed, where they focused it and for how long. The resulting data is transformed into a physical object through parametric modeling tools and digital fabrication processes. These objects are realized in the material most closely associated with the duration of the couple’s relationship at their anniversary.
A lifetime subscription model provides couples with an annual appointment to collect new gaze data and evolve their bespoke collection of highly symbolic designed objects.
ListeningCups
3D printed porcelain cups
2019
Sounds can be fun, irritating, relaxing or annoying. What we feel is inextricably linked to the outside stimuli. The recording and materialization of sounds preserves the feelings which they evoked. Sounds can be fun, irritating, relaxing or annoying. What we feel is inextricably linked to the outside stimuli. The recording and materialization of sounds preserves the feelings which they evoked.
The ListeningCups is a set of 3D printed porcelain cups embedded with datasets of everyday ambient sounds. This project emerged from a collaboration between a ceramic artist Timea Tihanyi from Slip Rabbit digital ceramic lab and an interaction design researcher Audrey Desjardins who were exploring meaning-making around everyday data. Data of home noise levels were used to drive the texture on the surface of the 3D printed cups. Bumps on the cup represent short snippets of sounds recorded in various everyday places: Desjardins’s home, a restaurant, Slip Rabbit Studio, or street. The joint work of Desjardins and Tihanyi consisted in developing a process of recording ambient sound data, namely recording volume levels (decibels) and converting this information into machine code information (G-code), which then produced a texture on a Potterbot7 ceramic 3D printer.
Check out how the cup was made.
The Rhythm of Heart
installation
2017
The heart reveals emotions, its rhythm naturally synchronizes with what we are experiencing. Our heartbeat is beyond our control, and that is why it is a true reflection of our feelings.
Varvara & Mar artists from Estonia monitored the function of their hearts for a year. They translated the heart rate monitoring data from FitBit bands into the rhythm of metronomes. The audience of the installation can listen to the beat of the artists' hearts, and even set the metronomes to a selected date and time to recreate the rhythm from a chosen moment from the artists' archives. What you hear is related to the emotions that the artists were experiencing at a given moment. It is a critical project, encouraging reflection on the desirability of data mining by third parties and draws our attention to the monetization of data in the times of capitalist surveillance. The artists also initiate a discussion on the meaning and changing form of personal archives.
Data as Design Material
“AI is not going to kill us; it might make us more human”—said Holly Herndon, American composer, graduate of Stanford University. Herndon has created a special Artificial Intelligence system that collaborates with her in composing music.
Data can become a source of expression and inspiration. They help to predict user behavior and create personalized solutions that are developed in real time in a continuous learning process. Designers have skills that allow them to cope with new challenges and understand the complexities that arise in human-data interactions. They can contribute to habituating people to data that often seem confusing and hostile.
Data are often demonized, dreaded, and may seem terrifying. And yet, texts and numbers are a symbolic record of human stories and behaviors, experiences, reflections and desires. However, data do not speak for themselves. In order to give them meaning, it is necessary to distance oneself from technology and create a narrative about data, which will be embedded in everyday life. We are at the heart of the digital revolution and still do not see all the possible consequences. It is not the first time this has happened. It is worth remembering what happened during the industrial revolution—to draw conclusions and try not to make the same mistakes.
“Numbers can dehumanize and data is often a matter of numbers. When we reduce people to numbers, we risk abstracting human pain, even human rights. I'm thinking of immigration reports. The importance of designing data is not to be underestimated. Like our own speech and body language, the communicative power is in how we speak and move even more perhaps that what we say or in this case, what the numbers say.” Susan Yelavich (Professor Emerita, Design Studies, Parsons School of Design, The New School in New York)
Artificial intelligence methods and techniques are the tool, and data are the material. When designers have mastered this new tool and new material, then we can expect rapid progress in data humanization, rediscovery of data and the emergence of solutions that we have not thought about before. Designers can support the world of new technologies in changing the perspective by shifting the focus from data to people.
Exponential Growth
the project of experiencing data
2020
As the story goes, the invention of chess was highly appreciated by the king, who allowed the inventor to choose any reward he wanted. The wise man asked the king to give him some rice: one grain for the first field of the chessboard, twice as many (2 grains) for the second field, twice as many (4 grains) for the third field, twice as many (8 grains) for the fourth field and so on, up to 64 fields. The king, deceived by such a small amount, agreed to pay out the prize, which turned out to be a trap—it would amount to 18,446,744,073,709,550,000 grains. The King could not tolerate such a trap and had the wise man beheaded.
Both the king and the wise man fell victim to exponential growth. It consists in the fact that with the passage of each interval (second, week, year, etc.), the measured entity increases a certain number of times (2 times, 20 times, by 5%, etc.). In our legend, the amount of rice doubled on each successive chessboard field. However, the increase may take any other value.
Let us take a recent example of the COVID-19 pandemic. If a new, similar pandemic were to occur today and go out of hand, the unrestrained virus would spread exponentially, increasing the number of infected people about three times every week. After 3 weeks 27 people would be infected, after 6 weeks—729 people and after 12 weeks—531 441 people. After 21 weeks, every person in the world would be infected.
Exponential growth deceives our intuition—it seems slow and insignificant to become fast and unstoppable after reaching the critical point. In reality, however, every growth will end one day. The real question is: when? This question is explored in the Exponential Growth.
Challenges, Opportunities, Future
There is a lot of questions to be answered. To what extent will algorithms replace designers? When will designers learn to work with algorithms? How will Artificial Intelligence change the design practice? When will a new area of Data-driven Design emerge?
Data pose many challenges for designers. Gathering data can be stressful: Shall I delete this file? Should I keep it? The more data we have, the more difficult it is to organize them and search. The omnipresence of data increasingly affects our attention span and time management. "You can get addicted to data," says Piotr Migdał, Data Scientist. We hear increasingly often that algorithms are biased. Is it really so, or rather the results of algorithms reflects human behavior and decisions? According to Andrew Ng, Professor at Stanford University, in practice it is easier to reduce bias in AI than in people.
Professor Piotr Płoszajski (Warsaw School of Economics) draws attention to the spectacular effects of using artificial intelligence in diagnostics. A database and the description of particular symptoms allow algorithms to detect autism in infants. The symptoms that can be observed by people will develop in children aged about 4. AI is also extremely effective in imaging diagnostics, as well as genetic and space research. The most fascinating thing about data analysis is that in large pools of various data there are interdependencies which cannot be found without artificial intelligence algorithms, which in turn may lead to groundbreaking phenomena.
Designers are faced with the challenge of going beyond the context of human-centred and human-oriented design. Perhaps we will design together with machines and data—"Technology can be not only the material, but also a participant in the design process". (Giaccardi & Redström, 2020). Designers will both use and cooperate with data in the design process. Perhaps, we are currently on the verge of exhausting the current design framework and structures.
Data analysis is a creative process, just like design. According to Ewa Bugajska, Senior Data Analyst at Brainly, "an analyst is a data guide who protects the designer from over-interpretation". A good analyst is a narrator. Designers also use narratives, for example when they create scenarios for using designed solutions, generate ideas or present concepts. Stories and narratives can become a bridge between the world of designers and data analysts. The clash of such narratives may contribute to a breakthrough in design.
In her Nobel Prize lecture, Olga Tokarczuk wrote: "It has turned out that we are not capable of bearing this enormity of information, which instead of uniting, generalizing and freeing, has differentiated, divided, enclosed in individual little bubbles, creating a multitude of stories that are incompatible with one another or even openly hostile toward each other, mutually antagonizing." Will designers be able to change that?
Optical Brain
prototype of an optical neural network based on optical fibers
2020
Designers' work has always consisted in extracting key observations from data, ordering complexity, finding meaning in the maze of information, combining seemingly distant phenomena. Designers visualize information, prioritize it. Similar words can be used to describe operation of many algorithms and systems based on Artificial Intelligence methods and techniques.
Will the design process become automated? Will designers specializing in problem solving replace algorithms? Michał Witkowski, Lead Game Economy Designer at Rage Quit Games shows an interesting trend: "Analytics is becoming another support system for design." But will people ever learn to work with algorithms which achieve unimaginable performance levels?
The installation of Jakub Mielczarek presents a prototype of an optical artificial neural network based on optical fiber. Such a simple optical neural network can be built by each of us, using commonly available elements for fiber-optic Internet networks. If we could simulate the operation of biological neural networks with light, they could process information about 2.5 million times faster than the human brain. One day of our lives would therefore correspond to about four hundredths of a second of the optical brain. One earthly year would take about 13 seconds in the optical simulation. In turn, in the optical world, a simulation of our entire life would take no longer than twenty minutes! Artificial Intelligence can become more powerful than human intelligence in terms of both quantity and the speed of information processing. Is it possible that such super-fast artificial intelligence will remain in contact with humans?
